[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:00:46.229679  7164 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.255.62:35591
I20260709 13:00:46.231087  7164 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:00:46.232077  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:46.248435  7173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:46.248426  7171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:46.250821  7164 server_base.cc:1061] running on GCE node
W20260709 13:00:46.249147  7170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:47.451802  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:47.452099  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:47.452309  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602047452282 us; error 0 us; skew 500 ppm
I20260709 13:00:47.453172  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:47.458703  7164 webserver.cc:533] Webserver started at http://127.6.255.62:44103/ using document root <none> and password file <none>
I20260709 13:00:47.459582  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:47.459822  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:47.460268  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:47.466010  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602046064212-7164-0/minicluster-data/master-0-root/instance:
uuid: "4127d5019f104ffc89c7721a5cfd02d1"
format_stamp: "Formatted at 2026-07-09 13:00:47 on dist-test-slave-rnv4"
I20260709 13:00:47.473726  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:00:47.479346  7179 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:47.480533  7164 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:00:47.480914  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602046064212-7164-0/minicluster-data/master-0-root
uuid: "4127d5019f104ffc89c7721a5cfd02d1"
format_stamp: "Formatted at 2026-07-09 13:00:47 on dist-test-slave-rnv4"
I20260709 13:00:47.481271  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602046064212-7164-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602046064212-7164-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602046064212-7164-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:47.533708  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:47.535251  7164 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:00:47.535737  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:47.608624  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.62:35591
I20260709 13:00:47.608708  7230 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.62:35591 every 8 connection(s)
I20260709 13:00:47.616739  7231 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:47.640657  7231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1: Bootstrap starting.
I20260709 13:00:47.647500  7231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:47.649473  7231 log.cc:826] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1: Log is configured to *not* fsync() on all Append() calls
I20260709 13:00:47.654313  7231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1: No bootstrap required, opened a new log
I20260709 13:00:47.676899  7231 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4127d5019f104ffc89c7721a5cfd02d1" member_type: VOTER }
I20260709 13:00:47.677562  7231 raft_consensus.cc:385] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:47.677795  7231 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4127d5019f104ffc89c7721a5cfd02d1, State: Initialized, Role: FOLLOWER
I20260709 13:00:47.678787  7231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4127d5019f104ffc89c7721a5cfd02d1" member_type: VOTER }
I20260709 13:00:47.679288  7231 raft_consensus.cc:399] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:00:47.679571  7231 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:00:47.679881  7231 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:47.684250  7231 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4127d5019f104ffc89c7721a5cfd02d1" member_type: VOTER }
I20260709 13:00:47.684984  7231 leader_election.cc:304] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4127d5019f104ffc89c7721a5cfd02d1; no voters: 
I20260709 13:00:47.686805  7231 leader_election.cc:290] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:00:47.687176  7234 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:47.689566  7234 raft_consensus.cc:697] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1 [term 1 LEADER]: Becoming Leader. State: Replica: 4127d5019f104ffc89c7721a5cfd02d1, State: Running, Role: LEADER
I20260709 13:00:47.690563  7234 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4127d5019f104ffc89c7721a5cfd02d1" member_type: VOTER }
I20260709 13:00:47.691720  7231 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:00:47.701118  7235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4127d5019f104ffc89c7721a5cfd02d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4127d5019f104ffc89c7721a5cfd02d1" member_type: VOTER } }
I20260709 13:00:47.701941  7235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1 [sys.catalog]: This master's current role is: LEADER
I20260709 13:00:47.702092  7236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4127d5019f104ffc89c7721a5cfd02d1. Latest consensus state: current_term: 1 leader_uuid: "4127d5019f104ffc89c7721a5cfd02d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4127d5019f104ffc89c7721a5cfd02d1" member_type: VOTER } }
I20260709 13:00:47.702659  7236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1 [sys.catalog]: This master's current role is: LEADER
I20260709 13:00:47.705098  7242 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:00:47.718379  7242 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:00:47.720784  7164 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:00:47.734710  7242 catalog_manager.cc:1383] Generated new cluster ID: 28a4a452d36e49b3915511c06b4be613
I20260709 13:00:47.735097  7242 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:00:47.763677  7242 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:00:47.765177  7242 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:00:47.786778  7242 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1: Generated new TSK 0
I20260709 13:00:47.787681  7242 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:00:47.851600  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:47.860319  7253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:47.863127  7254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:47.865593  7256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:47.866022  7164 server_base.cc:1061] running on GCE node
I20260709 13:00:47.866806  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:47.867043  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:47.867229  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602047867211 us; error 0 us; skew 500 ppm
I20260709 13:00:47.868162  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:47.872805  7164 webserver.cc:533] Webserver started at http://127.6.255.1:46841/ using document root <none> and password file <none>
I20260709 13:00:47.873356  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:47.873620  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:47.873901  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:47.874994  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602046064212-7164-0/minicluster-data/ts-0-root/instance:
uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
format_stamp: "Formatted at 2026-07-09 13:00:47 on dist-test-slave-rnv4"
I20260709 13:00:47.880842  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:00:47.885522  7261 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:47.886464  7164 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:00:47.886762  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602046064212-7164-0/minicluster-data/ts-0-root
uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
format_stamp: "Formatted at 2026-07-09 13:00:47 on dist-test-slave-rnv4"
I20260709 13:00:47.887064  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602046064212-7164-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602046064212-7164-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602046064212-7164-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:47.914033  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:47.915441  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:47.918388  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:47.923084  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:47.923318  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:47.923623  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:47.923794  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:48.031379  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.1:38881
I20260709 13:00:48.031505  7323 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.1:38881 every 8 connection(s)
I20260709 13:00:48.039444  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:48.052022  7328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:48.054903  7329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:48.059314  7331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:48.060089  7164 server_base.cc:1061] running on GCE node
I20260709 13:00:48.061214  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:48.061492  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:48.061685  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602048061663 us; error 0 us; skew 500 ppm
I20260709 13:00:48.062503  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:48.065653  7164 webserver.cc:533] Webserver started at http://127.6.255.2:44033/ using document root <none> and password file <none>
I20260709 13:00:48.066349  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:48.066608  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:48.066913  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:48.068219  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602046064212-7164-0/minicluster-data/ts-1-root/instance:
uuid: "97c6951c9f204a3db527b90376f3ea36"
format_stamp: "Formatted at 2026-07-09 13:00:48 on dist-test-slave-rnv4"
I20260709 13:00:48.069294  7324 heartbeater.cc:344] Connected to a master server at 127.6.255.62:35591
I20260709 13:00:48.069798  7324 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:48.070986  7324 heartbeater.cc:507] Master 127.6.255.62:35591 requested a full tablet report, sending...
I20260709 13:00:48.074327  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.005s
I20260709 13:00:48.074851  7196 ts_manager.cc:194] Registered new tserver with Master: ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881)
I20260709 13:00:48.077883  7196 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50498
I20260709 13:00:48.079839  7336 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:48.081064  7164 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.002s
I20260709 13:00:48.081390  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602046064212-7164-0/minicluster-data/ts-1-root
uuid: "97c6951c9f204a3db527b90376f3ea36"
format_stamp: "Formatted at 2026-07-09 13:00:48 on dist-test-slave-rnv4"
I20260709 13:00:48.081835  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602046064212-7164-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602046064212-7164-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602046064212-7164-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:48.102500  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:48.103781  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:48.105862  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:48.108788  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:48.109004  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:48.109242  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:48.109457  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:48.202877  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.2:46257
I20260709 13:00:48.203022  7398 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.2:46257 every 8 connection(s)
I20260709 13:00:48.208647  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:48.219039  7403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:48.222590  7399 heartbeater.cc:344] Connected to a master server at 127.6.255.62:35591
I20260709 13:00:48.222990  7399 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:48.223855  7399 heartbeater.cc:507] Master 127.6.255.62:35591 requested a full tablet report, sending...
I20260709 13:00:48.226128  7196 ts_manager.cc:194] Registered new tserver with Master: 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257)
I20260709 13:00:48.227864  7196 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50500
W20260709 13:00:48.230449  7402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:48.231299  7406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:48.231348  7164 server_base.cc:1061] running on GCE node
I20260709 13:00:48.232578  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:48.232801  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:48.232972  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602048232950 us; error 0 us; skew 500 ppm
I20260709 13:00:48.233577  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:48.235953  7164 webserver.cc:533] Webserver started at http://127.6.255.3:37667/ using document root <none> and password file <none>
I20260709 13:00:48.236490  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:48.236701  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:48.236963  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:48.238034  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602046064212-7164-0/minicluster-data/ts-2-root/instance:
uuid: "29546ef8491d4dcba75459ed3d780497"
format_stamp: "Formatted at 2026-07-09 13:00:48 on dist-test-slave-rnv4"
I20260709 13:00:48.242539  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:00:48.245848  7410 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:48.246670  7164 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:00:48.246954  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602046064212-7164-0/minicluster-data/ts-2-root
uuid: "29546ef8491d4dcba75459ed3d780497"
format_stamp: "Formatted at 2026-07-09 13:00:48 on dist-test-slave-rnv4"
I20260709 13:00:48.247244  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602046064212-7164-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602046064212-7164-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602046064212-7164-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:48.257521  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:48.258723  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:48.261292  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:48.264256  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:48.264508  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:48.264750  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:48.264918  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:48.368842  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.3:44593
I20260709 13:00:48.368997  7472 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.3:44593 every 8 connection(s)
I20260709 13:00:48.386531  7473 heartbeater.cc:344] Connected to a master server at 127.6.255.62:35591
I20260709 13:00:48.386914  7473 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:48.387668  7473 heartbeater.cc:507] Master 127.6.255.62:35591 requested a full tablet report, sending...
I20260709 13:00:48.389853  7196 ts_manager.cc:194] Registered new tserver with Master: 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:48.390689  7164 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016300618s
I20260709 13:00:48.391566  7196 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50514
I20260709 13:00:48.452081  7196 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50526:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:00:48.457142  7196 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:00:48.584628  7438 tablet_service.cc:1511] Processing CreateTablet for tablet e451ff7482b5487c8db99685c67f888d (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:00:48.591166  7435 tablet_service.cc:1511] Processing CreateTablet for tablet 7d16cc32774d49fbbd48d20cd2618904 (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:00:48.593116  7438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e451ff7482b5487c8db99685c67f888d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.591217  7288 tablet_service.cc:1511] Processing CreateTablet for tablet dbc48c8abc054511b1c78331b267f725 (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:00:48.587817  7289 tablet_service.cc:1511] Processing CreateTablet for tablet e451ff7482b5487c8db99685c67f888d (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:00:48.593247  7286 tablet_service.cc:1511] Processing CreateTablet for tablet f55dd47f5d644c3eb24678e018b317e1 (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:00:48.597009  7288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbc48c8abc054511b1c78331b267f725. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.595359  7287 tablet_service.cc:1511] Processing CreateTablet for tablet 7d16cc32774d49fbbd48d20cd2618904 (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:00:48.598723  7436 tablet_service.cc:1511] Processing CreateTablet for tablet f55dd47f5d644c3eb24678e018b317e1 (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:00:48.596979  7437 tablet_service.cc:1511] Processing CreateTablet for tablet dbc48c8abc054511b1c78331b267f725 (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:00:48.598342  7285 tablet_service.cc:1511] Processing CreateTablet for tablet d144c1f909d74a298d08c0f8b19cd9e0 (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:00:48.603681  7289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e451ff7482b5487c8db99685c67f888d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.603133  7434 tablet_service.cc:1511] Processing CreateTablet for tablet d144c1f909d74a298d08c0f8b19cd9e0 (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:00:48.606241  7286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f55dd47f5d644c3eb24678e018b317e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.616346  7287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d16cc32774d49fbbd48d20cd2618904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.616530  7435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d16cc32774d49fbbd48d20cd2618904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.620124  7284 tablet_service.cc:1511] Processing CreateTablet for tablet 3c646ace346640f8a3a4939d5c2aab5e (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:00:48.624461  7436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f55dd47f5d644c3eb24678e018b317e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.624450  7284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c646ace346640f8a3a4939d5c2aab5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.629498  7434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d144c1f909d74a298d08c0f8b19cd9e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.633504  7364 tablet_service.cc:1511] Processing CreateTablet for tablet e451ff7482b5487c8db99685c67f888d (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:00:48.633970  7437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbc48c8abc054511b1c78331b267f725. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.634028  7363 tablet_service.cc:1511] Processing CreateTablet for tablet dbc48c8abc054511b1c78331b267f725 (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:00:48.641696  7362 tablet_service.cc:1511] Processing CreateTablet for tablet f55dd47f5d644c3eb24678e018b317e1 (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:00:48.649255  7433 tablet_service.cc:1511] Processing CreateTablet for tablet 3c646ace346640f8a3a4939d5c2aab5e (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:00:48.653249  7363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbc48c8abc054511b1c78331b267f725. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.653518  7285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d144c1f909d74a298d08c0f8b19cd9e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.655050  7433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c646ace346640f8a3a4939d5c2aab5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.655033  7359 tablet_service.cc:1511] Processing CreateTablet for tablet 3c646ace346640f8a3a4939d5c2aab5e (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:00:48.652215  7361 tablet_service.cc:1511] Processing CreateTablet for tablet 7d16cc32774d49fbbd48d20cd2618904 (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:00:48.668525  7362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f55dd47f5d644c3eb24678e018b317e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.677662  7364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e451ff7482b5487c8db99685c67f888d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.673539  7283 tablet_service.cc:1511] Processing CreateTablet for tablet 1e97321e887f48a4b24dce648a4aa34d (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:00:48.678664  7432 tablet_service.cc:1511] Processing CreateTablet for tablet 1e97321e887f48a4b24dce648a4aa34d (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:00:48.671227  7360 tablet_service.cc:1511] Processing CreateTablet for tablet d144c1f909d74a298d08c0f8b19cd9e0 (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:00:48.681351  7358 tablet_service.cc:1511] Processing CreateTablet for tablet 1e97321e887f48a4b24dce648a4aa34d (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:00:48.681627  7432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e97321e887f48a4b24dce648a4aa34d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.682211  7283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e97321e887f48a4b24dce648a4aa34d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.692171  7361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d16cc32774d49fbbd48d20cd2618904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.705232  7360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d144c1f909d74a298d08c0f8b19cd9e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.712024  7358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e97321e887f48a4b24dce648a4aa34d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:00:48.714550  7244 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:48.716583  7359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c646ace346640f8a3a4939d5c2aab5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.718142  7282 tablet_service.cc:1511] Processing CreateTablet for tablet 69dba3b1b71e434cac316850c2da8c01 (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:00:48.719573  7282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69dba3b1b71e434cac316850c2da8c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.737413  7491 tablet_bootstrap.cc:492] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe: Bootstrap starting.
I20260709 13:00:48.764731  7357 tablet_service.cc:1511] Processing CreateTablet for tablet 69dba3b1b71e434cac316850c2da8c01 (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:00:48.768122  7357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69dba3b1b71e434cac316850c2da8c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.772809  7431 tablet_service.cc:1511] Processing CreateTablet for tablet 69dba3b1b71e434cac316850c2da8c01 (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:00:48.777806  7431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69dba3b1b71e434cac316850c2da8c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.797040  7282 tablet_service.cc:1511] Processing CreateTablet for tablet b90380b15f1a42f2b3c9736536c43e21 (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:00:48.804533  7282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b90380b15f1a42f2b3c9736536c43e21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.817992  7356 tablet_service.cc:1511] Processing CreateTablet for tablet b90380b15f1a42f2b3c9736536c43e21 (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:00:48.820048  7356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b90380b15f1a42f2b3c9736536c43e21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.815654  7491 tablet_bootstrap.cc:654] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:48.836469  7435 tablet_service.cc:1511] Processing CreateTablet for tablet b90380b15f1a42f2b3c9736536c43e21 (DEFAULT_TABLE table=test-workload [id=4aa6d5ab924741b29a78a8fb3110c29b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:00:48.837873  7435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b90380b15f1a42f2b3c9736536c43e21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.843798  7492 tablet_bootstrap.cc:492] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497: Bootstrap starting.
I20260709 13:00:48.884766  7493 tablet_bootstrap.cc:492] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36: Bootstrap starting.
I20260709 13:00:48.899770  7491 tablet_bootstrap.cc:492] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe: No bootstrap required, opened a new log
I20260709 13:00:48.900272  7491 ts_tablet_manager.cc:1403] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe: Time spent bootstrapping tablet: real 0.163s	user 0.008s	sys 0.013s
I20260709 13:00:48.902146  7492 tablet_bootstrap.cc:654] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:48.903543  7493 tablet_bootstrap.cc:654] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:48.903929  7491 raft_consensus.cc:359] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:48.904737  7491 raft_consensus.cc:385] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:48.905038  7491 raft_consensus.cc:740] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Initialized, Role: FOLLOWER
I20260709 13:00:48.905921  7491 consensus_queue.cc:260] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:48.915199  7492 tablet_bootstrap.cc:492] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497: No bootstrap required, opened a new log
I20260709 13:00:48.915900  7492 ts_tablet_manager.cc:1403] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497: Time spent bootstrapping tablet: real 0.083s	user 0.012s	sys 0.009s
I20260709 13:00:48.916302  7491 ts_tablet_manager.cc:1434] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.008s
I20260709 13:00:48.917145  7324 heartbeater.cc:499] Master 127.6.255.62:35591 was elected leader, sending a full tablet report...
I20260709 13:00:48.917528  7491 tablet_bootstrap.cc:492] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe: Bootstrap starting.
I20260709 13:00:48.920092  7492 raft_consensus.cc:359] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:48.926836  7492 raft_consensus.cc:385] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:48.927152  7492 raft_consensus.cc:740] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29546ef8491d4dcba75459ed3d780497, State: Initialized, Role: FOLLOWER
I20260709 13:00:48.928267  7493 tablet_bootstrap.cc:492] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36: No bootstrap required, opened a new log
I20260709 13:00:48.927995  7492 consensus_queue.cc:260] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:48.928736  7493 ts_tablet_manager.cc:1403] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36: Time spent bootstrapping tablet: real 0.045s	user 0.007s	sys 0.023s
I20260709 13:00:48.923714  7491 tablet_bootstrap.cc:654] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:48.931780  7493 raft_consensus.cc:359] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:48.932514  7493 raft_consensus.cc:385] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:48.932853  7493 raft_consensus.cc:740] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97c6951c9f204a3db527b90376f3ea36, State: Initialized, Role: FOLLOWER
I20260709 13:00:48.933611  7493 consensus_queue.cc:260] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:48.942638  7493 ts_tablet_manager.cc:1434] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36: Time spent starting tablet: real 0.014s	user 0.013s	sys 0.001s
I20260709 13:00:48.943403  7399 heartbeater.cc:499] Master 127.6.255.62:35591 was elected leader, sending a full tablet report...
I20260709 13:00:48.943588  7493 tablet_bootstrap.cc:492] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36: Bootstrap starting.
I20260709 13:00:48.951676  7493 tablet_bootstrap.cc:654] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:48.959205  7492 ts_tablet_manager.cc:1434] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497: Time spent starting tablet: real 0.043s	user 0.001s	sys 0.019s
W20260709 13:00:48.959532  7400 tablet.cc:2406] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:00:48.959786  7473 heartbeater.cc:499] Master 127.6.255.62:35591 was elected leader, sending a full tablet report...
I20260709 13:00:48.960225  7492 tablet_bootstrap.cc:492] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497: Bootstrap starting.
I20260709 13:00:48.963544  7493 tablet_bootstrap.cc:492] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36: No bootstrap required, opened a new log
I20260709 13:00:48.964017  7493 ts_tablet_manager.cc:1403] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.000s
I20260709 13:00:48.966702  7492 tablet_bootstrap.cc:654] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:48.967069  7493 raft_consensus.cc:359] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:48.969267  7493 raft_consensus.cc:385] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:48.969659  7493 raft_consensus.cc:740] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97c6951c9f204a3db527b90376f3ea36, State: Initialized, Role: FOLLOWER
I20260709 13:00:48.970445  7493 consensus_queue.cc:260] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:48.971989  7491 tablet_bootstrap.cc:492] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe: No bootstrap required, opened a new log
I20260709 13:00:48.972579  7491 ts_tablet_manager.cc:1403] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe: Time spent bootstrapping tablet: real 0.055s	user 0.014s	sys 0.009s
I20260709 13:00:48.972994  7493 ts_tablet_manager.cc:1434] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:00:48.973985  7493 tablet_bootstrap.cc:492] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36: Bootstrap starting.
I20260709 13:00:48.975521  7491 raft_consensus.cc:359] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:48.976271  7491 raft_consensus.cc:385] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:48.976574  7491 raft_consensus.cc:740] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Initialized, Role: FOLLOWER
I20260709 13:00:48.977332  7491 consensus_queue.cc:260] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:48.979866  7491 ts_tablet_manager.cc:1434] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:00:48.980801  7491 tablet_bootstrap.cc:492] T d144c1f909d74a298d08c0f8b19cd9e0 P ceaa51da210b427f92d4bb4a95b35bbe: Bootstrap starting.
I20260709 13:00:48.981554  7493 tablet_bootstrap.cc:654] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:48.985720  7492 tablet_bootstrap.cc:492] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497: No bootstrap required, opened a new log
I20260709 13:00:48.986336  7492 ts_tablet_manager.cc:1403] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.002s
I20260709 13:00:48.989699  7492 raft_consensus.cc:359] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:48.990409  7492 raft_consensus.cc:385] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:48.990703  7492 raft_consensus.cc:740] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29546ef8491d4dcba75459ed3d780497, State: Initialized, Role: FOLLOWER
I20260709 13:00:48.993784  7492 consensus_queue.cc:260] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:48.995759  7492 ts_tablet_manager.cc:1434] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:00:48.997571  7491 tablet_bootstrap.cc:654] T d144c1f909d74a298d08c0f8b19cd9e0 P ceaa51da210b427f92d4bb4a95b35bbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:48.999955  7493 tablet_bootstrap.cc:492] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36: No bootstrap required, opened a new log
I20260709 13:00:49.000408  7493 ts_tablet_manager.cc:1403] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36: Time spent bootstrapping tablet: real 0.027s	user 0.003s	sys 0.013s
I20260709 13:00:49.003317  7493 raft_consensus.cc:359] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } } peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } }
I20260709 13:00:49.004029  7493 raft_consensus.cc:385] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.004323  7493 raft_consensus.cc:740] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97c6951c9f204a3db527b90376f3ea36, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.005803  7492 tablet_bootstrap.cc:492] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497: Bootstrap starting.
I20260709 13:00:49.004984  7493 consensus_queue.cc:260] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } } peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } }
I20260709 13:00:49.011437  7493 ts_tablet_manager.cc:1434] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:00:49.012629  7493 tablet_bootstrap.cc:492] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36: Bootstrap starting.
I20260709 13:00:49.013008  7492 tablet_bootstrap.cc:654] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.016340  7498 raft_consensus.cc:493] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.016979  7498 raft_consensus.cc:515] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.019425  7491 tablet_bootstrap.cc:492] T d144c1f909d74a298d08c0f8b19cd9e0 P ceaa51da210b427f92d4bb4a95b35bbe: No bootstrap required, opened a new log
I20260709 13:00:49.019935  7491 ts_tablet_manager.cc:1403] T d144c1f909d74a298d08c0f8b19cd9e0 P ceaa51da210b427f92d4bb4a95b35bbe: Time spent bootstrapping tablet: real 0.039s	user 0.018s	sys 0.009s
I20260709 13:00:49.023108  7491 raft_consensus.cc:359] T d144c1f909d74a298d08c0f8b19cd9e0 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.023881  7491 raft_consensus.cc:385] T d144c1f909d74a298d08c0f8b19cd9e0 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.024165  7491 raft_consensus.cc:740] T d144c1f909d74a298d08c0f8b19cd9e0 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.025933  7498 leader_election.cc:290] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257)
I20260709 13:00:49.024825  7491 consensus_queue.cc:260] T d144c1f909d74a298d08c0f8b19cd9e0 P ceaa51da210b427f92d4bb4a95b35bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.034391  7492 tablet_bootstrap.cc:492] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497: No bootstrap required, opened a new log
I20260709 13:00:49.034865  7492 ts_tablet_manager.cc:1403] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.008s
I20260709 13:00:49.036887  7491 ts_tablet_manager.cc:1434] T d144c1f909d74a298d08c0f8b19cd9e0 P ceaa51da210b427f92d4bb4a95b35bbe: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.002s
I20260709 13:00:49.037930  7491 tablet_bootstrap.cc:492] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe: Bootstrap starting.
I20260709 13:00:49.037797  7492 raft_consensus.cc:359] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } } peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } }
I20260709 13:00:49.038550  7492 raft_consensus.cc:385] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.038843  7492 raft_consensus.cc:740] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29546ef8491d4dcba75459ed3d780497, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.039628  7492 consensus_queue.cc:260] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } } peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } }
I20260709 13:00:49.046937  7500 raft_consensus.cc:493] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.047776  7500 raft_consensus.cc:515] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.048476  7498 raft_consensus.cc:493] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.049464  7498 raft_consensus.cc:515] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.052011  7498 leader_election.cc:290] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257)
I20260709 13:00:49.054937  7500 leader_election.cc:290] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:49.079077  7491 tablet_bootstrap.cc:654] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.075856  7492 ts_tablet_manager.cc:1434] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497: Time spent starting tablet: real 0.041s	user 0.004s	sys 0.005s
I20260709 13:00:49.082118  7500 raft_consensus.cc:493] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.084419  7500 raft_consensus.cc:515] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.086498  7500 leader_election.cc:290] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:49.087971  7492 tablet_bootstrap.cc:492] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497: Bootstrap starting.
I20260709 13:00:49.091821  7374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55dd47f5d644c3eb24678e018b317e1" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c6951c9f204a3db527b90376f3ea36" is_pre_election: true
I20260709 13:00:49.092991  7493 tablet_bootstrap.cc:654] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.093859  7374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d144c1f909d74a298d08c0f8b19cd9e0" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c6951c9f204a3db527b90376f3ea36" is_pre_election: true
I20260709 13:00:49.098841  7374 raft_consensus.cc:2468] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29546ef8491d4dcba75459ed3d780497 in term 0.
W20260709 13:00:49.094918  7414 leader_election.cc:343] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:49.098152  7492 tablet_bootstrap.cc:654] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.100536  7414 leader_election.cc:304] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, 97c6951c9f204a3db527b90376f3ea36; no voters: 
I20260709 13:00:49.101584  7498 raft_consensus.cc:2804] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:49.101941  7498 raft_consensus.cc:493] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.102303  7498 raft_consensus.cc:3060] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.102926  7491 tablet_bootstrap.cc:492] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe: No bootstrap required, opened a new log
I20260709 13:00:49.103471  7491 ts_tablet_manager.cc:1403] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe: Time spent bootstrapping tablet: real 0.066s	user 0.008s	sys 0.003s
I20260709 13:00:49.109301  7498 raft_consensus.cc:515] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.111865  7498 leader_election.cc:290] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 1 election: Requested vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257)
I20260709 13:00:49.113219  7374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d144c1f909d74a298d08c0f8b19cd9e0" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
I20260709 13:00:49.115562  7514 raft_consensus.cc:493] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.115885  7299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d144c1f909d74a298d08c0f8b19cd9e0" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" is_pre_election: true
I20260709 13:00:49.115585  7491 raft_consensus.cc:359] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.117007  7374 raft_consensus.cc:3060] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.117273  7491 raft_consensus.cc:385] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.117209  7299 raft_consensus.cc:2468] T d144c1f909d74a298d08c0f8b19cd9e0 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97c6951c9f204a3db527b90376f3ea36 in term 0.
I20260709 13:00:49.117518  7298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbc48c8abc054511b1c78331b267f725" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" is_pre_election: true
I20260709 13:00:49.117787  7491 raft_consensus.cc:740] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.118445  7497 raft_consensus.cc:493] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.119086  7298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55dd47f5d644c3eb24678e018b317e1" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" is_pre_election: true
I20260709 13:00:49.118885  7491 consensus_queue.cc:260] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.118891  7497 raft_consensus.cc:515] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.119935  7298 raft_consensus.cc:2468] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29546ef8491d4dcba75459ed3d780497 in term 0.
I20260709 13:00:49.121352  7298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d144c1f909d74a298d08c0f8b19cd9e0" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" is_pre_election: true
I20260709 13:00:49.122021  7298 raft_consensus.cc:2468] T d144c1f909d74a298d08c0f8b19cd9e0 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29546ef8491d4dcba75459ed3d780497 in term 0.
I20260709 13:00:49.123683  7497 leader_election.cc:290] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:49.124385  7298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d144c1f909d74a298d08c0f8b19cd9e0" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
I20260709 13:00:49.124938  7298 raft_consensus.cc:3060] T d144c1f909d74a298d08c0f8b19cd9e0 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.124075  7374 raft_consensus.cc:2468] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29546ef8491d4dcba75459ed3d780497 in term 1.
I20260709 13:00:49.126061  7493 tablet_bootstrap.cc:492] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36: No bootstrap required, opened a new log
I20260709 13:00:49.126636  7493 ts_tablet_manager.cc:1403] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36: Time spent bootstrapping tablet: real 0.114s	user 0.014s	sys 0.004s
I20260709 13:00:49.130074  7338 leader_election.cc:304] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97c6951c9f204a3db527b90376f3ea36, ceaa51da210b427f92d4bb4a95b35bbe; no voters: 
I20260709 13:00:49.130878  7500 raft_consensus.cc:2764] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
W20260709 13:00:49.131264  7338 leader_election.cc:343] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:49.131506  7298 raft_consensus.cc:2468] T d144c1f909d74a298d08c0f8b19cd9e0 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29546ef8491d4dcba75459ed3d780497 in term 1.
I20260709 13:00:49.132398  7412 leader_election.cc:304] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, ceaa51da210b427f92d4bb4a95b35bbe; no voters: 97c6951c9f204a3db527b90376f3ea36
I20260709 13:00:49.132262  7493 raft_consensus.cc:359] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.133217  7493 raft_consensus.cc:385] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.133600  7493 raft_consensus.cc:740] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97c6951c9f204a3db527b90376f3ea36, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.129652  7514 raft_consensus.cc:515] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } } peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } }
I20260709 13:00:49.137091  7299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e451ff7482b5487c8db99685c67f888d" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" is_pre_election: true
I20260709 13:00:49.138298  7498 raft_consensus.cc:2804] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:49.138363  7448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d144c1f909d74a298d08c0f8b19cd9e0" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29546ef8491d4dcba75459ed3d780497" is_pre_election: true
I20260709 13:00:49.138823  7498 raft_consensus.cc:493] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.138729  7447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbc48c8abc054511b1c78331b267f725" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29546ef8491d4dcba75459ed3d780497" is_pre_election: true
I20260709 13:00:49.139416  7448 raft_consensus.cc:2393] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97c6951c9f204a3db527b90376f3ea36 in current term 1: Already voted for candidate 29546ef8491d4dcba75459ed3d780497 in this term.
W20260709 13:00:49.139825  7412 leader_election.cc:343] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:49.140880  7337 leader_election.cc:343] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:49.141041  7498 raft_consensus.cc:3060] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.141323  7337 leader_election.cc:304] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97c6951c9f204a3db527b90376f3ea36; no voters: 29546ef8491d4dcba75459ed3d780497, ceaa51da210b427f92d4bb4a95b35bbe
I20260709 13:00:49.142114  7500 raft_consensus.cc:2749] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:49.144169  7414 leader_election.cc:304] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, 97c6951c9f204a3db527b90376f3ea36; no voters: 
I20260709 13:00:49.148404  7497 raft_consensus.cc:493] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.149467  7492 tablet_bootstrap.cc:492] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497: No bootstrap required, opened a new log
I20260709 13:00:49.144650  7514 leader_election.cc:290] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257)
I20260709 13:00:49.149401  7497 raft_consensus.cc:515] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.150146  7492 ts_tablet_manager.cc:1403] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497: Time spent bootstrapping tablet: real 0.062s	user 0.012s	sys 0.005s
I20260709 13:00:49.153260  7497 leader_election.cc:290] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:49.151578  7514 raft_consensus.cc:2804] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:49.153935  7514 raft_consensus.cc:697] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 1 LEADER]: Becoming Leader. State: Replica: 29546ef8491d4dcba75459ed3d780497, State: Running, Role: LEADER
I20260709 13:00:49.154858  7492 raft_consensus.cc:359] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.154744  7514 consensus_queue.cc:237] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.155522  7492 raft_consensus.cc:385] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.145920  7493 consensus_queue.cc:260] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.156597  7492 raft_consensus.cc:740] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29546ef8491d4dcba75459ed3d780497, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.158452  7492 consensus_queue.cc:260] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.158938  7491 ts_tablet_manager.cc:1434] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe: Time spent starting tablet: real 0.055s	user 0.009s	sys 0.010s
I20260709 13:00:49.162328  7491 tablet_bootstrap.cc:492] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe: Bootstrap starting.
I20260709 13:00:49.164418  7493 ts_tablet_manager.cc:1434] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.002s
I20260709 13:00:49.156456  7498 raft_consensus.cc:515] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.169942  7498 leader_election.cc:290] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 1 election: Requested vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257)
I20260709 13:00:49.170663  7298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55dd47f5d644c3eb24678e018b317e1" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
I20260709 13:00:49.171264  7298 raft_consensus.cc:3060] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.178571  7493 tablet_bootstrap.cc:492] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36: Bootstrap starting.
I20260709 13:00:49.195619  7492 ts_tablet_manager.cc:1434] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497: Time spent starting tablet: real 0.044s	user 0.005s	sys 0.018s
I20260709 13:00:49.196673  7492 tablet_bootstrap.cc:492] T 3c646ace346640f8a3a4939d5c2aab5e P 29546ef8491d4dcba75459ed3d780497: Bootstrap starting.
I20260709 13:00:49.200304  7298 raft_consensus.cc:2468] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29546ef8491d4dcba75459ed3d780497 in term 1.
I20260709 13:00:49.202026  7412 leader_election.cc:304] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, ceaa51da210b427f92d4bb4a95b35bbe; no voters: 
I20260709 13:00:49.204003  7520 raft_consensus.cc:2804] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:49.204710  7520 raft_consensus.cc:697] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 1 LEADER]: Becoming Leader. State: Replica: 29546ef8491d4dcba75459ed3d780497, State: Running, Role: LEADER
I20260709 13:00:49.205824  7491 tablet_bootstrap.cc:654] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.207253  7374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c646ace346640f8a3a4939d5c2aab5e" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c6951c9f204a3db527b90376f3ea36" is_pre_election: true
I20260709 13:00:49.216631  7493 tablet_bootstrap.cc:654] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.221315  7374 raft_consensus.cc:2468] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceaa51da210b427f92d4bb4a95b35bbe in term 0.
W20260709 13:00:49.221328  7414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:46257, user_credentials={real_user=slave}} blocked reactor thread for 77203.8us
I20260709 13:00:49.208660  7373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dba3b1b71e434cac316850c2da8c01" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c6951c9f204a3db527b90376f3ea36" is_pre_election: true
I20260709 13:00:49.223357  7265 leader_election.cc:304] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97c6951c9f204a3db527b90376f3ea36, ceaa51da210b427f92d4bb4a95b35bbe; no voters: 
I20260709 13:00:49.223966  7374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e451ff7482b5487c8db99685c67f888d" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c6951c9f204a3db527b90376f3ea36" is_pre_election: true
I20260709 13:00:49.224255  7492 tablet_bootstrap.cc:654] T 3c646ace346640f8a3a4939d5c2aab5e P 29546ef8491d4dcba75459ed3d780497: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.225001  7515 raft_consensus.cc:2804] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:49.225970  7515 raft_consensus.cc:493] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.206506  7520 consensus_queue.cc:237] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.231148  7373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55dd47f5d644c3eb24678e018b317e1" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
I20260709 13:00:49.231725  7448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c646ace346640f8a3a4939d5c2aab5e" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29546ef8491d4dcba75459ed3d780497" is_pre_election: true
I20260709 13:00:49.232565  7447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dba3b1b71e434cac316850c2da8c01" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29546ef8491d4dcba75459ed3d780497" is_pre_election: true
W20260709 13:00:49.233685  7265 leader_election.cc:343] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:49.230094  7515 raft_consensus.cc:3060] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.230087  7374 raft_consensus.cc:2468] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29546ef8491d4dcba75459ed3d780497 in term 0.
W20260709 13:00:49.237563  7262 leader_election.cc:343] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:49.239346  7195 catalog_manager.cc:5697] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29546ef8491d4dcba75459ed3d780497 (127.6.255.3). New cstate: current_term: 1 leader_uuid: "29546ef8491d4dcba75459ed3d780497" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: HEALTHY } } }
W20260709 13:00:49.239449  7262 leader_election.cc:343] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:49.242645  7262 leader_election.cc:304] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceaa51da210b427f92d4bb4a95b35bbe; no voters: 29546ef8491d4dcba75459ed3d780497, 97c6951c9f204a3db527b90376f3ea36
I20260709 13:00:49.243563  7497 raft_consensus.cc:2749] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:00:49.241137  7414 leader_election.cc:343] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:49.249747  7414 leader_election.cc:304] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, 97c6951c9f204a3db527b90376f3ea36; no voters: ceaa51da210b427f92d4bb4a95b35bbe
I20260709 13:00:49.246393  7515 raft_consensus.cc:515] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.254793  7519 raft_consensus.cc:2804] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:49.255406  7519 raft_consensus.cc:493] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.255782  7519 raft_consensus.cc:3060] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.257825  7515 leader_election.cc:290] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 1 election: Requested vote from peers 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:49.258816  7374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c646ace346640f8a3a4939d5c2aab5e" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
I20260709 13:00:49.265833  7374 raft_consensus.cc:3060] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.266017  7447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c646ace346640f8a3a4939d5c2aab5e" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29546ef8491d4dcba75459ed3d780497"
W20260709 13:00:49.268326  7262 leader_election.cc:343] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:49.285038  7493 tablet_bootstrap.cc:492] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36: No bootstrap required, opened a new log
I20260709 13:00:49.285404  7491 tablet_bootstrap.cc:492] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe: No bootstrap required, opened a new log
I20260709 13:00:49.285619  7493 ts_tablet_manager.cc:1403] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36: Time spent bootstrapping tablet: real 0.107s	user 0.010s	sys 0.008s
I20260709 13:00:49.286054  7491 ts_tablet_manager.cc:1403] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe: Time spent bootstrapping tablet: real 0.124s	user 0.017s	sys 0.022s
I20260709 13:00:49.286989  7374 raft_consensus.cc:2468] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceaa51da210b427f92d4bb4a95b35bbe in term 1.
I20260709 13:00:49.284655  7519 raft_consensus.cc:515] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } } peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } }
I20260709 13:00:49.289135  7493 raft_consensus.cc:359] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.289925  7493 raft_consensus.cc:385] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.290251  7493 raft_consensus.cc:740] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97c6951c9f204a3db527b90376f3ea36, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.289942  7491 raft_consensus.cc:359] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.291141  7265 leader_election.cc:304] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97c6951c9f204a3db527b90376f3ea36, ceaa51da210b427f92d4bb4a95b35bbe; no voters: 29546ef8491d4dcba75459ed3d780497
I20260709 13:00:49.291685  7491 raft_consensus.cc:385] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.291097  7493 consensus_queue.cc:260] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.291982  7491 raft_consensus.cc:740] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.292707  7491 consensus_queue.cc:260] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.293694  7493 ts_tablet_manager.cc:1434] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:00:49.293982  7497 raft_consensus.cc:2804] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:49.294780  7493 tablet_bootstrap.cc:492] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36: Bootstrap starting.
I20260709 13:00:49.294586  7298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e451ff7482b5487c8db99685c67f888d" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
I20260709 13:00:49.296221  7491 ts_tablet_manager.cc:1434] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
W20260709 13:00:49.296795  7412 leader_election.cc:343] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:49.298074  7519 leader_election.cc:290] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 1 election: Requested vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257)
I20260709 13:00:49.299365  7491 tablet_bootstrap.cc:492] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe: Bootstrap starting.
I20260709 13:00:49.299877  7374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e451ff7482b5487c8db99685c67f888d" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
I20260709 13:00:49.305265  7374 raft_consensus.cc:3060] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.311472  7492 tablet_bootstrap.cc:492] T 3c646ace346640f8a3a4939d5c2aab5e P 29546ef8491d4dcba75459ed3d780497: No bootstrap required, opened a new log
I20260709 13:00:49.312062  7492 ts_tablet_manager.cc:1403] T 3c646ace346640f8a3a4939d5c2aab5e P 29546ef8491d4dcba75459ed3d780497: Time spent bootstrapping tablet: real 0.116s	user 0.008s	sys 0.013s
I20260709 13:00:49.312318  7491 tablet_bootstrap.cc:654] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.304404  7493 tablet_bootstrap.cc:654] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.320529  7374 raft_consensus.cc:2468] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29546ef8491d4dcba75459ed3d780497 in term 1.
I20260709 13:00:49.322192  7414 leader_election.cc:304] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, 97c6951c9f204a3db527b90376f3ea36; no voters: ceaa51da210b427f92d4bb4a95b35bbe
I20260709 13:00:49.327841  7497 raft_consensus.cc:697] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [term 1 LEADER]: Becoming Leader. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Running, Role: LEADER
I20260709 13:00:49.329169  7497 consensus_queue.cc:237] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.334474  7491 tablet_bootstrap.cc:492] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe: No bootstrap required, opened a new log
I20260709 13:00:49.335147  7491 ts_tablet_manager.cc:1403] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe: Time spent bootstrapping tablet: real 0.036s	user 0.010s	sys 0.003s
I20260709 13:00:49.338351  7491 raft_consensus.cc:359] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.338873  7491 raft_consensus.cc:385] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.339049  7491 raft_consensus.cc:740] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.339624  7491 consensus_queue.cc:260] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.337868  7520 raft_consensus.cc:2804] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:49.345793  7520 raft_consensus.cc:697] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 1 LEADER]: Becoming Leader. State: Replica: 29546ef8491d4dcba75459ed3d780497, State: Running, Role: LEADER
I20260709 13:00:49.346714  7520 consensus_queue.cc:237] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } } peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } }
I20260709 13:00:49.341835  7491 ts_tablet_manager.cc:1434] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.003s
I20260709 13:00:49.347172  7194 catalog_manager.cc:5697] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe reported cstate change: term changed from 0 to 1, leader changed from <none> to ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1). New cstate: current_term: 1 leader_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:49.349334  7491 tablet_bootstrap.cc:492] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe: Bootstrap starting.
I20260709 13:00:49.353926  7492 raft_consensus.cc:359] T 3c646ace346640f8a3a4939d5c2aab5e P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.360013  7491 tablet_bootstrap.cc:654] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.362159  7193 catalog_manager.cc:5697] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29546ef8491d4dcba75459ed3d780497 (127.6.255.3). New cstate: current_term: 1 leader_uuid: "29546ef8491d4dcba75459ed3d780497" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:49.371141  7492 raft_consensus.cc:385] T 3c646ace346640f8a3a4939d5c2aab5e P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.371549  7492 raft_consensus.cc:740] T 3c646ace346640f8a3a4939d5c2aab5e P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29546ef8491d4dcba75459ed3d780497, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.372548  7492 consensus_queue.cc:260] T 3c646ace346640f8a3a4939d5c2aab5e P 29546ef8491d4dcba75459ed3d780497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.375548  7492 ts_tablet_manager.cc:1434] T 3c646ace346640f8a3a4939d5c2aab5e P 29546ef8491d4dcba75459ed3d780497: Time spent starting tablet: real 0.063s	user 0.000s	sys 0.006s
I20260709 13:00:49.376874  7492 tablet_bootstrap.cc:492] T 1e97321e887f48a4b24dce648a4aa34d P 29546ef8491d4dcba75459ed3d780497: Bootstrap starting.
I20260709 13:00:49.379634  7491 tablet_bootstrap.cc:492] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe: No bootstrap required, opened a new log
I20260709 13:00:49.380147  7491 ts_tablet_manager.cc:1403] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe: Time spent bootstrapping tablet: real 0.031s	user 0.009s	sys 0.005s
I20260709 13:00:49.383209  7491 raft_consensus.cc:359] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.383926  7491 raft_consensus.cc:385] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.384313  7491 raft_consensus.cc:740] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.385073  7491 consensus_queue.cc:260] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.387388  7491 ts_tablet_manager.cc:1434] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:00:49.388381  7491 tablet_bootstrap.cc:492] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe: Bootstrap starting.
I20260709 13:00:49.389204  7493 tablet_bootstrap.cc:492] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36: No bootstrap required, opened a new log
I20260709 13:00:49.389776  7493 ts_tablet_manager.cc:1403] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36: Time spent bootstrapping tablet: real 0.095s	user 0.033s	sys 0.031s
I20260709 13:00:49.393018  7493 raft_consensus.cc:359] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.394045  7493 raft_consensus.cc:385] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.394690  7491 tablet_bootstrap.cc:654] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.394577  7493 raft_consensus.cc:740] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97c6951c9f204a3db527b90376f3ea36, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.392781  7194 catalog_manager.cc:5697] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29546ef8491d4dcba75459ed3d780497 (127.6.255.3). New cstate: current_term: 1 leader_uuid: "29546ef8491d4dcba75459ed3d780497" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:49.395789  7493 consensus_queue.cc:260] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.402556  7520 raft_consensus.cc:493] T 3c646ace346640f8a3a4939d5c2aab5e P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.403059  7520 raft_consensus.cc:515] T 3c646ace346640f8a3a4939d5c2aab5e P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.405540  7520 leader_election.cc:290] T 3c646ace346640f8a3a4939d5c2aab5e P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257)
I20260709 13:00:49.406744  7374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c646ace346640f8a3a4939d5c2aab5e" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c6951c9f204a3db527b90376f3ea36" is_pre_election: true
I20260709 13:00:49.407617  7374 raft_consensus.cc:2393] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29546ef8491d4dcba75459ed3d780497 in current term 1: Already voted for candidate ceaa51da210b427f92d4bb4a95b35bbe in this term.
I20260709 13:00:49.409441  7298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c646ace346640f8a3a4939d5c2aab5e" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" is_pre_election: true
I20260709 13:00:49.411219  7412 leader_election.cc:304] T 3c646ace346640f8a3a4939d5c2aab5e P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497; no voters: 97c6951c9f204a3db527b90376f3ea36, ceaa51da210b427f92d4bb4a95b35bbe
I20260709 13:00:49.412190  7493 ts_tablet_manager.cc:1434] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 13:00:49.413288  7493 tablet_bootstrap.cc:492] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36: Bootstrap starting.
I20260709 13:00:49.413215  7520 raft_consensus.cc:3060] T 3c646ace346640f8a3a4939d5c2aab5e P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.420715  7493 tablet_bootstrap.cc:654] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.422396  7520 raft_consensus.cc:2749] T 3c646ace346640f8a3a4939d5c2aab5e P 29546ef8491d4dcba75459ed3d780497 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:49.423585  7492 tablet_bootstrap.cc:654] T 1e97321e887f48a4b24dce648a4aa34d P 29546ef8491d4dcba75459ed3d780497: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.425837  7491 tablet_bootstrap.cc:492] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe: No bootstrap required, opened a new log
I20260709 13:00:49.426301  7491 ts_tablet_manager.cc:1403] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe: Time spent bootstrapping tablet: real 0.038s	user 0.009s	sys 0.010s
I20260709 13:00:49.429095  7491 raft_consensus.cc:359] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } } peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } }
I20260709 13:00:49.429764  7491 raft_consensus.cc:385] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.430009  7491 raft_consensus.cc:740] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.430615  7491 consensus_queue.cc:260] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } } peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } }
I20260709 13:00:49.437357  7492 tablet_bootstrap.cc:492] T 1e97321e887f48a4b24dce648a4aa34d P 29546ef8491d4dcba75459ed3d780497: No bootstrap required, opened a new log
I20260709 13:00:49.437695  7493 tablet_bootstrap.cc:492] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36: No bootstrap required, opened a new log
I20260709 13:00:49.437929  7492 ts_tablet_manager.cc:1403] T 1e97321e887f48a4b24dce648a4aa34d P 29546ef8491d4dcba75459ed3d780497: Time spent bootstrapping tablet: real 0.061s	user 0.011s	sys 0.011s
I20260709 13:00:49.439546  7491 ts_tablet_manager.cc:1434] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.002s
I20260709 13:00:49.441143  7492 raft_consensus.cc:359] T 1e97321e887f48a4b24dce648a4aa34d P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.442023  7492 raft_consensus.cc:385] T 1e97321e887f48a4b24dce648a4aa34d P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.438300  7493 ts_tablet_manager.cc:1403] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.010s
I20260709 13:00:49.442351  7492 raft_consensus.cc:740] T 1e97321e887f48a4b24dce648a4aa34d P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29546ef8491d4dcba75459ed3d780497, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.443256  7492 consensus_queue.cc:260] T 1e97321e887f48a4b24dce648a4aa34d P 29546ef8491d4dcba75459ed3d780497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.445780  7492 ts_tablet_manager.cc:1434] T 1e97321e887f48a4b24dce648a4aa34d P 29546ef8491d4dcba75459ed3d780497: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:00:49.446861  7492 tablet_bootstrap.cc:492] T dbc48c8abc054511b1c78331b267f725 P 29546ef8491d4dcba75459ed3d780497: Bootstrap starting.
I20260709 13:00:49.447194  7491 tablet_bootstrap.cc:492] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe: Bootstrap starting.
I20260709 13:00:49.453493  7491 tablet_bootstrap.cc:654] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.454224  7492 tablet_bootstrap.cc:654] T dbc48c8abc054511b1c78331b267f725 P 29546ef8491d4dcba75459ed3d780497: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.457230  7493 raft_consensus.cc:359] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.458324  7493 raft_consensus.cc:385] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.458755  7493 raft_consensus.cc:740] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97c6951c9f204a3db527b90376f3ea36, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.459609  7493 consensus_queue.cc:260] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.461292  7500 raft_consensus.cc:493] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.461812  7500 raft_consensus.cc:515] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.464600  7500 leader_election.cc:290] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:49.466042  7447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55dd47f5d644c3eb24678e018b317e1" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29546ef8491d4dcba75459ed3d780497" is_pre_election: true
I20260709 13:00:49.466743  7493 ts_tablet_manager.cc:1434] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:00:49.467741  7520 raft_consensus.cc:493] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.467993  7298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55dd47f5d644c3eb24678e018b317e1" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" is_pre_election: true
I20260709 13:00:49.468787  7298 raft_consensus.cc:2393] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97c6951c9f204a3db527b90376f3ea36 in current term 1: Already voted for candidate 29546ef8491d4dcba75459ed3d780497 in this term.
I20260709 13:00:49.468247  7520 raft_consensus.cc:515] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.470252  7338 leader_election.cc:304] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97c6951c9f204a3db527b90376f3ea36; no voters: 29546ef8491d4dcba75459ed3d780497, ceaa51da210b427f92d4bb4a95b35bbe
I20260709 13:00:49.471189  7520 leader_election.cc:290] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257)
I20260709 13:00:49.472398  7500 raft_consensus.cc:3060] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.472855  7374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d16cc32774d49fbbd48d20cd2618904" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c6951c9f204a3db527b90376f3ea36" is_pre_election: true
I20260709 13:00:49.473529  7497 raft_consensus.cc:493] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.473959  7497 raft_consensus.cc:515] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
W20260709 13:00:49.474392  7414 leader_election.cc:343] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:49.475306  7298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d16cc32774d49fbbd48d20cd2618904" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" is_pre_election: true
I20260709 13:00:49.475927  7497 leader_election.cc:290] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:49.475950  7298 raft_consensus.cc:2468] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29546ef8491d4dcba75459ed3d780497 in term 0.
I20260709 13:00:49.476766  7374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbc48c8abc054511b1c78331b267f725" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c6951c9f204a3db527b90376f3ea36" is_pre_election: true
I20260709 13:00:49.477392  7374 raft_consensus.cc:2468] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceaa51da210b427f92d4bb4a95b35bbe in term 0.
I20260709 13:00:49.477490  7412 leader_election.cc:304] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, ceaa51da210b427f92d4bb4a95b35bbe; no voters: 97c6951c9f204a3db527b90376f3ea36
I20260709 13:00:49.478809  7265 leader_election.cc:304] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97c6951c9f204a3db527b90376f3ea36, ceaa51da210b427f92d4bb4a95b35bbe; no voters: 
I20260709 13:00:49.479257  7447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbc48c8abc054511b1c78331b267f725" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29546ef8491d4dcba75459ed3d780497" is_pre_election: true
I20260709 13:00:49.479794  7520 raft_consensus.cc:2804] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:49.480331  7520 raft_consensus.cc:493] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:49.480705  7262 leader_election.cc:343] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:49.482493  7497 raft_consensus.cc:2804] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:49.480768  7520 raft_consensus.cc:3060] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.482856  7497 raft_consensus.cc:493] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.483425  7497 raft_consensus.cc:3060] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.486001  7493 tablet_bootstrap.cc:492] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36: Bootstrap starting.
I20260709 13:00:49.494227  7500 raft_consensus.cc:2749] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:49.494729  7491 tablet_bootstrap.cc:492] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe: No bootstrap required, opened a new log
I20260709 13:00:49.494061  7520 raft_consensus.cc:515] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.498169  7492 tablet_bootstrap.cc:492] T dbc48c8abc054511b1c78331b267f725 P 29546ef8491d4dcba75459ed3d780497: No bootstrap required, opened a new log
I20260709 13:00:49.498634  7298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d16cc32774d49fbbd48d20cd2618904" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
I20260709 13:00:49.499254  7298 raft_consensus.cc:3060] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.505601  7520 leader_election.cc:290] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 1 election: Requested vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257)
I20260709 13:00:49.506597  7497 raft_consensus.cc:515] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.507326  7493 tablet_bootstrap.cc:654] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.508031  7491 ts_tablet_manager.cc:1403] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe: Time spent bootstrapping tablet: real 0.061s	user 0.013s	sys 0.021s
I20260709 13:00:49.508293  7492 ts_tablet_manager.cc:1403] T dbc48c8abc054511b1c78331b267f725 P 29546ef8491d4dcba75459ed3d780497: Time spent bootstrapping tablet: real 0.062s	user 0.023s	sys 0.004s
I20260709 13:00:49.509318  7374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d16cc32774d49fbbd48d20cd2618904" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
W20260709 13:00:49.511479  7414 leader_election.cc:343] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:49.512511  7491 raft_consensus.cc:359] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.513163  7491 raft_consensus.cc:385] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.513456  7491 raft_consensus.cc:740] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.513634  7492 raft_consensus.cc:359] T dbc48c8abc054511b1c78331b267f725 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.514099  7491 consensus_queue.cc:260] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.514605  7492 raft_consensus.cc:385] T dbc48c8abc054511b1c78331b267f725 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.515380  7492 raft_consensus.cc:740] T dbc48c8abc054511b1c78331b267f725 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29546ef8491d4dcba75459ed3d780497, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.516083  7491 ts_tablet_manager.cc:1434] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:00:49.515931  7298 raft_consensus.cc:2468] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29546ef8491d4dcba75459ed3d780497 in term 1.
I20260709 13:00:49.517902  7497 leader_election.cc:290] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 1 election: Requested vote from peers 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:49.518839  7412 leader_election.cc:304] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, ceaa51da210b427f92d4bb4a95b35bbe; no voters: 97c6951c9f204a3db527b90376f3ea36
I20260709 13:00:49.518992  7374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbc48c8abc054511b1c78331b267f725" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
I20260709 13:00:49.519656  7374 raft_consensus.cc:3060] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.520678  7520 raft_consensus.cc:2804] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:49.521205  7520 raft_consensus.cc:697] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 1 LEADER]: Becoming Leader. State: Replica: 29546ef8491d4dcba75459ed3d780497, State: Running, Role: LEADER
I20260709 13:00:49.521848  7447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbc48c8abc054511b1c78331b267f725" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29546ef8491d4dcba75459ed3d780497"
I20260709 13:00:49.522096  7520 consensus_queue.cc:237] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.516431  7492 consensus_queue.cc:260] T dbc48c8abc054511b1c78331b267f725 P 29546ef8491d4dcba75459ed3d780497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.526986  7492 ts_tablet_manager.cc:1434] T dbc48c8abc054511b1c78331b267f725 P 29546ef8491d4dcba75459ed3d780497: Time spent starting tablet: real 0.018s	user 0.000s	sys 0.007s
I20260709 13:00:49.527490  7374 raft_consensus.cc:2468] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceaa51da210b427f92d4bb4a95b35bbe in term 1.
I20260709 13:00:49.528374  7492 tablet_bootstrap.cc:492] T 69dba3b1b71e434cac316850c2da8c01 P 29546ef8491d4dcba75459ed3d780497: Bootstrap starting.
I20260709 13:00:49.528616  7265 leader_election.cc:304] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97c6951c9f204a3db527b90376f3ea36, ceaa51da210b427f92d4bb4a95b35bbe; no voters: 
I20260709 13:00:49.529385  7497 raft_consensus.cc:2804] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:49.530354  7497 raft_consensus.cc:697] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 LEADER]: Becoming Leader. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Running, Role: LEADER
I20260709 13:00:49.531131  7497 consensus_queue.cc:237] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.531510  7447 raft_consensus.cc:3060] T dbc48c8abc054511b1c78331b267f725 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.545473  7519 consensus_queue.cc:1048] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:49.559710  7193 catalog_manager.cc:5697] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe reported cstate change: term changed from 0 to 1, leader changed from <none> to ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1). New cstate: current_term: 1 leader_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:49.563746  7493 tablet_bootstrap.cc:492] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36: No bootstrap required, opened a new log
I20260709 13:00:49.564229  7493 ts_tablet_manager.cc:1403] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36: Time spent bootstrapping tablet: real 0.079s	user 0.009s	sys 0.010s
I20260709 13:00:49.564365  7447 raft_consensus.cc:2468] T dbc48c8abc054511b1c78331b267f725 P 29546ef8491d4dcba75459ed3d780497 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceaa51da210b427f92d4bb4a95b35bbe in term 1.
I20260709 13:00:49.572321  7492 tablet_bootstrap.cc:654] T 69dba3b1b71e434cac316850c2da8c01 P 29546ef8491d4dcba75459ed3d780497: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.572690  7493 raft_consensus.cc:359] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.573367  7493 raft_consensus.cc:385] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.573661  7493 raft_consensus.cc:740] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97c6951c9f204a3db527b90376f3ea36, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.574394  7493 consensus_queue.cc:260] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.576762  7493 ts_tablet_manager.cc:1434] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.005s
I20260709 13:00:49.578480  7493 tablet_bootstrap.cc:492] T b90380b15f1a42f2b3c9736536c43e21 P 97c6951c9f204a3db527b90376f3ea36: Bootstrap starting.
I20260709 13:00:49.588352  7492 tablet_bootstrap.cc:492] T 69dba3b1b71e434cac316850c2da8c01 P 29546ef8491d4dcba75459ed3d780497: No bootstrap required, opened a new log
I20260709 13:00:49.589025  7492 ts_tablet_manager.cc:1403] T 69dba3b1b71e434cac316850c2da8c01 P 29546ef8491d4dcba75459ed3d780497: Time spent bootstrapping tablet: real 0.061s	user 0.002s	sys 0.011s
I20260709 13:00:49.592099  7194 catalog_manager.cc:5697] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29546ef8491d4dcba75459ed3d780497 (127.6.255.3). New cstate: current_term: 1 leader_uuid: "29546ef8491d4dcba75459ed3d780497" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:49.599545  7492 raft_consensus.cc:359] T 69dba3b1b71e434cac316850c2da8c01 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.600298  7492 raft_consensus.cc:385] T 69dba3b1b71e434cac316850c2da8c01 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.600596  7492 raft_consensus.cc:740] T 69dba3b1b71e434cac316850c2da8c01 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29546ef8491d4dcba75459ed3d780497, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.601328  7492 consensus_queue.cc:260] T 69dba3b1b71e434cac316850c2da8c01 P 29546ef8491d4dcba75459ed3d780497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.603570  7493 tablet_bootstrap.cc:654] T b90380b15f1a42f2b3c9736536c43e21 P 97c6951c9f204a3db527b90376f3ea36: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.603535  7492 ts_tablet_manager.cc:1434] T 69dba3b1b71e434cac316850c2da8c01 P 29546ef8491d4dcba75459ed3d780497: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:00:49.604743  7492 tablet_bootstrap.cc:492] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497: Bootstrap starting.
I20260709 13:00:49.617660  7514 consensus_queue.cc:1048] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:00:49.624130  7492 tablet_bootstrap.cc:654] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.625922  7493 tablet_bootstrap.cc:492] T b90380b15f1a42f2b3c9736536c43e21 P 97c6951c9f204a3db527b90376f3ea36: No bootstrap required, opened a new log
I20260709 13:00:49.626425  7493 ts_tablet_manager.cc:1403] T b90380b15f1a42f2b3c9736536c43e21 P 97c6951c9f204a3db527b90376f3ea36: Time spent bootstrapping tablet: real 0.049s	user 0.009s	sys 0.005s
I20260709 13:00:49.628947  7493 raft_consensus.cc:359] T b90380b15f1a42f2b3c9736536c43e21 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.629652  7493 raft_consensus.cc:385] T b90380b15f1a42f2b3c9736536c43e21 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.629987  7493 raft_consensus.cc:740] T b90380b15f1a42f2b3c9736536c43e21 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97c6951c9f204a3db527b90376f3ea36, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.630762  7493 consensus_queue.cc:260] T b90380b15f1a42f2b3c9736536c43e21 P 97c6951c9f204a3db527b90376f3ea36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.633496  7493 ts_tablet_manager.cc:1434] T b90380b15f1a42f2b3c9736536c43e21 P 97c6951c9f204a3db527b90376f3ea36: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:00:49.638794  7492 tablet_bootstrap.cc:492] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497: No bootstrap required, opened a new log
I20260709 13:00:49.639348  7492 ts_tablet_manager.cc:1403] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497: Time spent bootstrapping tablet: real 0.035s	user 0.009s	sys 0.005s
I20260709 13:00:49.643014  7492 raft_consensus.cc:359] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.643615  7492 raft_consensus.cc:385] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.643908  7492 raft_consensus.cc:740] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29546ef8491d4dcba75459ed3d780497, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.644876  7492 consensus_queue.cc:260] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.647441  7520 consensus_queue.cc:1048] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:49.647228  7492 ts_tablet_manager.cc:1434] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:00:49.664827  7514 consensus_queue.cc:1048] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:49.681906  7497 raft_consensus.cc:493] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.682422  7497 raft_consensus.cc:515] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.684166  7497 leader_election.cc:290] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:49.684767  7373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e97321e887f48a4b24dce648a4aa34d" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c6951c9f204a3db527b90376f3ea36" is_pre_election: true
I20260709 13:00:49.685256  7447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e97321e887f48a4b24dce648a4aa34d" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29546ef8491d4dcba75459ed3d780497" is_pre_election: true
I20260709 13:00:49.685341  7373 raft_consensus.cc:2468] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceaa51da210b427f92d4bb4a95b35bbe in term 0.
I20260709 13:00:49.685935  7447 raft_consensus.cc:2468] T 1e97321e887f48a4b24dce648a4aa34d P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceaa51da210b427f92d4bb4a95b35bbe in term 0.
I20260709 13:00:49.686538  7265 leader_election.cc:304] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97c6951c9f204a3db527b90376f3ea36, ceaa51da210b427f92d4bb4a95b35bbe; no voters: 
I20260709 13:00:49.687254  7497 raft_consensus.cc:2804] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:49.687562  7497 raft_consensus.cc:493] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.687841  7497 raft_consensus.cc:3060] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.692268  7497 raft_consensus.cc:515] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.693920  7497 leader_election.cc:290] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 1 election: Requested vote from peers 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:49.694660  7373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e97321e887f48a4b24dce648a4aa34d" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
I20260709 13:00:49.694963  7447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e97321e887f48a4b24dce648a4aa34d" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29546ef8491d4dcba75459ed3d780497"
I20260709 13:00:49.695221  7373 raft_consensus.cc:3060] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.695472  7447 raft_consensus.cc:3060] T 1e97321e887f48a4b24dce648a4aa34d P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.699985  7447 raft_consensus.cc:2468] T 1e97321e887f48a4b24dce648a4aa34d P 29546ef8491d4dcba75459ed3d780497 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceaa51da210b427f92d4bb4a95b35bbe in term 1.
I20260709 13:00:49.700922  7262 leader_election.cc:304] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, ceaa51da210b427f92d4bb4a95b35bbe; no voters: 
I20260709 13:00:49.701575  7497 raft_consensus.cc:2804] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:49.701815  7373 raft_consensus.cc:2468] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceaa51da210b427f92d4bb4a95b35bbe in term 1.
I20260709 13:00:49.701990  7497 raft_consensus.cc:697] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [term 1 LEADER]: Becoming Leader. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Running, Role: LEADER
I20260709 13:00:49.702831  7497 consensus_queue.cc:237] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.710223  7193 catalog_manager.cc:5697] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe reported cstate change: term changed from 0 to 1, leader changed from <none> to ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1). New cstate: current_term: 1 leader_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:49.718591  7500 raft_consensus.cc:493] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.719099  7500 raft_consensus.cc:515] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.722921  7447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dba3b1b71e434cac316850c2da8c01" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29546ef8491d4dcba75459ed3d780497" is_pre_election: true
I20260709 13:00:49.723610  7447 raft_consensus.cc:2468] T 69dba3b1b71e434cac316850c2da8c01 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97c6951c9f204a3db527b90376f3ea36 in term 0.
I20260709 13:00:49.723763  7298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dba3b1b71e434cac316850c2da8c01" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" is_pre_election: true
I20260709 13:00:49.724447  7298 raft_consensus.cc:2468] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97c6951c9f204a3db527b90376f3ea36 in term 0.
I20260709 13:00:49.724915  7337 leader_election.cc:304] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, 97c6951c9f204a3db527b90376f3ea36; no voters: 
I20260709 13:00:49.725793  7500 leader_election.cc:290] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:49.726284  7500 raft_consensus.cc:2804] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:49.726622  7500 raft_consensus.cc:493] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.726969  7500 raft_consensus.cc:3060] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:49.727535  7244 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:49.733671  7500 raft_consensus.cc:515] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.736560  7298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dba3b1b71e434cac316850c2da8c01" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
I20260709 13:00:49.736788  7447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dba3b1b71e434cac316850c2da8c01" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29546ef8491d4dcba75459ed3d780497"
I20260709 13:00:49.735980  7500 leader_election.cc:290] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 1 election: Requested vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:49.737185  7298 raft_consensus.cc:3060] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.737349  7447 raft_consensus.cc:3060] T 69dba3b1b71e434cac316850c2da8c01 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.741858  7447 raft_consensus.cc:2468] T 69dba3b1b71e434cac316850c2da8c01 P 29546ef8491d4dcba75459ed3d780497 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c6951c9f204a3db527b90376f3ea36 in term 1.
I20260709 13:00:49.742830  7337 leader_election.cc:304] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, 97c6951c9f204a3db527b90376f3ea36; no voters: 
I20260709 13:00:49.743505  7500 raft_consensus.cc:2804] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:49.743971  7500 raft_consensus.cc:697] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [term 1 LEADER]: Becoming Leader. State: Replica: 97c6951c9f204a3db527b90376f3ea36, State: Running, Role: LEADER
I20260709 13:00:49.745114  7500 consensus_queue.cc:237] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.744328  7298 raft_consensus.cc:2468] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c6951c9f204a3db527b90376f3ea36 in term 1.
I20260709 13:00:49.752990  7193 catalog_manager.cc:5697] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "97c6951c9f204a3db527b90376f3ea36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:49.767453  7298 raft_consensus.cc:3060] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.772908  7519 consensus_queue.cc:1048] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:49.776077  7497 raft_consensus.cc:493] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.776539  7497 raft_consensus.cc:515] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.778715  7497 leader_election.cc:290] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:49.779875  7373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b90380b15f1a42f2b3c9736536c43e21" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c6951c9f204a3db527b90376f3ea36" is_pre_election: true
I20260709 13:00:49.780239  7447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b90380b15f1a42f2b3c9736536c43e21" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29546ef8491d4dcba75459ed3d780497" is_pre_election: true
I20260709 13:00:49.780503  7373 raft_consensus.cc:2468] T b90380b15f1a42f2b3c9736536c43e21 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceaa51da210b427f92d4bb4a95b35bbe in term 0.
I20260709 13:00:49.780862  7447 raft_consensus.cc:2468] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceaa51da210b427f92d4bb4a95b35bbe in term 0.
I20260709 13:00:49.781744  7265 leader_election.cc:304] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97c6951c9f204a3db527b90376f3ea36, ceaa51da210b427f92d4bb4a95b35bbe; no voters: 
I20260709 13:00:49.782546  7497 raft_consensus.cc:2804] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:49.782881  7497 raft_consensus.cc:493] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.784137  7497 raft_consensus.cc:3060] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.788728  7520 consensus_queue.cc:1048] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:49.799633  7497 raft_consensus.cc:515] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.802117  7497 leader_election.cc:290] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 1 election: Requested vote from peers 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:49.802861  7447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b90380b15f1a42f2b3c9736536c43e21" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29546ef8491d4dcba75459ed3d780497"
I20260709 13:00:49.803330  7447 raft_consensus.cc:3060] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.803498  7373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b90380b15f1a42f2b3c9736536c43e21" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
I20260709 13:00:49.804162  7373 raft_consensus.cc:3060] T b90380b15f1a42f2b3c9736536c43e21 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.810499  7373 raft_consensus.cc:2468] T b90380b15f1a42f2b3c9736536c43e21 P 97c6951c9f204a3db527b90376f3ea36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceaa51da210b427f92d4bb4a95b35bbe in term 1.
I20260709 13:00:49.810602  7447 raft_consensus.cc:2468] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceaa51da210b427f92d4bb4a95b35bbe in term 1.
I20260709 13:00:49.811909  7262 leader_election.cc:304] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, ceaa51da210b427f92d4bb4a95b35bbe; no voters: 
I20260709 13:00:49.812839  7497 raft_consensus.cc:2804] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:49.813341  7497 raft_consensus.cc:697] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 LEADER]: Becoming Leader. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Running, Role: LEADER
I20260709 13:00:49.814057  7497 consensus_queue.cc:237] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:49.820668  7194 catalog_manager.cc:5697] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe reported cstate change: term changed from 0 to 1, leader changed from <none> to ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1). New cstate: current_term: 1 leader_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:49.890720  7497 consensus_queue.cc:1048] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:49.903213  7560 consensus_queue.cc:1048] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:49.933537  7373 raft_consensus.cc:3060] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.941478  7520 consensus_queue.cc:1048] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:49.953415  7560 consensus_queue.cc:1048] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:49.955375  7520 consensus_queue.cc:1048] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:49.978085  7529 consensus_queue.cc:1048] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:50.013878  7448 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e451ff7482b5487c8db99685c67f888d"
dest_uuid: "29546ef8491d4dcba75459ed3d780497"
mode: GRACEFUL
new_leader_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
 from {username='slave'} at 127.0.0.1:38616
I20260709 13:00:50.014599  7448 raft_consensus.cc:606] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 1 LEADER]: Received request to transfer leadership to TS ceaa51da210b427f92d4bb4a95b35bbe
I20260709 13:00:50.023442  7448 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f55dd47f5d644c3eb24678e018b317e1"
dest_uuid: "29546ef8491d4dcba75459ed3d780497"
mode: GRACEFUL
new_leader_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
 from {username='slave'} at 127.0.0.1:38616
I20260709 13:00:50.024052  7448 raft_consensus.cc:606] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 1 LEADER]: Received request to transfer leadership to TS ceaa51da210b427f92d4bb4a95b35bbe
I20260709 13:00:50.029358  7448 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d16cc32774d49fbbd48d20cd2618904"
dest_uuid: "29546ef8491d4dcba75459ed3d780497"
mode: GRACEFUL
new_leader_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
 from {username='slave'} at 127.0.0.1:38616
I20260709 13:00:50.029874  7448 raft_consensus.cc:606] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 1 LEADER]: Received request to transfer leadership to TS ceaa51da210b427f92d4bb4a95b35bbe
I20260709 13:00:50.034775  7448 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d144c1f909d74a298d08c0f8b19cd9e0"
dest_uuid: "29546ef8491d4dcba75459ed3d780497"
mode: GRACEFUL
new_leader_uuid: "97c6951c9f204a3db527b90376f3ea36"
 from {username='slave'} at 127.0.0.1:38616
I20260709 13:00:50.035251  7448 raft_consensus.cc:606] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 1 LEADER]: Received request to transfer leadership to TS 97c6951c9f204a3db527b90376f3ea36
I20260709 13:00:50.048710  7298 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3c646ace346640f8a3a4939d5c2aab5e"
dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
mode: GRACEFUL
new_leader_uuid: "97c6951c9f204a3db527b90376f3ea36"
 from {username='slave'} at 127.0.0.1:37710
I20260709 13:00:50.049258  7298 raft_consensus.cc:606] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [term 1 LEADER]: Received request to transfer leadership to TS 97c6951c9f204a3db527b90376f3ea36
I20260709 13:00:50.054708  7298 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1e97321e887f48a4b24dce648a4aa34d"
dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
mode: GRACEFUL
new_leader_uuid: "97c6951c9f204a3db527b90376f3ea36"
 from {username='slave'} at 127.0.0.1:37710
I20260709 13:00:50.055138  7298 raft_consensus.cc:606] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [term 1 LEADER]: Received request to transfer leadership to TS 97c6951c9f204a3db527b90376f3ea36
I20260709 13:00:50.063156  7298 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b90380b15f1a42f2b3c9736536c43e21"
dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
mode: GRACEFUL
new_leader_uuid: "29546ef8491d4dcba75459ed3d780497"
 from {username='slave'} at 127.0.0.1:37710
I20260709 13:00:50.063617  7298 raft_consensus.cc:606] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 LEADER]: Received request to transfer leadership to TS 29546ef8491d4dcba75459ed3d780497
I20260709 13:00:50.199636  7520 raft_consensus.cc:993] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497: : Instructing follower ceaa51da210b427f92d4bb4a95b35bbe to start an election
I20260709 13:00:50.200214  7514 raft_consensus.cc:1081] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 1 LEADER]: Signalling peer ceaa51da210b427f92d4bb4a95b35bbe to start an election
I20260709 13:00:50.202136  7298 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e451ff7482b5487c8db99685c67f888d"
dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
 from {username='slave'} at 127.0.0.1:37706
I20260709 13:00:50.202689  7298 raft_consensus.cc:493] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:50.203037  7298 raft_consensus.cc:3060] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.209518  7298 raft_consensus.cc:515] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } } peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } }
I20260709 13:00:50.211500  7298 leader_election.cc:290] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 2 election: Requested vote from peers 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593), 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257)
I20260709 13:00:50.212109  7448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e451ff7482b5487c8db99685c67f888d" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "29546ef8491d4dcba75459ed3d780497"
I20260709 13:00:50.212497  7374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e451ff7482b5487c8db99685c67f888d" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
I20260709 13:00:50.212764  7448 raft_consensus.cc:3055] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:50.213065  7374 raft_consensus.cc:3060] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.213150  7448 raft_consensus.cc:740] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 29546ef8491d4dcba75459ed3d780497, State: Running, Role: LEADER
I20260709 13:00:50.213932  7448 consensus_queue.cc:260] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } } peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } }
I20260709 13:00:50.215011  7448 raft_consensus.cc:3060] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.219359  7448 raft_consensus.cc:2468] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceaa51da210b427f92d4bb4a95b35bbe in term 2.
I20260709 13:00:50.219659  7374 raft_consensus.cc:2468] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceaa51da210b427f92d4bb4a95b35bbe in term 2.
I20260709 13:00:50.220402  7262 leader_election.cc:304] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, ceaa51da210b427f92d4bb4a95b35bbe; no voters: 
I20260709 13:00:50.221134  7529 raft_consensus.cc:2804] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:50.221602  7529 raft_consensus.cc:697] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [term 2 LEADER]: Becoming Leader. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Running, Role: LEADER
I20260709 13:00:50.222321  7529 consensus_queue.cc:237] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } } peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } }
I20260709 13:00:50.229094  7193 catalog_manager.cc:5697] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe reported cstate change: term changed from 1 to 2, leader changed from 29546ef8491d4dcba75459ed3d780497 (127.6.255.3) to ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1). New cstate: current_term: 2 leader_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:50.247828  7500 consensus_queue.cc:1048] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:50.267198  7520 raft_consensus.cc:993] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497: : Instructing follower ceaa51da210b427f92d4bb4a95b35bbe to start an election
I20260709 13:00:50.267669  7514 raft_consensus.cc:1081] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 1 LEADER]: Signalling peer ceaa51da210b427f92d4bb4a95b35bbe to start an election
I20260709 13:00:50.269315  7299 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f55dd47f5d644c3eb24678e018b317e1"
dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
 from {username='slave'} at 127.0.0.1:37706
I20260709 13:00:50.268678  7571 consensus_queue.cc:1048] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:50.269873  7299 raft_consensus.cc:493] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:50.270287  7299 raft_consensus.cc:3060] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.277061  7299 raft_consensus.cc:515] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:50.280300  7299 leader_election.cc:290] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 2 election: Requested vote from peers 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:50.280212  7448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55dd47f5d644c3eb24678e018b317e1" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "29546ef8491d4dcba75459ed3d780497"
I20260709 13:00:50.279866  7374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55dd47f5d644c3eb24678e018b317e1" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
I20260709 13:00:50.280954  7448 raft_consensus.cc:3055] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:50.281386  7448 raft_consensus.cc:740] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 29546ef8491d4dcba75459ed3d780497, State: Running, Role: LEADER
I20260709 13:00:50.282292  7448 consensus_queue.cc:260] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:50.283320  7374 raft_consensus.cc:3060] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.283380  7448 raft_consensus.cc:3060] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.288317  7560 consensus_queue.cc:1048] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:50.290380  7374 raft_consensus.cc:2468] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceaa51da210b427f92d4bb4a95b35bbe in term 2.
I20260709 13:00:50.290380  7448 raft_consensus.cc:2468] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceaa51da210b427f92d4bb4a95b35bbe in term 2.
I20260709 13:00:50.291692  7265 leader_election.cc:304] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97c6951c9f204a3db527b90376f3ea36, ceaa51da210b427f92d4bb4a95b35bbe; no voters: 
I20260709 13:00:50.292474  7529 raft_consensus.cc:2804] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:50.292919  7529 raft_consensus.cc:697] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 LEADER]: Becoming Leader. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Running, Role: LEADER
I20260709 13:00:50.293761  7529 consensus_queue.cc:237] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:50.294762  7497 consensus_queue.cc:1048] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:50.302697  7194 catalog_manager.cc:5697] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe reported cstate change: term changed from 1 to 2, leader changed from 29546ef8491d4dcba75459ed3d780497 (127.6.255.3) to ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1). New cstate: current_term: 2 leader_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:50.316563  7560 consensus_queue.cc:1048] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:50.319273  7529 raft_consensus.cc:993] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe: : Instructing follower 97c6951c9f204a3db527b90376f3ea36 to start an election
I20260709 13:00:50.319741  7529 raft_consensus.cc:1081] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [term 1 LEADER]: Signalling peer 97c6951c9f204a3db527b90376f3ea36 to start an election
I20260709 13:00:50.324311  7529 consensus_queue.cc:1048] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:50.324746  7372 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1e97321e887f48a4b24dce648a4aa34d"
dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
 from {username='slave'} at 127.0.0.1:40294
I20260709 13:00:50.328665  7372 raft_consensus.cc:493] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:50.329212  7372 raft_consensus.cc:3060] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.344947  7372 raft_consensus.cc:515] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:50.352217  7299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e97321e887f48a4b24dce648a4aa34d" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
I20260709 13:00:50.352905  7299 raft_consensus.cc:3055] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:50.353302  7299 raft_consensus.cc:740] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Running, Role: LEADER
I20260709 13:00:50.353718  7447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e97321e887f48a4b24dce648a4aa34d" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "29546ef8491d4dcba75459ed3d780497"
I20260709 13:00:50.354355  7447 raft_consensus.cc:3060] T 1e97321e887f48a4b24dce648a4aa34d P 29546ef8491d4dcba75459ed3d780497 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.354267  7299 consensus_queue.cc:260] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:50.355549  7299 raft_consensus.cc:3060] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.364771  7447 raft_consensus.cc:2468] T 1e97321e887f48a4b24dce648a4aa34d P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c6951c9f204a3db527b90376f3ea36 in term 2.
I20260709 13:00:50.366171  7299 raft_consensus.cc:2468] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c6951c9f204a3db527b90376f3ea36 in term 2.
I20260709 13:00:50.371402  7337 leader_election.cc:304] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, 97c6951c9f204a3db527b90376f3ea36; no voters: 
I20260709 13:00:50.367967  7497 raft_consensus.cc:993] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe: : Instructing follower 29546ef8491d4dcba75459ed3d780497 to start an election
I20260709 13:00:50.372185  7497 raft_consensus.cc:1081] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 LEADER]: Signalling peer 29546ef8491d4dcba75459ed3d780497 to start an election
I20260709 13:00:50.372298  7571 raft_consensus.cc:2804] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:50.372747  7571 raft_consensus.cc:697] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36 [term 2 LEADER]: Becoming Leader. State: Replica: 97c6951c9f204a3db527b90376f3ea36, State: Running, Role: LEADER
I20260709 13:00:50.365478  7372 leader_election.cc:290] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 2 election: Requested vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:50.374398  7571 consensus_queue.cc:237] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:50.376951  7447 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b90380b15f1a42f2b3c9736536c43e21"
dest_uuid: "29546ef8491d4dcba75459ed3d780497"
 from {username='slave'} at 127.0.0.1:38608
I20260709 13:00:50.377684  7447 raft_consensus.cc:493] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:50.378135  7447 raft_consensus.cc:3060] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.387831  7193 catalog_manager.cc:5697] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36 reported cstate change: term changed from 1 to 2, leader changed from ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1) to 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "97c6951c9f204a3db527b90376f3ea36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:50.391023  7447 raft_consensus.cc:515] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:50.393400  7447 leader_election.cc:290] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 2 election: Requested vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257)
I20260709 13:00:50.395088  7299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b90380b15f1a42f2b3c9736536c43e21" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
I20260709 13:00:50.395730  7299 raft_consensus.cc:3055] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:50.396128  7299 raft_consensus.cc:740] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Running, Role: LEADER
I20260709 13:00:50.397079  7371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b90380b15f1a42f2b3c9736536c43e21" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
I20260709 13:00:50.396986  7299 consensus_queue.cc:260] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:50.397760  7371 raft_consensus.cc:3060] T b90380b15f1a42f2b3c9736536c43e21 P 97c6951c9f204a3db527b90376f3ea36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.398447  7299 raft_consensus.cc:3060] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.404796  7371 raft_consensus.cc:2468] T b90380b15f1a42f2b3c9736536c43e21 P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29546ef8491d4dcba75459ed3d780497 in term 2.
I20260709 13:00:50.405656  7299 raft_consensus.cc:2468] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29546ef8491d4dcba75459ed3d780497 in term 2.
I20260709 13:00:50.406095  7414 leader_election.cc:304] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, 97c6951c9f204a3db527b90376f3ea36; no voters: 
I20260709 13:00:50.406991  7514 raft_consensus.cc:2804] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:50.407541  7514 raft_consensus.cc:697] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497 [term 2 LEADER]: Becoming Leader. State: Replica: 29546ef8491d4dcba75459ed3d780497, State: Running, Role: LEADER
I20260709 13:00:50.408643  7514 consensus_queue.cc:237] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:50.414950  7193 catalog_manager.cc:5697] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497 reported cstate change: term changed from 1 to 2, leader changed from ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1) to 29546ef8491d4dcba75459ed3d780497 (127.6.255.3). New cstate: current_term: 2 leader_uuid: "29546ef8491d4dcba75459ed3d780497" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:50.421334  7529 raft_consensus.cc:993] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe: : Instructing follower 97c6951c9f204a3db527b90376f3ea36 to start an election
I20260709 13:00:50.421810  7575 raft_consensus.cc:1081] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [term 1 LEADER]: Signalling peer 97c6951c9f204a3db527b90376f3ea36 to start an election
I20260709 13:00:50.423542  7371 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3c646ace346640f8a3a4939d5c2aab5e"
dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
 from {username='slave'} at 127.0.0.1:40294
I20260709 13:00:50.424080  7371 raft_consensus.cc:493] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:50.424417  7371 raft_consensus.cc:3060] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.430819  7371 raft_consensus.cc:515] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:50.433364  7371 leader_election.cc:290] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 2 election: Requested vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:50.433921  7447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c646ace346640f8a3a4939d5c2aab5e" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "29546ef8491d4dcba75459ed3d780497"
I20260709 13:00:50.433840  7299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c646ace346640f8a3a4939d5c2aab5e" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
I20260709 13:00:50.434542  7447 raft_consensus.cc:3060] T 3c646ace346640f8a3a4939d5c2aab5e P 29546ef8491d4dcba75459ed3d780497 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.434626  7299 raft_consensus.cc:3055] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:50.435021  7299 raft_consensus.cc:740] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Running, Role: LEADER
I20260709 13:00:50.435995  7299 consensus_queue.cc:260] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:50.437314  7299 raft_consensus.cc:3060] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.442247  7447 raft_consensus.cc:2468] T 3c646ace346640f8a3a4939d5c2aab5e P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c6951c9f204a3db527b90376f3ea36 in term 2.
I20260709 13:00:50.443607  7337 leader_election.cc:304] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, 97c6951c9f204a3db527b90376f3ea36; no voters: 
I20260709 13:00:50.444545  7571 raft_consensus.cc:2804] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:50.445040  7571 raft_consensus.cc:697] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36 [term 2 LEADER]: Becoming Leader. State: Replica: 97c6951c9f204a3db527b90376f3ea36, State: Running, Role: LEADER
I20260709 13:00:50.446051  7571 consensus_queue.cc:237] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:50.449259  7299 raft_consensus.cc:2468] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c6951c9f204a3db527b90376f3ea36 in term 2.
I20260709 13:00:50.452502  7520 raft_consensus.cc:993] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497: : Instructing follower 97c6951c9f204a3db527b90376f3ea36 to start an election
I20260709 13:00:50.452965  7514 raft_consensus.cc:1081] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 1 LEADER]: Signalling peer 97c6951c9f204a3db527b90376f3ea36 to start an election
I20260709 13:00:50.454882  7371 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d144c1f909d74a298d08c0f8b19cd9e0"
dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:00:50.455641  7371 raft_consensus.cc:493] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:50.455546  7194 catalog_manager.cc:5697] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36 reported cstate change: term changed from 1 to 2, leader changed from ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1) to 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "97c6951c9f204a3db527b90376f3ea36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:50.456178  7371 raft_consensus.cc:3060] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.465083  7371 raft_consensus.cc:515] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:50.468523  7299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d144c1f909d74a298d08c0f8b19cd9e0" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
I20260709 13:00:50.470518  7447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d144c1f909d74a298d08c0f8b19cd9e0" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "29546ef8491d4dcba75459ed3d780497"
I20260709 13:00:50.476313  7371 leader_election.cc:290] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 2 election: Requested vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:50.476930  7299 raft_consensus.cc:3060] T d144c1f909d74a298d08c0f8b19cd9e0 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.477358  7447 raft_consensus.cc:3055] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:50.477784  7447 raft_consensus.cc:740] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 29546ef8491d4dcba75459ed3d780497, State: Running, Role: LEADER
I20260709 13:00:50.478698  7447 consensus_queue.cc:260] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:50.479861  7447 raft_consensus.cc:3060] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.483728  7299 raft_consensus.cc:2468] T d144c1f909d74a298d08c0f8b19cd9e0 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c6951c9f204a3db527b90376f3ea36 in term 2.
I20260709 13:00:50.484969  7338 leader_election.cc:304] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97c6951c9f204a3db527b90376f3ea36, ceaa51da210b427f92d4bb4a95b35bbe; no voters: 
I20260709 13:00:50.486047  7571 raft_consensus.cc:2804] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:50.486671  7571 raft_consensus.cc:697] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 2 LEADER]: Becoming Leader. State: Replica: 97c6951c9f204a3db527b90376f3ea36, State: Running, Role: LEADER
I20260709 13:00:50.487552  7571 consensus_queue.cc:237] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:50.489522  7447 raft_consensus.cc:2468] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c6951c9f204a3db527b90376f3ea36 in term 2.
I20260709 13:00:50.495301  7194 catalog_manager.cc:5697] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 reported cstate change: term changed from 1 to 2, leader changed from 29546ef8491d4dcba75459ed3d780497 (127.6.255.3) to 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "97c6951c9f204a3db527b90376f3ea36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:50.510382  7520 raft_consensus.cc:993] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497: : Instructing follower ceaa51da210b427f92d4bb4a95b35bbe to start an election
I20260709 13:00:50.510829  7520 raft_consensus.cc:1081] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 1 LEADER]: Signalling peer ceaa51da210b427f92d4bb4a95b35bbe to start an election
I20260709 13:00:50.512401  7299 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7d16cc32774d49fbbd48d20cd2618904"
dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
 from {username='slave'} at 127.0.0.1:37706
I20260709 13:00:50.513048  7299 raft_consensus.cc:493] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:50.513513  7299 raft_consensus.cc:3060] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.518484  7299 raft_consensus.cc:515] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:50.520030  7299 leader_election.cc:290] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 2 election: Requested vote from peers 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:50.520921  7371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d16cc32774d49fbbd48d20cd2618904" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
I20260709 13:00:50.521427  7447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d16cc32774d49fbbd48d20cd2618904" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "29546ef8491d4dcba75459ed3d780497"
I20260709 13:00:50.521524  7371 raft_consensus.cc:3060] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.522089  7447 raft_consensus.cc:3055] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:50.522421  7447 raft_consensus.cc:740] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 29546ef8491d4dcba75459ed3d780497, State: Running, Role: LEADER
I20260709 13:00:50.523202  7447 consensus_queue.cc:260] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:50.524365  7447 raft_consensus.cc:3060] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.528371  7371 raft_consensus.cc:2468] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceaa51da210b427f92d4bb4a95b35bbe in term 2.
I20260709 13:00:50.529597  7265 leader_election.cc:304] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97c6951c9f204a3db527b90376f3ea36, ceaa51da210b427f92d4bb4a95b35bbe; no voters: 
I20260709 13:00:50.530372  7575 raft_consensus.cc:2804] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:50.530839  7575 raft_consensus.cc:697] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 LEADER]: Becoming Leader. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Running, Role: LEADER
I20260709 13:00:50.531502  7447 raft_consensus.cc:2468] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceaa51da210b427f92d4bb4a95b35bbe in term 2.
I20260709 13:00:50.531632  7575 consensus_queue.cc:237] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:50.538141  7194 catalog_manager.cc:5697] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe reported cstate change: term changed from 1 to 2, leader changed from 29546ef8491d4dcba75459ed3d780497 (127.6.255.3) to ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1). New cstate: current_term: 2 leader_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:50.642251  7371 raft_consensus.cc:1275] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Refusing update from remote peer ceaa51da210b427f92d4bb4a95b35bbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:50.643577  7575 consensus_queue.cc:1048] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:50.650658  7447 raft_consensus.cc:1275] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Refusing update from remote peer ceaa51da210b427f92d4bb4a95b35bbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:50.652285  7560 consensus_queue.cc:1048] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:50.780368  7371 raft_consensus.cc:1275] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Refusing update from remote peer ceaa51da210b427f92d4bb4a95b35bbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:50.781622  7560 consensus_queue.cc:1048] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:50.790978  7447 raft_consensus.cc:1275] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Refusing update from remote peer ceaa51da210b427f92d4bb4a95b35bbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:50.792373  7560 consensus_queue.cc:1048] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:50.807734  7371 raft_consensus.cc:1275] T b90380b15f1a42f2b3c9736536c43e21 P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Refusing update from remote peer 29546ef8491d4dcba75459ed3d780497: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:50.808890  7514 consensus_queue.cc:1048] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:50.816254  7299 raft_consensus.cc:1275] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Refusing update from remote peer 29546ef8491d4dcba75459ed3d780497: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:50.817929  7520 consensus_queue.cc:1048] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:50.874647  7299 raft_consensus.cc:1275] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Refusing update from remote peer 97c6951c9f204a3db527b90376f3ea36: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:50.876045  7571 consensus_queue.cc:1048] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:50.879096  7447 raft_consensus.cc:1275] T 3c646ace346640f8a3a4939d5c2aab5e P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Refusing update from remote peer 97c6951c9f204a3db527b90376f3ea36: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:50.881516  7570 consensus_queue.cc:1048] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:50.884938  7447 raft_consensus.cc:1275] T 1e97321e887f48a4b24dce648a4aa34d P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Refusing update from remote peer 97c6951c9f204a3db527b90376f3ea36: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:50.887797  7580 consensus_queue.cc:1048] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:50.892688  7299 raft_consensus.cc:1275] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Refusing update from remote peer 97c6951c9f204a3db527b90376f3ea36: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:50.894243  7571 consensus_queue.cc:1048] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:50.966575  7448 raft_consensus.cc:1275] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Refusing update from remote peer ceaa51da210b427f92d4bb4a95b35bbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:50.967958  7575 consensus_queue.cc:1048] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:50.977952  7371 raft_consensus.cc:1275] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Refusing update from remote peer ceaa51da210b427f92d4bb4a95b35bbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:50.979463  7529 consensus_queue.cc:1048] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:51.099622  7448 raft_consensus.cc:1275] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Refusing update from remote peer 97c6951c9f204a3db527b90376f3ea36: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:51.101048  7570 consensus_queue.cc:1048] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:51.102751  7299 raft_consensus.cc:1275] T d144c1f909d74a298d08c0f8b19cd9e0 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Refusing update from remote peer 97c6951c9f204a3db527b90376f3ea36: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:51.104218  7570 consensus_queue.cc:1048] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:53.094622  7164 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:00:53.095501  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:53.116029  7371 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d144c1f909d74a298d08c0f8b19cd9e0"
dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
mode: GRACEFUL
new_leader_uuid: "29546ef8491d4dcba75459ed3d780497"
 from {username='slave'} at 127.0.0.1:50932
I20260709 13:00:53.116586  7371 raft_consensus.cc:606] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 2 LEADER]: Received request to transfer leadership to TS 29546ef8491d4dcba75459ed3d780497
I20260709 13:00:53.128196  7298 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f55dd47f5d644c3eb24678e018b317e1"
dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
mode: GRACEFUL
new_leader_uuid: "29546ef8491d4dcba75459ed3d780497"
 from {username='slave'} at 127.0.0.1:39938
I20260709 13:00:53.128767  7298 raft_consensus.cc:606] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 LEADER]: Received request to transfer leadership to TS 29546ef8491d4dcba75459ed3d780497
I20260709 13:00:53.130393  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:00:53.130905  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:53.429131  7529 raft_consensus.cc:993] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe: : Instructing follower 29546ef8491d4dcba75459ed3d780497 to start an election
I20260709 13:00:53.429615  7590 raft_consensus.cc:1081] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 LEADER]: Signalling peer 29546ef8491d4dcba75459ed3d780497 to start an election
I20260709 13:00:53.431141  7448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f55dd47f5d644c3eb24678e018b317e1"
dest_uuid: "29546ef8491d4dcba75459ed3d780497"
 from {username='slave'} at 127.0.0.1:38608
I20260709 13:00:53.431651  7448 raft_consensus.cc:493] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:53.431984  7448 raft_consensus.cc:3060] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:53.436376  7448 raft_consensus.cc:515] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:53.438170  7448 leader_election.cc:290] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 3 election: Requested vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257)
I20260709 13:00:53.438952  7298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55dd47f5d644c3eb24678e018b317e1" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
I20260709 13:00:53.439229  7371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55dd47f5d644c3eb24678e018b317e1" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
I20260709 13:00:53.439603  7298 raft_consensus.cc:3055] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:53.439805  7371 raft_consensus.cc:3060] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:53.439963  7298 raft_consensus.cc:740] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Running, Role: LEADER
I20260709 13:00:53.440866  7298 consensus_queue.cc:260] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:53.441798  7298 raft_consensus.cc:3060] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:53.446444  7371 raft_consensus.cc:2468] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29546ef8491d4dcba75459ed3d780497 in term 3.
I20260709 13:00:53.446633  7298 raft_consensus.cc:2468] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29546ef8491d4dcba75459ed3d780497 in term 3.
I20260709 13:00:53.447607  7414 leader_election.cc:304] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, 97c6951c9f204a3db527b90376f3ea36; no voters: 
I20260709 13:00:53.448421  7585 raft_consensus.cc:2804] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:53.449877  7585 raft_consensus.cc:697] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 3 LEADER]: Becoming Leader. State: Replica: 29546ef8491d4dcba75459ed3d780497, State: Running, Role: LEADER
I20260709 13:00:53.450640  7585 consensus_queue.cc:237] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:53.457926  7193 catalog_manager.cc:5697] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 reported cstate change: term changed from 2 to 3, leader changed from ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1) to 29546ef8491d4dcba75459ed3d780497 (127.6.255.3). New cstate: current_term: 3 leader_uuid: "29546ef8491d4dcba75459ed3d780497" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:53.666196  7500 raft_consensus.cc:993] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36: : Instructing follower 29546ef8491d4dcba75459ed3d780497 to start an election
I20260709 13:00:53.666553  7500 raft_consensus.cc:1081] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 2 LEADER]: Signalling peer 29546ef8491d4dcba75459ed3d780497 to start an election
I20260709 13:00:53.668057  7448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d144c1f909d74a298d08c0f8b19cd9e0"
dest_uuid: "29546ef8491d4dcba75459ed3d780497"
 from {username='slave'} at 127.0.0.1:38600
I20260709 13:00:53.668588  7448 raft_consensus.cc:493] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:53.668901  7448 raft_consensus.cc:3060] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:53.673251  7448 raft_consensus.cc:515] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:53.674875  7448 leader_election.cc:290] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 3 election: Requested vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257)
I20260709 13:00:53.675810  7298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d144c1f909d74a298d08c0f8b19cd9e0" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
I20260709 13:00:53.675925  7371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d144c1f909d74a298d08c0f8b19cd9e0" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
I20260709 13:00:53.676293  7298 raft_consensus.cc:3060] T d144c1f909d74a298d08c0f8b19cd9e0 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:53.676549  7371 raft_consensus.cc:3055] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:53.676898  7371 raft_consensus.cc:740] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 97c6951c9f204a3db527b90376f3ea36, State: Running, Role: LEADER
I20260709 13:00:53.677701  7371 consensus_queue.cc:260] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:53.678656  7371 raft_consensus.cc:3060] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:53.680840  7298 raft_consensus.cc:2468] T d144c1f909d74a298d08c0f8b19cd9e0 P ceaa51da210b427f92d4bb4a95b35bbe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29546ef8491d4dcba75459ed3d780497 in term 3.
I20260709 13:00:53.681811  7412 leader_election.cc:304] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, ceaa51da210b427f92d4bb4a95b35bbe; no voters: 
I20260709 13:00:53.682511  7585 raft_consensus.cc:2804] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:53.683043  7585 raft_consensus.cc:697] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 3 LEADER]: Becoming Leader. State: Replica: 29546ef8491d4dcba75459ed3d780497, State: Running, Role: LEADER
I20260709 13:00:53.684110  7371 raft_consensus.cc:2468] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29546ef8491d4dcba75459ed3d780497 in term 3.
I20260709 13:00:53.683984  7585 consensus_queue.cc:237] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:53.691676  7193 catalog_manager.cc:5697] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 reported cstate change: term changed from 2 to 3, leader changed from 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2) to 29546ef8491d4dcba75459ed3d780497 (127.6.255.3). New cstate: current_term: 3 leader_uuid: "29546ef8491d4dcba75459ed3d780497" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:53.914980  7298 raft_consensus.cc:1275] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 3 FOLLOWER]: Refusing update from remote peer 29546ef8491d4dcba75459ed3d780497: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:53.916133  7585 consensus_queue.cc:1048] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:53.926935  7372 raft_consensus.cc:1275] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 3 FOLLOWER]: Refusing update from remote peer 29546ef8491d4dcba75459ed3d780497: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:53.928411  7591 consensus_queue.cc:1048] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:54.131773  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:54.142035  7299 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e451ff7482b5487c8db99685c67f888d"
dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
mode: GRACEFUL
new_leader_uuid: "97c6951c9f204a3db527b90376f3ea36"
 from {username='slave'} at 127.0.0.1:37710
I20260709 13:00:54.142583  7299 raft_consensus.cc:606] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [term 2 LEADER]: Received request to transfer leadership to TS 97c6951c9f204a3db527b90376f3ea36
I20260709 13:00:54.147109  7299 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dbc48c8abc054511b1c78331b267f725"
dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
mode: GRACEFUL
new_leader_uuid: "97c6951c9f204a3db527b90376f3ea36"
 from {username='slave'} at 127.0.0.1:37710
I20260709 13:00:54.147643  7299 raft_consensus.cc:606] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 LEADER]: Received request to transfer leadership to TS 97c6951c9f204a3db527b90376f3ea36
I20260709 13:00:54.152489  7448 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f55dd47f5d644c3eb24678e018b317e1"
dest_uuid: "29546ef8491d4dcba75459ed3d780497"
mode: GRACEFUL
new_leader_uuid: "97c6951c9f204a3db527b90376f3ea36"
 from {username='slave'} at 127.0.0.1:38616
I20260709 13:00:54.153020  7448 raft_consensus.cc:606] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 3 LEADER]: Received request to transfer leadership to TS 97c6951c9f204a3db527b90376f3ea36
I20260709 13:00:54.157473  7299 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d16cc32774d49fbbd48d20cd2618904"
dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
mode: GRACEFUL
new_leader_uuid: "97c6951c9f204a3db527b90376f3ea36"
 from {username='slave'} at 127.0.0.1:37710
I20260709 13:00:54.157998  7299 raft_consensus.cc:606] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 LEADER]: Received request to transfer leadership to TS 97c6951c9f204a3db527b90376f3ea36
I20260709 13:00:54.162667  7448 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d144c1f909d74a298d08c0f8b19cd9e0"
dest_uuid: "29546ef8491d4dcba75459ed3d780497"
mode: GRACEFUL
new_leader_uuid: "97c6951c9f204a3db527b90376f3ea36"
 from {username='slave'} at 127.0.0.1:38616
I20260709 13:00:54.163237  7448 raft_consensus.cc:606] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 3 LEADER]: Received request to transfer leadership to TS 97c6951c9f204a3db527b90376f3ea36
I20260709 13:00:54.272984  7590 raft_consensus.cc:993] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe: : Instructing follower 97c6951c9f204a3db527b90376f3ea36 to start an election
I20260709 13:00:54.273407  7590 raft_consensus.cc:1081] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [term 2 LEADER]: Signalling peer 97c6951c9f204a3db527b90376f3ea36 to start an election
I20260709 13:00:54.274864  7372 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e451ff7482b5487c8db99685c67f888d"
dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
 from {username='slave'} at 127.0.0.1:40294
I20260709 13:00:54.275291  7372 raft_consensus.cc:493] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:54.275532  7372 raft_consensus.cc:3060] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:54.279747  7372 raft_consensus.cc:515] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } } peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } }
I20260709 13:00:54.281178  7372 leader_election.cc:290] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 3 election: Requested vote from peers 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593), ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881)
I20260709 13:00:54.282135  7448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e451ff7482b5487c8db99685c67f888d" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "29546ef8491d4dcba75459ed3d780497"
I20260709 13:00:54.282217  7299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e451ff7482b5487c8db99685c67f888d" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
I20260709 13:00:54.282737  7448 raft_consensus.cc:3060] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:54.282804  7299 raft_consensus.cc:3055] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:54.283316  7299 raft_consensus.cc:740] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Running, Role: LEADER
I20260709 13:00:54.284149  7299 consensus_queue.cc:260] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } } peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } }
I20260709 13:00:54.285081  7299 raft_consensus.cc:3060] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:54.288153  7448 raft_consensus.cc:2468] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c6951c9f204a3db527b90376f3ea36 in term 3.
I20260709 13:00:54.289402  7337 leader_election.cc:304] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, 97c6951c9f204a3db527b90376f3ea36; no voters: 
I20260709 13:00:54.290225  7500 raft_consensus.cc:2804] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:54.290792  7500 raft_consensus.cc:697] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [term 3 LEADER]: Becoming Leader. State: Replica: 97c6951c9f204a3db527b90376f3ea36, State: Running, Role: LEADER
I20260709 13:00:54.291749  7299 raft_consensus.cc:2468] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c6951c9f204a3db527b90376f3ea36 in term 3.
I20260709 13:00:54.291841  7500 consensus_queue.cc:237] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } } peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } }
I20260709 13:00:54.294157  7372 raft_consensus.cc:1275] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 3 FOLLOWER]: Refusing update from remote peer 29546ef8491d4dcba75459ed3d780497: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:54.295459  7591 consensus_queue.cc:1048] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:54.300082  7193 catalog_manager.cc:5697] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 reported cstate change: term changed from 2 to 3, leader changed from ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1) to 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2). New cstate: current_term: 3 leader_uuid: "97c6951c9f204a3db527b90376f3ea36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:54.303381  7299 raft_consensus.cc:1275] T d144c1f909d74a298d08c0f8b19cd9e0 P ceaa51da210b427f92d4bb4a95b35bbe [term 3 FOLLOWER]: Refusing update from remote peer 29546ef8491d4dcba75459ed3d780497: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:54.305267  7608 consensus_queue.cc:1048] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:54.308758  7591 raft_consensus.cc:993] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497: : Instructing follower 97c6951c9f204a3db527b90376f3ea36 to start an election
I20260709 13:00:54.309299  7591 raft_consensus.cc:1081] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 3 LEADER]: Signalling peer 97c6951c9f204a3db527b90376f3ea36 to start an election
I20260709 13:00:54.311820  7372 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d144c1f909d74a298d08c0f8b19cd9e0"
dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:00:54.312531  7372 raft_consensus.cc:493] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:54.312992  7372 raft_consensus.cc:3060] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:54.322094  7372 raft_consensus.cc:515] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:54.325892  7298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d144c1f909d74a298d08c0f8b19cd9e0" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
I20260709 13:00:54.326143  7448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d144c1f909d74a298d08c0f8b19cd9e0" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "29546ef8491d4dcba75459ed3d780497"
I20260709 13:00:54.326915  7448 raft_consensus.cc:3055] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:54.327335  7448 raft_consensus.cc:740] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 29546ef8491d4dcba75459ed3d780497, State: Running, Role: LEADER
I20260709 13:00:54.328980  7372 leader_election.cc:290] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 4 election: Requested vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:54.328336  7448 consensus_queue.cc:260] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:54.330729  7448 raft_consensus.cc:3060] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:54.337456  7448 raft_consensus.cc:2468] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c6951c9f204a3db527b90376f3ea36 in term 4.
I20260709 13:00:54.338620  7337 leader_election.cc:304] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, 97c6951c9f204a3db527b90376f3ea36; no voters: ceaa51da210b427f92d4bb4a95b35bbe
I20260709 13:00:54.339419  7500 raft_consensus.cc:2804] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:54.339957  7500 raft_consensus.cc:697] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 4 LEADER]: Becoming Leader. State: Replica: 97c6951c9f204a3db527b90376f3ea36, State: Running, Role: LEADER
I20260709 13:00:54.340696  7500 consensus_queue.cc:237] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:54.346592  7193 catalog_manager.cc:5697] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 reported cstate change: term changed from 3 to 4, leader changed from 29546ef8491d4dcba75459ed3d780497 (127.6.255.3) to 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2). New cstate: current_term: 4 leader_uuid: "97c6951c9f204a3db527b90376f3ea36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:54.354396  7590 raft_consensus.cc:993] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe: : Instructing follower 97c6951c9f204a3db527b90376f3ea36 to start an election
I20260709 13:00:54.354851  7604 raft_consensus.cc:1081] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 LEADER]: Signalling peer 97c6951c9f204a3db527b90376f3ea36 to start an election
I20260709 13:00:54.356416  7372 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dbc48c8abc054511b1c78331b267f725"
dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
 from {username='slave'} at 127.0.0.1:40294
I20260709 13:00:54.356935  7372 raft_consensus.cc:493] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:54.357245  7372 raft_consensus.cc:3060] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:54.361721  7372 raft_consensus.cc:515] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:54.363255  7372 leader_election.cc:290] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 2 election: Requested vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:54.364177  7299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbc48c8abc054511b1c78331b267f725" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
I20260709 13:00:54.364404  7448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbc48c8abc054511b1c78331b267f725" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "29546ef8491d4dcba75459ed3d780497"
I20260709 13:00:54.364681  7299 raft_consensus.cc:3055] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:54.364928  7448 raft_consensus.cc:3060] T dbc48c8abc054511b1c78331b267f725 P 29546ef8491d4dcba75459ed3d780497 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:54.365012  7299 raft_consensus.cc:740] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Running, Role: LEADER
I20260709 13:00:54.365808  7299 consensus_queue.cc:260] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:54.366786  7299 raft_consensus.cc:3060] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:54.369119  7448 raft_consensus.cc:2468] T dbc48c8abc054511b1c78331b267f725 P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c6951c9f204a3db527b90376f3ea36 in term 2.
I20260709 13:00:54.370086  7337 leader_election.cc:304] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, 97c6951c9f204a3db527b90376f3ea36; no voters: 
I20260709 13:00:54.370764  7500 raft_consensus.cc:2804] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:54.371114  7299 raft_consensus.cc:2468] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c6951c9f204a3db527b90376f3ea36 in term 2.
I20260709 13:00:54.371147  7500 raft_consensus.cc:697] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [term 2 LEADER]: Becoming Leader. State: Replica: 97c6951c9f204a3db527b90376f3ea36, State: Running, Role: LEADER
I20260709 13:00:54.372134  7500 consensus_queue.cc:237] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:54.378863  7193 catalog_manager.cc:5697] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 reported cstate change: term changed from 1 to 2, leader changed from ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1) to 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "97c6951c9f204a3db527b90376f3ea36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:54.398921  7590 raft_consensus.cc:993] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe: : Instructing follower 97c6951c9f204a3db527b90376f3ea36 to start an election
I20260709 13:00:54.399394  7590 raft_consensus.cc:1081] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 LEADER]: Signalling peer 97c6951c9f204a3db527b90376f3ea36 to start an election
I20260709 13:00:54.401120  7372 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7d16cc32774d49fbbd48d20cd2618904"
dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
 from {username='slave'} at 127.0.0.1:40294
I20260709 13:00:54.401711  7372 raft_consensus.cc:493] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:54.402098  7372 raft_consensus.cc:3060] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:54.409761  7372 raft_consensus.cc:515] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:54.412678  7372 leader_election.cc:290] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 3 election: Requested vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:54.413677  7299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d16cc32774d49fbbd48d20cd2618904" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
I20260709 13:00:54.414414  7448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d16cc32774d49fbbd48d20cd2618904" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "29546ef8491d4dcba75459ed3d780497"
I20260709 13:00:54.414467  7299 raft_consensus.cc:3055] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:54.414978  7448 raft_consensus.cc:3060] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:54.414925  7299 raft_consensus.cc:740] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Running, Role: LEADER
I20260709 13:00:54.415844  7299 consensus_queue.cc:260] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:54.417039  7299 raft_consensus.cc:3060] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:54.421073  7448 raft_consensus.cc:2468] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c6951c9f204a3db527b90376f3ea36 in term 3.
I20260709 13:00:54.422365  7337 leader_election.cc:304] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, 97c6951c9f204a3db527b90376f3ea36; no voters: 
I20260709 13:00:54.423244  7500 raft_consensus.cc:2804] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:54.423583  7299 raft_consensus.cc:2468] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c6951c9f204a3db527b90376f3ea36 in term 3.
I20260709 13:00:54.423812  7500 raft_consensus.cc:697] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [term 3 LEADER]: Becoming Leader. State: Replica: 97c6951c9f204a3db527b90376f3ea36, State: Running, Role: LEADER
I20260709 13:00:54.424690  7500 consensus_queue.cc:237] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:54.436728  7193 catalog_manager.cc:5697] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 reported cstate change: term changed from 2 to 3, leader changed from ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1) to 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2). New cstate: current_term: 3 leader_uuid: "97c6951c9f204a3db527b90376f3ea36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:54.465512  7585 raft_consensus.cc:993] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497: : Instructing follower 97c6951c9f204a3db527b90376f3ea36 to start an election
I20260709 13:00:54.466033  7608 raft_consensus.cc:1081] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 3 LEADER]: Signalling peer 97c6951c9f204a3db527b90376f3ea36 to start an election
I20260709 13:00:54.467964  7372 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f55dd47f5d644c3eb24678e018b317e1"
dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:00:54.468474  7372 raft_consensus.cc:493] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:54.468782  7372 raft_consensus.cc:3060] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:54.473850  7372 raft_consensus.cc:515] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:54.476153  7372 leader_election.cc:290] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 4 election: Requested vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:54.476583  7299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55dd47f5d644c3eb24678e018b317e1" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
I20260709 13:00:54.476953  7448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55dd47f5d644c3eb24678e018b317e1" candidate_uuid: "97c6951c9f204a3db527b90376f3ea36" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "29546ef8491d4dcba75459ed3d780497"
I20260709 13:00:54.477196  7299 raft_consensus.cc:3060] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:54.477548  7448 raft_consensus.cc:3055] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:54.477913  7448 raft_consensus.cc:740] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 29546ef8491d4dcba75459ed3d780497, State: Running, Role: LEADER
I20260709 13:00:54.478663  7448 consensus_queue.cc:260] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:54.479528  7448 raft_consensus.cc:3060] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:54.483155  7299 raft_consensus.cc:2468] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c6951c9f204a3db527b90376f3ea36 in term 4.
I20260709 13:00:54.484200  7448 raft_consensus.cc:2468] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c6951c9f204a3db527b90376f3ea36 in term 4.
I20260709 13:00:54.484263  7338 leader_election.cc:304] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97c6951c9f204a3db527b90376f3ea36, ceaa51da210b427f92d4bb4a95b35bbe; no voters: 
I20260709 13:00:54.485074  7500 raft_consensus.cc:2804] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:54.485575  7500 raft_consensus.cc:697] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 4 LEADER]: Becoming Leader. State: Replica: 97c6951c9f204a3db527b90376f3ea36, State: Running, Role: LEADER
I20260709 13:00:54.486295  7500 consensus_queue.cc:237] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:54.491658  7193 catalog_manager.cc:5697] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 reported cstate change: term changed from 3 to 4, leader changed from 29546ef8491d4dcba75459ed3d780497 (127.6.255.3) to 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2). New cstate: current_term: 4 leader_uuid: "97c6951c9f204a3db527b90376f3ea36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:54.774114  7299 raft_consensus.cc:1275] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [term 3 FOLLOWER]: Refusing update from remote peer 97c6951c9f204a3db527b90376f3ea36: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:54.775513  7500 consensus_queue.cc:1048] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:54.787329  7448 raft_consensus.cc:1275] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 3 FOLLOWER]: Refusing update from remote peer 97c6951c9f204a3db527b90376f3ea36: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:54.791685  7299 raft_consensus.cc:1275] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Refusing update from remote peer 97c6951c9f204a3db527b90376f3ea36: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:54.792001  7616 consensus_queue.cc:1048] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:54.792915  7500 consensus_queue.cc:1048] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:54.803800  7447 raft_consensus.cc:1275] T dbc48c8abc054511b1c78331b267f725 P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Refusing update from remote peer 97c6951c9f204a3db527b90376f3ea36: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:54.805231  7616 consensus_queue.cc:1048] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:54.811158  7299 raft_consensus.cc:3060] T d144c1f909d74a298d08c0f8b19cd9e0 P ceaa51da210b427f92d4bb4a95b35bbe [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:54.816123  7299 raft_consensus.cc:1275] T d144c1f909d74a298d08c0f8b19cd9e0 P ceaa51da210b427f92d4bb4a95b35bbe [term 4 FOLLOWER]: Refusing update from remote peer 97c6951c9f204a3db527b90376f3ea36: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:54.817358  7605 consensus_queue.cc:1048] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:54.824064  7448 raft_consensus.cc:1275] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 4 FOLLOWER]: Refusing update from remote peer 97c6951c9f204a3db527b90376f3ea36: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:54.825641  7616 consensus_queue.cc:1048] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:54.898581  7448 raft_consensus.cc:1275] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 3 FOLLOWER]: Refusing update from remote peer 97c6951c9f204a3db527b90376f3ea36: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:54.899873  7616 consensus_queue.cc:1048] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:54.907905  7299 raft_consensus.cc:1275] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [term 3 FOLLOWER]: Refusing update from remote peer 97c6951c9f204a3db527b90376f3ea36: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:54.909751  7500 consensus_queue.cc:1048] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:54.998719  7448 raft_consensus.cc:1275] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 4 FOLLOWER]: Refusing update from remote peer 97c6951c9f204a3db527b90376f3ea36: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:54.999897  7500 consensus_queue.cc:1048] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:55.007632  7299 raft_consensus.cc:1275] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 4 FOLLOWER]: Refusing update from remote peer 97c6951c9f204a3db527b90376f3ea36: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:55.009487  7500 consensus_queue.cc:1048] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:57.175015  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:57.182281  7372 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "69dba3b1b71e434cac316850c2da8c01"
dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
mode: GRACEFUL
new_leader_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
 from {username='slave'} at 127.0.0.1:50932
I20260709 13:00:57.183045  7372 raft_consensus.cc:606] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [term 1 LEADER]: Received request to transfer leadership to TS ceaa51da210b427f92d4bb4a95b35bbe
I20260709 13:00:57.186347  7372 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d16cc32774d49fbbd48d20cd2618904"
dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
mode: GRACEFUL
new_leader_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
 from {username='slave'} at 127.0.0.1:50932
I20260709 13:00:57.186916  7372 raft_consensus.cc:606] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [term 3 LEADER]: Received request to transfer leadership to TS ceaa51da210b427f92d4bb4a95b35bbe
I20260709 13:00:57.189364  7372 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dbc48c8abc054511b1c78331b267f725"
dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
mode: GRACEFUL
new_leader_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
 from {username='slave'} at 127.0.0.1:50932
I20260709 13:00:57.189890  7372 raft_consensus.cc:606] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [term 2 LEADER]: Received request to transfer leadership to TS ceaa51da210b427f92d4bb4a95b35bbe
I20260709 13:00:57.192147  7372 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e451ff7482b5487c8db99685c67f888d"
dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
mode: GRACEFUL
new_leader_uuid: "29546ef8491d4dcba75459ed3d780497"
 from {username='slave'} at 127.0.0.1:50932
I20260709 13:00:57.192591  7372 raft_consensus.cc:606] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [term 3 LEADER]: Received request to transfer leadership to TS 29546ef8491d4dcba75459ed3d780497
I20260709 13:00:57.194802  7372 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f55dd47f5d644c3eb24678e018b317e1"
dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
mode: GRACEFUL
new_leader_uuid: "29546ef8491d4dcba75459ed3d780497"
 from {username='slave'} at 127.0.0.1:50932
I20260709 13:00:57.195235  7372 raft_consensus.cc:606] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 4 LEADER]: Received request to transfer leadership to TS 29546ef8491d4dcba75459ed3d780497
I20260709 13:00:57.196416  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:00:57.196877  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:57.212569  7605 raft_consensus.cc:993] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36: : Instructing follower ceaa51da210b427f92d4bb4a95b35bbe to start an election
I20260709 13:00:57.213011  7605 raft_consensus.cc:1081] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [term 2 LEADER]: Signalling peer ceaa51da210b427f92d4bb4a95b35bbe to start an election
I20260709 13:00:57.214512  7299 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dbc48c8abc054511b1c78331b267f725"
dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
 from {username='slave'} at 127.0.0.1:37708
I20260709 13:00:57.214983  7299 raft_consensus.cc:493] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:57.215246  7299 raft_consensus.cc:3060] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:57.220216  7299 raft_consensus.cc:515] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:57.221900  7299 leader_election.cc:290] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 3 election: Requested vote from peers 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:57.223248  7448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbc48c8abc054511b1c78331b267f725" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "29546ef8491d4dcba75459ed3d780497"
I20260709 13:00:57.223052  7372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbc48c8abc054511b1c78331b267f725" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
I20260709 13:00:57.223839  7448 raft_consensus.cc:3060] T dbc48c8abc054511b1c78331b267f725 P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:57.223915  7372 raft_consensus.cc:3055] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:57.224268  7372 raft_consensus.cc:740] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 97c6951c9f204a3db527b90376f3ea36, State: Running, Role: LEADER
I20260709 13:00:57.225131  7372 consensus_queue.cc:260] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:57.226490  7372 raft_consensus.cc:3060] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:57.230494  7448 raft_consensus.cc:2468] T dbc48c8abc054511b1c78331b267f725 P 29546ef8491d4dcba75459ed3d780497 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceaa51da210b427f92d4bb4a95b35bbe in term 3.
I20260709 13:00:57.231829  7262 leader_election.cc:304] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, ceaa51da210b427f92d4bb4a95b35bbe; no voters: 
I20260709 13:00:57.232806  7372 raft_consensus.cc:2468] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceaa51da210b427f92d4bb4a95b35bbe in term 3.
I20260709 13:00:57.233549  7627 raft_consensus.cc:2804] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:57.234633  7627 raft_consensus.cc:697] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [term 3 LEADER]: Becoming Leader. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Running, Role: LEADER
I20260709 13:00:57.235576  7627 consensus_queue.cc:237] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:57.242489  7194 catalog_manager.cc:5697] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe reported cstate change: term changed from 2 to 3, leader changed from 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2) to ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1). New cstate: current_term: 3 leader_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:57.245321  7626 raft_consensus.cc:993] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36: : Instructing follower 29546ef8491d4dcba75459ed3d780497 to start an election
I20260709 13:00:57.245839  7605 raft_consensus.cc:1081] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [term 3 LEADER]: Signalling peer 29546ef8491d4dcba75459ed3d780497 to start an election
I20260709 13:00:57.247336  7448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e451ff7482b5487c8db99685c67f888d"
dest_uuid: "29546ef8491d4dcba75459ed3d780497"
 from {username='slave'} at 127.0.0.1:38600
I20260709 13:00:57.247782  7448 raft_consensus.cc:493] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:57.248055  7448 raft_consensus.cc:3060] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:57.254101  7448 raft_consensus.cc:515] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } } peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } }
I20260709 13:00:57.256742  7448 leader_election.cc:290] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 4 election: Requested vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257)
I20260709 13:00:57.257165  7299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e451ff7482b5487c8db99685c67f888d" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
I20260709 13:00:57.257860  7299 raft_consensus.cc:3060] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:57.257808  7372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e451ff7482b5487c8db99685c67f888d" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
I20260709 13:00:57.258445  7372 raft_consensus.cc:3055] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:57.258785  7372 raft_consensus.cc:740] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 97c6951c9f204a3db527b90376f3ea36, State: Running, Role: LEADER
I20260709 13:00:57.259581  7372 consensus_queue.cc:260] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } } peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } }
I20260709 13:00:57.260766  7372 raft_consensus.cc:3060] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:57.264192  7299 raft_consensus.cc:2468] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29546ef8491d4dcba75459ed3d780497 in term 4.
I20260709 13:00:57.265614  7412 leader_election.cc:304] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, ceaa51da210b427f92d4bb4a95b35bbe; no voters: 
I20260709 13:00:57.266567  7608 raft_consensus.cc:2804] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:57.267452  7372 raft_consensus.cc:2468] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29546ef8491d4dcba75459ed3d780497 in term 4.
I20260709 13:00:57.268014  7608 raft_consensus.cc:697] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 4 LEADER]: Becoming Leader. State: Replica: 29546ef8491d4dcba75459ed3d780497, State: Running, Role: LEADER
I20260709 13:00:57.268982  7608 consensus_queue.cc:237] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } } peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } }
I20260709 13:00:57.275964  7194 catalog_manager.cc:5697] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 reported cstate change: term changed from 3 to 4, leader changed from 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2) to 29546ef8491d4dcba75459ed3d780497 (127.6.255.3). New cstate: current_term: 4 leader_uuid: "29546ef8491d4dcba75459ed3d780497" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:57.338423  7626 raft_consensus.cc:993] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36: : Instructing follower ceaa51da210b427f92d4bb4a95b35bbe to start an election
I20260709 13:00:57.338878  7605 raft_consensus.cc:1081] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [term 1 LEADER]: Signalling peer ceaa51da210b427f92d4bb4a95b35bbe to start an election
I20260709 13:00:57.340238  7299 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "69dba3b1b71e434cac316850c2da8c01"
dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
 from {username='slave'} at 127.0.0.1:37708
I20260709 13:00:57.340688  7299 raft_consensus.cc:493] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:57.340981  7299 raft_consensus.cc:3060] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:57.344836  7299 raft_consensus.cc:515] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:57.346419  7299 leader_election.cc:290] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 2 election: Requested vote from peers 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:57.347357  7372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dba3b1b71e434cac316850c2da8c01" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
I20260709 13:00:57.347579  7448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dba3b1b71e434cac316850c2da8c01" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "29546ef8491d4dcba75459ed3d780497"
I20260709 13:00:57.347972  7372 raft_consensus.cc:3055] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:57.348079  7448 raft_consensus.cc:3060] T 69dba3b1b71e434cac316850c2da8c01 P 29546ef8491d4dcba75459ed3d780497 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:57.348320  7372 raft_consensus.cc:740] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 97c6951c9f204a3db527b90376f3ea36, State: Running, Role: LEADER
I20260709 13:00:57.349229  7372 consensus_queue.cc:260] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:57.350351  7372 raft_consensus.cc:3060] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:57.352283  7448 raft_consensus.cc:2468] T 69dba3b1b71e434cac316850c2da8c01 P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceaa51da210b427f92d4bb4a95b35bbe in term 2.
I20260709 13:00:57.353277  7262 leader_election.cc:304] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, ceaa51da210b427f92d4bb4a95b35bbe; no voters: 
I20260709 13:00:57.354103  7627 raft_consensus.cc:2804] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:57.354522  7627 raft_consensus.cc:697] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 LEADER]: Becoming Leader. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Running, Role: LEADER
I20260709 13:00:57.355356  7627 consensus_queue.cc:237] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:57.359027  7372 raft_consensus.cc:2468] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceaa51da210b427f92d4bb4a95b35bbe in term 2.
I20260709 13:00:57.362650  7194 catalog_manager.cc:5697] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe reported cstate change: term changed from 1 to 2, leader changed from 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2) to ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1). New cstate: current_term: 2 leader_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:57.484980  7605 raft_consensus.cc:993] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36: : Instructing follower ceaa51da210b427f92d4bb4a95b35bbe to start an election
I20260709 13:00:57.485482  7626 raft_consensus.cc:1081] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [term 3 LEADER]: Signalling peer ceaa51da210b427f92d4bb4a95b35bbe to start an election
I20260709 13:00:57.487093  7299 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7d16cc32774d49fbbd48d20cd2618904"
dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
 from {username='slave'} at 127.0.0.1:37708
I20260709 13:00:57.487560  7299 raft_consensus.cc:493] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:57.487838  7299 raft_consensus.cc:3060] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:57.492436  7299 raft_consensus.cc:515] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:57.494391  7299 leader_election.cc:290] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 4 election: Requested vote from peers 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257), 29546ef8491d4dcba75459ed3d780497 (127.6.255.3:44593)
I20260709 13:00:57.494978  7372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d16cc32774d49fbbd48d20cd2618904" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
I20260709 13:00:57.495580  7372 raft_consensus.cc:3055] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:57.495594  7448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d16cc32774d49fbbd48d20cd2618904" candidate_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "29546ef8491d4dcba75459ed3d780497"
I20260709 13:00:57.495997  7372 raft_consensus.cc:740] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 97c6951c9f204a3db527b90376f3ea36, State: Running, Role: LEADER
I20260709 13:00:57.496884  7372 consensus_queue.cc:260] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:57.496404  7448 raft_consensus.cc:3060] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:57.497994  7372 raft_consensus.cc:3060] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:57.498713  7605 raft_consensus.cc:993] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36: : Instructing follower 29546ef8491d4dcba75459ed3d780497 to start an election
I20260709 13:00:57.499224  7626 raft_consensus.cc:1081] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 4 LEADER]: Signalling peer 29546ef8491d4dcba75459ed3d780497 to start an election
I20260709 13:00:57.500924  7447 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f55dd47f5d644c3eb24678e018b317e1"
dest_uuid: "29546ef8491d4dcba75459ed3d780497"
 from {username='slave'} at 127.0.0.1:38600
I20260709 13:00:57.501513  7447 raft_consensus.cc:493] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:57.501869  7447 raft_consensus.cc:3060] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:57.503613  7448 raft_consensus.cc:2468] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceaa51da210b427f92d4bb4a95b35bbe in term 4.
I20260709 13:00:57.504360  7372 raft_consensus.cc:2468] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceaa51da210b427f92d4bb4a95b35bbe in term 4.
I20260709 13:00:57.504707  7262 leader_election.cc:304] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, ceaa51da210b427f92d4bb4a95b35bbe; no voters: 
I20260709 13:00:57.505880  7627 raft_consensus.cc:2804] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:57.506273  7627 raft_consensus.cc:697] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [term 4 LEADER]: Becoming Leader. State: Replica: ceaa51da210b427f92d4bb4a95b35bbe, State: Running, Role: LEADER
I20260709 13:00:57.507067  7627 consensus_queue.cc:237] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:57.507881  7447 raft_consensus.cc:515] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:57.510838  7299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55dd47f5d644c3eb24678e018b317e1" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "ceaa51da210b427f92d4bb4a95b35bbe"
I20260709 13:00:57.510943  7447 leader_election.cc:290] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 5 election: Requested vote from peers ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881), 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257)
I20260709 13:00:57.511464  7299 raft_consensus.cc:3060] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:57.512506  7372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55dd47f5d644c3eb24678e018b317e1" candidate_uuid: "29546ef8491d4dcba75459ed3d780497" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "97c6951c9f204a3db527b90376f3ea36"
I20260709 13:00:57.513172  7372 raft_consensus.cc:3055] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:57.513540  7372 raft_consensus.cc:740] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 97c6951c9f204a3db527b90376f3ea36, State: Running, Role: LEADER
I20260709 13:00:57.514348  7372 consensus_queue.cc:260] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:57.515638  7372 raft_consensus.cc:3060] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:57.515264  7194 catalog_manager.cc:5697] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe reported cstate change: term changed from 3 to 4, leader changed from 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2) to ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1). New cstate: current_term: 4 leader_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:57.516252  7299 raft_consensus.cc:2468] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29546ef8491d4dcba75459ed3d780497 in term 5.
I20260709 13:00:57.517231  7412 leader_election.cc:304] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29546ef8491d4dcba75459ed3d780497, ceaa51da210b427f92d4bb4a95b35bbe; no voters: 
I20260709 13:00:57.518033  7608 raft_consensus.cc:2804] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:57.518477  7608 raft_consensus.cc:697] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 5 LEADER]: Becoming Leader. State: Replica: 29546ef8491d4dcba75459ed3d780497, State: Running, Role: LEADER
I20260709 13:00:57.519330  7608 consensus_queue.cc:237] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } }
I20260709 13:00:57.521862  7372 raft_consensus.cc:2468] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29546ef8491d4dcba75459ed3d780497 in term 5.
I20260709 13:00:57.528326  7194 catalog_manager.cc:5697] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 reported cstate change: term changed from 4 to 5, leader changed from 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2) to 29546ef8491d4dcba75459ed3d780497 (127.6.255.3). New cstate: current_term: 5 leader_uuid: "29546ef8491d4dcba75459ed3d780497" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:57.700039  7372 raft_consensus.cc:1275] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [term 3 FOLLOWER]: Refusing update from remote peer ceaa51da210b427f92d4bb4a95b35bbe: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:57.701318  7627 consensus_queue.cc:1048] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:57.713250  7447 raft_consensus.cc:1275] T dbc48c8abc054511b1c78331b267f725 P 29546ef8491d4dcba75459ed3d780497 [term 3 FOLLOWER]: Refusing update from remote peer ceaa51da210b427f92d4bb4a95b35bbe: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:57.716491  7641 consensus_queue.cc:1048] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:57.829118  7447 raft_consensus.cc:1275] T 69dba3b1b71e434cac316850c2da8c01 P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Refusing update from remote peer ceaa51da210b427f92d4bb4a95b35bbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:57.830706  7641 consensus_queue.cc:1048] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:57.839969  7372 raft_consensus.cc:1275] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Refusing update from remote peer ceaa51da210b427f92d4bb4a95b35bbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:57.841331  7628 consensus_queue.cc:1048] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:57.847978  7299 raft_consensus.cc:1275] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [term 4 FOLLOWER]: Refusing update from remote peer 29546ef8491d4dcba75459ed3d780497: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:57.849947  7608 consensus_queue.cc:1048] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:57.861099  7372 raft_consensus.cc:1275] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [term 4 FOLLOWER]: Refusing update from remote peer 29546ef8491d4dcba75459ed3d780497: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:57.863546  7608 consensus_queue.cc:1048] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:57.973362  7447 raft_consensus.cc:1275] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 4 FOLLOWER]: Refusing update from remote peer ceaa51da210b427f92d4bb4a95b35bbe: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:57.974680  7628 consensus_queue.cc:1048] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "29546ef8491d4dcba75459ed3d780497" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:57.983834  7371 raft_consensus.cc:1275] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [term 4 FOLLOWER]: Refusing update from remote peer ceaa51da210b427f92d4bb4a95b35bbe: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:57.985630  7628 consensus_queue.cc:1048] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:58.000476  7371 raft_consensus.cc:1275] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 5 FOLLOWER]: Refusing update from remote peer 29546ef8491d4dcba75459ed3d780497: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:58.001791  7634 consensus_queue.cc:1048] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97c6951c9f204a3db527b90376f3ea36" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:58.010758  7299 raft_consensus.cc:1275] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 5 FOLLOWER]: Refusing update from remote peer 29546ef8491d4dcba75459ed3d780497: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:58.012495  7648 consensus_queue.cc:1048] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceaa51da210b427f92d4bb4a95b35bbe" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 38881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:58.197731  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:58.203027  7164 tablet_server.cc:179] TabletServer@127.6.255.1:0 shutting down...
I20260709 13:00:58.232260  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:00:58.233166  7164 tablet_replica.cc:333] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe: stopping tablet replica
I20260709 13:00:58.233991  7164 raft_consensus.cc:2243] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:58.234627  7164 raft_consensus.cc:2272] T 1e97321e887f48a4b24dce648a4aa34d P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.236969  7164 tablet_replica.cc:333] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe: stopping tablet replica
I20260709 13:00:58.237514  7164 raft_consensus.cc:2243] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:00:58.238166  7164 raft_consensus.cc:2272] T dbc48c8abc054511b1c78331b267f725 P ceaa51da210b427f92d4bb4a95b35bbe [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.240049  7164 tablet_replica.cc:333] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe: stopping tablet replica
I20260709 13:00:58.240610  7164 raft_consensus.cc:2243] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:58.240962  7164 raft_consensus.cc:2272] T e451ff7482b5487c8db99685c67f888d P ceaa51da210b427f92d4bb4a95b35bbe [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.243026  7164 tablet_replica.cc:333] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe: stopping tablet replica
I20260709 13:00:58.243486  7164 raft_consensus.cc:2243] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:00:58.244100  7164 raft_consensus.cc:2272] T 7d16cc32774d49fbbd48d20cd2618904 P ceaa51da210b427f92d4bb4a95b35bbe [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.246018  7164 tablet_replica.cc:333] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe: stopping tablet replica
I20260709 13:00:58.246438  7164 raft_consensus.cc:2243] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:58.246773  7164 raft_consensus.cc:2272] T b90380b15f1a42f2b3c9736536c43e21 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.248238  7164 tablet_replica.cc:333] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe: stopping tablet replica
I20260709 13:00:58.248673  7164 raft_consensus.cc:2243] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:58.249082  7164 raft_consensus.cc:2272] T f55dd47f5d644c3eb24678e018b317e1 P ceaa51da210b427f92d4bb4a95b35bbe [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.251116  7164 tablet_replica.cc:333] T d144c1f909d74a298d08c0f8b19cd9e0 P ceaa51da210b427f92d4bb4a95b35bbe: stopping tablet replica
I20260709 13:00:58.251626  7164 raft_consensus.cc:2243] T d144c1f909d74a298d08c0f8b19cd9e0 P ceaa51da210b427f92d4bb4a95b35bbe [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:58.252033  7164 raft_consensus.cc:2272] T d144c1f909d74a298d08c0f8b19cd9e0 P ceaa51da210b427f92d4bb4a95b35bbe [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.254145  7164 tablet_replica.cc:333] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe: stopping tablet replica
I20260709 13:00:58.254755  7164 raft_consensus.cc:2243] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:00:58.255658  7164 raft_consensus.cc:2272] T 69dba3b1b71e434cac316850c2da8c01 P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.257711  7164 tablet_replica.cc:333] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe: stopping tablet replica
I20260709 13:00:58.258188  7164 raft_consensus.cc:2243] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:58.258601  7164 raft_consensus.cc:2272] T 3c646ace346640f8a3a4939d5c2aab5e P ceaa51da210b427f92d4bb4a95b35bbe [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:00:58.367748  7412 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.255.1:38881: connect: Connection refused (error 111)
I20260709 13:00:58.369311  7164 tablet_server.cc:196] TabletServer@127.6.255.1:0 shutdown complete.
W20260709 13:00:58.371845  7412 consensus_peers.cc:597] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 -> Peer ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881): Couldn't send request to peer ceaa51da210b427f92d4bb4a95b35bbe. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:38881: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:58.389832  7164 tablet_server.cc:179] TabletServer@127.6.255.2:0 shutting down...
W20260709 13:00:58.417938  7414 consensus_peers.cc:597] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 -> Peer 97c6951c9f204a3db527b90376f3ea36 (127.6.255.2:46257): Couldn't send request to peer 97c6951c9f204a3db527b90376f3ea36. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:58.418692  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:00:58.419379  7164 tablet_replica.cc:333] T b90380b15f1a42f2b3c9736536c43e21 P 97c6951c9f204a3db527b90376f3ea36: stopping tablet replica
I20260709 13:00:58.420264  7164 raft_consensus.cc:2243] T b90380b15f1a42f2b3c9736536c43e21 P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:58.420872  7164 raft_consensus.cc:2272] T b90380b15f1a42f2b3c9736536c43e21 P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.422694  7164 tablet_replica.cc:333] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36: stopping tablet replica
I20260709 13:00:58.423153  7164 raft_consensus.cc:2243] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:58.423486  7164 raft_consensus.cc:2272] T f55dd47f5d644c3eb24678e018b317e1 P 97c6951c9f204a3db527b90376f3ea36 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.425848  7164 tablet_replica.cc:333] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36: stopping tablet replica
I20260709 13:00:58.426307  7164 raft_consensus.cc:2243] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:00:58.426975  7164 raft_consensus.cc:2272] T 3c646ace346640f8a3a4939d5c2aab5e P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.428534  7164 tablet_replica.cc:333] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36: stopping tablet replica
I20260709 13:00:58.428959  7164 raft_consensus.cc:2243] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:58.429286  7164 raft_consensus.cc:2272] T 7d16cc32774d49fbbd48d20cd2618904 P 97c6951c9f204a3db527b90376f3ea36 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.431123  7164 tablet_replica.cc:333] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36: stopping tablet replica
I20260709 13:00:58.431550  7164 raft_consensus.cc:2243] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:58.431883  7164 raft_consensus.cc:2272] T e451ff7482b5487c8db99685c67f888d P 97c6951c9f204a3db527b90376f3ea36 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.433972  7164 tablet_replica.cc:333] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36: stopping tablet replica
I20260709 13:00:58.434414  7164 raft_consensus.cc:2243] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:58.434734  7164 raft_consensus.cc:2272] T dbc48c8abc054511b1c78331b267f725 P 97c6951c9f204a3db527b90376f3ea36 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.436686  7164 tablet_replica.cc:333] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36: stopping tablet replica
I20260709 13:00:58.437294  7164 raft_consensus.cc:2243] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:00:58.438076  7164 raft_consensus.cc:2272] T 1e97321e887f48a4b24dce648a4aa34d P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.439849  7164 tablet_replica.cc:333] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36: stopping tablet replica
I20260709 13:00:58.440367  7164 raft_consensus.cc:2243] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:58.440747  7164 raft_consensus.cc:2272] T 69dba3b1b71e434cac316850c2da8c01 P 97c6951c9f204a3db527b90376f3ea36 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.442799  7164 tablet_replica.cc:333] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36: stopping tablet replica
I20260709 13:00:58.443253  7164 raft_consensus.cc:2243] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:00:58.443948  7164 raft_consensus.cc:2272] T d144c1f909d74a298d08c0f8b19cd9e0 P 97c6951c9f204a3db527b90376f3ea36 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.471643  7164 tablet_server.cc:196] TabletServer@127.6.255.2:0 shutdown complete.
W20260709 13:00:58.484164  7412 consensus_peers.cc:597] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 -> Peer ceaa51da210b427f92d4bb4a95b35bbe (127.6.255.1:38881): Couldn't send request to peer ceaa51da210b427f92d4bb4a95b35bbe. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:38881: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:58.493182  7164 tablet_server.cc:179] TabletServer@127.6.255.3:0 shutting down...
I20260709 13:00:58.519212  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:00:58.520020  7164 tablet_replica.cc:333] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497: stopping tablet replica
I20260709 13:00:58.520682  7164 raft_consensus.cc:2243] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:00:58.521411  7164 raft_consensus.cc:2272] T b90380b15f1a42f2b3c9736536c43e21 P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.523162  7164 tablet_replica.cc:333] T dbc48c8abc054511b1c78331b267f725 P 29546ef8491d4dcba75459ed3d780497: stopping tablet replica
I20260709 13:00:58.523658  7164 raft_consensus.cc:2243] T dbc48c8abc054511b1c78331b267f725 P 29546ef8491d4dcba75459ed3d780497 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:58.524032  7164 raft_consensus.cc:2272] T dbc48c8abc054511b1c78331b267f725 P 29546ef8491d4dcba75459ed3d780497 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.526055  7164 tablet_replica.cc:333] T 1e97321e887f48a4b24dce648a4aa34d P 29546ef8491d4dcba75459ed3d780497: stopping tablet replica
I20260709 13:00:58.526557  7164 raft_consensus.cc:2243] T 1e97321e887f48a4b24dce648a4aa34d P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:58.526947  7164 raft_consensus.cc:2272] T 1e97321e887f48a4b24dce648a4aa34d P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.528463  7164 tablet_replica.cc:333] T 3c646ace346640f8a3a4939d5c2aab5e P 29546ef8491d4dcba75459ed3d780497: stopping tablet replica
I20260709 13:00:58.528939  7164 raft_consensus.cc:2243] T 3c646ace346640f8a3a4939d5c2aab5e P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:58.529311  7164 raft_consensus.cc:2272] T 3c646ace346640f8a3a4939d5c2aab5e P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.530870  7164 tablet_replica.cc:333] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497: stopping tablet replica
I20260709 13:00:58.531364  7164 raft_consensus.cc:2243] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:58.531769  7164 raft_consensus.cc:2272] T 7d16cc32774d49fbbd48d20cd2618904 P 29546ef8491d4dcba75459ed3d780497 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.533649  7164 tablet_replica.cc:333] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497: stopping tablet replica
I20260709 13:00:58.534132  7164 raft_consensus.cc:2243] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:00:58.534789  7164 raft_consensus.cc:2272] T e451ff7482b5487c8db99685c67f888d P 29546ef8491d4dcba75459ed3d780497 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.536597  7164 tablet_replica.cc:333] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497: stopping tablet replica
I20260709 13:00:58.537065  7164 raft_consensus.cc:2243] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:00:58.537876  7164 raft_consensus.cc:2272] T f55dd47f5d644c3eb24678e018b317e1 P 29546ef8491d4dcba75459ed3d780497 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.539597  7164 tablet_replica.cc:333] T 69dba3b1b71e434cac316850c2da8c01 P 29546ef8491d4dcba75459ed3d780497: stopping tablet replica
I20260709 13:00:58.540059  7164 raft_consensus.cc:2243] T 69dba3b1b71e434cac316850c2da8c01 P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:58.540433  7164 raft_consensus.cc:2272] T 69dba3b1b71e434cac316850c2da8c01 P 29546ef8491d4dcba75459ed3d780497 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.542177  7164 tablet_replica.cc:333] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497: stopping tablet replica
I20260709 13:00:58.542618  7164 raft_consensus.cc:2243] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:58.543056  7164 raft_consensus.cc:2272] T d144c1f909d74a298d08c0f8b19cd9e0 P 29546ef8491d4dcba75459ed3d780497 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.569000  7164 tablet_server.cc:196] TabletServer@127.6.255.3:0 shutdown complete.
I20260709 13:00:58.585821  7164 master.cc:562] Master@127.6.255.62:35591 shutting down...
I20260709 13:00:58.717633  7164 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:00:58.718223  7164 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.718582  7164 tablet_replica.cc:333] T 00000000000000000000000000000000 P 4127d5019f104ffc89c7721a5cfd02d1: stopping tablet replica
I20260709 13:00:58.738664  7164 master.cc:584] Master@127.6.255.62:35591 shutdown complete.
29546ef8491d4dcba75459ed3d780497  1
97c6951c9f204a3db527b90376f3ea36  4
ceaa51da210b427f92d4bb4a95b35bbe  4
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (12552 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:00:58.778928  7164 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.255.62:41293
I20260709 13:00:58.779985  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:58.785578  7656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:58.785696  7657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:58.786568  7659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:58.787369  7164 server_base.cc:1061] running on GCE node
I20260709 13:00:58.788154  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:58.788334  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:58.788446  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602058788435 us; error 0 us; skew 500 ppm
I20260709 13:00:58.788883  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:58.791220  7164 webserver.cc:533] Webserver started at http://127.6.255.62:39351/ using document root <none> and password file <none>
I20260709 13:00:58.791649  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:58.791793  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:58.791997  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:58.793032  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602046064212-7164-0/minicluster-data/master-0-root/instance:
uuid: "775e031c99604f14afc254078b5e195c"
format_stamp: "Formatted at 2026-07-09 13:00:58 on dist-test-slave-rnv4"
I20260709 13:00:58.797324  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.002s
I20260709 13:00:58.800694  7664 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:58.801604  7164 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.003s
I20260709 13:00:58.801908  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602046064212-7164-0/minicluster-data/master-0-root
uuid: "775e031c99604f14afc254078b5e195c"
format_stamp: "Formatted at 2026-07-09 13:00:58 on dist-test-slave-rnv4"
I20260709 13:00:58.802198  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602046064212-7164-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602046064212-7164-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602046064212-7164-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:58.828382  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:58.829842  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:58.870442  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.62:41293
I20260709 13:00:58.870559  7715 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.62:41293 every 8 connection(s)
I20260709 13:00:58.874809  7716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.884972  7716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 775e031c99604f14afc254078b5e195c: Bootstrap starting.
I20260709 13:00:58.889660  7716 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 775e031c99604f14afc254078b5e195c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:58.894052  7716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 775e031c99604f14afc254078b5e195c: No bootstrap required, opened a new log
I20260709 13:00:58.896418  7716 raft_consensus.cc:359] T 00000000000000000000000000000000 P 775e031c99604f14afc254078b5e195c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "775e031c99604f14afc254078b5e195c" member_type: VOTER }
I20260709 13:00:58.896795  7716 raft_consensus.cc:385] T 00000000000000000000000000000000 P 775e031c99604f14afc254078b5e195c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:58.897073  7716 raft_consensus.cc:740] T 00000000000000000000000000000000 P 775e031c99604f14afc254078b5e195c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 775e031c99604f14afc254078b5e195c, State: Initialized, Role: FOLLOWER
I20260709 13:00:58.897693  7716 consensus_queue.cc:260] T 00000000000000000000000000000000 P 775e031c99604f14afc254078b5e195c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "775e031c99604f14afc254078b5e195c" member_type: VOTER }
I20260709 13:00:58.898100  7716 raft_consensus.cc:399] T 00000000000000000000000000000000 P 775e031c99604f14afc254078b5e195c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:00:58.898355  7716 raft_consensus.cc:493] T 00000000000000000000000000000000 P 775e031c99604f14afc254078b5e195c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:00:58.898633  7716 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 775e031c99604f14afc254078b5e195c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:58.902746  7716 raft_consensus.cc:515] T 00000000000000000000000000000000 P 775e031c99604f14afc254078b5e195c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "775e031c99604f14afc254078b5e195c" member_type: VOTER }
I20260709 13:00:58.903273  7716 leader_election.cc:304] T 00000000000000000000000000000000 P 775e031c99604f14afc254078b5e195c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 775e031c99604f14afc254078b5e195c; no voters: 
I20260709 13:00:58.904564  7716 leader_election.cc:290] T 00000000000000000000000000000000 P 775e031c99604f14afc254078b5e195c [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:00:58.904938  7719 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 775e031c99604f14afc254078b5e195c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:58.906345  7719 raft_consensus.cc:697] T 00000000000000000000000000000000 P 775e031c99604f14afc254078b5e195c [term 1 LEADER]: Becoming Leader. State: Replica: 775e031c99604f14afc254078b5e195c, State: Running, Role: LEADER
I20260709 13:00:58.906935  7719 consensus_queue.cc:237] T 00000000000000000000000000000000 P 775e031c99604f14afc254078b5e195c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "775e031c99604f14afc254078b5e195c" member_type: VOTER }
I20260709 13:00:58.907498  7716 sys_catalog.cc:565] T 00000000000000000000000000000000 P 775e031c99604f14afc254078b5e195c [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:00:58.911675  7721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 775e031c99604f14afc254078b5e195c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 775e031c99604f14afc254078b5e195c. Latest consensus state: current_term: 1 leader_uuid: "775e031c99604f14afc254078b5e195c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "775e031c99604f14afc254078b5e195c" member_type: VOTER } }
I20260709 13:00:58.912479  7721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 775e031c99604f14afc254078b5e195c [sys.catalog]: This master's current role is: LEADER
I20260709 13:00:58.912978  7720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 775e031c99604f14afc254078b5e195c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "775e031c99604f14afc254078b5e195c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "775e031c99604f14afc254078b5e195c" member_type: VOTER } }
I20260709 13:00:58.913676  7720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 775e031c99604f14afc254078b5e195c [sys.catalog]: This master's current role is: LEADER
I20260709 13:00:58.915714  7727 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:00:58.923700  7727 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:00:58.924656  7164 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:00:58.933475  7727 catalog_manager.cc:1383] Generated new cluster ID: 6f98be67f8fc404c80007c7ba32130b6
I20260709 13:00:58.933808  7727 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:00:58.954476  7727 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:00:58.955698  7727 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:00:58.971123  7727 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 775e031c99604f14afc254078b5e195c: Generated new TSK 0
I20260709 13:00:58.971684  7727 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:00:58.991156  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:58.997609  7737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:58.998775  7738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:59.000156  7740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:59.001127  7164 server_base.cc:1061] running on GCE node
I20260709 13:00:59.001982  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:59.002164  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:59.002405  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602059002361 us; error 0 us; skew 500 ppm
I20260709 13:00:59.003166  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:59.005781  7164 webserver.cc:533] Webserver started at http://127.6.255.1:43119/ using document root <none> and password file <none>
I20260709 13:00:59.006273  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:59.006459  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:59.006757  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:59.007794  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602046064212-7164-0/minicluster-data/ts-0-root/instance:
uuid: "68c88031f63c411a9a349b00f53b211d"
format_stamp: "Formatted at 2026-07-09 13:00:59 on dist-test-slave-rnv4"
I20260709 13:00:59.012594  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.003s
I20260709 13:00:59.016381  7745 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:59.017494  7164 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:00:59.017763  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602046064212-7164-0/minicluster-data/ts-0-root
uuid: "68c88031f63c411a9a349b00f53b211d"
format_stamp: "Formatted at 2026-07-09 13:00:59 on dist-test-slave-rnv4"
I20260709 13:00:59.018002  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602046064212-7164-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602046064212-7164-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602046064212-7164-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:59.033619  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:59.034838  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:59.036303  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:59.038945  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:59.039136  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:59.039438  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:59.039624  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:59.083271  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.1:37239
I20260709 13:00:59.083367  7807 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.1:37239 every 8 connection(s)
I20260709 13:00:59.088567  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:59.096241  7812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:59.097947  7813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:59.100888  7164 server_base.cc:1061] running on GCE node
W20260709 13:00:59.101239  7815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:59.102404  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:59.102715  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:59.102941  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602059102917 us; error 0 us; skew 500 ppm
I20260709 13:00:59.103657  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:59.106707  7164 webserver.cc:533] Webserver started at http://127.6.255.2:39997/ using document root <none> and password file <none>
I20260709 13:00:59.107023  7808 heartbeater.cc:344] Connected to a master server at 127.6.255.62:41293
I20260709 13:00:59.107376  7808 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:59.107345  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:59.107801  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:59.108115  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:59.108240  7808 heartbeater.cc:507] Master 127.6.255.62:41293 requested a full tablet report, sending...
I20260709 13:00:59.109680  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602046064212-7164-0/minicluster-data/ts-1-root/instance:
uuid: "c4ca9afe41ff4eb298062aa06f346b95"
format_stamp: "Formatted at 2026-07-09 13:00:59 on dist-test-slave-rnv4"
I20260709 13:00:59.110563  7681 ts_manager.cc:194] Registered new tserver with Master: 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239)
I20260709 13:00:59.112334  7681 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52414
I20260709 13:00:59.115023  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:00:59.118687  7820 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:59.119740  7164 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:00:59.120047  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602046064212-7164-0/minicluster-data/ts-1-root
uuid: "c4ca9afe41ff4eb298062aa06f346b95"
format_stamp: "Formatted at 2026-07-09 13:00:59 on dist-test-slave-rnv4"
I20260709 13:00:59.120355  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602046064212-7164-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602046064212-7164-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602046064212-7164-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:59.146181  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:59.147392  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:59.148864  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:59.151047  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:59.151271  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:59.151518  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:59.151703  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:59.191270  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.2:33185
I20260709 13:00:59.191361  7882 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.2:33185 every 8 connection(s)
I20260709 13:00:59.196142  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:59.203276  7886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:59.204861  7887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:59.207016  7889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:59.207536  7164 server_base.cc:1061] running on GCE node
I20260709 13:00:59.208173  7883 heartbeater.cc:344] Connected to a master server at 127.6.255.62:41293
I20260709 13:00:59.208564  7883 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:59.208573  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:59.208957  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:59.209158  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602059209140 us; error 0 us; skew 500 ppm
I20260709 13:00:59.209605  7883 heartbeater.cc:507] Master 127.6.255.62:41293 requested a full tablet report, sending...
I20260709 13:00:59.209903  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:59.211971  7681 ts_manager.cc:194] Registered new tserver with Master: c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185)
I20260709 13:00:59.213089  7164 webserver.cc:533] Webserver started at http://127.6.255.3:35313/ using document root <none> and password file <none>
I20260709 13:00:59.213729  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:59.213871  7681 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52422
I20260709 13:00:59.213979  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:59.214402  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:59.215864  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602046064212-7164-0/minicluster-data/ts-2-root/instance:
uuid: "9aec419c47a640edb247a753aaa6ad5b"
format_stamp: "Formatted at 2026-07-09 13:00:59 on dist-test-slave-rnv4"
I20260709 13:00:59.220331  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.003s
I20260709 13:00:59.223799  7894 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:59.224620  7164 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:00:59.224877  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602046064212-7164-0/minicluster-data/ts-2-root
uuid: "9aec419c47a640edb247a753aaa6ad5b"
format_stamp: "Formatted at 2026-07-09 13:00:59 on dist-test-slave-rnv4"
I20260709 13:00:59.225157  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602046064212-7164-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602046064212-7164-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602046064212-7164-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:59.237311  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:59.238540  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:59.240065  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:59.242347  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:59.242524  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:59.242834  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:59.242975  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:59.284006  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.3:41505
I20260709 13:00:59.284111  7956 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.3:41505 every 8 connection(s)
I20260709 13:00:59.298592  7957 heartbeater.cc:344] Connected to a master server at 127.6.255.62:41293
I20260709 13:00:59.298949  7957 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:59.299665  7957 heartbeater.cc:507] Master 127.6.255.62:41293 requested a full tablet report, sending...
I20260709 13:00:59.301846  7681 ts_manager.cc:194] Registered new tserver with Master: 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
I20260709 13:00:59.302173  7164 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01481011s
I20260709 13:00:59.303686  7681 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52438
I20260709 13:00:59.329532  7681 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52442:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:00:59.332329  7681 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:00:59.411988  7773 tablet_service.cc:1511] Processing CreateTablet for tablet 428de0e6e6d1418ebcbc8ff1fbf24a26 (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:00:59.412436  7772 tablet_service.cc:1511] Processing CreateTablet for tablet 6c817cd2a564431e89c98f5500a9bd37 (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:00:59.413604  7773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 428de0e6e6d1418ebcbc8ff1fbf24a26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.415319  7771 tablet_service.cc:1511] Processing CreateTablet for tablet bcb544d4966e409c97ce8eff34e7521e (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:00:59.416628  7771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcb544d4966e409c97ce8eff34e7521e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.417716  7772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c817cd2a564431e89c98f5500a9bd37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.419492  7770 tablet_service.cc:1511] Processing CreateTablet for tablet 55538fcb866c4dd1a3cd12db54eb946b (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:00:59.420960  7770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55538fcb866c4dd1a3cd12db54eb946b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.440414  7772 tablet_service.cc:1511] Processing CreateTablet for tablet 494fef1b35024d30b17d4c0aee058999 (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:00:59.440531  7922 tablet_service.cc:1511] Processing CreateTablet for tablet 428de0e6e6d1418ebcbc8ff1fbf24a26 (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:00:59.441304  7920 tablet_service.cc:1511] Processing CreateTablet for tablet bcb544d4966e409c97ce8eff34e7521e (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:00:59.442078  7922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 428de0e6e6d1418ebcbc8ff1fbf24a26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.442215  7770 tablet_service.cc:1511] Processing CreateTablet for tablet c4ced2215e2544faa2ecda7ceba5de27 (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:00:59.442770  7920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcb544d4966e409c97ce8eff34e7521e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.438939  7769 tablet_service.cc:1511] Processing CreateTablet for tablet 20e3b50df0d7462da76007b962677370 (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:00:59.443394  7770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4ced2215e2544faa2ecda7ceba5de27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.443997  7769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20e3b50df0d7462da76007b962677370. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.437400  7972 tablet_bootstrap.cc:492] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d: Bootstrap starting.
I20260709 13:00:59.447085  7919 tablet_service.cc:1511] Processing CreateTablet for tablet 55538fcb866c4dd1a3cd12db54eb946b (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:00:59.448413  7919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55538fcb866c4dd1a3cd12db54eb946b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.450139  7772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 494fef1b35024d30b17d4c0aee058999. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.452383  7917 tablet_service.cc:1511] Processing CreateTablet for tablet c4ced2215e2544faa2ecda7ceba5de27 (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:00:59.442385  7921 tablet_service.cc:1511] Processing CreateTablet for tablet 6c817cd2a564431e89c98f5500a9bd37 (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:00:59.453671  7917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4ced2215e2544faa2ecda7ceba5de27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.454325  7921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c817cd2a564431e89c98f5500a9bd37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.450630  7918 tablet_service.cc:1511] Processing CreateTablet for tablet 20e3b50df0d7462da76007b962677370 (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:00:59.457151  7918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20e3b50df0d7462da76007b962677370. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.466506  7973 tablet_bootstrap.cc:492] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b: Bootstrap starting.
I20260709 13:00:59.451164  7972 tablet_bootstrap.cc:654] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.474205  7973 tablet_bootstrap.cc:654] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.481757  7972 tablet_bootstrap.cc:492] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d: No bootstrap required, opened a new log
I20260709 13:00:59.482220  7972 ts_tablet_manager.cc:1403] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d: Time spent bootstrapping tablet: real 0.045s	user 0.009s	sys 0.003s
I20260709 13:00:59.485006  7972 raft_consensus.cc:359] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } } peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } }
I20260709 13:00:59.485749  7972 raft_consensus.cc:385] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.487582  7921 tablet_service.cc:1511] Processing CreateTablet for tablet 494fef1b35024d30b17d4c0aee058999 (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:00:59.488832  7921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 494fef1b35024d30b17d4c0aee058999. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.486053  7972 raft_consensus.cc:740] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.494773  7972 consensus_queue.cc:260] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } } peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } }
I20260709 13:00:59.496284  7973 tablet_bootstrap.cc:492] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b: No bootstrap required, opened a new log
I20260709 13:00:59.496738  7973 ts_tablet_manager.cc:1403] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.008s
I20260709 13:00:59.497227  7772 tablet_service.cc:1511] Processing CreateTablet for tablet 5d8273641a344bfd96c1bee2c72380ef (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:00:59.498554  7772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d8273641a344bfd96c1bee2c72380ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.499765  7973 raft_consensus.cc:359] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } } peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } }
I20260709 13:00:59.500497  7973 raft_consensus.cc:385] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.500789  7973 raft_consensus.cc:740] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aec419c47a640edb247a753aaa6ad5b, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.504331  7922 tablet_service.cc:1511] Processing CreateTablet for tablet 441ce35ef600431fa3fa881500cb9563 (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:00:59.504753  7916 tablet_service.cc:1511] Processing CreateTablet for tablet 6e2882ee34dd4e6dbd67b64cea774679 (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:00:59.505007  7917 tablet_service.cc:1511] Processing CreateTablet for tablet 5d8273641a344bfd96c1bee2c72380ef (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:00:59.505684  7922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 441ce35ef600431fa3fa881500cb9563. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.506315  7916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e2882ee34dd4e6dbd67b64cea774679. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.507066  7769 tablet_service.cc:1511] Processing CreateTablet for tablet 6e2882ee34dd4e6dbd67b64cea774679 (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:00:59.508301  7769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e2882ee34dd4e6dbd67b64cea774679. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.510502  7848 tablet_service.cc:1511] Processing CreateTablet for tablet 428de0e6e6d1418ebcbc8ff1fbf24a26 (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:00:59.511734  7848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 428de0e6e6d1418ebcbc8ff1fbf24a26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.508863  7917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d8273641a344bfd96c1bee2c72380ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.508615  7973 consensus_queue.cc:260] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } } peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } }
I20260709 13:00:59.510062  7770 tablet_service.cc:1511] Processing CreateTablet for tablet 441ce35ef600431fa3fa881500cb9563 (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:00:59.515434  7842 tablet_service.cc:1511] Processing CreateTablet for tablet 494fef1b35024d30b17d4c0aee058999 (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:00:59.517496  7843 tablet_service.cc:1511] Processing CreateTablet for tablet c4ced2215e2544faa2ecda7ceba5de27 (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:00:59.517863  7842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 494fef1b35024d30b17d4c0aee058999. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.517797  7846 tablet_service.cc:1511] Processing CreateTablet for tablet bcb544d4966e409c97ce8eff34e7521e (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:00:59.518793  7843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4ced2215e2544faa2ecda7ceba5de27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.521943  7845 tablet_service.cc:1511] Processing CreateTablet for tablet 55538fcb866c4dd1a3cd12db54eb946b (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:00:59.522773  7841 tablet_service.cc:1511] Processing CreateTablet for tablet 5d8273641a344bfd96c1bee2c72380ef (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:00:59.523584  7839 tablet_service.cc:1511] Processing CreateTablet for tablet 6e2882ee34dd4e6dbd67b64cea774679 (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:00:59.524128  7847 tablet_service.cc:1511] Processing CreateTablet for tablet 6c817cd2a564431e89c98f5500a9bd37 (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:00:59.525007  7840 tablet_service.cc:1511] Processing CreateTablet for tablet 441ce35ef600431fa3fa881500cb9563 (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:00:59.525540  7844 tablet_service.cc:1511] Processing CreateTablet for tablet 20e3b50df0d7462da76007b962677370 (DEFAULT_TABLE table=test-workload [id=144dce7831a1456ebef58e2a57b65c09]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:00:59.523254  7845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55538fcb866c4dd1a3cd12db54eb946b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.526769  7839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e2882ee34dd4e6dbd67b64cea774679. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.529824  7847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c817cd2a564431e89c98f5500a9bd37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.530500  7841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d8273641a344bfd96c1bee2c72380ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.531975  7840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 441ce35ef600431fa3fa881500cb9563. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.533798  7846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcb544d4966e409c97ce8eff34e7521e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.537791  7844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20e3b50df0d7462da76007b962677370. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.543916  7770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 441ce35ef600431fa3fa881500cb9563. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:59.556077  7808 heartbeater.cc:499] Master 127.6.255.62:41293 was elected leader, sending a full tablet report...
I20260709 13:00:59.574406  7972 ts_tablet_manager.cc:1434] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d: Time spent starting tablet: real 0.092s	user 0.009s	sys 0.005s
I20260709 13:00:59.575474  7972 tablet_bootstrap.cc:492] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d: Bootstrap starting.
I20260709 13:00:59.612387  7972 tablet_bootstrap.cc:654] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.615324  7977 raft_consensus.cc:493] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.615908  7977 raft_consensus.cc:515] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } } peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } }
I20260709 13:00:59.621184  7977 leader_election.cc:290] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185)
I20260709 13:00:59.622103  7976 raft_consensus.cc:493] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.622304  7957 heartbeater.cc:499] Master 127.6.255.62:41293 was elected leader, sending a full tablet report...
I20260709 13:00:59.622611  7976 raft_consensus.cc:515] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } } peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } }
I20260709 13:00:59.625180  7976 leader_election.cc:290] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505), c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185)
I20260709 13:00:59.651283  7973 ts_tablet_manager.cc:1434] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b: Time spent starting tablet: real 0.154s	user 0.016s	sys 0.020s
I20260709 13:00:59.651995  7973 tablet_bootstrap.cc:492] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b: Bootstrap starting.
I20260709 13:00:59.663628  7972 tablet_bootstrap.cc:492] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d: No bootstrap required, opened a new log
I20260709 13:00:59.664160  7972 ts_tablet_manager.cc:1403] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d: Time spent bootstrapping tablet: real 0.089s	user 0.015s	sys 0.007s
I20260709 13:00:59.670359  7973 tablet_bootstrap.cc:654] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.670459  7972 raft_consensus.cc:359] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.671001  7979 tablet_bootstrap.cc:492] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P c4ca9afe41ff4eb298062aa06f346b95: Bootstrap starting.
I20260709 13:00:59.671288  7972 raft_consensus.cc:385] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.672767  7972 raft_consensus.cc:740] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.674012  7972 consensus_queue.cc:260] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.676443  7972 ts_tablet_manager.cc:1434] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.002s
I20260709 13:00:59.677481  7972 tablet_bootstrap.cc:492] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d: Bootstrap starting.
I20260709 13:00:59.684939  7783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428de0e6e6d1418ebcbc8ff1fbf24a26" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c88031f63c411a9a349b00f53b211d" is_pre_election: true
I20260709 13:00:59.686622  7783 raft_consensus.cc:2468] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aec419c47a640edb247a753aaa6ad5b in term 0.
I20260709 13:00:59.694020  7895 leader_election.cc:304] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d, 9aec419c47a640edb247a753aaa6ad5b; no voters: 
I20260709 13:00:59.694936  7972 tablet_bootstrap.cc:654] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.695768  7858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428de0e6e6d1418ebcbc8ff1fbf24a26" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95" is_pre_election: true
W20260709 13:00:59.697408  7895 leader_election.cc:343] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:59.697614  7977 raft_consensus.cc:2804] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:59.698743  7977 raft_consensus.cc:493] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.699071  7977 raft_consensus.cc:3060] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:59.700631  7979 tablet_bootstrap.cc:654] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P c4ca9afe41ff4eb298062aa06f346b95: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.702389  7932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428de0e6e6d1418ebcbc8ff1fbf24a26" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aec419c47a640edb247a753aaa6ad5b" is_pre_election: true
I20260709 13:00:59.709950  7973 tablet_bootstrap.cc:492] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b: No bootstrap required, opened a new log
I20260709 13:00:59.710438  7973 ts_tablet_manager.cc:1403] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b: Time spent bootstrapping tablet: real 0.059s	user 0.011s	sys 0.001s
I20260709 13:00:59.709306  7977 raft_consensus.cc:515] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } } peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } }
I20260709 13:00:59.711532  7932 raft_consensus.cc:2393] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 68c88031f63c411a9a349b00f53b211d in current term 1: Already voted for candidate 9aec419c47a640edb247a753aaa6ad5b in this term.
I20260709 13:00:59.713496  7973 raft_consensus.cc:359] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.714186  7973 raft_consensus.cc:385] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.714462  7973 raft_consensus.cc:740] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aec419c47a640edb247a753aaa6ad5b, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.715273  7973 consensus_queue.cc:260] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.716498  7977 leader_election.cc:290] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 election: Requested vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185)
I20260709 13:00:59.717522  7858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428de0e6e6d1418ebcbc8ff1fbf24a26" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
I20260709 13:00:59.720129  7973 ts_tablet_manager.cc:1434] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:00:59.718192  7783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428de0e6e6d1418ebcbc8ff1fbf24a26" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c88031f63c411a9a349b00f53b211d"
I20260709 13:00:59.719288  7857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428de0e6e6d1418ebcbc8ff1fbf24a26" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95" is_pre_election: true
I20260709 13:00:59.721120  7973 tablet_bootstrap.cc:492] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b: Bootstrap starting.
I20260709 13:00:59.721032  7783 raft_consensus.cc:3060] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:59.722306  7746 leader_election.cc:343] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:59.722908  7746 leader_election.cc:304] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d; no voters: 9aec419c47a640edb247a753aaa6ad5b, c4ca9afe41ff4eb298062aa06f346b95
W20260709 13:00:59.723868  7895 leader_election.cc:343] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:59.731695  7973 tablet_bootstrap.cc:654] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.732404  7783 raft_consensus.cc:2468] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aec419c47a640edb247a753aaa6ad5b in term 1.
I20260709 13:00:59.733335  7972 tablet_bootstrap.cc:492] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d: No bootstrap required, opened a new log
I20260709 13:00:59.733763  7976 raft_consensus.cc:2749] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:59.733981  7972 ts_tablet_manager.cc:1403] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d: Time spent bootstrapping tablet: real 0.057s	user 0.025s	sys 0.020s
I20260709 13:00:59.735378  7895 leader_election.cc:304] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d, 9aec419c47a640edb247a753aaa6ad5b; no voters: c4ca9afe41ff4eb298062aa06f346b95
I20260709 13:00:59.736640  7977 raft_consensus.cc:2804] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:59.737287  7977 raft_consensus.cc:697] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 1 LEADER]: Becoming Leader. State: Replica: 9aec419c47a640edb247a753aaa6ad5b, State: Running, Role: LEADER
I20260709 13:00:59.737344  7972 raft_consensus.cc:359] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.738866  7972 raft_consensus.cc:385] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.739198  7972 raft_consensus.cc:740] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.739276  7977 consensus_queue.cc:237] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } } peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } }
I20260709 13:00:59.740193  7972 consensus_queue.cc:260] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.742313  7972 ts_tablet_manager.cc:1434] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:00:59.743285  7972 tablet_bootstrap.cc:492] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d: Bootstrap starting.
I20260709 13:00:59.746613  7979 tablet_bootstrap.cc:492] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P c4ca9afe41ff4eb298062aa06f346b95: No bootstrap required, opened a new log
I20260709 13:00:59.747474  7979 ts_tablet_manager.cc:1403] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P c4ca9afe41ff4eb298062aa06f346b95: Time spent bootstrapping tablet: real 0.077s	user 0.015s	sys 0.009s
I20260709 13:00:59.751892  7973 tablet_bootstrap.cc:492] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b: No bootstrap required, opened a new log
I20260709 13:00:59.751528  7979 raft_consensus.cc:359] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } } peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } }
I20260709 13:00:59.752293  7979 raft_consensus.cc:385] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.752400  7973 ts_tablet_manager.cc:1403] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b: Time spent bootstrapping tablet: real 0.031s	user 0.007s	sys 0.006s
I20260709 13:00:59.754103  7972 tablet_bootstrap.cc:654] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.755465  7973 raft_consensus.cc:359] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.756203  7973 raft_consensus.cc:385] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.756524  7973 raft_consensus.cc:740] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aec419c47a640edb247a753aaa6ad5b, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.755719  7679 catalog_manager.cc:5697] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3). New cstate: current_term: 1 leader_uuid: "9aec419c47a640edb247a753aaa6ad5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:59.752616  7979 raft_consensus.cc:740] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4ca9afe41ff4eb298062aa06f346b95, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.757483  7979 consensus_queue.cc:260] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P c4ca9afe41ff4eb298062aa06f346b95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } } peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } }
I20260709 13:00:59.759893  7972 tablet_bootstrap.cc:492] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d: No bootstrap required, opened a new log
I20260709 13:00:59.760337  7972 ts_tablet_manager.cc:1403] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d: Time spent bootstrapping tablet: real 0.017s	user 0.005s	sys 0.005s
I20260709 13:00:59.762113  7979 ts_tablet_manager.cc:1434] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P c4ca9afe41ff4eb298062aa06f346b95: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:00:59.763371  7979 tablet_bootstrap.cc:492] T 6c817cd2a564431e89c98f5500a9bd37 P c4ca9afe41ff4eb298062aa06f346b95: Bootstrap starting.
I20260709 13:00:59.763177  7972 raft_consensus.cc:359] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.763897  7972 raft_consensus.cc:385] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.764204  7972 raft_consensus.cc:740] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.764557  7883 heartbeater.cc:499] Master 127.6.255.62:41293 was elected leader, sending a full tablet report...
I20260709 13:00:59.764920  7972 consensus_queue.cc:260] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.766091  7991 raft_consensus.cc:493] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.767409  7972 ts_tablet_manager.cc:1434] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:00:59.767184  7991 raft_consensus.cc:515] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } } peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } }
I20260709 13:00:59.768383  7972 tablet_bootstrap.cc:492] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d: Bootstrap starting.
I20260709 13:00:59.770112  7973 consensus_queue.cc:260] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.775590  7979 tablet_bootstrap.cc:654] T 6c817cd2a564431e89c98f5500a9bd37 P c4ca9afe41ff4eb298062aa06f346b95: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.775662  7972 tablet_bootstrap.cc:654] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.785537  7977 raft_consensus.cc:493] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.786532  7973 ts_tablet_manager.cc:1434] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.001s
I20260709 13:00:59.786284  7977 raft_consensus.cc:515] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.788022  7991 leader_election.cc:290] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P c4ca9afe41ff4eb298062aa06f346b95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505), 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239)
I20260709 13:00:59.807087  7783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4ced2215e2544faa2ecda7ceba5de27" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c88031f63c411a9a349b00f53b211d" is_pre_election: true
W20260709 13:00:59.808720  7895 leader_election.cc:343] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:59.807732  7977 leader_election.cc:290] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185)
I20260709 13:00:59.810330  7973 tablet_bootstrap.cc:492] T 20e3b50df0d7462da76007b962677370 P 9aec419c47a640edb247a753aaa6ad5b: Bootstrap starting.
I20260709 13:00:59.811911  7979 tablet_bootstrap.cc:492] T 6c817cd2a564431e89c98f5500a9bd37 P c4ca9afe41ff4eb298062aa06f346b95: No bootstrap required, opened a new log
I20260709 13:00:59.812566  7979 ts_tablet_manager.cc:1403] T 6c817cd2a564431e89c98f5500a9bd37 P c4ca9afe41ff4eb298062aa06f346b95: Time spent bootstrapping tablet: real 0.049s	user 0.015s	sys 0.006s
I20260709 13:00:59.816776  7979 raft_consensus.cc:359] T 6c817cd2a564431e89c98f5500a9bd37 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.819716  7979 raft_consensus.cc:385] T 6c817cd2a564431e89c98f5500a9bd37 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.820091  7979 raft_consensus.cc:740] T 6c817cd2a564431e89c98f5500a9bd37 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4ca9afe41ff4eb298062aa06f346b95, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.826467  7857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4ced2215e2544faa2ecda7ceba5de27" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95" is_pre_election: true
I20260709 13:00:59.829128  7979 consensus_queue.cc:260] T 6c817cd2a564431e89c98f5500a9bd37 P c4ca9afe41ff4eb298062aa06f346b95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
W20260709 13:00:59.829614  7895 leader_election.cc:343] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:59.830163  7895 leader_election.cc:304] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aec419c47a640edb247a753aaa6ad5b; no voters: 68c88031f63c411a9a349b00f53b211d, c4ca9afe41ff4eb298062aa06f346b95
I20260709 13:00:59.831041  7973 tablet_bootstrap.cc:654] T 20e3b50df0d7462da76007b962677370 P 9aec419c47a640edb247a753aaa6ad5b: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.831260  7977 raft_consensus.cc:2749] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:59.831429  7979 ts_tablet_manager.cc:1434] T 6c817cd2a564431e89c98f5500a9bd37 P c4ca9afe41ff4eb298062aa06f346b95: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.003s
I20260709 13:00:59.831542  7972 tablet_bootstrap.cc:492] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d: No bootstrap required, opened a new log
I20260709 13:00:59.832803  7972 ts_tablet_manager.cc:1403] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d: Time spent bootstrapping tablet: real 0.065s	user 0.017s	sys 0.023s
I20260709 13:00:59.833145  7979 tablet_bootstrap.cc:492] T 6e2882ee34dd4e6dbd67b64cea774679 P c4ca9afe41ff4eb298062aa06f346b95: Bootstrap starting.
I20260709 13:00:59.835651  7972 raft_consensus.cc:359] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.836386  7972 raft_consensus.cc:385] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.836692  7972 raft_consensus.cc:740] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.839376  7932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428de0e6e6d1418ebcbc8ff1fbf24a26" candidate_uuid: "c4ca9afe41ff4eb298062aa06f346b95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aec419c47a640edb247a753aaa6ad5b" is_pre_election: true
I20260709 13:00:59.837594  7972 consensus_queue.cc:260] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.842286  7972 ts_tablet_manager.cc:1434] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:00:59.843290  7972 tablet_bootstrap.cc:492] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d: Bootstrap starting.
I20260709 13:00:59.840785  7979 tablet_bootstrap.cc:654] T 6e2882ee34dd4e6dbd67b64cea774679 P c4ca9afe41ff4eb298062aa06f346b95: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.849946  7972 tablet_bootstrap.cc:654] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.851168  7973 tablet_bootstrap.cc:492] T 20e3b50df0d7462da76007b962677370 P 9aec419c47a640edb247a753aaa6ad5b: No bootstrap required, opened a new log
I20260709 13:00:59.851666  7973 ts_tablet_manager.cc:1403] T 20e3b50df0d7462da76007b962677370 P 9aec419c47a640edb247a753aaa6ad5b: Time spent bootstrapping tablet: real 0.045s	user 0.011s	sys 0.013s
I20260709 13:00:59.853904  7976 raft_consensus.cc:493] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.854390  7976 raft_consensus.cc:515] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.854817  7972 tablet_bootstrap.cc:492] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d: No bootstrap required, opened a new log
I20260709 13:00:59.854615  7973 raft_consensus.cc:359] T 20e3b50df0d7462da76007b962677370 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.855273  7972 ts_tablet_manager.cc:1403] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d: Time spent bootstrapping tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:00:59.855324  7973 raft_consensus.cc:385] T 20e3b50df0d7462da76007b962677370 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.855726  7973 raft_consensus.cc:740] T 20e3b50df0d7462da76007b962677370 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aec419c47a640edb247a753aaa6ad5b, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.856504  7976 leader_election.cc:290] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
I20260709 13:00:59.856982  7783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428de0e6e6d1418ebcbc8ff1fbf24a26" candidate_uuid: "c4ca9afe41ff4eb298062aa06f346b95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c88031f63c411a9a349b00f53b211d" is_pre_election: true
I20260709 13:00:59.857937  7783 raft_consensus.cc:2393] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c4ca9afe41ff4eb298062aa06f346b95 in current term 1: Already voted for candidate 9aec419c47a640edb247a753aaa6ad5b in this term.
I20260709 13:00:59.858381  7857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c817cd2a564431e89c98f5500a9bd37" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95" is_pre_election: true
I20260709 13:00:59.858462  7972 raft_consensus.cc:359] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.859064  7857 raft_consensus.cc:2468] T 6c817cd2a564431e89c98f5500a9bd37 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68c88031f63c411a9a349b00f53b211d in term 0.
I20260709 13:00:59.859194  7821 leader_election.cc:304] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P c4ca9afe41ff4eb298062aa06f346b95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4ca9afe41ff4eb298062aa06f346b95; no voters: 68c88031f63c411a9a349b00f53b211d, 9aec419c47a640edb247a753aaa6ad5b
I20260709 13:00:59.859272  7972 raft_consensus.cc:385] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.859608  7972 raft_consensus.cc:740] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.859983  7992 raft_consensus.cc:3060] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:59.860481  7972 consensus_queue.cc:260] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.860774  7746 leader_election.cc:304] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d, c4ca9afe41ff4eb298062aa06f346b95; no voters: 
I20260709 13:00:59.856981  7973 consensus_queue.cc:260] T 20e3b50df0d7462da76007b962677370 P 9aec419c47a640edb247a753aaa6ad5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.862159  7932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c817cd2a564431e89c98f5500a9bd37" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aec419c47a640edb247a753aaa6ad5b" is_pre_election: true
I20260709 13:00:59.862768  7972 ts_tablet_manager.cc:1434] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:00:59.863659  7972 tablet_bootstrap.cc:492] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d: Bootstrap starting.
I20260709 13:00:59.863792  7973 ts_tablet_manager.cc:1434] T 20e3b50df0d7462da76007b962677370 P 9aec419c47a640edb247a753aaa6ad5b: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:00:59.862780  7932 raft_consensus.cc:2468] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68c88031f63c411a9a349b00f53b211d in term 0.
I20260709 13:00:59.864873  7973 tablet_bootstrap.cc:492] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b: Bootstrap starting.
I20260709 13:00:59.858598  7979 tablet_bootstrap.cc:492] T 6e2882ee34dd4e6dbd67b64cea774679 P c4ca9afe41ff4eb298062aa06f346b95: No bootstrap required, opened a new log
I20260709 13:00:59.866916  7979 ts_tablet_manager.cc:1403] T 6e2882ee34dd4e6dbd67b64cea774679 P c4ca9afe41ff4eb298062aa06f346b95: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.001s
I20260709 13:00:59.869936  7979 raft_consensus.cc:359] T 6e2882ee34dd4e6dbd67b64cea774679 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.870640  7979 raft_consensus.cc:385] T 6e2882ee34dd4e6dbd67b64cea774679 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.871012  7972 tablet_bootstrap.cc:654] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.870944  7979 raft_consensus.cc:740] T 6e2882ee34dd4e6dbd67b64cea774679 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4ca9afe41ff4eb298062aa06f346b95, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.874151  7976 raft_consensus.cc:2804] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:59.874598  7976 raft_consensus.cc:493] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.874982  7976 raft_consensus.cc:3060] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:59.875645  7979 consensus_queue.cc:260] T 6e2882ee34dd4e6dbd67b64cea774679 P c4ca9afe41ff4eb298062aa06f346b95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.883167  7972 tablet_bootstrap.cc:492] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d: No bootstrap required, opened a new log
I20260709 13:00:59.884805  7979 ts_tablet_manager.cc:1434] T 6e2882ee34dd4e6dbd67b64cea774679 P c4ca9afe41ff4eb298062aa06f346b95: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:00:59.885749  7972 ts_tablet_manager.cc:1403] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.001s
I20260709 13:00:59.879618  7992 raft_consensus.cc:2749] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:59.887760  7979 tablet_bootstrap.cc:492] T 55538fcb866c4dd1a3cd12db54eb946b P c4ca9afe41ff4eb298062aa06f346b95: Bootstrap starting.
I20260709 13:00:59.889020  7972 raft_consensus.cc:359] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.889818  7972 raft_consensus.cc:385] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.890287  7972 raft_consensus.cc:740] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.891360  7972 consensus_queue.cc:260] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.893797  7973 tablet_bootstrap.cc:654] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.898046  7972 ts_tablet_manager.cc:1434] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.003s
I20260709 13:00:59.899329  7972 tablet_bootstrap.cc:492] T 6e2882ee34dd4e6dbd67b64cea774679 P 68c88031f63c411a9a349b00f53b211d: Bootstrap starting.
I20260709 13:00:59.895030  7976 raft_consensus.cc:515] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.902287  7979 tablet_bootstrap.cc:654] T 55538fcb866c4dd1a3cd12db54eb946b P c4ca9afe41ff4eb298062aa06f346b95: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.904090  8002 raft_consensus.cc:493] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.904678  8002 raft_consensus.cc:515] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.910818  8002 leader_election.cc:290] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
W20260709 13:00:59.914047  7746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:33185, user_credentials={real_user=slave}} blocked reactor thread for 53354us
I20260709 13:00:59.914731  7976 leader_election.cc:290] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 election: Requested vote from peers c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
I20260709 13:00:59.916210  7857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c817cd2a564431e89c98f5500a9bd37" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
I20260709 13:00:59.916806  7857 raft_consensus.cc:3060] T 6c817cd2a564431e89c98f5500a9bd37 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:59.918128  7972 tablet_bootstrap.cc:654] T 6e2882ee34dd4e6dbd67b64cea774679 P 68c88031f63c411a9a349b00f53b211d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.920711  7973 tablet_bootstrap.cc:492] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b: No bootstrap required, opened a new log
I20260709 13:00:59.921258  7973 ts_tablet_manager.cc:1403] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b: Time spent bootstrapping tablet: real 0.057s	user 0.012s	sys 0.010s
I20260709 13:00:59.922066  7992 raft_consensus.cc:493] T 6e2882ee34dd4e6dbd67b64cea774679 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.922533  7992 raft_consensus.cc:515] T 6e2882ee34dd4e6dbd67b64cea774679 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
W20260709 13:00:59.924561  7729 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:59.924708  7992 leader_election.cc:290] T 6e2882ee34dd4e6dbd67b64cea774679 P c4ca9afe41ff4eb298062aa06f346b95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
I20260709 13:00:59.925050  7857 raft_consensus.cc:2468] T 6c817cd2a564431e89c98f5500a9bd37 P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68c88031f63c411a9a349b00f53b211d in term 1.
I20260709 13:00:59.925781  7973 raft_consensus.cc:359] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.926748  7932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e2882ee34dd4e6dbd67b64cea774679" candidate_uuid: "c4ca9afe41ff4eb298062aa06f346b95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aec419c47a640edb247a753aaa6ad5b" is_pre_election: true
I20260709 13:00:59.926774  7783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e2882ee34dd4e6dbd67b64cea774679" candidate_uuid: "c4ca9afe41ff4eb298062aa06f346b95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c88031f63c411a9a349b00f53b211d" is_pre_election: true
I20260709 13:00:59.927207  7857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20e3b50df0d7462da76007b962677370" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95" is_pre_election: true
I20260709 13:00:59.926833  7973 raft_consensus.cc:385] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.928890  7931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20e3b50df0d7462da76007b962677370" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aec419c47a640edb247a753aaa6ad5b" is_pre_election: true
W20260709 13:00:59.935323  7821 leader_election.cc:343] T 6e2882ee34dd4e6dbd67b64cea774679 P c4ca9afe41ff4eb298062aa06f346b95 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:59.937162  7973 raft_consensus.cc:740] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aec419c47a640edb247a753aaa6ad5b, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.937388  7931 raft_consensus.cc:2468] T 20e3b50df0d7462da76007b962677370 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68c88031f63c411a9a349b00f53b211d in term 0.
W20260709 13:00:59.937716  7821 leader_election.cc:343] T 6e2882ee34dd4e6dbd67b64cea774679 P c4ca9afe41ff4eb298062aa06f346b95 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:59.938210  7821 leader_election.cc:304] T 6e2882ee34dd4e6dbd67b64cea774679 P c4ca9afe41ff4eb298062aa06f346b95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4ca9afe41ff4eb298062aa06f346b95; no voters: 68c88031f63c411a9a349b00f53b211d, 9aec419c47a640edb247a753aaa6ad5b
I20260709 13:00:59.938217  7973 consensus_queue.cc:260] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.939254  7992 raft_consensus.cc:2749] T 6e2882ee34dd4e6dbd67b64cea774679 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:59.935348  7972 tablet_bootstrap.cc:492] T 6e2882ee34dd4e6dbd67b64cea774679 P 68c88031f63c411a9a349b00f53b211d: No bootstrap required, opened a new log
I20260709 13:00:59.939450  7746 leader_election.cc:304] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d, 9aec419c47a640edb247a753aaa6ad5b; no voters: 
I20260709 13:00:59.940176  7972 ts_tablet_manager.cc:1403] T 6e2882ee34dd4e6dbd67b64cea774679 P 68c88031f63c411a9a349b00f53b211d: Time spent bootstrapping tablet: real 0.041s	user 0.013s	sys 0.000s
I20260709 13:00:59.936517  7932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c817cd2a564431e89c98f5500a9bd37" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
I20260709 13:00:59.940821  7932 raft_consensus.cc:3060] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:59.943163  7972 raft_consensus.cc:359] T 6e2882ee34dd4e6dbd67b64cea774679 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.944013  7972 raft_consensus.cc:385] T 6e2882ee34dd4e6dbd67b64cea774679 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.944382  7972 raft_consensus.cc:740] T 6e2882ee34dd4e6dbd67b64cea774679 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.945139  7972 consensus_queue.cc:260] T 6e2882ee34dd4e6dbd67b64cea774679 P 68c88031f63c411a9a349b00f53b211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.947396  7972 ts_tablet_manager.cc:1434] T 6e2882ee34dd4e6dbd67b64cea774679 P 68c88031f63c411a9a349b00f53b211d: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:00:59.948354  7972 tablet_bootstrap.cc:492] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d: Bootstrap starting.
I20260709 13:00:59.948735  7973 ts_tablet_manager.cc:1434] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.002s
I20260709 13:00:59.948989  7979 tablet_bootstrap.cc:492] T 55538fcb866c4dd1a3cd12db54eb946b P c4ca9afe41ff4eb298062aa06f346b95: No bootstrap required, opened a new log
I20260709 13:00:59.949482  7979 ts_tablet_manager.cc:1403] T 55538fcb866c4dd1a3cd12db54eb946b P c4ca9afe41ff4eb298062aa06f346b95: Time spent bootstrapping tablet: real 0.062s	user 0.009s	sys 0.004s
I20260709 13:00:59.949784  7976 raft_consensus.cc:2804] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:59.950176  7976 raft_consensus.cc:493] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.950505  7976 raft_consensus.cc:3060] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:59.952337  7979 raft_consensus.cc:359] T 55538fcb866c4dd1a3cd12db54eb946b P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.953050  7979 raft_consensus.cc:385] T 55538fcb866c4dd1a3cd12db54eb946b P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.953338  7979 raft_consensus.cc:740] T 55538fcb866c4dd1a3cd12db54eb946b P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4ca9afe41ff4eb298062aa06f346b95, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.953577  8000 raft_consensus.cc:493] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.954414  7972 tablet_bootstrap.cc:654] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.954089  8000 raft_consensus.cc:515] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.954212  7979 consensus_queue.cc:260] T 55538fcb866c4dd1a3cd12db54eb946b P c4ca9afe41ff4eb298062aa06f346b95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.956485  7979 ts_tablet_manager.cc:1434] T 55538fcb866c4dd1a3cd12db54eb946b P c4ca9afe41ff4eb298062aa06f346b95: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:00:59.956471  8000 leader_election.cc:290] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
I20260709 13:00:59.963173  7979 tablet_bootstrap.cc:492] T 20e3b50df0d7462da76007b962677370 P c4ca9afe41ff4eb298062aa06f346b95: Bootstrap starting.
I20260709 13:00:59.965091  7976 raft_consensus.cc:515] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.967337  7976 leader_election.cc:290] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 election: Requested vote from peers c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
I20260709 13:00:59.968851  7857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55538fcb866c4dd1a3cd12db54eb946b" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95" is_pre_election: true
I20260709 13:00:59.970002  7979 tablet_bootstrap.cc:654] T 20e3b50df0d7462da76007b962677370 P c4ca9afe41ff4eb298062aa06f346b95: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.970288  7931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55538fcb866c4dd1a3cd12db54eb946b" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aec419c47a640edb247a753aaa6ad5b" is_pre_election: true
I20260709 13:00:59.971712  7858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20e3b50df0d7462da76007b962677370" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
I20260709 13:00:59.972602  7930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20e3b50df0d7462da76007b962677370" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
I20260709 13:00:59.975759  7930 raft_consensus.cc:3060] T 20e3b50df0d7462da76007b962677370 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:59.977628  7973 tablet_bootstrap.cc:492] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b: Bootstrap starting.
I20260709 13:00:59.978019  7932 raft_consensus.cc:2468] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68c88031f63c411a9a349b00f53b211d in term 1.
I20260709 13:00:59.983157  7930 raft_consensus.cc:2468] T 20e3b50df0d7462da76007b962677370 P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68c88031f63c411a9a349b00f53b211d in term 1.
I20260709 13:00:59.972683  7746 leader_election.cc:304] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d, c4ca9afe41ff4eb298062aa06f346b95; no voters: 
I20260709 13:00:59.973469  7976 raft_consensus.cc:493] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.973495  7857 raft_consensus.cc:2468] T 55538fcb866c4dd1a3cd12db54eb946b P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68c88031f63c411a9a349b00f53b211d in term 0.
I20260709 13:00:59.984300  7973 tablet_bootstrap.cc:654] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.973877  7972 tablet_bootstrap.cc:492] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d: No bootstrap required, opened a new log
I20260709 13:00:59.985023  7972 ts_tablet_manager.cc:1403] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d: Time spent bootstrapping tablet: real 0.037s	user 0.017s	sys 0.007s
I20260709 13:00:59.984371  7976 raft_consensus.cc:515] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.987936  7976 leader_election.cc:290] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
W20260709 13:00:59.988977  7746 leader_election.cc:343] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:59.988843  7972 raft_consensus.cc:359] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.989683  7972 raft_consensus.cc:385] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.989894  7972 raft_consensus.cc:740] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Initialized, Role: FOLLOWER
W20260709 13:00:59.990021  7746 leader_election.cc:343] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:59.990648  7972 consensus_queue.cc:260] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.990762  7746 leader_election.cc:304] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d, c4ca9afe41ff4eb298062aa06f346b95; no voters: 
I20260709 13:00:59.991410  8000 raft_consensus.cc:2804] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:59.991787  8002 raft_consensus.cc:2804] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:59.991851  8000 raft_consensus.cc:697] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [term 1 LEADER]: Becoming Leader. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Running, Role: LEADER
I20260709 13:00:59.992228  8002 raft_consensus.cc:493] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.992559  7857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb544d4966e409c97ce8eff34e7521e" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95" is_pre_election: true
I20260709 13:00:59.992770  8000 consensus_queue.cc:237] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
W20260709 13:00:59.994479  7746 leader_election.cc:343] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:59.995294  7930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb544d4966e409c97ce8eff34e7521e" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aec419c47a640edb247a753aaa6ad5b" is_pre_election: true
I20260709 13:00:59.997157  7746 leader_election.cc:304] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d, 9aec419c47a640edb247a753aaa6ad5b; no voters: c4ca9afe41ff4eb298062aa06f346b95
I20260709 13:00:59.998075  7976 raft_consensus.cc:2804] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:59.998931  7973 tablet_bootstrap.cc:492] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b: No bootstrap required, opened a new log
I20260709 13:00:59.998977  7979 tablet_bootstrap.cc:492] T 20e3b50df0d7462da76007b962677370 P c4ca9afe41ff4eb298062aa06f346b95: No bootstrap required, opened a new log
I20260709 13:00:59.999634  7979 ts_tablet_manager.cc:1403] T 20e3b50df0d7462da76007b962677370 P c4ca9afe41ff4eb298062aa06f346b95: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.004s
I20260709 13:00:59.999634  7973 ts_tablet_manager.cc:1403] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b: Time spent bootstrapping tablet: real 0.022s	user 0.005s	sys 0.008s
W20260709 13:01:00.000979  7746 leader_election.cc:343] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:00.001507  8000 raft_consensus.cc:493] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:00.001689  7746 leader_election.cc:343] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:00.002120  7746 leader_election.cc:304] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d; no voters: 9aec419c47a640edb247a753aaa6ad5b, c4ca9afe41ff4eb298062aa06f346b95
I20260709 13:01:00.001998  8000 raft_consensus.cc:515] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.992607  8002 raft_consensus.cc:3060] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.002805  7979 raft_consensus.cc:359] T 20e3b50df0d7462da76007b962677370 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.003564  7979 raft_consensus.cc:385] T 20e3b50df0d7462da76007b962677370 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:00.003123  7973 raft_consensus.cc:359] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.004376  7973 raft_consensus.cc:385] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:00.004510  7976 raft_consensus.cc:697] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d [term 1 LEADER]: Becoming Leader. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Running, Role: LEADER
I20260709 13:01:00.003897  7979 raft_consensus.cc:740] T 20e3b50df0d7462da76007b962677370 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4ca9afe41ff4eb298062aa06f346b95, State: Initialized, Role: FOLLOWER
I20260709 13:01:00.004889  8010 raft_consensus.cc:2749] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:00.005717  7979 consensus_queue.cc:260] T 20e3b50df0d7462da76007b962677370 P c4ca9afe41ff4eb298062aa06f346b95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:00:59.992563  7972 ts_tablet_manager.cc:1434] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:01:00.005537  7976 consensus_queue.cc:237] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.007113  7972 tablet_bootstrap.cc:492] T 441ce35ef600431fa3fa881500cb9563 P 68c88031f63c411a9a349b00f53b211d: Bootstrap starting.
I20260709 13:01:00.004801  7973 raft_consensus.cc:740] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aec419c47a640edb247a753aaa6ad5b, State: Initialized, Role: FOLLOWER
I20260709 13:01:00.010442  7979 ts_tablet_manager.cc:1434] T 20e3b50df0d7462da76007b962677370 P c4ca9afe41ff4eb298062aa06f346b95: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:01:00.010469  7973 consensus_queue.cc:260] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.011631  7979 tablet_bootstrap.cc:492] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95: Bootstrap starting.
I20260709 13:01:00.013368  7972 tablet_bootstrap.cc:654] T 441ce35ef600431fa3fa881500cb9563 P 68c88031f63c411a9a349b00f53b211d: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:00.015829  7973 ts_tablet_manager.cc:1434] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:01:00.016983  7973 tablet_bootstrap.cc:492] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b: Bootstrap starting.
I20260709 13:01:00.018788  7979 tablet_bootstrap.cc:654] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:00.021987  7972 tablet_bootstrap.cc:492] T 441ce35ef600431fa3fa881500cb9563 P 68c88031f63c411a9a349b00f53b211d: No bootstrap required, opened a new log
I20260709 13:01:00.022537  7972 ts_tablet_manager.cc:1403] T 441ce35ef600431fa3fa881500cb9563 P 68c88031f63c411a9a349b00f53b211d: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.000s
I20260709 13:01:00.024785  7973 tablet_bootstrap.cc:654] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:00.025120  7972 raft_consensus.cc:359] T 441ce35ef600431fa3fa881500cb9563 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.025880  7972 raft_consensus.cc:385] T 441ce35ef600431fa3fa881500cb9563 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:00.026207  7972 raft_consensus.cc:740] T 441ce35ef600431fa3fa881500cb9563 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Initialized, Role: FOLLOWER
I20260709 13:01:00.027069  7972 consensus_queue.cc:260] T 441ce35ef600431fa3fa881500cb9563 P 68c88031f63c411a9a349b00f53b211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.029173  7972 ts_tablet_manager.cc:1434] T 441ce35ef600431fa3fa881500cb9563 P 68c88031f63c411a9a349b00f53b211d: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:01:00.035539  7679 catalog_manager.cc:5697] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d reported cstate change: term changed from 0 to 1, leader changed from <none> to 68c88031f63c411a9a349b00f53b211d (127.6.255.1). New cstate: current_term: 1 leader_uuid: "68c88031f63c411a9a349b00f53b211d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:00.037458  7679 catalog_manager.cc:5697] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d reported cstate change: term changed from 0 to 1, leader changed from <none> to 68c88031f63c411a9a349b00f53b211d (127.6.255.1). New cstate: current_term: 1 leader_uuid: "68c88031f63c411a9a349b00f53b211d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:00.038888  7973 tablet_bootstrap.cc:492] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b: No bootstrap required, opened a new log
I20260709 13:01:00.038955  8002 raft_consensus.cc:515] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.039868  7973 ts_tablet_manager.cc:1403] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b: Time spent bootstrapping tablet: real 0.023s	user 0.021s	sys 0.000s
I20260709 13:01:00.042824  7930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55538fcb866c4dd1a3cd12db54eb946b" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
I20260709 13:01:00.044194  7858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d8273641a344bfd96c1bee2c72380ef" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95" is_pre_election: true
I20260709 13:01:00.044576  7979 tablet_bootstrap.cc:492] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95: No bootstrap required, opened a new log
W20260709 13:01:00.044627  7746 leader_election.cc:343] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:00.045197  7979 ts_tablet_manager.cc:1403] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.015s
I20260709 13:01:00.042824  7857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55538fcb866c4dd1a3cd12db54eb946b" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
I20260709 13:01:00.045671  8000 leader_election.cc:290] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
I20260709 13:01:00.041301  8002 leader_election.cc:290] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 election: Requested vote from peers c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
I20260709 13:01:00.045998  7857 raft_consensus.cc:3060] T 55538fcb866c4dd1a3cd12db54eb946b P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.048697  7979 raft_consensus.cc:359] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.049500  7979 raft_consensus.cc:385] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:00.049866  7979 raft_consensus.cc:740] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4ca9afe41ff4eb298062aa06f346b95, State: Initialized, Role: FOLLOWER
I20260709 13:01:00.049680  7973 raft_consensus.cc:359] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
W20260709 13:01:00.050237  7746 leader_election.cc:343] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:00.050411  7973 raft_consensus.cc:385] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:00.050758  7973 raft_consensus.cc:740] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aec419c47a640edb247a753aaa6ad5b, State: Initialized, Role: FOLLOWER
I20260709 13:01:00.050825  7979 consensus_queue.cc:260] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.051234  7930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d8273641a344bfd96c1bee2c72380ef" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aec419c47a640edb247a753aaa6ad5b" is_pre_election: true
I20260709 13:01:00.051582  7973 consensus_queue.cc:260] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.053404  7979 ts_tablet_manager.cc:1434] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.004s
I20260709 13:01:00.053867  7973 ts_tablet_manager.cc:1434] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:01:00.054524  7979 tablet_bootstrap.cc:492] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95: Bootstrap starting.
W20260709 13:01:00.055303  7746 leader_election.cc:343] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:00.055794  7746 leader_election.cc:304] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d; no voters: 9aec419c47a640edb247a753aaa6ad5b, c4ca9afe41ff4eb298062aa06f346b95
I20260709 13:01:00.059235  8002 raft_consensus.cc:2749] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:00.060669  7973 tablet_bootstrap.cc:492] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b: Bootstrap starting.
I20260709 13:01:00.061956  7979 tablet_bootstrap.cc:654] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:00.061887  7857 raft_consensus.cc:2468] T 55538fcb866c4dd1a3cd12db54eb946b P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68c88031f63c411a9a349b00f53b211d in term 1.
I20260709 13:01:00.063539  7746 leader_election.cc:304] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d, c4ca9afe41ff4eb298062aa06f346b95; no voters: 9aec419c47a640edb247a753aaa6ad5b
I20260709 13:01:00.064838  8002 raft_consensus.cc:2804] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:00.066630  8002 raft_consensus.cc:697] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [term 1 LEADER]: Becoming Leader. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Running, Role: LEADER
I20260709 13:01:00.068078  8002 consensus_queue.cc:237] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.070341  7973 tablet_bootstrap.cc:654] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:00.077610  7979 tablet_bootstrap.cc:492] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95: No bootstrap required, opened a new log
I20260709 13:01:00.078140  7979 ts_tablet_manager.cc:1403] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.008s
I20260709 13:01:00.081346  7979 raft_consensus.cc:359] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.082190  7979 raft_consensus.cc:385] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:00.082532  7979 raft_consensus.cc:740] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4ca9afe41ff4eb298062aa06f346b95, State: Initialized, Role: FOLLOWER
I20260709 13:01:00.083469  7979 consensus_queue.cc:260] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.084373  7977 raft_consensus.cc:493] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.083770  7680 catalog_manager.cc:5697] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d reported cstate change: term changed from 0 to 1, leader changed from <none> to 68c88031f63c411a9a349b00f53b211d (127.6.255.1). New cstate: current_term: 1 leader_uuid: "68c88031f63c411a9a349b00f53b211d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:00.085007  7977 raft_consensus.cc:515] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.087596  7979 ts_tablet_manager.cc:1434] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:01:00.088656  7979 tablet_bootstrap.cc:492] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95: Bootstrap starting.
I20260709 13:01:00.094517  7857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "494fef1b35024d30b17d4c0aee058999" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95" is_pre_election: true
I20260709 13:01:00.093290  7783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "494fef1b35024d30b17d4c0aee058999" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c88031f63c411a9a349b00f53b211d" is_pre_election: true
I20260709 13:01:00.096050  7979 tablet_bootstrap.cc:654] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:00.096467  7783 raft_consensus.cc:2468] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aec419c47a640edb247a753aaa6ad5b in term 0.
I20260709 13:01:00.098239  7895 leader_election.cc:304] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d, 9aec419c47a640edb247a753aaa6ad5b; no voters: 
W20260709 13:01:00.099833  7895 leader_election.cc:343] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:00.087641  7977 leader_election.cc:290] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185)
I20260709 13:01:00.100693  7978 raft_consensus.cc:2804] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:00.101109  7978 raft_consensus.cc:493] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:00.101792  7978 raft_consensus.cc:3060] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.106149  7973 tablet_bootstrap.cc:492] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b: No bootstrap required, opened a new log
I20260709 13:01:00.106766  7973 ts_tablet_manager.cc:1403] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b: Time spent bootstrapping tablet: real 0.046s	user 0.024s	sys 0.016s
I20260709 13:01:00.106912  7979 tablet_bootstrap.cc:492] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95: No bootstrap required, opened a new log
I20260709 13:01:00.107336  7979 ts_tablet_manager.cc:1403] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.009s
I20260709 13:01:00.109885  7973 raft_consensus.cc:359] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.110020  7979 raft_consensus.cc:359] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.110589  7973 raft_consensus.cc:385] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:00.110631  7979 raft_consensus.cc:385] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:00.110970  7973 raft_consensus.cc:740] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aec419c47a640edb247a753aaa6ad5b, State: Initialized, Role: FOLLOWER
I20260709 13:01:00.111716  7978 raft_consensus.cc:515] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.111766  7973 consensus_queue.cc:260] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.113662  7973 ts_tablet_manager.cc:1434] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:01:00.113675  7978 leader_election.cc:290] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 election: Requested vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185)
I20260709 13:01:00.114655  7973 tablet_bootstrap.cc:492] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b: Bootstrap starting.
I20260709 13:01:00.114974  7783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "494fef1b35024d30b17d4c0aee058999" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c88031f63c411a9a349b00f53b211d"
I20260709 13:01:00.115512  7783 raft_consensus.cc:3060] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.111006  7979 raft_consensus.cc:740] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4ca9afe41ff4eb298062aa06f346b95, State: Initialized, Role: FOLLOWER
I20260709 13:01:00.117925  7979 consensus_queue.cc:260] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.119938  7979 ts_tablet_manager.cc:1434] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 13:01:00.120914  7973 tablet_bootstrap.cc:654] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:00.121269  7977 raft_consensus.cc:493] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.121757  7977 raft_consensus.cc:515] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.122099  7979 tablet_bootstrap.cc:492] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95: Bootstrap starting.
I20260709 13:01:00.122624  7783 raft_consensus.cc:2468] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aec419c47a640edb247a753aaa6ad5b in term 1.
I20260709 13:01:00.123669  7977 leader_election.cc:290] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185)
I20260709 13:01:00.124128  7977 raft_consensus.cc:493] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.124527  7977 raft_consensus.cc:515] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.126477  7895 leader_election.cc:304] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d, 9aec419c47a640edb247a753aaa6ad5b; no voters: 
I20260709 13:01:00.127096  7973 tablet_bootstrap.cc:492] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b: No bootstrap required, opened a new log
I20260709 13:01:00.126982  7858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e2882ee34dd4e6dbd67b64cea774679" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95" is_pre_election: true
I20260709 13:01:00.127525  7973 ts_tablet_manager.cc:1403] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b: Time spent bootstrapping tablet: real 0.013s	user 0.007s	sys 0.004s
I20260709 13:01:00.127600  7858 raft_consensus.cc:2468] T 6e2882ee34dd4e6dbd67b64cea774679 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aec419c47a640edb247a753aaa6ad5b in term 0.
I20260709 13:01:00.127990  7783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e2882ee34dd4e6dbd67b64cea774679" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c88031f63c411a9a349b00f53b211d" is_pre_election: true
I20260709 13:01:00.128492  7979 tablet_bootstrap.cc:654] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:00.128615  7783 raft_consensus.cc:2468] T 6e2882ee34dd4e6dbd67b64cea774679 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aec419c47a640edb247a753aaa6ad5b in term 0.
I20260709 13:01:00.128858  7895 leader_election.cc:304] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aec419c47a640edb247a753aaa6ad5b, c4ca9afe41ff4eb298062aa06f346b95; no voters: 
I20260709 13:01:00.129385  7857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "494fef1b35024d30b17d4c0aee058999" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
I20260709 13:01:00.130353  7973 raft_consensus.cc:359] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.131220  7973 raft_consensus.cc:385] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:00.131569  7973 raft_consensus.cc:740] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aec419c47a640edb247a753aaa6ad5b, State: Initialized, Role: FOLLOWER
I20260709 13:01:00.132316  7973 consensus_queue.cc:260] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.136407  7977 leader_election.cc:290] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185)
I20260709 13:01:00.137247  7977 raft_consensus.cc:2804] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:00.137908  7977 raft_consensus.cc:697] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 1 LEADER]: Becoming Leader. State: Replica: 9aec419c47a640edb247a753aaa6ad5b, State: Running, Role: LEADER
I20260709 13:01:00.139106  7977 consensus_queue.cc:237] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.140338  7978 raft_consensus.cc:2804] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:00.142172  7978 raft_consensus.cc:493] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:00.142707  7978 raft_consensus.cc:3060] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.156142  7978 raft_consensus.cc:515] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.171602  7783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb544d4966e409c97ce8eff34e7521e" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c88031f63c411a9a349b00f53b211d" is_pre_election: true
I20260709 13:01:00.174078  7783 raft_consensus.cc:2468] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aec419c47a640edb247a753aaa6ad5b in term 0.
I20260709 13:01:00.172780  7979 tablet_bootstrap.cc:492] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95: No bootstrap required, opened a new log
W20260709 13:01:00.174671  7895 leader_election.cc:343] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:00.175132  7978 leader_election.cc:290] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 election: Requested vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185)
I20260709 13:01:00.175310  7979 ts_tablet_manager.cc:1403] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95: Time spent bootstrapping tablet: real 0.053s	user 0.015s	sys 0.035s
I20260709 13:01:00.173112  7973 ts_tablet_manager.cc:1434] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.038s
I20260709 13:01:00.173091  7857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb544d4966e409c97ce8eff34e7521e" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95" is_pre_election: true
I20260709 13:01:00.182473  7973 tablet_bootstrap.cc:492] T 441ce35ef600431fa3fa881500cb9563 P 9aec419c47a640edb247a753aaa6ad5b: Bootstrap starting.
I20260709 13:01:00.183193  7783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e2882ee34dd4e6dbd67b64cea774679" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c88031f63c411a9a349b00f53b211d"
I20260709 13:01:00.183853  7783 raft_consensus.cc:3060] T 6e2882ee34dd4e6dbd67b64cea774679 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.184283  7979 raft_consensus.cc:359] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.185153  7979 raft_consensus.cc:385] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:00.184311  7680 catalog_manager.cc:5697] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3). New cstate: current_term: 1 leader_uuid: "9aec419c47a640edb247a753aaa6ad5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:00.185519  7979 raft_consensus.cc:740] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4ca9afe41ff4eb298062aa06f346b95, State: Initialized, Role: FOLLOWER
I20260709 13:01:00.186329  7979 consensus_queue.cc:260] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.188524  7979 ts_tablet_manager.cc:1434] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95: Time spent starting tablet: real 0.013s	user 0.000s	sys 0.010s
I20260709 13:01:00.189471  7973 tablet_bootstrap.cc:654] T 441ce35ef600431fa3fa881500cb9563 P 9aec419c47a640edb247a753aaa6ad5b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:00.189549  7979 tablet_bootstrap.cc:492] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95: Bootstrap starting.
I20260709 13:01:00.192143  8002 raft_consensus.cc:493] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.192763  8002 raft_consensus.cc:515] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.194821  7783 raft_consensus.cc:2468] T 6e2882ee34dd4e6dbd67b64cea774679 P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aec419c47a640edb247a753aaa6ad5b in term 1.
I20260709 13:01:00.195278  8002 leader_election.cc:290] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
I20260709 13:01:00.197711  7858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4ced2215e2544faa2ecda7ceba5de27" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95" is_pre_election: true
I20260709 13:01:00.197999  7930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4ced2215e2544faa2ecda7ceba5de27" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aec419c47a640edb247a753aaa6ad5b" is_pre_election: true
I20260709 13:01:00.198351  7858 raft_consensus.cc:2468] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68c88031f63c411a9a349b00f53b211d in term 0.
I20260709 13:01:00.198534  7930 raft_consensus.cc:2468] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68c88031f63c411a9a349b00f53b211d in term 0.
I20260709 13:01:00.199947  7746 leader_election.cc:304] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d, 9aec419c47a640edb247a753aaa6ad5b; no voters: 
I20260709 13:01:00.201265  8002 raft_consensus.cc:2804] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:00.201640  8002 raft_consensus.cc:493] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:00.201861  7973 tablet_bootstrap.cc:492] T 441ce35ef600431fa3fa881500cb9563 P 9aec419c47a640edb247a753aaa6ad5b: No bootstrap required, opened a new log
I20260709 13:01:00.201987  8002 raft_consensus.cc:3060] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.202517  7973 ts_tablet_manager.cc:1403] T 441ce35ef600431fa3fa881500cb9563 P 9aec419c47a640edb247a753aaa6ad5b: Time spent bootstrapping tablet: real 0.020s	user 0.006s	sys 0.005s
I20260709 13:01:00.205904  7973 raft_consensus.cc:359] T 441ce35ef600431fa3fa881500cb9563 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.207645  7973 raft_consensus.cc:385] T 441ce35ef600431fa3fa881500cb9563 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:00.207988  7973 raft_consensus.cc:740] T 441ce35ef600431fa3fa881500cb9563 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aec419c47a640edb247a753aaa6ad5b, State: Initialized, Role: FOLLOWER
I20260709 13:01:00.208711  7973 consensus_queue.cc:260] T 441ce35ef600431fa3fa881500cb9563 P 9aec419c47a640edb247a753aaa6ad5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.209385  8002 raft_consensus.cc:515] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.207356  7979 tablet_bootstrap.cc:654] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:00.213167  7858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4ced2215e2544faa2ecda7ceba5de27" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
I20260709 13:01:00.213829  7858 raft_consensus.cc:3060] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.214361  7973 ts_tablet_manager.cc:1434] T 441ce35ef600431fa3fa881500cb9563 P 9aec419c47a640edb247a753aaa6ad5b: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.002s
I20260709 13:01:00.217823  8002 leader_election.cc:290] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 election: Requested vote from peers c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
I20260709 13:01:00.219050  7930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4ced2215e2544faa2ecda7ceba5de27" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
I20260709 13:01:00.219821  7930 raft_consensus.cc:3060] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.220152  7979 tablet_bootstrap.cc:492] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95: No bootstrap required, opened a new log
I20260709 13:01:00.220594  7979 ts_tablet_manager.cc:1403] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95: Time spent bootstrapping tablet: real 0.031s	user 0.004s	sys 0.008s
I20260709 13:01:00.223320  7858 raft_consensus.cc:2468] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68c88031f63c411a9a349b00f53b211d in term 1.
I20260709 13:01:00.223165  7979 raft_consensus.cc:359] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.224771  7979 raft_consensus.cc:385] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:00.224833  7746 leader_election.cc:304] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d, c4ca9afe41ff4eb298062aa06f346b95; no voters: 
I20260709 13:01:00.225097  7979 raft_consensus.cc:740] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4ca9afe41ff4eb298062aa06f346b95, State: Initialized, Role: FOLLOWER
I20260709 13:01:00.226060  8002 raft_consensus.cc:2804] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:00.226724  8002 raft_consensus.cc:697] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 1 LEADER]: Becoming Leader. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Running, Role: LEADER
I20260709 13:01:00.226989  7979 consensus_queue.cc:260] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.227852  8002 consensus_queue.cc:237] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.228307  7930 raft_consensus.cc:2468] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68c88031f63c411a9a349b00f53b211d in term 1.
I20260709 13:01:00.229947  7979 ts_tablet_manager.cc:1434] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:01:00.233047  7857 raft_consensus.cc:2468] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aec419c47a640edb247a753aaa6ad5b in term 0.
I20260709 13:01:00.233562  7992 raft_consensus.cc:493] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.233978  7992 raft_consensus.cc:515] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.234419  7895 leader_election.cc:304] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d, 9aec419c47a640edb247a753aaa6ad5b; no voters: 
I20260709 13:01:00.235046  7857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e2882ee34dd4e6dbd67b64cea774679" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
I20260709 13:01:00.235302  7895 leader_election.cc:304] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d, 9aec419c47a640edb247a753aaa6ad5b; no voters: 
I20260709 13:01:00.235538  7857 raft_consensus.cc:3060] T 6e2882ee34dd4e6dbd67b64cea774679 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.236220  8022 raft_consensus.cc:2804] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:00.236604  8022 raft_consensus.cc:493] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:00.236948  8022 raft_consensus.cc:3060] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.236941  7978 raft_consensus.cc:2804] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:00.237692  7978 raft_consensus.cc:697] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b [term 1 LEADER]: Becoming Leader. State: Replica: 9aec419c47a640edb247a753aaa6ad5b, State: Running, Role: LEADER
I20260709 13:01:00.238569  7978 consensus_queue.cc:237] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.240763  7992 leader_election.cc:290] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
I20260709 13:01:00.242184  7930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb544d4966e409c97ce8eff34e7521e" candidate_uuid: "c4ca9afe41ff4eb298062aa06f346b95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aec419c47a640edb247a753aaa6ad5b" is_pre_election: true
I20260709 13:01:00.242817  7857 raft_consensus.cc:2468] T 6e2882ee34dd4e6dbd67b64cea774679 P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aec419c47a640edb247a753aaa6ad5b in term 1.
I20260709 13:01:00.247267  8022 raft_consensus.cc:515] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.245604  7783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb544d4966e409c97ce8eff34e7521e" candidate_uuid: "c4ca9afe41ff4eb298062aa06f346b95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c88031f63c411a9a349b00f53b211d" is_pre_election: true
I20260709 13:01:00.249655  7930 raft_consensus.cc:2393] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c4ca9afe41ff4eb298062aa06f346b95 in current term 1: Already voted for candidate 9aec419c47a640edb247a753aaa6ad5b in this term.
I20260709 13:01:00.251084  8020 raft_consensus.cc:493] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.251583  8020 raft_consensus.cc:515] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.252070  7677 catalog_manager.cc:5697] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d reported cstate change: term changed from 0 to 1, leader changed from <none> to 68c88031f63c411a9a349b00f53b211d (127.6.255.1). New cstate: current_term: 1 leader_uuid: "68c88031f63c411a9a349b00f53b211d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:00.253789  8020 leader_election.cc:290] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185)
I20260709 13:01:00.254751  7782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d8273641a344bfd96c1bee2c72380ef" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c88031f63c411a9a349b00f53b211d" is_pre_election: true
I20260709 13:01:00.254902  7857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d8273641a344bfd96c1bee2c72380ef" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95" is_pre_election: true
I20260709 13:01:00.255380  7782 raft_consensus.cc:2468] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aec419c47a640edb247a753aaa6ad5b in term 0.
I20260709 13:01:00.255724  7857 raft_consensus.cc:2468] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aec419c47a640edb247a753aaa6ad5b in term 0.
I20260709 13:01:00.256850  7895 leader_election.cc:304] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d, 9aec419c47a640edb247a753aaa6ad5b; no voters: 
I20260709 13:01:00.257634  8020 raft_consensus.cc:2804] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:00.257983  8020 raft_consensus.cc:493] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:00.258257  8020 raft_consensus.cc:3060] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.260389  8022 leader_election.cc:290] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 election: Requested vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185)
I20260709 13:01:00.261011  7782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb544d4966e409c97ce8eff34e7521e" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c88031f63c411a9a349b00f53b211d"
I20260709 13:01:00.264178  7857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb544d4966e409c97ce8eff34e7521e" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
I20260709 13:01:00.264827  7857 raft_consensus.cc:3060] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.264623  7680 catalog_manager.cc:5697] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3). New cstate: current_term: 1 leader_uuid: "9aec419c47a640edb247a753aaa6ad5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:00.258914  7783 raft_consensus.cc:2468] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4ca9afe41ff4eb298062aa06f346b95 in term 0.
I20260709 13:01:00.268287  7821 leader_election.cc:304] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d, c4ca9afe41ff4eb298062aa06f346b95; no voters: 9aec419c47a640edb247a753aaa6ad5b
I20260709 13:01:00.265055  8020 raft_consensus.cc:515] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.270686  8020 leader_election.cc:290] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 election: Requested vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185)
I20260709 13:01:00.271337  7783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d8273641a344bfd96c1bee2c72380ef" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c88031f63c411a9a349b00f53b211d"
I20260709 13:01:00.271912  7783 raft_consensus.cc:3060] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.272142  7858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d8273641a344bfd96c1bee2c72380ef" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
I20260709 13:01:00.272701  7858 raft_consensus.cc:3060] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.277796  7991 raft_consensus.cc:493] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.278630  7783 raft_consensus.cc:2468] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aec419c47a640edb247a753aaa6ad5b in term 1.
I20260709 13:01:00.278409  7991 raft_consensus.cc:515] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.280454  7858 raft_consensus.cc:2468] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aec419c47a640edb247a753aaa6ad5b in term 1.
I20260709 13:01:00.280541  7991 leader_election.cc:290] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
I20260709 13:01:00.280900  7857 raft_consensus.cc:2468] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aec419c47a640edb247a753aaa6ad5b in term 1.
I20260709 13:01:00.281765  7992 raft_consensus.cc:2764] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:01:00.281641  7895 leader_election.cc:304] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aec419c47a640edb247a753aaa6ad5b, c4ca9afe41ff4eb298062aa06f346b95; no voters: 
I20260709 13:01:00.282258  7930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441ce35ef600431fa3fa881500cb9563" candidate_uuid: "c4ca9afe41ff4eb298062aa06f346b95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aec419c47a640edb247a753aaa6ad5b" is_pre_election: true
I20260709 13:01:00.282444  7782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441ce35ef600431fa3fa881500cb9563" candidate_uuid: "c4ca9afe41ff4eb298062aa06f346b95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c88031f63c411a9a349b00f53b211d" is_pre_election: true
I20260709 13:01:00.283113  7930 raft_consensus.cc:2468] T 441ce35ef600431fa3fa881500cb9563 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4ca9afe41ff4eb298062aa06f346b95 in term 0.
I20260709 13:01:00.283214  7782 raft_consensus.cc:2468] T 441ce35ef600431fa3fa881500cb9563 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4ca9afe41ff4eb298062aa06f346b95 in term 0.
I20260709 13:01:00.283604  7895 leader_election.cc:304] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aec419c47a640edb247a753aaa6ad5b, c4ca9afe41ff4eb298062aa06f346b95; no voters: 68c88031f63c411a9a349b00f53b211d
I20260709 13:01:00.284431  8022 raft_consensus.cc:2804] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:00.284499  8020 raft_consensus.cc:2804] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:00.284425  7821 leader_election.cc:304] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d, c4ca9afe41ff4eb298062aa06f346b95; no voters: 
I20260709 13:01:00.285197  8022 raft_consensus.cc:697] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [term 1 LEADER]: Becoming Leader. State: Replica: 9aec419c47a640edb247a753aaa6ad5b, State: Running, Role: LEADER
I20260709 13:01:00.285389  8020 raft_consensus.cc:697] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 1 LEADER]: Becoming Leader. State: Replica: 9aec419c47a640edb247a753aaa6ad5b, State: Running, Role: LEADER
I20260709 13:01:00.285584  7992 raft_consensus.cc:2804] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:00.285988  7992 raft_consensus.cc:493] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:00.286338  7992 raft_consensus.cc:3060] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.286247  8022 consensus_queue.cc:237] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.286253  8020 consensus_queue.cc:237] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.291996  7992 raft_consensus.cc:515] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.295308  7783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441ce35ef600431fa3fa881500cb9563" candidate_uuid: "c4ca9afe41ff4eb298062aa06f346b95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c88031f63c411a9a349b00f53b211d"
I20260709 13:01:00.296036  7783 raft_consensus.cc:3060] T 441ce35ef600431fa3fa881500cb9563 P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.296836  7930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441ce35ef600431fa3fa881500cb9563" candidate_uuid: "c4ca9afe41ff4eb298062aa06f346b95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
I20260709 13:01:00.297616  7930 raft_consensus.cc:3060] T 441ce35ef600431fa3fa881500cb9563 P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.298408  7992 leader_election.cc:290] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95 [CANDIDATE]: Term 1 election: Requested vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
I20260709 13:01:00.302759  7930 raft_consensus.cc:2468] T 441ce35ef600431fa3fa881500cb9563 P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4ca9afe41ff4eb298062aa06f346b95 in term 1.
I20260709 13:01:00.302906  7783 raft_consensus.cc:2468] T 441ce35ef600431fa3fa881500cb9563 P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4ca9afe41ff4eb298062aa06f346b95 in term 1.
I20260709 13:01:00.302155  7680 catalog_manager.cc:5697] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3). New cstate: current_term: 1 leader_uuid: "9aec419c47a640edb247a753aaa6ad5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:00.303915  7821 leader_election.cc:304] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aec419c47a640edb247a753aaa6ad5b, c4ca9afe41ff4eb298062aa06f346b95; no voters: 
I20260709 13:01:00.304730  7992 raft_consensus.cc:2804] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:00.304170  7680 catalog_manager.cc:5697] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3). New cstate: current_term: 1 leader_uuid: "9aec419c47a640edb247a753aaa6ad5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:00.305199  7992 raft_consensus.cc:697] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95 [term 1 LEADER]: Becoming Leader. State: Replica: c4ca9afe41ff4eb298062aa06f346b95, State: Running, Role: LEADER
I20260709 13:01:00.306056  7992 consensus_queue.cc:237] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.309306  8022 consensus_queue.cc:1048] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:00.326804  8020 consensus_queue.cc:1048] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:00.332413  7680 catalog_manager.cc:5697] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "c4ca9afe41ff4eb298062aa06f346b95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:00.432158  7930 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "428de0e6e6d1418ebcbc8ff1fbf24a26"
dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
mode: GRACEFUL
new_leader_uuid: "68c88031f63c411a9a349b00f53b211d"
 from {username='slave'} at 127.0.0.1:42908
I20260709 13:01:00.432672  7930 raft_consensus.cc:606] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 1 LEADER]: Received request to transfer leadership to TS 68c88031f63c411a9a349b00f53b211d
I20260709 13:01:00.440847  7930 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bcb544d4966e409c97ce8eff34e7521e"
dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
mode: GRACEFUL
new_leader_uuid: "68c88031f63c411a9a349b00f53b211d"
 from {username='slave'} at 127.0.0.1:42908
I20260709 13:01:00.441454  7930 raft_consensus.cc:606] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 1 LEADER]: Received request to transfer leadership to TS 68c88031f63c411a9a349b00f53b211d
I20260709 13:01:00.460002  7783 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4ced2215e2544faa2ecda7ceba5de27"
dest_uuid: "68c88031f63c411a9a349b00f53b211d"
mode: GRACEFUL
new_leader_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
 from {username='slave'} at 127.0.0.1:54254
I20260709 13:01:00.460523  7783 raft_consensus.cc:606] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 1 LEADER]: Received request to transfer leadership to TS c4ca9afe41ff4eb298062aa06f346b95
I20260709 13:01:00.466042  7930 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "494fef1b35024d30b17d4c0aee058999"
dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
mode: GRACEFUL
new_leader_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
 from {username='slave'} at 127.0.0.1:42908
I20260709 13:01:00.466648  7930 raft_consensus.cc:606] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 1 LEADER]: Received request to transfer leadership to TS c4ca9afe41ff4eb298062aa06f346b95
I20260709 13:01:00.472894  7930 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d8273641a344bfd96c1bee2c72380ef"
dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
mode: GRACEFUL
new_leader_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
 from {username='slave'} at 127.0.0.1:42908
I20260709 13:01:00.473556  7930 raft_consensus.cc:606] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [term 1 LEADER]: Received request to transfer leadership to TS c4ca9afe41ff4eb298062aa06f346b95
I20260709 13:01:00.504679  8020 raft_consensus.cc:493] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.505314  8020 raft_consensus.cc:515] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.505712  7992 raft_consensus.cc:493] T 20e3b50df0d7462da76007b962677370 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.506259  7992 raft_consensus.cc:515] T 20e3b50df0d7462da76007b962677370 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.507354  8020 leader_election.cc:290] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185)
I20260709 13:01:00.507992  7992 leader_election.cc:290] T 20e3b50df0d7462da76007b962677370 P c4ca9afe41ff4eb298062aa06f346b95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
I20260709 13:01:00.508294  7783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55538fcb866c4dd1a3cd12db54eb946b" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c88031f63c411a9a349b00f53b211d" is_pre_election: true
I20260709 13:01:00.508589  7857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55538fcb866c4dd1a3cd12db54eb946b" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95" is_pre_election: true
I20260709 13:01:00.509351  7857 raft_consensus.cc:2393] T 55538fcb866c4dd1a3cd12db54eb946b P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aec419c47a640edb247a753aaa6ad5b in current term 1: Already voted for candidate 68c88031f63c411a9a349b00f53b211d in this term.
I20260709 13:01:00.509394  7782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20e3b50df0d7462da76007b962677370" candidate_uuid: "c4ca9afe41ff4eb298062aa06f346b95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c88031f63c411a9a349b00f53b211d" is_pre_election: true
I20260709 13:01:00.509681  7930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20e3b50df0d7462da76007b962677370" candidate_uuid: "c4ca9afe41ff4eb298062aa06f346b95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aec419c47a640edb247a753aaa6ad5b" is_pre_election: true
I20260709 13:01:00.510408  7930 raft_consensus.cc:2393] T 20e3b50df0d7462da76007b962677370 P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c4ca9afe41ff4eb298062aa06f346b95 in current term 1: Already voted for candidate 68c88031f63c411a9a349b00f53b211d in this term.
I20260709 13:01:00.510435  7895 leader_election.cc:304] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aec419c47a640edb247a753aaa6ad5b; no voters: 68c88031f63c411a9a349b00f53b211d, c4ca9afe41ff4eb298062aa06f346b95
I20260709 13:01:00.511308  8020 raft_consensus.cc:3060] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.511431  7821 leader_election.cc:304] T 20e3b50df0d7462da76007b962677370 P c4ca9afe41ff4eb298062aa06f346b95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4ca9afe41ff4eb298062aa06f346b95; no voters: 68c88031f63c411a9a349b00f53b211d, 9aec419c47a640edb247a753aaa6ad5b
I20260709 13:01:00.512197  7992 raft_consensus.cc:3060] T 20e3b50df0d7462da76007b962677370 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.518015  8020 raft_consensus.cc:2749] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:00.518667  7992 raft_consensus.cc:2749] T 20e3b50df0d7462da76007b962677370 P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:00.520291  8002 consensus_queue.cc:1048] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:00.530372  8000 consensus_queue.cc:1048] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:00.531781  7976 consensus_queue.cc:1048] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:00.545935  8002 consensus_queue.cc:1048] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:00.552500  7858 raft_consensus.cc:3060] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.556150  8002 consensus_queue.cc:1048] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:00.559924  8020 consensus_queue.cc:1048] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:00.568972  8002 consensus_queue.cc:1048] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:00.588186  7977 raft_consensus.cc:993] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b: : Instructing follower c4ca9afe41ff4eb298062aa06f346b95 to start an election
I20260709 13:01:00.588677  7978 raft_consensus.cc:1081] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 1 LEADER]: Signalling peer c4ca9afe41ff4eb298062aa06f346b95 to start an election
I20260709 13:01:00.592171  7856 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "494fef1b35024d30b17d4c0aee058999"
dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
 from {username='slave'} at 127.0.0.1:42592
I20260709 13:01:00.592648  7856 raft_consensus.cc:493] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:00.592960  7856 raft_consensus.cc:3060] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:00.600728  7978 consensus_queue.cc:1048] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:00.600817  7856 raft_consensus.cc:515] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.603601  7783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "494fef1b35024d30b17d4c0aee058999" candidate_uuid: "c4ca9afe41ff4eb298062aa06f346b95" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "68c88031f63c411a9a349b00f53b211d"
I20260709 13:01:00.609105  7856 leader_election.cc:290] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [CANDIDATE]: Term 2 election: Requested vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
I20260709 13:01:00.609671  7930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "494fef1b35024d30b17d4c0aee058999" candidate_uuid: "c4ca9afe41ff4eb298062aa06f346b95" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
I20260709 13:01:00.610194  7930 raft_consensus.cc:3055] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:00.610496  7930 raft_consensus.cc:740] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aec419c47a640edb247a753aaa6ad5b, State: Running, Role: LEADER
I20260709 13:01:00.611157  7930 consensus_queue.cc:260] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.612110  7930 raft_consensus.cc:3060] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:00.617902  7930 raft_consensus.cc:2468] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4ca9afe41ff4eb298062aa06f346b95 in term 2.
I20260709 13:01:00.619032  7821 leader_election.cc:304] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aec419c47a640edb247a753aaa6ad5b, c4ca9afe41ff4eb298062aa06f346b95; no voters: 68c88031f63c411a9a349b00f53b211d
I20260709 13:01:00.619951  7991 raft_consensus.cc:2804] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:00.620414  7991 raft_consensus.cc:697] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [term 2 LEADER]: Becoming Leader. State: Replica: c4ca9afe41ff4eb298062aa06f346b95, State: Running, Role: LEADER
I20260709 13:01:00.621186  7991 consensus_queue.cc:237] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.628355  7677 catalog_manager.cc:5697] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 reported cstate change: term changed from 1 to 2, leader changed from 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3) to c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "c4ca9afe41ff4eb298062aa06f346b95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:00.678750  7978 consensus_queue.cc:1048] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:00.689198  7978 consensus_queue.cc:1048] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:00.717028  7991 consensus_queue.cc:1048] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:00.728688  8050 consensus_queue.cc:1048] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:00.736800  7976 consensus_queue.cc:1048] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:00.746538  8002 raft_consensus.cc:993] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d: : Instructing follower c4ca9afe41ff4eb298062aa06f346b95 to start an election
I20260709 13:01:00.747054  8000 raft_consensus.cc:1081] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 1 LEADER]: Signalling peer c4ca9afe41ff4eb298062aa06f346b95 to start an election
I20260709 13:01:00.748844  7858 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c4ced2215e2544faa2ecda7ceba5de27"
dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
 from {username='slave'} at 127.0.0.1:42594
I20260709 13:01:00.749583  7858 raft_consensus.cc:493] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:00.749969  7858 raft_consensus.cc:3060] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:00.756711  7976 consensus_queue.cc:1048] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:00.755848  7858 raft_consensus.cc:515] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.758291  7782 raft_consensus.cc:3060] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.759107  7858 leader_election.cc:290] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [CANDIDATE]: Term 2 election: Requested vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
I20260709 13:01:00.761262  7783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4ced2215e2544faa2ecda7ceba5de27" candidate_uuid: "c4ca9afe41ff4eb298062aa06f346b95" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "68c88031f63c411a9a349b00f53b211d"
I20260709 13:01:00.761734  7932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4ced2215e2544faa2ecda7ceba5de27" candidate_uuid: "c4ca9afe41ff4eb298062aa06f346b95" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
I20260709 13:01:00.761973  7783 raft_consensus.cc:3055] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:00.762344  7783 raft_consensus.cc:740] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Running, Role: LEADER
I20260709 13:01:00.763190  7783 consensus_queue.cc:260] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.764354  7783 raft_consensus.cc:3060] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:00.771804  7978 consensus_queue.cc:1048] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:00.772353  7783 raft_consensus.cc:2468] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4ca9afe41ff4eb298062aa06f346b95 in term 2.
I20260709 13:01:00.773733  7821 leader_election.cc:304] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d, c4ca9afe41ff4eb298062aa06f346b95; no voters: 9aec419c47a640edb247a753aaa6ad5b
I20260709 13:01:00.774546  8050 raft_consensus.cc:2804] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:00.775106  8050 raft_consensus.cc:697] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [term 2 LEADER]: Becoming Leader. State: Replica: c4ca9afe41ff4eb298062aa06f346b95, State: Running, Role: LEADER
I20260709 13:01:00.776010  8050 consensus_queue.cc:237] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.782788  7977 raft_consensus.cc:993] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b: : Instructing follower 68c88031f63c411a9a349b00f53b211d to start an election
I20260709 13:01:00.783296  7977 raft_consensus.cc:1081] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 1 LEADER]: Signalling peer 68c88031f63c411a9a349b00f53b211d to start an election
I20260709 13:01:00.785156  7677 catalog_manager.cc:5697] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 reported cstate change: term changed from 1 to 2, leader changed from 68c88031f63c411a9a349b00f53b211d (127.6.255.1) to c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "c4ca9afe41ff4eb298062aa06f346b95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:00.786991  7783 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bcb544d4966e409c97ce8eff34e7521e"
dest_uuid: "68c88031f63c411a9a349b00f53b211d"
 from {username='slave'} at 127.0.0.1:54240
I20260709 13:01:00.787606  7783 raft_consensus.cc:493] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:00.787985  7783 raft_consensus.cc:3060] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:00.791559  7978 consensus_queue.cc:1048] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:00.795050  7783 raft_consensus.cc:515] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.797394  7783 leader_election.cc:290] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 2 election: Requested vote from peers c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
I20260709 13:01:00.798112  7858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb544d4966e409c97ce8eff34e7521e" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
I20260709 13:01:00.798483  7930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb544d4966e409c97ce8eff34e7521e" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
I20260709 13:01:00.799059  7930 raft_consensus.cc:3055] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:00.799516  7930 raft_consensus.cc:740] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aec419c47a640edb247a753aaa6ad5b, State: Running, Role: LEADER
I20260709 13:01:00.799011  7782 raft_consensus.cc:1240] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Rejecting Update request from peer 9aec419c47a640edb247a753aaa6ad5b for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:00.800361  7930 consensus_queue.cc:260] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.801075  7930 raft_consensus.cc:3060] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:00.807165  7930 raft_consensus.cc:2468] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68c88031f63c411a9a349b00f53b211d in term 2.
I20260709 13:01:00.808146  7746 leader_election.cc:304] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d, 9aec419c47a640edb247a753aaa6ad5b; no voters: c4ca9afe41ff4eb298062aa06f346b95
I20260709 13:01:00.808941  7976 raft_consensus.cc:2804] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:00.809371  7976 raft_consensus.cc:697] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 2 LEADER]: Becoming Leader. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Running, Role: LEADER
I20260709 13:01:00.810086  7976 consensus_queue.cc:237] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.814551  7978 consensus_queue.cc:1048] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:00.816361  7677 catalog_manager.cc:5697] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d reported cstate change: term changed from 1 to 2, leader changed from 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3) to 68c88031f63c411a9a349b00f53b211d (127.6.255.1). New cstate: current_term: 2 leader_uuid: "68c88031f63c411a9a349b00f53b211d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:00.824612  8020 consensus_queue.cc:1048] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:00.834023  8020 raft_consensus.cc:993] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b: : Instructing follower c4ca9afe41ff4eb298062aa06f346b95 to start an election
I20260709 13:01:00.834576  7977 raft_consensus.cc:1081] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [term 1 LEADER]: Signalling peer c4ca9afe41ff4eb298062aa06f346b95 to start an election
I20260709 13:01:00.837015  7856 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5d8273641a344bfd96c1bee2c72380ef"
dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
 from {username='slave'} at 127.0.0.1:42592
I20260709 13:01:00.837553  7856 raft_consensus.cc:493] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:00.837869  7856 raft_consensus.cc:3060] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:00.843078  7856 raft_consensus.cc:515] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.844769  7856 leader_election.cc:290] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [CANDIDATE]: Term 2 election: Requested vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
I20260709 13:01:00.845916  7930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d8273641a344bfd96c1bee2c72380ef" candidate_uuid: "c4ca9afe41ff4eb298062aa06f346b95" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
I20260709 13:01:00.846604  7930 raft_consensus.cc:3055] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:00.846890  7930 raft_consensus.cc:740] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aec419c47a640edb247a753aaa6ad5b, State: Running, Role: LEADER
I20260709 13:01:00.847573  7930 consensus_queue.cc:260] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.848618  7930 raft_consensus.cc:3060] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:00.851382  8020 raft_consensus.cc:993] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b: : Instructing follower 68c88031f63c411a9a349b00f53b211d to start an election
I20260709 13:01:00.851898  7977 raft_consensus.cc:1081] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 1 LEADER]: Signalling peer 68c88031f63c411a9a349b00f53b211d to start an election
I20260709 13:01:00.845947  7782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d8273641a344bfd96c1bee2c72380ef" candidate_uuid: "c4ca9afe41ff4eb298062aa06f346b95" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "68c88031f63c411a9a349b00f53b211d"
I20260709 13:01:00.853477  7783 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "428de0e6e6d1418ebcbc8ff1fbf24a26"
dest_uuid: "68c88031f63c411a9a349b00f53b211d"
 from {username='slave'} at 127.0.0.1:54240
I20260709 13:01:00.853756  7782 raft_consensus.cc:3060] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:00.854041  7783 raft_consensus.cc:493] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:00.854481  7783 raft_consensus.cc:3060] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:00.854436  7930 raft_consensus.cc:2468] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4ca9afe41ff4eb298062aa06f346b95 in term 2.
I20260709 13:01:00.855834  7821 leader_election.cc:304] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aec419c47a640edb247a753aaa6ad5b, c4ca9afe41ff4eb298062aa06f346b95; no voters: 
I20260709 13:01:00.856633  8050 raft_consensus.cc:2804] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:00.857161  8050 raft_consensus.cc:697] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [term 2 LEADER]: Becoming Leader. State: Replica: c4ca9afe41ff4eb298062aa06f346b95, State: Running, Role: LEADER
I20260709 13:01:00.858150  8050 consensus_queue.cc:237] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:00.861101  7782 raft_consensus.cc:2468] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4ca9afe41ff4eb298062aa06f346b95 in term 2.
I20260709 13:01:00.861259  7783 raft_consensus.cc:515] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } } peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } }
I20260709 13:01:00.863330  7783 leader_election.cc:290] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 2 election: Requested vote from peers 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505), c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185)
I20260709 13:01:00.864108  7930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428de0e6e6d1418ebcbc8ff1fbf24a26" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
I20260709 13:01:00.864710  7930 raft_consensus.cc:3055] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:00.864749  7856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428de0e6e6d1418ebcbc8ff1fbf24a26" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
I20260709 13:01:00.865087  7930 raft_consensus.cc:740] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aec419c47a640edb247a753aaa6ad5b, State: Running, Role: LEADER
I20260709 13:01:00.865406  7856 raft_consensus.cc:3060] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:00.865919  7930 consensus_queue.cc:260] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } } peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } }
I20260709 13:01:00.866818  7930 raft_consensus.cc:3060] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:00.866667  7677 catalog_manager.cc:5697] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 reported cstate change: term changed from 1 to 2, leader changed from 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3) to c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "c4ca9afe41ff4eb298062aa06f346b95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:00.871183  7856 raft_consensus.cc:2468] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P c4ca9afe41ff4eb298062aa06f346b95 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68c88031f63c411a9a349b00f53b211d in term 2.
I20260709 13:01:00.872277  7930 raft_consensus.cc:2468] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68c88031f63c411a9a349b00f53b211d in term 2.
I20260709 13:01:00.873373  7746 leader_election.cc:304] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d, 9aec419c47a640edb247a753aaa6ad5b; no voters: 
I20260709 13:01:00.874190  7976 raft_consensus.cc:2804] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:00.874785  7976 raft_consensus.cc:697] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [term 2 LEADER]: Becoming Leader. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Running, Role: LEADER
I20260709 13:01:00.875698  7976 consensus_queue.cc:237] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } } peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } }
I20260709 13:01:00.882221  7677 catalog_manager.cc:5697] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d reported cstate change: term changed from 1 to 2, leader changed from 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3) to 68c88031f63c411a9a349b00f53b211d (127.6.255.1). New cstate: current_term: 2 leader_uuid: "68c88031f63c411a9a349b00f53b211d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:01.208300  7783 raft_consensus.cc:3060] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:01.213471  7783 raft_consensus.cc:1275] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Refusing update from remote peer c4ca9afe41ff4eb298062aa06f346b95: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:01.214831  8050 consensus_queue.cc:1048] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:01.222585  7930 raft_consensus.cc:1275] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 2 FOLLOWER]: Refusing update from remote peer c4ca9afe41ff4eb298062aa06f346b95: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:01.224545  7991 consensus_queue.cc:1048] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:01.245644  7783 raft_consensus.cc:1275] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Refusing update from remote peer c4ca9afe41ff4eb298062aa06f346b95: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:01.246945  7991 consensus_queue.cc:1048] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:01.253804  7930 raft_consensus.cc:3060] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:01.260969  7930 raft_consensus.cc:1275] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b [term 2 FOLLOWER]: Refusing update from remote peer c4ca9afe41ff4eb298062aa06f346b95: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:01.262413  7992 consensus_queue.cc:1048] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:01.266700  7932 raft_consensus.cc:1275] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 2 FOLLOWER]: Refusing update from remote peer 68c88031f63c411a9a349b00f53b211d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:01.268229  8002 consensus_queue.cc:1048] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:01.275961  7856 raft_consensus.cc:3060] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:01.282745  7856 raft_consensus.cc:1275] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95 [term 2 FOLLOWER]: Refusing update from remote peer 68c88031f63c411a9a349b00f53b211d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:01.284018  7976 consensus_queue.cc:1048] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:01.307130  7783 raft_consensus.cc:1275] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Refusing update from remote peer c4ca9afe41ff4eb298062aa06f346b95: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:01.308554  7992 consensus_queue.cc:1048] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:01.315721  7930 raft_consensus.cc:1275] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [term 2 FOLLOWER]: Refusing update from remote peer c4ca9afe41ff4eb298062aa06f346b95: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:01.317245  8050 consensus_queue.cc:1048] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:01.383944  7930 raft_consensus.cc:1275] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 2 FOLLOWER]: Refusing update from remote peer 68c88031f63c411a9a349b00f53b211d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:01.385082  7976 consensus_queue.cc:1048] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:01.395625  7856 raft_consensus.cc:1275] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P c4ca9afe41ff4eb298062aa06f346b95 [term 2 FOLLOWER]: Refusing update from remote peer 68c88031f63c411a9a349b00f53b211d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:01.396814  7976 consensus_queue.cc:1048] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:03.515084  7164 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:01:03.515889  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:03.533205  7782 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "55538fcb866c4dd1a3cd12db54eb946b"
dest_uuid: "68c88031f63c411a9a349b00f53b211d"
mode: GRACEFUL
new_leader_uuid: "9aec419c47a640edb247a753aaa6ad5b"
 from {username='slave'} at 127.0.0.1:37936
I20260709 13:01:03.533743  7782 raft_consensus.cc:606] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [term 1 LEADER]: Received request to transfer leadership to TS 9aec419c47a640edb247a753aaa6ad5b
I20260709 13:01:03.536098  7782 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bcb544d4966e409c97ce8eff34e7521e"
dest_uuid: "68c88031f63c411a9a349b00f53b211d"
mode: GRACEFUL
new_leader_uuid: "9aec419c47a640edb247a753aaa6ad5b"
 from {username='slave'} at 127.0.0.1:37936
I20260709 13:01:03.536516  7782 raft_consensus.cc:606] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 2 LEADER]: Received request to transfer leadership to TS 9aec419c47a640edb247a753aaa6ad5b
I20260709 13:01:03.537405  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:01:03.537858  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:03.720559  7976 raft_consensus.cc:993] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d: : Instructing follower 9aec419c47a640edb247a753aaa6ad5b to start an election
I20260709 13:01:03.720956  7976 raft_consensus.cc:1081] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [term 1 LEADER]: Signalling peer 9aec419c47a640edb247a753aaa6ad5b to start an election
I20260709 13:01:03.722543  7930 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "55538fcb866c4dd1a3cd12db54eb946b"
dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
 from {username='slave'} at 127.0.0.1:42896
I20260709 13:01:03.722961  7930 raft_consensus.cc:493] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:03.723233  7930 raft_consensus.cc:3060] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:03.727330  7930 raft_consensus.cc:515] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:03.728781  7930 leader_election.cc:290] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 2 election: Requested vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185)
I20260709 13:01:03.729506  7782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55538fcb866c4dd1a3cd12db54eb946b" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "68c88031f63c411a9a349b00f53b211d"
I20260709 13:01:03.729813  7856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55538fcb866c4dd1a3cd12db54eb946b" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
I20260709 13:01:03.730052  7782 raft_consensus.cc:3055] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:03.730357  7856 raft_consensus.cc:3060] T 55538fcb866c4dd1a3cd12db54eb946b P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:03.730468  7782 raft_consensus.cc:740] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Running, Role: LEADER
I20260709 13:01:03.731333  7782 consensus_queue.cc:260] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:03.732478  7782 raft_consensus.cc:3060] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:03.735925  7856 raft_consensus.cc:2468] T 55538fcb866c4dd1a3cd12db54eb946b P c4ca9afe41ff4eb298062aa06f346b95 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aec419c47a640edb247a753aaa6ad5b in term 2.
I20260709 13:01:03.737030  7895 leader_election.cc:304] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aec419c47a640edb247a753aaa6ad5b, c4ca9afe41ff4eb298062aa06f346b95; no voters: 
I20260709 13:01:03.737792  7978 raft_consensus.cc:2804] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:03.737784  7782 raft_consensus.cc:2468] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aec419c47a640edb247a753aaa6ad5b in term 2.
I20260709 13:01:03.739075  7978 raft_consensus.cc:697] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [term 2 LEADER]: Becoming Leader. State: Replica: 9aec419c47a640edb247a753aaa6ad5b, State: Running, Role: LEADER
I20260709 13:01:03.740003  7978 consensus_queue.cc:237] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:03.746977  7677 catalog_manager.cc:5697] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b reported cstate change: term changed from 1 to 2, leader changed from 68c88031f63c411a9a349b00f53b211d (127.6.255.1) to 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3). New cstate: current_term: 2 leader_uuid: "9aec419c47a640edb247a753aaa6ad5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:03.838874  8063 raft_consensus.cc:993] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d: : Instructing follower 9aec419c47a640edb247a753aaa6ad5b to start an election
I20260709 13:01:03.839411  8063 raft_consensus.cc:1081] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 2 LEADER]: Signalling peer 9aec419c47a640edb247a753aaa6ad5b to start an election
I20260709 13:01:03.841015  7930 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bcb544d4966e409c97ce8eff34e7521e"
dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
 from {username='slave'} at 127.0.0.1:42896
I20260709 13:01:03.841583  7930 raft_consensus.cc:493] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:03.841892  7930 raft_consensus.cc:3060] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:03.845858  7930 raft_consensus.cc:515] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:03.847407  7930 leader_election.cc:290] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 3 election: Requested vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185)
I20260709 13:01:03.848340  7782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb544d4966e409c97ce8eff34e7521e" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "68c88031f63c411a9a349b00f53b211d"
I20260709 13:01:03.848922  7782 raft_consensus.cc:3055] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:03.849239  7782 raft_consensus.cc:740] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Running, Role: LEADER
I20260709 13:01:03.849339  7856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb544d4966e409c97ce8eff34e7521e" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
I20260709 13:01:03.849908  7856 raft_consensus.cc:3060] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:03.850147  7782 consensus_queue.cc:260] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:03.851409  7782 raft_consensus.cc:3060] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:03.856678  7856 raft_consensus.cc:2468] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aec419c47a640edb247a753aaa6ad5b in term 3.
I20260709 13:01:03.858001  7782 raft_consensus.cc:2468] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aec419c47a640edb247a753aaa6ad5b in term 3.
I20260709 13:01:03.857963  7895 leader_election.cc:304] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aec419c47a640edb247a753aaa6ad5b, c4ca9afe41ff4eb298062aa06f346b95; no voters: 
I20260709 13:01:03.858784  7978 raft_consensus.cc:2804] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:03.859282  7978 raft_consensus.cc:697] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 3 LEADER]: Becoming Leader. State: Replica: 9aec419c47a640edb247a753aaa6ad5b, State: Running, Role: LEADER
I20260709 13:01:03.859987  7978 consensus_queue.cc:237] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:03.867585  7677 catalog_manager.cc:5697] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b reported cstate change: term changed from 2 to 3, leader changed from 68c88031f63c411a9a349b00f53b211d (127.6.255.1) to 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3). New cstate: current_term: 3 leader_uuid: "9aec419c47a640edb247a753aaa6ad5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:04.138993  7782 raft_consensus.cc:1275] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Refusing update from remote peer 9aec419c47a640edb247a753aaa6ad5b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:04.140396  7978 consensus_queue.cc:1048] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:04.153645  7856 raft_consensus.cc:1275] T 55538fcb866c4dd1a3cd12db54eb946b P c4ca9afe41ff4eb298062aa06f346b95 [term 2 FOLLOWER]: Refusing update from remote peer 9aec419c47a640edb247a753aaa6ad5b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:04.155131  8064 consensus_queue.cc:1048] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:04.438336  7782 raft_consensus.cc:1275] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 3 FOLLOWER]: Refusing update from remote peer 9aec419c47a640edb247a753aaa6ad5b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:04.439934  8073 consensus_queue.cc:1048] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:04.449338  7858 raft_consensus.cc:1275] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95 [term 3 FOLLOWER]: Refusing update from remote peer 9aec419c47a640edb247a753aaa6ad5b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:04.451141  7978 consensus_queue.cc:1048] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:04.538623  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:04.554657  7932 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bcb544d4966e409c97ce8eff34e7521e"
dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
mode: GRACEFUL
new_leader_uuid: "68c88031f63c411a9a349b00f53b211d"
 from {username='slave'} at 127.0.0.1:42908
I20260709 13:01:04.555133  7932 raft_consensus.cc:606] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 3 LEADER]: Received request to transfer leadership to TS 68c88031f63c411a9a349b00f53b211d
I20260709 13:01:04.559862  7932 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "55538fcb866c4dd1a3cd12db54eb946b"
dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
mode: GRACEFUL
new_leader_uuid: "68c88031f63c411a9a349b00f53b211d"
 from {username='slave'} at 127.0.0.1:42908
I20260709 13:01:04.560297  7932 raft_consensus.cc:606] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [term 2 LEADER]: Received request to transfer leadership to TS 68c88031f63c411a9a349b00f53b211d
I20260709 13:01:04.575377  7858 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4ced2215e2544faa2ecda7ceba5de27"
dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
mode: GRACEFUL
new_leader_uuid: "68c88031f63c411a9a349b00f53b211d"
 from {username='slave'} at 127.0.0.1:33292
I20260709 13:01:04.575866  7858 raft_consensus.cc:606] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [term 2 LEADER]: Received request to transfer leadership to TS 68c88031f63c411a9a349b00f53b211d
I20260709 13:01:04.580475  7858 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "494fef1b35024d30b17d4c0aee058999"
dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
mode: GRACEFUL
new_leader_uuid: "68c88031f63c411a9a349b00f53b211d"
 from {username='slave'} at 127.0.0.1:33292
I20260709 13:01:04.580921  7858 raft_consensus.cc:606] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [term 2 LEADER]: Received request to transfer leadership to TS 68c88031f63c411a9a349b00f53b211d
I20260709 13:01:04.585630  7858 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d8273641a344bfd96c1bee2c72380ef"
dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
mode: GRACEFUL
new_leader_uuid: "68c88031f63c411a9a349b00f53b211d"
 from {username='slave'} at 127.0.0.1:33292
I20260709 13:01:04.586048  7858 raft_consensus.cc:606] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [term 2 LEADER]: Received request to transfer leadership to TS 68c88031f63c411a9a349b00f53b211d
I20260709 13:01:04.679718  7978 raft_consensus.cc:993] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b: : Instructing follower 68c88031f63c411a9a349b00f53b211d to start an election
I20260709 13:01:04.680183  7978 raft_consensus.cc:1081] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [term 2 LEADER]: Signalling peer 68c88031f63c411a9a349b00f53b211d to start an election
I20260709 13:01:04.681617  7782 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "55538fcb866c4dd1a3cd12db54eb946b"
dest_uuid: "68c88031f63c411a9a349b00f53b211d"
 from {username='slave'} at 127.0.0.1:54240
I20260709 13:01:04.682034  7782 raft_consensus.cc:493] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:04.682296  7782 raft_consensus.cc:3060] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:04.686920  7782 raft_consensus.cc:515] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:04.688534  7782 leader_election.cc:290] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 3 election: Requested vote from peers c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
I20260709 13:01:04.689342  7858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55538fcb866c4dd1a3cd12db54eb946b" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
I20260709 13:01:04.689695  7932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55538fcb866c4dd1a3cd12db54eb946b" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
I20260709 13:01:04.689939  7858 raft_consensus.cc:3060] T 55538fcb866c4dd1a3cd12db54eb946b P c4ca9afe41ff4eb298062aa06f346b95 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:04.690217  7932 raft_consensus.cc:3055] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:04.690510  7932 raft_consensus.cc:740] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aec419c47a640edb247a753aaa6ad5b, State: Running, Role: LEADER
I20260709 13:01:04.691195  7932 consensus_queue.cc:260] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:04.692108  7932 raft_consensus.cc:3060] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:04.694582  7858 raft_consensus.cc:2468] T 55538fcb866c4dd1a3cd12db54eb946b P c4ca9afe41ff4eb298062aa06f346b95 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68c88031f63c411a9a349b00f53b211d in term 3.
I20260709 13:01:04.695580  7746 leader_election.cc:304] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d, c4ca9afe41ff4eb298062aa06f346b95; no voters: 
I20260709 13:01:04.696287  8063 raft_consensus.cc:2804] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:04.697012  7932 raft_consensus.cc:2468] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68c88031f63c411a9a349b00f53b211d in term 3.
I20260709 13:01:04.697679  8063 raft_consensus.cc:697] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [term 3 LEADER]: Becoming Leader. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Running, Role: LEADER
I20260709 13:01:04.698506  8063 consensus_queue.cc:237] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:04.705135  7678 catalog_manager.cc:5697] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d reported cstate change: term changed from 2 to 3, leader changed from 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3) to 68c88031f63c411a9a349b00f53b211d (127.6.255.1). New cstate: current_term: 3 leader_uuid: "68c88031f63c411a9a349b00f53b211d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:04.851274  8085 raft_consensus.cc:993] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95: : Instructing follower 68c88031f63c411a9a349b00f53b211d to start an election
I20260709 13:01:04.851864  7991 raft_consensus.cc:1081] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [term 2 LEADER]: Signalling peer 68c88031f63c411a9a349b00f53b211d to start an election
I20260709 13:01:04.853600  7782 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c4ced2215e2544faa2ecda7ceba5de27"
dest_uuid: "68c88031f63c411a9a349b00f53b211d"
 from {username='slave'} at 127.0.0.1:54248
I20260709 13:01:04.854171  7782 raft_consensus.cc:493] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:04.854473  7782 raft_consensus.cc:3060] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:04.858906  7782 raft_consensus.cc:515] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:04.860504  7782 leader_election.cc:290] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 3 election: Requested vote from peers c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
I20260709 13:01:04.861174  7858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4ced2215e2544faa2ecda7ceba5de27" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
I20260709 13:01:04.861510  7930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4ced2215e2544faa2ecda7ceba5de27" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
I20260709 13:01:04.861778  7858 raft_consensus.cc:3055] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:04.861982  7930 raft_consensus.cc:3060] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:04.862123  7858 raft_consensus.cc:740] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c4ca9afe41ff4eb298062aa06f346b95, State: Running, Role: LEADER
I20260709 13:01:04.862926  7858 consensus_queue.cc:260] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:04.863809  7858 raft_consensus.cc:3060] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:04.866088  7930 raft_consensus.cc:2468] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68c88031f63c411a9a349b00f53b211d in term 3.
I20260709 13:01:04.866988  7746 leader_election.cc:304] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d, 9aec419c47a640edb247a753aaa6ad5b; no voters: 
I20260709 13:01:04.867648  8063 raft_consensus.cc:2804] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:04.868045  8063 raft_consensus.cc:697] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 3 LEADER]: Becoming Leader. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Running, Role: LEADER
I20260709 13:01:04.868569  7858 raft_consensus.cc:2468] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68c88031f63c411a9a349b00f53b211d in term 3.
I20260709 13:01:04.868734  8063 consensus_queue.cc:237] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:04.876652  7678 catalog_manager.cc:5697] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d reported cstate change: term changed from 2 to 3, leader changed from c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2) to 68c88031f63c411a9a349b00f53b211d (127.6.255.1). New cstate: current_term: 3 leader_uuid: "68c88031f63c411a9a349b00f53b211d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:04.893110  8085 raft_consensus.cc:993] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95: : Instructing follower 68c88031f63c411a9a349b00f53b211d to start an election
I20260709 13:01:04.893615  7991 raft_consensus.cc:1081] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [term 2 LEADER]: Signalling peer 68c88031f63c411a9a349b00f53b211d to start an election
I20260709 13:01:04.895076  7782 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "494fef1b35024d30b17d4c0aee058999"
dest_uuid: "68c88031f63c411a9a349b00f53b211d"
 from {username='slave'} at 127.0.0.1:54248
I20260709 13:01:04.895545  7782 raft_consensus.cc:493] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:04.895917  7782 raft_consensus.cc:3060] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:04.900942  7782 raft_consensus.cc:515] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:04.902797  7782 leader_election.cc:290] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 3 election: Requested vote from peers c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
I20260709 13:01:04.903685  7858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "494fef1b35024d30b17d4c0aee058999" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
I20260709 13:01:04.904076  7930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "494fef1b35024d30b17d4c0aee058999" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
I20260709 13:01:04.904279  7858 raft_consensus.cc:3055] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:04.904614  7930 raft_consensus.cc:3060] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:04.904657  7858 raft_consensus.cc:740] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c4ca9afe41ff4eb298062aa06f346b95, State: Running, Role: LEADER
I20260709 13:01:04.905601  7858 consensus_queue.cc:260] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:04.906803  7858 raft_consensus.cc:3060] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:04.911026  7930 raft_consensus.cc:2468] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68c88031f63c411a9a349b00f53b211d in term 3.
I20260709 13:01:04.912257  7746 leader_election.cc:304] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d, 9aec419c47a640edb247a753aaa6ad5b; no voters: 
I20260709 13:01:04.913015  8063 raft_consensus.cc:2804] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:04.913404  7858 raft_consensus.cc:2468] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68c88031f63c411a9a349b00f53b211d in term 3.
I20260709 13:01:04.913435  8063 raft_consensus.cc:697] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [term 3 LEADER]: Becoming Leader. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Running, Role: LEADER
I20260709 13:01:04.914494  8063 consensus_queue.cc:237] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:04.921276  7678 catalog_manager.cc:5697] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d reported cstate change: term changed from 2 to 3, leader changed from c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2) to 68c88031f63c411a9a349b00f53b211d (127.6.255.1). New cstate: current_term: 3 leader_uuid: "68c88031f63c411a9a349b00f53b211d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:05.035718  8064 raft_consensus.cc:993] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b: : Instructing follower 68c88031f63c411a9a349b00f53b211d to start an election
I20260709 13:01:05.036120  8088 raft_consensus.cc:1081] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 3 LEADER]: Signalling peer 68c88031f63c411a9a349b00f53b211d to start an election
I20260709 13:01:05.037709  7782 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bcb544d4966e409c97ce8eff34e7521e"
dest_uuid: "68c88031f63c411a9a349b00f53b211d"
 from {username='slave'} at 127.0.0.1:54240
I20260709 13:01:05.038218  7782 raft_consensus.cc:493] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:05.038512  7782 raft_consensus.cc:3060] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:05.043504  7782 raft_consensus.cc:515] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:05.045176  7782 leader_election.cc:290] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 4 election: Requested vote from peers c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
I20260709 13:01:05.046072  7858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb544d4966e409c97ce8eff34e7521e" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
I20260709 13:01:05.046388  7930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb544d4966e409c97ce8eff34e7521e" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
I20260709 13:01:05.046672  7858 raft_consensus.cc:3060] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:05.046873  7930 raft_consensus.cc:3055] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:05.047175  7930 raft_consensus.cc:740] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aec419c47a640edb247a753aaa6ad5b, State: Running, Role: LEADER
I20260709 13:01:05.047899  7930 consensus_queue.cc:260] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:05.048978  7930 raft_consensus.cc:3060] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:05.053457  7858 raft_consensus.cc:2468] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68c88031f63c411a9a349b00f53b211d in term 4.
I20260709 13:01:05.054694  7746 leader_election.cc:304] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d, c4ca9afe41ff4eb298062aa06f346b95; no voters: 
I20260709 13:01:05.054981  7930 raft_consensus.cc:2468] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68c88031f63c411a9a349b00f53b211d in term 4.
I20260709 13:01:05.055392  8063 raft_consensus.cc:2804] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:05.055912  8063 raft_consensus.cc:697] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 4 LEADER]: Becoming Leader. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Running, Role: LEADER
I20260709 13:01:05.056675  8063 consensus_queue.cc:237] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:05.062431  7678 catalog_manager.cc:5697] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d reported cstate change: term changed from 3 to 4, leader changed from 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3) to 68c88031f63c411a9a349b00f53b211d (127.6.255.1). New cstate: current_term: 4 leader_uuid: "68c88031f63c411a9a349b00f53b211d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:05.079325  8085 raft_consensus.cc:993] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95: : Instructing follower 68c88031f63c411a9a349b00f53b211d to start an election
I20260709 13:01:05.079767  7991 raft_consensus.cc:1081] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [term 2 LEADER]: Signalling peer 68c88031f63c411a9a349b00f53b211d to start an election
I20260709 13:01:05.081303  7782 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5d8273641a344bfd96c1bee2c72380ef"
dest_uuid: "68c88031f63c411a9a349b00f53b211d"
 from {username='slave'} at 127.0.0.1:54248
I20260709 13:01:05.081810  7782 raft_consensus.cc:493] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:05.082064  7782 raft_consensus.cc:3060] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:05.086871  7782 raft_consensus.cc:515] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:05.088304  7782 leader_election.cc:290] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 3 election: Requested vote from peers c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
I20260709 13:01:05.089298  7858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d8273641a344bfd96c1bee2c72380ef" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
I20260709 13:01:05.089637  7930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d8273641a344bfd96c1bee2c72380ef" candidate_uuid: "68c88031f63c411a9a349b00f53b211d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
I20260709 13:01:05.089951  7858 raft_consensus.cc:3055] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:05.090161  7930 raft_consensus.cc:3060] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:05.090382  7858 raft_consensus.cc:740] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c4ca9afe41ff4eb298062aa06f346b95, State: Running, Role: LEADER
I20260709 13:01:05.091223  7858 consensus_queue.cc:260] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:05.092172  7858 raft_consensus.cc:3060] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:05.094614  7930 raft_consensus.cc:2468] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68c88031f63c411a9a349b00f53b211d in term 3.
I20260709 13:01:05.095575  7746 leader_election.cc:304] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c88031f63c411a9a349b00f53b211d, 9aec419c47a640edb247a753aaa6ad5b; no voters: 
I20260709 13:01:05.096329  8063 raft_consensus.cc:2804] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:05.096839  8063 raft_consensus.cc:697] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [term 3 LEADER]: Becoming Leader. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Running, Role: LEADER
I20260709 13:01:05.098069  7858 raft_consensus.cc:2468] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68c88031f63c411a9a349b00f53b211d in term 3.
I20260709 13:01:05.097767  8063 consensus_queue.cc:237] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:05.101651  7930 raft_consensus.cc:1275] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [term 3 FOLLOWER]: Refusing update from remote peer 68c88031f63c411a9a349b00f53b211d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:05.103235  8063 consensus_queue.cc:1048] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:05.105515  7678 catalog_manager.cc:5697] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d reported cstate change: term changed from 2 to 3, leader changed from c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2) to 68c88031f63c411a9a349b00f53b211d (127.6.255.1). New cstate: current_term: 3 leader_uuid: "68c88031f63c411a9a349b00f53b211d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:05.114024  7858 raft_consensus.cc:1275] T 55538fcb866c4dd1a3cd12db54eb946b P c4ca9afe41ff4eb298062aa06f346b95 [term 3 FOLLOWER]: Refusing update from remote peer 68c88031f63c411a9a349b00f53b211d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:05.118992  8063 consensus_queue.cc:1048] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:05.265369  7856 raft_consensus.cc:1275] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [term 3 FOLLOWER]: Refusing update from remote peer 68c88031f63c411a9a349b00f53b211d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:05.266656  8063 consensus_queue.cc:1048] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:05.276217  7930 raft_consensus.cc:1275] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b [term 3 FOLLOWER]: Refusing update from remote peer 68c88031f63c411a9a349b00f53b211d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:05.278188  8081 consensus_queue.cc:1048] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:05.376567  7856 raft_consensus.cc:1275] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [term 3 FOLLOWER]: Refusing update from remote peer 68c88031f63c411a9a349b00f53b211d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:05.377928  8081 consensus_queue.cc:1048] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:05.389358  7930 raft_consensus.cc:1275] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 3 FOLLOWER]: Refusing update from remote peer 68c88031f63c411a9a349b00f53b211d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:05.390611  8081 consensus_queue.cc:1048] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:05.517354  7930 raft_consensus.cc:1275] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [term 3 FOLLOWER]: Refusing update from remote peer 68c88031f63c411a9a349b00f53b211d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:05.518515  8081 consensus_queue.cc:1048] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:05.520121  7930 raft_consensus.cc:1275] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 4 FOLLOWER]: Refusing update from remote peer 68c88031f63c411a9a349b00f53b211d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:05.521180  8081 consensus_queue.cc:1048] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:05.527357  7856 raft_consensus.cc:1275] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [term 3 FOLLOWER]: Refusing update from remote peer 68c88031f63c411a9a349b00f53b211d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:05.529196  8063 consensus_queue.cc:1048] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:05.532781  7858 raft_consensus.cc:1275] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95 [term 4 FOLLOWER]: Refusing update from remote peer 68c88031f63c411a9a349b00f53b211d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:05.534759  8063 consensus_queue.cc:1048] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:07.593309  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:07.600629  7782 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "428de0e6e6d1418ebcbc8ff1fbf24a26"
dest_uuid: "68c88031f63c411a9a349b00f53b211d"
mode: GRACEFUL
new_leader_uuid: "9aec419c47a640edb247a753aaa6ad5b"
 from {username='slave'} at 127.0.0.1:37936
I20260709 13:01:07.601198  7782 raft_consensus.cc:606] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [term 2 LEADER]: Received request to transfer leadership to TS 9aec419c47a640edb247a753aaa6ad5b
I20260709 13:01:07.603969  7782 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "494fef1b35024d30b17d4c0aee058999"
dest_uuid: "68c88031f63c411a9a349b00f53b211d"
mode: GRACEFUL
new_leader_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
 from {username='slave'} at 127.0.0.1:37936
I20260709 13:01:07.604403  7782 raft_consensus.cc:606] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [term 3 LEADER]: Received request to transfer leadership to TS c4ca9afe41ff4eb298062aa06f346b95
I20260709 13:01:07.606670  7782 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c817cd2a564431e89c98f5500a9bd37"
dest_uuid: "68c88031f63c411a9a349b00f53b211d"
mode: GRACEFUL
new_leader_uuid: "9aec419c47a640edb247a753aaa6ad5b"
 from {username='slave'} at 127.0.0.1:37936
I20260709 13:01:07.607064  7782 raft_consensus.cc:606] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [term 1 LEADER]: Received request to transfer leadership to TS 9aec419c47a640edb247a753aaa6ad5b
I20260709 13:01:07.609145  7782 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4ced2215e2544faa2ecda7ceba5de27"
dest_uuid: "68c88031f63c411a9a349b00f53b211d"
mode: GRACEFUL
new_leader_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
 from {username='slave'} at 127.0.0.1:37936
I20260709 13:01:07.609652  7782 raft_consensus.cc:606] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 3 LEADER]: Received request to transfer leadership to TS c4ca9afe41ff4eb298062aa06f346b95
I20260709 13:01:07.610807  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:01:07.611297  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:07.809698  8081 raft_consensus.cc:993] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d: : Instructing follower 9aec419c47a640edb247a753aaa6ad5b to start an election
I20260709 13:01:07.810245  8104 raft_consensus.cc:1081] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [term 1 LEADER]: Signalling peer 9aec419c47a640edb247a753aaa6ad5b to start an election
I20260709 13:01:07.811621  7930 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c817cd2a564431e89c98f5500a9bd37"
dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
 from {username='slave'} at 127.0.0.1:42896
I20260709 13:01:07.812062  7930 raft_consensus.cc:493] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:07.812366  7930 raft_consensus.cc:3060] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.817370  7930 raft_consensus.cc:515] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:07.819101  7930 leader_election.cc:290] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 2 election: Requested vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185)
I20260709 13:01:07.820024  7782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c817cd2a564431e89c98f5500a9bd37" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "68c88031f63c411a9a349b00f53b211d"
I20260709 13:01:07.820410  7856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c817cd2a564431e89c98f5500a9bd37" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
I20260709 13:01:07.820729  7782 raft_consensus.cc:3055] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:07.820952  7856 raft_consensus.cc:3060] T 6c817cd2a564431e89c98f5500a9bd37 P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.821033  7782 raft_consensus.cc:740] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Running, Role: LEADER
I20260709 13:01:07.821911  7782 consensus_queue.cc:260] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:07.822953  7782 raft_consensus.cc:3060] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.825809  7856 raft_consensus.cc:2468] T 6c817cd2a564431e89c98f5500a9bd37 P c4ca9afe41ff4eb298062aa06f346b95 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aec419c47a640edb247a753aaa6ad5b in term 2.
I20260709 13:01:07.827042  7895 leader_election.cc:304] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aec419c47a640edb247a753aaa6ad5b, c4ca9afe41ff4eb298062aa06f346b95; no voters: 
I20260709 13:01:07.827852  8088 raft_consensus.cc:2804] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:07.828107  7782 raft_consensus.cc:2468] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aec419c47a640edb247a753aaa6ad5b in term 2.
I20260709 13:01:07.828902  8088 raft_consensus.cc:697] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b [term 2 LEADER]: Becoming Leader. State: Replica: 9aec419c47a640edb247a753aaa6ad5b, State: Running, Role: LEADER
I20260709 13:01:07.829864  8088 consensus_queue.cc:237] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:07.836673  7677 catalog_manager.cc:5697] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b reported cstate change: term changed from 1 to 2, leader changed from 68c88031f63c411a9a349b00f53b211d (127.6.255.1) to 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3). New cstate: current_term: 2 leader_uuid: "9aec419c47a640edb247a753aaa6ad5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:07.922866  8081 raft_consensus.cc:993] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d: : Instructing follower c4ca9afe41ff4eb298062aa06f346b95 to start an election
I20260709 13:01:07.923307  8081 raft_consensus.cc:1081] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [term 3 LEADER]: Signalling peer c4ca9afe41ff4eb298062aa06f346b95 to start an election
I20260709 13:01:07.924710  7856 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "494fef1b35024d30b17d4c0aee058999"
dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
 from {username='slave'} at 127.0.0.1:42594
I20260709 13:01:07.925242  7856 raft_consensus.cc:493] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:07.925565  7856 raft_consensus.cc:3060] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:07.930395  7856 raft_consensus.cc:515] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:07.931910  7856 leader_election.cc:290] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [CANDIDATE]: Term 4 election: Requested vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
I20260709 13:01:07.932741  7782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "494fef1b35024d30b17d4c0aee058999" candidate_uuid: "c4ca9afe41ff4eb298062aa06f346b95" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "68c88031f63c411a9a349b00f53b211d"
I20260709 13:01:07.932989  7930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "494fef1b35024d30b17d4c0aee058999" candidate_uuid: "c4ca9afe41ff4eb298062aa06f346b95" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
I20260709 13:01:07.933233  7782 raft_consensus.cc:3055] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:07.933519  7930 raft_consensus.cc:3060] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:07.933591  7782 raft_consensus.cc:740] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Running, Role: LEADER
I20260709 13:01:07.934430  7782 consensus_queue.cc:260] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:07.935588  7782 raft_consensus.cc:3060] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:07.938004  7930 raft_consensus.cc:2468] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4ca9afe41ff4eb298062aa06f346b95 in term 4.
I20260709 13:01:07.939119  7821 leader_election.cc:304] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aec419c47a640edb247a753aaa6ad5b, c4ca9afe41ff4eb298062aa06f346b95; no voters: 
I20260709 13:01:07.939924  7991 raft_consensus.cc:2804] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:07.940992  7782 raft_consensus.cc:2468] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4ca9afe41ff4eb298062aa06f346b95 in term 4.
I20260709 13:01:07.941052  7991 raft_consensus.cc:697] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [term 4 LEADER]: Becoming Leader. State: Replica: c4ca9afe41ff4eb298062aa06f346b95, State: Running, Role: LEADER
I20260709 13:01:07.942163  7991 consensus_queue.cc:237] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:07.949478  7677 catalog_manager.cc:5697] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 reported cstate change: term changed from 3 to 4, leader changed from 68c88031f63c411a9a349b00f53b211d (127.6.255.1) to c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2). New cstate: current_term: 4 leader_uuid: "c4ca9afe41ff4eb298062aa06f346b95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:08.013566  8104 raft_consensus.cc:993] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d: : Instructing follower 9aec419c47a640edb247a753aaa6ad5b to start an election
I20260709 13:01:08.014039  8104 raft_consensus.cc:1081] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [term 2 LEADER]: Signalling peer 9aec419c47a640edb247a753aaa6ad5b to start an election
I20260709 13:01:08.015625  7930 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "428de0e6e6d1418ebcbc8ff1fbf24a26"
dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
 from {username='slave'} at 127.0.0.1:42896
I20260709 13:01:08.016165  7930 raft_consensus.cc:493] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:08.016520  7930 raft_consensus.cc:3060] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:08.021064  7930 raft_consensus.cc:515] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } } peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } }
I20260709 13:01:08.022624  7930 leader_election.cc:290] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 3 election: Requested vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185)
I20260709 13:01:08.023727  7782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428de0e6e6d1418ebcbc8ff1fbf24a26" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "68c88031f63c411a9a349b00f53b211d"
I20260709 13:01:08.023811  7856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428de0e6e6d1418ebcbc8ff1fbf24a26" candidate_uuid: "9aec419c47a640edb247a753aaa6ad5b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
I20260709 13:01:08.024294  7782 raft_consensus.cc:3055] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:08.024317  7856 raft_consensus.cc:3060] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P c4ca9afe41ff4eb298062aa06f346b95 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:08.024814  7782 raft_consensus.cc:740] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Running, Role: LEADER
I20260709 13:01:08.025695  7782 consensus_queue.cc:260] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } } peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } }
I20260709 13:01:08.027012  7782 raft_consensus.cc:3060] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:08.031203  7856 raft_consensus.cc:2468] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P c4ca9afe41ff4eb298062aa06f346b95 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aec419c47a640edb247a753aaa6ad5b in term 3.
I20260709 13:01:08.032471  7895 leader_election.cc:304] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aec419c47a640edb247a753aaa6ad5b, c4ca9afe41ff4eb298062aa06f346b95; no voters: 
I20260709 13:01:08.033252  8088 raft_consensus.cc:2804] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:08.033715  8088 raft_consensus.cc:697] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 3 LEADER]: Becoming Leader. State: Replica: 9aec419c47a640edb247a753aaa6ad5b, State: Running, Role: LEADER
I20260709 13:01:08.034422  7782 raft_consensus.cc:2468] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aec419c47a640edb247a753aaa6ad5b in term 3.
I20260709 13:01:08.034557  8088 consensus_queue.cc:237] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } } peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } }
I20260709 13:01:08.041903  7677 catalog_manager.cc:5697] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b reported cstate change: term changed from 2 to 3, leader changed from 68c88031f63c411a9a349b00f53b211d (127.6.255.1) to 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3). New cstate: current_term: 3 leader_uuid: "9aec419c47a640edb247a753aaa6ad5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:08.044960  8081 raft_consensus.cc:993] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d: : Instructing follower c4ca9afe41ff4eb298062aa06f346b95 to start an election
I20260709 13:01:08.045410  8104 raft_consensus.cc:1081] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 3 LEADER]: Signalling peer c4ca9afe41ff4eb298062aa06f346b95 to start an election
I20260709 13:01:08.046981  7856 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c4ced2215e2544faa2ecda7ceba5de27"
dest_uuid: "c4ca9afe41ff4eb298062aa06f346b95"
 from {username='slave'} at 127.0.0.1:42594
I20260709 13:01:08.047410  7856 raft_consensus.cc:493] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:08.047734  7856 raft_consensus.cc:3060] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:08.052681  7856 raft_consensus.cc:515] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:08.054196  7856 leader_election.cc:290] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [CANDIDATE]: Term 4 election: Requested vote from peers 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239), 9aec419c47a640edb247a753aaa6ad5b (127.6.255.3:41505)
I20260709 13:01:08.054841  7782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4ced2215e2544faa2ecda7ceba5de27" candidate_uuid: "c4ca9afe41ff4eb298062aa06f346b95" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "68c88031f63c411a9a349b00f53b211d"
I20260709 13:01:08.055234  7930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4ced2215e2544faa2ecda7ceba5de27" candidate_uuid: "c4ca9afe41ff4eb298062aa06f346b95" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9aec419c47a640edb247a753aaa6ad5b"
I20260709 13:01:08.055365  7782 raft_consensus.cc:3055] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:08.055688  7782 raft_consensus.cc:740] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 68c88031f63c411a9a349b00f53b211d, State: Running, Role: LEADER
I20260709 13:01:08.055728  7930 raft_consensus.cc:3060] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:08.056345  7782 consensus_queue.cc:260] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:08.057453  7782 raft_consensus.cc:3060] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:08.060007  7930 raft_consensus.cc:2468] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4ca9afe41ff4eb298062aa06f346b95 in term 4.
I20260709 13:01:08.060945  7821 leader_election.cc:304] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aec419c47a640edb247a753aaa6ad5b, c4ca9afe41ff4eb298062aa06f346b95; no voters: 
I20260709 13:01:08.061733  7991 raft_consensus.cc:2804] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:08.062160  7991 raft_consensus.cc:697] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [term 4 LEADER]: Becoming Leader. State: Replica: c4ca9afe41ff4eb298062aa06f346b95, State: Running, Role: LEADER
I20260709 13:01:08.062198  7782 raft_consensus.cc:2468] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4ca9afe41ff4eb298062aa06f346b95 in term 4.
I20260709 13:01:08.063032  7991 consensus_queue.cc:237] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } }
I20260709 13:01:08.070534  7677 catalog_manager.cc:5697] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 reported cstate change: term changed from 3 to 4, leader changed from 68c88031f63c411a9a349b00f53b211d (127.6.255.1) to c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2). New cstate: current_term: 4 leader_uuid: "c4ca9afe41ff4eb298062aa06f346b95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:08.307966  7856 raft_consensus.cc:1275] T 6c817cd2a564431e89c98f5500a9bd37 P c4ca9afe41ff4eb298062aa06f346b95 [term 2 FOLLOWER]: Refusing update from remote peer 9aec419c47a640edb247a753aaa6ad5b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:08.309280  8088 consensus_queue.cc:1048] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:08.320214  7782 raft_consensus.cc:1275] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Refusing update from remote peer 9aec419c47a640edb247a753aaa6ad5b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:08.322019  8088 consensus_queue.cc:1048] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:08.395893  7782 raft_consensus.cc:1275] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [term 4 FOLLOWER]: Refusing update from remote peer c4ca9afe41ff4eb298062aa06f346b95: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:08.397152  7991 consensus_queue.cc:1048] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:08.405964  7930 raft_consensus.cc:1275] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 4 FOLLOWER]: Refusing update from remote peer c4ca9afe41ff4eb298062aa06f346b95: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:08.407603  8111 consensus_queue.cc:1048] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:08.420092  7856 raft_consensus.cc:1275] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P c4ca9afe41ff4eb298062aa06f346b95 [term 3 FOLLOWER]: Refusing update from remote peer 9aec419c47a640edb247a753aaa6ad5b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:08.421391  8088 consensus_queue.cc:1048] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4ca9afe41ff4eb298062aa06f346b95" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:08.430850  7782 raft_consensus.cc:1275] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [term 3 FOLLOWER]: Refusing update from remote peer 9aec419c47a640edb247a753aaa6ad5b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:08.433547  8105 consensus_queue.cc:1048] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:08.553911  7930 raft_consensus.cc:1275] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b [term 4 FOLLOWER]: Refusing update from remote peer c4ca9afe41ff4eb298062aa06f346b95: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:08.555029  8121 consensus_queue.cc:1048] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aec419c47a640edb247a753aaa6ad5b" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 41505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:08.566560  7782 raft_consensus.cc:1275] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 4 FOLLOWER]: Refusing update from remote peer c4ca9afe41ff4eb298062aa06f346b95: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:08.568084  8111 consensus_queue.cc:1048] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68c88031f63c411a9a349b00f53b211d" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 37239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:08.612092  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:08.618171  7164 tablet_server.cc:179] TabletServer@127.6.255.1:0 shutting down...
I20260709 13:01:08.643229  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:01:08.644068  7164 tablet_replica.cc:333] T 6e2882ee34dd4e6dbd67b64cea774679 P 68c88031f63c411a9a349b00f53b211d: stopping tablet replica
I20260709 13:01:08.644646  7164 raft_consensus.cc:2243] T 6e2882ee34dd4e6dbd67b64cea774679 P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:08.645123  7164 raft_consensus.cc:2272] T 6e2882ee34dd4e6dbd67b64cea774679 P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.647022  7164 tablet_replica.cc:333] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d: stopping tablet replica
I20260709 13:01:08.647526  7164 raft_consensus.cc:2243] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:08.647902  7164 raft_consensus.cc:2272] T c4ced2215e2544faa2ecda7ceba5de27 P 68c88031f63c411a9a349b00f53b211d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.649986  7164 tablet_replica.cc:333] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d: stopping tablet replica
I20260709 13:01:08.650496  7164 raft_consensus.cc:2243] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:01:08.651212  7164 raft_consensus.cc:2272] T 5d8273641a344bfd96c1bee2c72380ef P 68c88031f63c411a9a349b00f53b211d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.652837  7164 tablet_replica.cc:333] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d: stopping tablet replica
I20260709 13:01:08.653321  7164 raft_consensus.cc:2243] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:08.653811  7164 raft_consensus.cc:2272] T 494fef1b35024d30b17d4c0aee058999 P 68c88031f63c411a9a349b00f53b211d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.655923  7164 tablet_replica.cc:333] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d: stopping tablet replica
I20260709 13:01:08.656515  7164 raft_consensus.cc:2243] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:01:08.657218  7164 raft_consensus.cc:2272] T 20e3b50df0d7462da76007b962677370 P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.658869  7164 tablet_replica.cc:333] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d: stopping tablet replica
I20260709 13:01:08.659344  7164 raft_consensus.cc:2243] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:01:08.660039  7164 raft_consensus.cc:2272] T 55538fcb866c4dd1a3cd12db54eb946b P 68c88031f63c411a9a349b00f53b211d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.661752  7164 tablet_replica.cc:333] T 441ce35ef600431fa3fa881500cb9563 P 68c88031f63c411a9a349b00f53b211d: stopping tablet replica
I20260709 13:01:08.662228  7164 raft_consensus.cc:2243] T 441ce35ef600431fa3fa881500cb9563 P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:08.662674  7164 raft_consensus.cc:2272] T 441ce35ef600431fa3fa881500cb9563 P 68c88031f63c411a9a349b00f53b211d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.664331  7164 tablet_replica.cc:333] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d: stopping tablet replica
I20260709 13:01:08.664816  7164 raft_consensus.cc:2243] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:08.665195  7164 raft_consensus.cc:2272] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 68c88031f63c411a9a349b00f53b211d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.667146  7164 tablet_replica.cc:333] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d: stopping tablet replica
I20260709 13:01:08.667603  7164 raft_consensus.cc:2243] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:08.667973  7164 raft_consensus.cc:2272] T 6c817cd2a564431e89c98f5500a9bd37 P 68c88031f63c411a9a349b00f53b211d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.669883  7164 tablet_replica.cc:333] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d: stopping tablet replica
I20260709 13:01:08.670315  7164 raft_consensus.cc:2243] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:01:08.671032  7164 raft_consensus.cc:2272] T bcb544d4966e409c97ce8eff34e7521e P 68c88031f63c411a9a349b00f53b211d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.700452  7164 tablet_server.cc:196] TabletServer@127.6.255.1:0 shutdown complete.
I20260709 13:01:08.720520  7164 tablet_server.cc:179] TabletServer@127.6.255.2:0 shutting down...
W20260709 13:01:08.731663  7895 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 2 similar messages]
W20260709 13:01:08.735670  7895 consensus_peers.cc:597] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b -> Peer c4ca9afe41ff4eb298062aa06f346b95 (127.6.255.2:33185): Couldn't send request to peer c4ca9afe41ff4eb298062aa06f346b95. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:08.746675  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:01:08.747359  7164 tablet_replica.cc:333] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95: stopping tablet replica
I20260709 13:01:08.747962  7164 raft_consensus.cc:2243] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:01:08.748723  7164 raft_consensus.cc:2272] T c4ced2215e2544faa2ecda7ceba5de27 P c4ca9afe41ff4eb298062aa06f346b95 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.750614  7164 tablet_replica.cc:333] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95: stopping tablet replica
I20260709 13:01:08.751147  7164 raft_consensus.cc:2243] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:01:08.751861  7164 raft_consensus.cc:2272] T 494fef1b35024d30b17d4c0aee058999 P c4ca9afe41ff4eb298062aa06f346b95 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.753684  7164 tablet_replica.cc:333] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95: stopping tablet replica
I20260709 13:01:08.754169  7164 raft_consensus.cc:2243] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:08.754585  7164 raft_consensus.cc:2272] T 5d8273641a344bfd96c1bee2c72380ef P c4ca9afe41ff4eb298062aa06f346b95 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.756170  7164 tablet_replica.cc:333] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95: stopping tablet replica
I20260709 13:01:08.756641  7164 raft_consensus.cc:2243] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:08.757026  7164 raft_consensus.cc:2272] T bcb544d4966e409c97ce8eff34e7521e P c4ca9afe41ff4eb298062aa06f346b95 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.758642  7164 tablet_replica.cc:333] T 55538fcb866c4dd1a3cd12db54eb946b P c4ca9afe41ff4eb298062aa06f346b95: stopping tablet replica
I20260709 13:01:08.759110  7164 raft_consensus.cc:2243] T 55538fcb866c4dd1a3cd12db54eb946b P c4ca9afe41ff4eb298062aa06f346b95 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:08.759511  7164 raft_consensus.cc:2272] T 55538fcb866c4dd1a3cd12db54eb946b P c4ca9afe41ff4eb298062aa06f346b95 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.761133  7164 tablet_replica.cc:333] T 20e3b50df0d7462da76007b962677370 P c4ca9afe41ff4eb298062aa06f346b95: stopping tablet replica
I20260709 13:01:08.761669  7164 raft_consensus.cc:2243] T 20e3b50df0d7462da76007b962677370 P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:08.762070  7164 raft_consensus.cc:2272] T 20e3b50df0d7462da76007b962677370 P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.763590  7164 tablet_replica.cc:333] T 6e2882ee34dd4e6dbd67b64cea774679 P c4ca9afe41ff4eb298062aa06f346b95: stopping tablet replica
I20260709 13:01:08.764038  7164 raft_consensus.cc:2243] T 6e2882ee34dd4e6dbd67b64cea774679 P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:08.764402  7164 raft_consensus.cc:2272] T 6e2882ee34dd4e6dbd67b64cea774679 P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.765998  7164 tablet_replica.cc:333] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95: stopping tablet replica
I20260709 13:01:08.766453  7164 raft_consensus.cc:2243] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:01:08.767122  7164 raft_consensus.cc:2272] T 441ce35ef600431fa3fa881500cb9563 P c4ca9afe41ff4eb298062aa06f346b95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.768631  7164 tablet_replica.cc:333] T 6c817cd2a564431e89c98f5500a9bd37 P c4ca9afe41ff4eb298062aa06f346b95: stopping tablet replica
I20260709 13:01:08.769085  7164 raft_consensus.cc:2243] T 6c817cd2a564431e89c98f5500a9bd37 P c4ca9afe41ff4eb298062aa06f346b95 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:08.769501  7164 raft_consensus.cc:2272] T 6c817cd2a564431e89c98f5500a9bd37 P c4ca9afe41ff4eb298062aa06f346b95 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.772228  7164 tablet_replica.cc:333] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P c4ca9afe41ff4eb298062aa06f346b95: stopping tablet replica
I20260709 13:01:08.772807  7164 raft_consensus.cc:2243] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P c4ca9afe41ff4eb298062aa06f346b95 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:08.773258  7164 raft_consensus.cc:2272] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P c4ca9afe41ff4eb298062aa06f346b95 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:01:08.775943  7895 consensus_peers.cc:597] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b -> Peer 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239): Couldn't send request to peer 68c88031f63c411a9a349b00f53b211d. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:37239: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:08.802699  7164 tablet_server.cc:196] TabletServer@127.6.255.2:0 shutdown complete.
I20260709 13:01:08.820829  7164 tablet_server.cc:179] TabletServer@127.6.255.3:0 shutting down...
I20260709 13:01:08.843209  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:01:08.843946  7164 tablet_replica.cc:333] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b: stopping tablet replica
I20260709 13:01:08.844822  7164 raft_consensus.cc:2243] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:08.845517  7164 raft_consensus.cc:2272] T 55538fcb866c4dd1a3cd12db54eb946b P 9aec419c47a640edb247a753aaa6ad5b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.847246  7164 tablet_replica.cc:333] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b: stopping tablet replica
I20260709 13:01:08.847700  7164 raft_consensus.cc:2243] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:01:08.848369  7164 raft_consensus.cc:2272] T 6e2882ee34dd4e6dbd67b64cea774679 P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.850040  7164 tablet_replica.cc:333] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b: stopping tablet replica
I20260709 13:01:08.850481  7164 raft_consensus.cc:2243] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:08.850826  7164 raft_consensus.cc:2272] T bcb544d4966e409c97ce8eff34e7521e P 9aec419c47a640edb247a753aaa6ad5b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.852325  7164 tablet_replica.cc:333] T 20e3b50df0d7462da76007b962677370 P 9aec419c47a640edb247a753aaa6ad5b: stopping tablet replica
I20260709 13:01:08.852790  7164 raft_consensus.cc:2243] T 20e3b50df0d7462da76007b962677370 P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:08.853129  7164 raft_consensus.cc:2272] T 20e3b50df0d7462da76007b962677370 P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.854708  7164 tablet_replica.cc:333] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b: stopping tablet replica
I20260709 13:01:08.855139  7164 raft_consensus.cc:2243] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:08.855535  7164 raft_consensus.cc:2272] T 5d8273641a344bfd96c1bee2c72380ef P 9aec419c47a640edb247a753aaa6ad5b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.857003  7164 tablet_replica.cc:333] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b: stopping tablet replica
I20260709 13:01:08.857528  7164 raft_consensus.cc:2243] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:08.857897  7164 raft_consensus.cc:2272] T 494fef1b35024d30b17d4c0aee058999 P 9aec419c47a640edb247a753aaa6ad5b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.859740  7164 tablet_replica.cc:333] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b: stopping tablet replica
I20260709 13:01:08.860162  7164 raft_consensus.cc:2243] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:08.860509  7164 raft_consensus.cc:2272] T c4ced2215e2544faa2ecda7ceba5de27 P 9aec419c47a640edb247a753aaa6ad5b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.863008  7164 tablet_replica.cc:333] T 441ce35ef600431fa3fa881500cb9563 P 9aec419c47a640edb247a753aaa6ad5b: stopping tablet replica
I20260709 13:01:08.863720  7164 raft_consensus.cc:2243] T 441ce35ef600431fa3fa881500cb9563 P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:08.864230  7164 raft_consensus.cc:2272] T 441ce35ef600431fa3fa881500cb9563 P 9aec419c47a640edb247a753aaa6ad5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.866453  7164 tablet_replica.cc:333] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b: stopping tablet replica
I20260709 13:01:08.866963  7164 raft_consensus.cc:2243] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:01:08.866998  7895 consensus_peers.cc:597] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b -> Peer 68c88031f63c411a9a349b00f53b211d (127.6.255.1:37239): Couldn't send request to peer 68c88031f63c411a9a349b00f53b211d. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:37239: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:08.867852  7164 raft_consensus.cc:2272] T 6c817cd2a564431e89c98f5500a9bd37 P 9aec419c47a640edb247a753aaa6ad5b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.869755  7164 tablet_replica.cc:333] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b: stopping tablet replica
I20260709 13:01:08.870131  7164 raft_consensus.cc:2243] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:01:08.870752  7164 raft_consensus.cc:2272] T 428de0e6e6d1418ebcbc8ff1fbf24a26 P 9aec419c47a640edb247a753aaa6ad5b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.897951  7164 tablet_server.cc:196] TabletServer@127.6.255.3:0 shutdown complete.
I20260709 13:01:08.915880  7164 master.cc:562] Master@127.6.255.62:41293 shutting down...
I20260709 13:01:08.934666  7164 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 775e031c99604f14afc254078b5e195c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:01:08.935148  7164 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 775e031c99604f14afc254078b5e195c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.935417  7164 tablet_replica.cc:333] T 00000000000000000000000000000000 P 775e031c99604f14afc254078b5e195c: stopping tablet replica
I20260709 13:01:08.955610  7164 master.cc:584] Master@127.6.255.62:41293 shutdown complete.
68c88031f63c411a9a349b00f53b211d  5
9aec419c47a640edb247a753aaa6ad5b  1
c4ca9afe41ff4eb298062aa06f346b95  4
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10217 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:01:08.996624  7164 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.255.62:40337
I20260709 13:01:08.997810  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:09.003851  8131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:09.004429  8130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:09.004604  8133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:09.005020  7164 server_base.cc:1061] running on GCE node
I20260709 13:01:09.005873  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:09.006078  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:09.006197  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602069006186 us; error 0 us; skew 500 ppm
I20260709 13:01:09.006645  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:09.008993  7164 webserver.cc:533] Webserver started at http://127.6.255.62:34645/ using document root <none> and password file <none>
I20260709 13:01:09.009483  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:09.009653  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:09.009881  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:09.010955  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/master-0-root/instance:
uuid: "b3bd0e8d7a084e649e387e729c0adabc"
format_stamp: "Formatted at 2026-07-09 13:01:09 on dist-test-slave-rnv4"
I20260709 13:01:09.015257  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:01:09.018702  8138 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:09.019510  7164 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:01:09.019821  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/master-0-root
uuid: "b3bd0e8d7a084e649e387e729c0adabc"
format_stamp: "Formatted at 2026-07-09 13:01:09 on dist-test-slave-rnv4"
I20260709 13:01:09.020107  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:09.036521  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:09.037827  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:09.076745  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.62:40337
I20260709 13:01:09.076907  8189 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.62:40337 every 8 connection(s)
I20260709 13:01:09.080580  8190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.090394  8190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc: Bootstrap starting.
I20260709 13:01:09.094805  8190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:09.098768  8190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc: No bootstrap required, opened a new log
I20260709 13:01:09.100840  8190 raft_consensus.cc:359] T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3bd0e8d7a084e649e387e729c0adabc" member_type: VOTER }
I20260709 13:01:09.101248  8190 raft_consensus.cc:385] T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:09.101485  8190 raft_consensus.cc:740] T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3bd0e8d7a084e649e387e729c0adabc, State: Initialized, Role: FOLLOWER
I20260709 13:01:09.102134  8190 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3bd0e8d7a084e649e387e729c0adabc" member_type: VOTER }
I20260709 13:01:09.102586  8190 raft_consensus.cc:399] T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:01:09.102818  8190 raft_consensus.cc:493] T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:01:09.103083  8190 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:09.107250  8190 raft_consensus.cc:515] T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3bd0e8d7a084e649e387e729c0adabc" member_type: VOTER }
I20260709 13:01:09.107777  8190 leader_election.cc:304] T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3bd0e8d7a084e649e387e729c0adabc; no voters: 
I20260709 13:01:09.108964  8190 leader_election.cc:290] T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:01:09.109375  8193 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:09.110702  8193 raft_consensus.cc:697] T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc [term 1 LEADER]: Becoming Leader. State: Replica: b3bd0e8d7a084e649e387e729c0adabc, State: Running, Role: LEADER
I20260709 13:01:09.111297  8193 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3bd0e8d7a084e649e387e729c0adabc" member_type: VOTER }
I20260709 13:01:09.111980  8190 sys_catalog.cc:565] T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:01:09.116187  8194 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3bd0e8d7a084e649e387e729c0adabc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3bd0e8d7a084e649e387e729c0adabc" member_type: VOTER } }
I20260709 13:01:09.116580  8195 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3bd0e8d7a084e649e387e729c0adabc. Latest consensus state: current_term: 1 leader_uuid: "b3bd0e8d7a084e649e387e729c0adabc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3bd0e8d7a084e649e387e729c0adabc" member_type: VOTER } }
I20260709 13:01:09.117034  8194 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc [sys.catalog]: This master's current role is: LEADER
I20260709 13:01:09.117319  8195 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc [sys.catalog]: This master's current role is: LEADER
I20260709 13:01:09.119832  8199 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:01:09.124894  8199 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:01:09.128968  7164 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:01:09.133765  8199 catalog_manager.cc:1383] Generated new cluster ID: 287c380455ad46ccb3b1fac27f88030b
I20260709 13:01:09.134061  8199 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:01:09.146632  8199 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:01:09.147866  8199 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:01:09.160172  8199 catalog_manager.cc:6070] T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc: Generated new TSK 0
I20260709 13:01:09.160761  8199 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:01:09.195236  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:09.201146  8211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:09.202265  8212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:09.204308  8214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:09.205120  7164 server_base.cc:1061] running on GCE node
I20260709 13:01:09.206374  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:09.206552  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:09.206667  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602069206656 us; error 0 us; skew 500 ppm
I20260709 13:01:09.207113  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:09.209390  7164 webserver.cc:533] Webserver started at http://127.6.255.1:41707/ using document root <none> and password file <none>
I20260709 13:01:09.209882  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:09.210034  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:09.210309  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:09.211278  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-0-root/instance:
uuid: "8c42259d7c4b463a9f485f42c320c81b"
format_stamp: "Formatted at 2026-07-09 13:01:09 on dist-test-slave-rnv4"
I20260709 13:01:09.215688  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:01:09.219187  8219 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:09.219914  7164 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:01:09.220141  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-0-root
uuid: "8c42259d7c4b463a9f485f42c320c81b"
format_stamp: "Formatted at 2026-07-09 13:01:09 on dist-test-slave-rnv4"
I20260709 13:01:09.220391  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:09.238603  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:09.239799  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:09.241200  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:09.243454  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:09.243624  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:09.243855  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:09.244010  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:09.283766  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.1:44679
I20260709 13:01:09.283922  8281 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.1:44679 every 8 connection(s)
I20260709 13:01:09.288555  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:09.295792  8286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:09.297467  8287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:09.300765  7164 server_base.cc:1061] running on GCE node
W20260709 13:01:09.300993  8289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:09.301658  8282 heartbeater.cc:344] Connected to a master server at 127.6.255.62:40337
I20260709 13:01:09.302069  8282 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:09.302425  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:09.302659  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:09.302781  8282 heartbeater.cc:507] Master 127.6.255.62:40337 requested a full tablet report, sending...
I20260709 13:01:09.302917  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602069302901 us; error 0 us; skew 500 ppm
I20260709 13:01:09.303541  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:09.304975  8155 ts_manager.cc:194] Registered new tserver with Master: 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679)
I20260709 13:01:09.306396  7164 webserver.cc:533] Webserver started at http://127.6.255.2:46653/ using document root <none> and password file <none>
I20260709 13:01:09.306891  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:09.306872  8155 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45434
I20260709 13:01:09.307261  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:09.307603  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:09.308983  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-1-root/instance:
uuid: "ff7a6fe533e94c1bae555b4cae427728"
format_stamp: "Formatted at 2026-07-09 13:01:09 on dist-test-slave-rnv4"
I20260709 13:01:09.313413  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.001s
I20260709 13:01:09.316545  8294 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:09.317288  7164 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:01:09.317631  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-1-root
uuid: "ff7a6fe533e94c1bae555b4cae427728"
format_stamp: "Formatted at 2026-07-09 13:01:09 on dist-test-slave-rnv4"
I20260709 13:01:09.317915  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:09.335520  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:09.336692  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:09.338162  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:09.340309  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:09.340485  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:09.340747  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:09.340924  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:09.379580  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.2:38599
I20260709 13:01:09.379685  8356 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.2:38599 every 8 connection(s)
I20260709 13:01:09.384080  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:09.390347  8360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:09.392628  8361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:09.394011  7164 server_base.cc:1061] running on GCE node
W20260709 13:01:09.394912  8364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:09.395915  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:09.396157  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:09.396359  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602069396339 us; error 0 us; skew 500 ppm
I20260709 13:01:09.396474  8357 heartbeater.cc:344] Connected to a master server at 127.6.255.62:40337
I20260709 13:01:09.396858  8357 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:09.396988  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:09.397619  8357 heartbeater.cc:507] Master 127.6.255.62:40337 requested a full tablet report, sending...
I20260709 13:01:09.399708  7164 webserver.cc:533] Webserver started at http://127.6.255.3:46411/ using document root <none> and password file <none>
I20260709 13:01:09.399780  8155 ts_manager.cc:194] Registered new tserver with Master: ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:09.400274  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:09.400492  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:09.400777  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:09.401273  8155 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45448
I20260709 13:01:09.402027  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-2-root/instance:
uuid: "e64761f43e4240319c41117cf886d22f"
format_stamp: "Formatted at 2026-07-09 13:01:09 on dist-test-slave-rnv4"
I20260709 13:01:09.406420  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.003s
I20260709 13:01:09.409751  8368 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:09.410466  7164 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.002s
I20260709 13:01:09.410779  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-2-root
uuid: "e64761f43e4240319c41117cf886d22f"
format_stamp: "Formatted at 2026-07-09 13:01:09 on dist-test-slave-rnv4"
I20260709 13:01:09.411065  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:09.425673  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:09.426856  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:09.428310  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:09.430794  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:09.431028  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:09.431279  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:09.431460  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:09.470651  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.3:37367
I20260709 13:01:09.470738  8430 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.3:37367 every 8 connection(s)
I20260709 13:01:09.483575  8431 heartbeater.cc:344] Connected to a master server at 127.6.255.62:40337
I20260709 13:01:09.483933  8431 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:09.484594  8431 heartbeater.cc:507] Master 127.6.255.62:40337 requested a full tablet report, sending...
I20260709 13:01:09.486404  8155 ts_manager.cc:194] Registered new tserver with Master: e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:09.487346  7164 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013529442s
I20260709 13:01:09.487901  8155 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45456
I20260709 13:01:09.518478  8155 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45464:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:01:09.525931  8155 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:01:09.724079  8245 tablet_service.cc:1511] Processing CreateTablet for tablet 6a2e021fd91643b49aa8e553436c4b69 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:01:09.724931  8247 tablet_service.cc:1511] Processing CreateTablet for tablet a1fdbe7b73a540ddb2e5cb473a8404de (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:01:09.725585  8245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a2e021fd91643b49aa8e553436c4b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.726263  8247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1fdbe7b73a540ddb2e5cb473a8404de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.727671  8244 tablet_service.cc:1511] Processing CreateTablet for tablet 007e904f46ef4a1a978f97d557d2a65b (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:01:09.728274  8246 tablet_service.cc:1511] Processing CreateTablet for tablet bf584dd56b3149a0b92539bc7115815b (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:01:09.730010  8243 tablet_service.cc:1511] Processing CreateTablet for tablet 338bba401dcd4d8c9642cd7b255dea63 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:01:09.730199  8392 tablet_service.cc:1511] Processing CreateTablet for tablet 338bba401dcd4d8c9642cd7b255dea63 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:01:09.730618  8395 tablet_service.cc:1511] Processing CreateTablet for tablet bf584dd56b3149a0b92539bc7115815b (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:01:09.731630  8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 338bba401dcd4d8c9642cd7b255dea63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.732393  8242 tablet_service.cc:1511] Processing CreateTablet for tablet d32240ced15f4c36a9ce65ca5a13634a (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:01:09.732785  8394 tablet_service.cc:1511] Processing CreateTablet for tablet 6a2e021fd91643b49aa8e553436c4b69 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:01:09.734160  8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a2e021fd91643b49aa8e553436c4b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.734633  8241 tablet_service.cc:1511] Processing CreateTablet for tablet ae82ced6ff014453bbcfd5f9f14b1f8d (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:01:09.737178  8390 tablet_service.cc:1511] Processing CreateTablet for tablet ae82ced6ff014453bbcfd5f9f14b1f8d (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:01:09.738451  8390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae82ced6ff014453bbcfd5f9f14b1f8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.740886  8393 tablet_service.cc:1511] Processing CreateTablet for tablet 007e904f46ef4a1a978f97d557d2a65b (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:01:09.742635  8393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 007e904f46ef4a1a978f97d557d2a65b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.745116  8391 tablet_service.cc:1511] Processing CreateTablet for tablet d32240ced15f4c36a9ce65ca5a13634a (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:01:09.746866  8391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d32240ced15f4c36a9ce65ca5a13634a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.749526  8396 tablet_service.cc:1511] Processing CreateTablet for tablet a1fdbe7b73a540ddb2e5cb473a8404de (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:01:09.751207  8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1fdbe7b73a540ddb2e5cb473a8404de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.755158  8240 tablet_service.cc:1511] Processing CreateTablet for tablet 10cffa7c6411440ca77cee6731faca67 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:01:09.757844  8244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 007e904f46ef4a1a978f97d557d2a65b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.759033  8389 tablet_service.cc:1511] Processing CreateTablet for tablet 10cffa7c6411440ca77cee6731faca67 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:01:09.760294  8389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10cffa7c6411440ca77cee6731faca67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.760948  8239 tablet_service.cc:1511] Processing CreateTablet for tablet 6c6a6afa04ab437cbbd6011a11681763 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:01:09.762181  8239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c6a6afa04ab437cbbd6011a11681763. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.764573  8388 tablet_service.cc:1511] Processing CreateTablet for tablet 6c6a6afa04ab437cbbd6011a11681763 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:01:09.766074  8388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c6a6afa04ab437cbbd6011a11681763. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.766578  8238 tablet_service.cc:1511] Processing CreateTablet for tablet 09943d7e76a54d4e80eb181cac80bf22 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:01:09.767827  8238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09943d7e76a54d4e80eb181cac80bf22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.770311  8387 tablet_service.cc:1511] Processing CreateTablet for tablet 09943d7e76a54d4e80eb181cac80bf22 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:01:09.771660  8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09943d7e76a54d4e80eb181cac80bf22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.777818  8240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10cffa7c6411440ca77cee6731faca67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.781827  8242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d32240ced15f4c36a9ce65ca5a13634a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.785825  8241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae82ced6ff014453bbcfd5f9f14b1f8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.785867  8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf584dd56b3149a0b92539bc7115815b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.797981  8246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf584dd56b3149a0b92539bc7115815b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.798656  8243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 338bba401dcd4d8c9642cd7b255dea63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.803439  8395 tablet_service.cc:1511] Processing CreateTablet for tablet 2323b22f124b407c95caf5525949dd6c (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:01:09.804869  8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2323b22f124b407c95caf5525949dd6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.830031  8247 tablet_service.cc:1511] Processing CreateTablet for tablet 2323b22f124b407c95caf5525949dd6c (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:01:09.830873  8322 tablet_service.cc:1511] Processing CreateTablet for tablet a1fdbe7b73a540ddb2e5cb473a8404de (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:01:09.832813  8322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1fdbe7b73a540ddb2e5cb473a8404de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.833200  8320 tablet_service.cc:1511] Processing CreateTablet for tablet 6a2e021fd91643b49aa8e553436c4b69 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:01:09.833492  8316 tablet_service.cc:1511] Processing CreateTablet for tablet ae82ced6ff014453bbcfd5f9f14b1f8d (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:01:09.834488  8320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a2e021fd91643b49aa8e553436c4b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.835170  8316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae82ced6ff014453bbcfd5f9f14b1f8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.838976  8315 tablet_service.cc:1511] Processing CreateTablet for tablet 10cffa7c6411440ca77cee6731faca67 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:01:09.840639  8315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10cffa7c6411440ca77cee6731faca67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.842231  8321 tablet_service.cc:1511] Processing CreateTablet for tablet bf584dd56b3149a0b92539bc7115815b (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:01:09.843538  8321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf584dd56b3149a0b92539bc7115815b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.843899  8392 tablet_service.cc:1511] Processing CreateTablet for tablet 39384c6d69ad4f658d2e777dfdcbb49c (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:01:09.845201  8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39384c6d69ad4f658d2e777dfdcbb49c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.846522  8317 tablet_service.cc:1511] Processing CreateTablet for tablet d32240ced15f4c36a9ce65ca5a13634a (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:01:09.847846  8317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d32240ced15f4c36a9ce65ca5a13634a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.850566  8319 tablet_service.cc:1511] Processing CreateTablet for tablet 007e904f46ef4a1a978f97d557d2a65b (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:01:09.851864  8319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 007e904f46ef4a1a978f97d557d2a65b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.854924  8318 tablet_service.cc:1511] Processing CreateTablet for tablet 338bba401dcd4d8c9642cd7b255dea63 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:01:09.856645  8318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 338bba401dcd4d8c9642cd7b255dea63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.859572  8314 tablet_service.cc:1511] Processing CreateTablet for tablet 6c6a6afa04ab437cbbd6011a11681763 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:01:09.875088  8313 tablet_service.cc:1511] Processing CreateTablet for tablet 09943d7e76a54d4e80eb181cac80bf22 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:01:09.877705  8314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c6a6afa04ab437cbbd6011a11681763. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.881963  8313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09943d7e76a54d4e80eb181cac80bf22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.883585  8388 tablet_service.cc:1511] Processing CreateTablet for tablet 104e8624a4bd4222aa8e1be9cba98372 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:01:09.884925  8388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 104e8624a4bd4222aa8e1be9cba98372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.892673  8394 tablet_service.cc:1511] Processing CreateTablet for tablet 5c7b40c72696450b884fa3a4659ccd84 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:01:09.893285  8396 tablet_service.cc:1511] Processing CreateTablet for tablet 20b7fa75b43141c0ba1e5e08972a3a8f (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:01:09.894068  8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c7b40c72696450b884fa3a4659ccd84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.885229  8393 tablet_service.cc:1511] Processing CreateTablet for tablet ed318aa932504f70b21cfa8e646283ac (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:01:09.913883  8239 tablet_service.cc:1511] Processing CreateTablet for tablet 39384c6d69ad4f658d2e777dfdcbb49c (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:01:09.914628  8389 tablet_service.cc:1511] Processing CreateTablet for tablet 60c80a6b33dc431eac0fe35b37d4ea23 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:01:09.920884  8446 tablet_bootstrap.cc:492] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:09.925865  8247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2323b22f124b407c95caf5525949dd6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.925335  8245 tablet_service.cc:1511] Processing CreateTablet for tablet 104e8624a4bd4222aa8e1be9cba98372 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:01:09.946768  8239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39384c6d69ad4f658d2e777dfdcbb49c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.947743  8245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 104e8624a4bd4222aa8e1be9cba98372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.951642  8317 tablet_service.cc:1511] Processing CreateTablet for tablet 2323b22f124b407c95caf5525949dd6c (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:01:09.955160  8319 tablet_service.cc:1511] Processing CreateTablet for tablet 39384c6d69ad4f658d2e777dfdcbb49c (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:01:09.955874  8317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2323b22f124b407c95caf5525949dd6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.926256  8390 tablet_service.cc:1511] Processing CreateTablet for tablet cbcc5b6425934c0aa85546e0b33f4abd (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:01:09.957823  8238 tablet_service.cc:1511] Processing CreateTablet for tablet ed318aa932504f70b21cfa8e646283ac (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:01:09.958379  8322 tablet_service.cc:1511] Processing CreateTablet for tablet 60c80a6b33dc431eac0fe35b37d4ea23 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:01:09.963125  8242 tablet_service.cc:1511] Processing CreateTablet for tablet 5c7b40c72696450b884fa3a4659ccd84 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:01:09.963775  8392 tablet_service.cc:1511] Processing CreateTablet for tablet 1263b3742fd541f8bda5a6b6c10a8596 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:01:09.964422  8242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c7b40c72696450b884fa3a4659ccd84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.966918  8238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed318aa932504f70b21cfa8e646283ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.969002  8240 tablet_service.cc:1511] Processing CreateTablet for tablet 20b7fa75b43141c0ba1e5e08972a3a8f (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:01:09.970359  8240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20b7fa75b43141c0ba1e5e08972a3a8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.958853  8389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60c80a6b33dc431eac0fe35b37d4ea23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.980108  8319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39384c6d69ad4f658d2e777dfdcbb49c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.981786  8447 tablet_bootstrap.cc:492] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:09.984730  8448 tablet_bootstrap.cc:492] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:09.985275  8322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60c80a6b33dc431eac0fe35b37d4ea23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.001724  8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20b7fa75b43141c0ba1e5e08972a3a8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.002609  8320 tablet_service.cc:1511] Processing CreateTablet for tablet ed318aa932504f70b21cfa8e646283ac (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:01:10.003386  8246 tablet_service.cc:1511] Processing CreateTablet for tablet cbcc5b6425934c0aa85546e0b33f4abd (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:01:10.003899  8320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed318aa932504f70b21cfa8e646283ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.004752  8448 tablet_bootstrap.cc:654] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:10.005234  8316 tablet_service.cc:1511] Processing CreateTablet for tablet 104e8624a4bd4222aa8e1be9cba98372 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:01:10.006481  8316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 104e8624a4bd4222aa8e1be9cba98372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.006631  8390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbcc5b6425934c0aa85546e0b33f4abd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.007604  8315 tablet_service.cc:1511] Processing CreateTablet for tablet 5c7b40c72696450b884fa3a4659ccd84 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:01:10.008811  8315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c7b40c72696450b884fa3a4659ccd84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.011399  8393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed318aa932504f70b21cfa8e646283ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.016276  8243 tablet_service.cc:1511] Processing CreateTablet for tablet 60c80a6b33dc431eac0fe35b37d4ea23 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:01:10.016996  8387 tablet_service.cc:1511] Processing CreateTablet for tablet 4516247b88e34a1d89ec9eb29e0ecb7f (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:01:10.020156  8391 tablet_service.cc:1511] Processing CreateTablet for tablet 07268be9b33649e8b0e8bddf7a3bcabe (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:01:10.029757  8247 tablet_service.cc:1511] Processing CreateTablet for tablet 1263b3742fd541f8bda5a6b6c10a8596 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:01:10.031971  8446 tablet_bootstrap.cc:654] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:10.034633  8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4516247b88e34a1d89ec9eb29e0ecb7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.057284  8317 tablet_service.cc:1511] Processing CreateTablet for tablet 20b7fa75b43141c0ba1e5e08972a3a8f (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:01:10.037137  8241 tablet_service.cc:1511] Processing CreateTablet for tablet 4516247b88e34a1d89ec9eb29e0ecb7f (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:01:10.042740  8395 tablet_service.cc:1511] Processing CreateTablet for tablet 625b0bdeab5d48eaa06a46928cbad7e4 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:01:10.035079  8246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbcc5b6425934c0aa85546e0b33f4abd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.057777  8391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07268be9b33649e8b0e8bddf7a3bcabe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.058580  8321 tablet_service.cc:1511] Processing CreateTablet for tablet cbcc5b6425934c0aa85546e0b33f4abd (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:01:10.059825  8321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbcc5b6425934c0aa85546e0b33f4abd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.065752  8241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4516247b88e34a1d89ec9eb29e0ecb7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.067224  8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 625b0bdeab5d48eaa06a46928cbad7e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.067852  8318 tablet_service.cc:1511] Processing CreateTablet for tablet 1263b3742fd541f8bda5a6b6c10a8596 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:01:10.068781  8243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60c80a6b33dc431eac0fe35b37d4ea23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.069151  8318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1263b3742fd541f8bda5a6b6c10a8596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.071451  8447 tablet_bootstrap.cc:654] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:10.078172  8247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1263b3742fd541f8bda5a6b6c10a8596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.079406  8244 tablet_service.cc:1511] Processing CreateTablet for tablet 625b0bdeab5d48eaa06a46928cbad7e4 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:01:10.080823  8244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 625b0bdeab5d48eaa06a46928cbad7e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.094885  8317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20b7fa75b43141c0ba1e5e08972a3a8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.098628  8238 tablet_service.cc:1511] Processing CreateTablet for tablet bf42cb9d00c6491eb6b0cf8b896f7c5b (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:01:10.078164  8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1263b3742fd541f8bda5a6b6c10a8596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.100091  8238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf42cb9d00c6491eb6b0cf8b896f7c5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.106159  8388 tablet_service.cc:1511] Processing CreateTablet for tablet bf42cb9d00c6491eb6b0cf8b896f7c5b (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:01:10.107988  8388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf42cb9d00c6491eb6b0cf8b896f7c5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.106235  8245 tablet_service.cc:1511] Processing CreateTablet for tablet 07268be9b33649e8b0e8bddf7a3bcabe (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:01:10.110612  8245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07268be9b33649e8b0e8bddf7a3bcabe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.113049  8242 tablet_service.cc:1511] Processing CreateTablet for tablet 82abcf8a2a574841972243a029be749d (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:01:10.114634  8242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82abcf8a2a574841972243a029be749d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.123639  8391 tablet_service.cc:1511] Processing CreateTablet for tablet 82abcf8a2a574841972243a029be749d (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:01:10.126972  8316 tablet_service.cc:1511] Processing CreateTablet for tablet 4516247b88e34a1d89ec9eb29e0ecb7f (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:01:10.128283  8316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4516247b88e34a1d89ec9eb29e0ecb7f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:01:10.131592  8203 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:10.133302  8314 tablet_service.cc:1511] Processing CreateTablet for tablet 07268be9b33649e8b0e8bddf7a3bcabe (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:01:10.134641  8314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07268be9b33649e8b0e8bddf7a3bcabe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.134716  8313 tablet_service.cc:1511] Processing CreateTablet for tablet 625b0bdeab5d48eaa06a46928cbad7e4 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:01:10.125031  8448 tablet_bootstrap.cc:492] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:10.141846  8448 ts_tablet_manager.cc:1403] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.157s	user 0.015s	sys 0.000s
I20260709 13:01:10.144986  8448 raft_consensus.cc:359] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:10.153149  8448 raft_consensus.cc:385] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:10.158444  8313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 625b0bdeab5d48eaa06a46928cbad7e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.158910  8240 tablet_service.cc:1511] Processing CreateTablet for tablet fade5d45b9804cc98377ac6ab69be94b (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:01:10.160161  8240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fade5d45b9804cc98377ac6ab69be94b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.182957  8446 tablet_bootstrap.cc:492] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:10.155278  8448 raft_consensus.cc:740] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:10.173418  8318 tablet_service.cc:1511] Processing CreateTablet for tablet bf42cb9d00c6491eb6b0cf8b896f7c5b (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:01:10.202980  8391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82abcf8a2a574841972243a029be749d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.209226  8318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf42cb9d00c6491eb6b0cf8b896f7c5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.202593  8242 tablet_service.cc:1511] Processing CreateTablet for tablet e9159db9f370400d98f8503d06a33a33 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:01:10.204404  8448 consensus_queue.cc:260] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:10.204629  8390 tablet_service.cc:1511] Processing CreateTablet for tablet fade5d45b9804cc98377ac6ab69be94b (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:01:10.218338  8390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fade5d45b9804cc98377ac6ab69be94b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.180730  8321 tablet_service.cc:1511] Processing CreateTablet for tablet 82abcf8a2a574841972243a029be749d (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:01:10.204873  8319 tablet_service.cc:1511] Processing CreateTablet for tablet e9159db9f370400d98f8503d06a33a33 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:01:10.197156  8446 ts_tablet_manager.cc:1403] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.276s	user 0.010s	sys 0.017s
I20260709 13:01:10.218360  8242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9159db9f370400d98f8503d06a33a33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.229928  8387 tablet_service.cc:1511] Processing CreateTablet for tablet e9159db9f370400d98f8503d06a33a33 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:01:10.231904  8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9159db9f370400d98f8503d06a33a33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.239357  8447 tablet_bootstrap.cc:492] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:10.239930  8447 ts_tablet_manager.cc:1403] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.259s	user 0.022s	sys 0.020s
I20260709 13:01:10.239989  8446 raft_consensus.cc:359] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:10.240777  8446 raft_consensus.cc:385] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:10.241072  8446 raft_consensus.cc:740] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:10.243021  8447 raft_consensus.cc:359] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:10.243752  8447 raft_consensus.cc:385] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:10.244091  8447 raft_consensus.cc:740] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:10.253706  8447 consensus_queue.cc:260] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:10.254665  8321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82abcf8a2a574841972243a029be749d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.261902  8315 tablet_service.cc:1511] Processing CreateTablet for tablet fade5d45b9804cc98377ac6ab69be94b (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:01:10.262671  8448 ts_tablet_manager.cc:1434] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.121s	user 0.012s	sys 0.033s
I20260709 13:01:10.263485  8357 heartbeater.cc:499] Master 127.6.255.62:40337 was elected leader, sending a full tablet report...
I20260709 13:01:10.263774  8448 tablet_bootstrap.cc:492] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:10.264712  8319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9159db9f370400d98f8503d06a33a33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.265969  8322 tablet_service.cc:1511] Processing CreateTablet for tablet 430fa172910841a49c0c80f2cbf76ae5 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:01:10.267124  8315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fade5d45b9804cc98377ac6ab69be94b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.267792  8322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 430fa172910841a49c0c80f2cbf76ae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.273649  8446 consensus_queue.cc:260] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:10.283532  8391 tablet_service.cc:1511] Processing CreateTablet for tablet 430fa172910841a49c0c80f2cbf76ae5 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:01:10.285061  8391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 430fa172910841a49c0c80f2cbf76ae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.291499  8238 tablet_service.cc:1511] Processing CreateTablet for tablet cb79894cecb541e3b7dd164271cadc16 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:01:10.293282  8238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb79894cecb541e3b7dd164271cadc16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.309816  8282 heartbeater.cc:499] Master 127.6.255.62:40337 was elected leader, sending a full tablet report...
I20260709 13:01:10.328449  8318 tablet_service.cc:1511] Processing CreateTablet for tablet cb79894cecb541e3b7dd164271cadc16 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:01:10.329757  8318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb79894cecb541e3b7dd164271cadc16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.332064  8452 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.332533  8452 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:10.334568  8446 ts_tablet_manager.cc:1434] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.112s	user 0.007s	sys 0.002s
I20260709 13:01:10.335471  8431 heartbeater.cc:499] Master 127.6.255.62:40337 was elected leader, sending a full tablet report...
I20260709 13:01:10.342049  8446 tablet_bootstrap.cc:492] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:10.356236  8452 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:10.356859  8387 tablet_service.cc:1511] Processing CreateTablet for tablet cb79894cecb541e3b7dd164271cadc16 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:01:10.350432  8448 tablet_bootstrap.cc:654] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:10.364645  8316 tablet_service.cc:1511] Processing CreateTablet for tablet eeecf7303d634c0d8983b810bb97c33c (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:01:10.366330  8316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeecf7303d634c0d8983b810bb97c33c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.366619  8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb79894cecb541e3b7dd164271cadc16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.379087  8447 ts_tablet_manager.cc:1434] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.139s	user 0.030s	sys 0.057s
I20260709 13:01:10.384539  8246 tablet_service.cc:1511] Processing CreateTablet for tablet eeecf7303d634c0d8983b810bb97c33c (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:01:10.385881  8446 tablet_bootstrap.cc:654] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:10.386034  8246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeecf7303d634c0d8983b810bb97c33c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.387915  8393 tablet_service.cc:1511] Processing CreateTablet for tablet 766ac43473084aa6a9fdce530520947c (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:01:10.392128  8447 tablet_bootstrap.cc:492] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:10.391335  8241 tablet_service.cc:1511] Processing CreateTablet for tablet 766ac43473084aa6a9fdce530520947c (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:01:10.392478  8393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 766ac43473084aa6a9fdce530520947c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.393175  8241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 766ac43473084aa6a9fdce530520947c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.393463  8390 tablet_service.cc:1511] Processing CreateTablet for tablet eeecf7303d634c0d8983b810bb97c33c (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:01:10.394811  8390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeecf7303d634c0d8983b810bb97c33c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.399017  8447 tablet_bootstrap.cc:654] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:10.382709  8240 tablet_service.cc:1511] Processing CreateTablet for tablet 430fa172910841a49c0c80f2cbf76ae5 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:01:10.406431  8244 tablet_service.cc:1511] Processing CreateTablet for tablet 757eb9e36091440581fbfc9a6b000e74 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:01:10.407675  8244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 757eb9e36091440581fbfc9a6b000e74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.409085  8388 tablet_service.cc:1511] Processing CreateTablet for tablet 757eb9e36091440581fbfc9a6b000e74 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:01:10.412825  8388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 757eb9e36091440581fbfc9a6b000e74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.413746  8313 tablet_service.cc:1511] Processing CreateTablet for tablet 757eb9e36091440581fbfc9a6b000e74 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:01:10.415333  8313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 757eb9e36091440581fbfc9a6b000e74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.446863  8240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 430fa172910841a49c0c80f2cbf76ae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.463320  8453 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.463882  8453 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:10.391332  8314 tablet_service.cc:1511] Processing CreateTablet for tablet 766ac43473084aa6a9fdce530520947c (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:01:10.468400  8314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 766ac43473084aa6a9fdce530520947c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.473934  8238 tablet_service.cc:1511] Processing CreateTablet for tablet 954d2ad09dcb4697adccd661625a841b (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:01:10.475647  8238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 954d2ad09dcb4697adccd661625a841b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.481953  8453 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:10.500730  8390 tablet_service.cc:1511] Processing CreateTablet for tablet 954d2ad09dcb4697adccd661625a841b (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:01:10.507910  8396 tablet_service.cc:1511] Processing CreateTablet for tablet 81804f59e3124d27998ea80fc1c36663 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:01:10.510452  8245 tablet_service.cc:1511] Processing CreateTablet for tablet 81804f59e3124d27998ea80fc1c36663 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:01:10.511888  8245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81804f59e3124d27998ea80fc1c36663. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.511981  8247 tablet_service.cc:1511] Processing CreateTablet for tablet 83839ff3ce80447bb72b3c8d15b630ff (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:01:10.513298  8247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83839ff3ce80447bb72b3c8d15b630ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.514550  8448 tablet_bootstrap.cc:492] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:10.515014  8448 ts_tablet_manager.cc:1403] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.252s	user 0.010s	sys 0.039s
I20260709 13:01:10.508986  8319 tablet_service.cc:1511] Processing CreateTablet for tablet 954d2ad09dcb4697adccd661625a841b (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:01:10.510452  8313 tablet_service.cc:1511] Processing CreateTablet for tablet 81804f59e3124d27998ea80fc1c36663 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:01:10.521782  8319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 954d2ad09dcb4697adccd661625a841b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.522439  8313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81804f59e3124d27998ea80fc1c36663. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.525611  8322 tablet_service.cc:1511] Processing CreateTablet for tablet 83839ff3ce80447bb72b3c8d15b630ff (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:01:10.527215  8322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83839ff3ce80447bb72b3c8d15b630ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.529999  8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81804f59e3124d27998ea80fc1c36663. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.533793  8390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 954d2ad09dcb4697adccd661625a841b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.555649  8395 tablet_service.cc:1511] Processing CreateTablet for tablet 83839ff3ce80447bb72b3c8d15b630ff (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:01:10.559823  8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83839ff3ce80447bb72b3c8d15b630ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.561345  8446 tablet_bootstrap.cc:492] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:10.561868  8446 ts_tablet_manager.cc:1403] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.220s	user 0.012s	sys 0.017s
I20260709 13:01:10.562232  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:01:10.569399  8297 leader_election.cc:343] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:10.571290  8447 tablet_bootstrap.cc:492] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:10.571751  8447 ts_tablet_manager.cc:1403] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.180s	user 0.012s	sys 0.031s
I20260709 13:01:10.572494  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:10.573040  8242 tablet_service.cc:1511] Processing CreateTablet for tablet ec275b714efb4b8b92919025fbd79b1f (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
W20260709 13:01:10.573959  8295 leader_election.cc:343] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:10.574424  8295 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:10.574967  8447 raft_consensus.cc:359] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:01:10.575539  8452 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:10.575726  8447 raft_consensus.cc:385] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:10.576187  8447 raft_consensus.cc:740] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:10.576916  8447 consensus_queue.cc:260] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:01:10.574671  8242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec275b714efb4b8b92919025fbd79b1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.582767  8446 raft_consensus.cc:359] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:10.583568  8446 raft_consensus.cc:385] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:10.583716  8447 ts_tablet_manager.cc:1434] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:01:10.583904  8446 raft_consensus.cc:740] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:10.584894  8447 tablet_bootstrap.cc:492] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:10.585597  8446 consensus_queue.cc:260] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:10.590246  8446 ts_tablet_manager.cc:1434] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.002s
I20260709 13:01:10.591118  8446 tablet_bootstrap.cc:492] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:10.600770  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:10.601420  8406 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:10.598263  8388 tablet_service.cc:1511] Processing CreateTablet for tablet ec275b714efb4b8b92919025fbd79b1f (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:01:10.603844  8388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec275b714efb4b8b92919025fbd79b1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.580597  8448 raft_consensus.cc:359] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:01:10.605072  8448 raft_consensus.cc:385] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:10.605437  8448 raft_consensus.cc:740] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:10.606357  8448 consensus_queue.cc:260] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:01:10.608733  8448 ts_tablet_manager.cc:1434] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.093s	user 0.007s	sys 0.000s
I20260709 13:01:10.609697  8448 tablet_bootstrap.cc:492] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:10.608863  8318 tablet_service.cc:1511] Processing CreateTablet for tablet 3c1d459f4b98433682d7d19a825eb4ef (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:01:10.609887  8222 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:10.611500  8321 tablet_service.cc:1511] Processing CreateTablet for tablet ec275b714efb4b8b92919025fbd79b1f (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:01:10.601215  8241 tablet_service.cc:1511] Processing CreateTablet for tablet 3c1d459f4b98433682d7d19a825eb4ef (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:01:10.613282  8241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c1d459f4b98433682d7d19a825eb4ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.617663  8453 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:10.618041  8453 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:10.618382  8453 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:10.618723  8318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c1d459f4b98433682d7d19a825eb4ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.631127  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:10.652693  8319 tablet_service.cc:1511] Processing CreateTablet for tablet 4b9b125a55cf4a5d9db580313045edde (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:01:10.645466  8244 tablet_service.cc:1511] Processing CreateTablet for tablet 4b9b125a55cf4a5d9db580313045edde (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:01:10.634897  8446 tablet_bootstrap.cc:654] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:10.658326  8244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b9b125a55cf4a5d9db580313045edde. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:01:10.658258  8220 leader_election.cc:343] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:10.625713  8453 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:10.659366  8388 tablet_service.cc:1511] Processing CreateTablet for tablet 3c1d459f4b98433682d7d19a825eb4ef (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:01:10.660611  8388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c1d459f4b98433682d7d19a825eb4ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.661121  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:01:10.662562  8220 leader_election.cc:343] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:10.622676  8447 tablet_bootstrap.cc:654] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:10.665563  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:10.656437  8391 tablet_service.cc:1511] Processing CreateTablet for tablet 4b9b125a55cf4a5d9db580313045edde (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:01:10.666406  8406 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:10.666951  8391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b9b125a55cf4a5d9db580313045edde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.673389  8247 tablet_service.cc:1511] Processing CreateTablet for tablet eeca7eb827a04189af4de94af55ffb91 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:01:10.674863  8247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeca7eb827a04189af4de94af55ffb91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.675045  8406 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:10.676375  8222 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:10.622274  8321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec275b714efb4b8b92919025fbd79b1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.681530  8319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b9b125a55cf4a5d9db580313045edde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.657282  8448 tablet_bootstrap.cc:654] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:10.689980  8453 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:10.694716  8453 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:10.695171  8453 raft_consensus.cc:697] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:10.696019  8453 consensus_queue.cc:237] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:10.704176  8454 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.704726  8454 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:10.707194  8313 tablet_service.cc:1511] Processing CreateTablet for tablet eeca7eb827a04189af4de94af55ffb91 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:01:10.708513  8313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeca7eb827a04189af4de94af55ffb91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.713231  8454 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:10.750574  8238 tablet_service.cc:1511] Processing CreateTablet for tablet b23a66e34de947348d5940084701c646 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:01:10.753010  8446 tablet_bootstrap.cc:492] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:10.752516  8238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b23a66e34de947348d5940084701c646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.753504  8446 ts_tablet_manager.cc:1403] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.163s	user 0.020s	sys 0.027s
I20260709 13:01:10.756707  8447 tablet_bootstrap.cc:492] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:10.757258  8447 ts_tablet_manager.cc:1403] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.173s	user 0.010s	sys 0.014s
I20260709 13:01:10.756934  8446 raft_consensus.cc:359] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:10.765686  8446 raft_consensus.cc:385] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:10.766108  8446 raft_consensus.cc:740] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:10.767086  8446 consensus_queue.cc:260] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:10.767661  8242 tablet_service.cc:1511] Processing CreateTablet for tablet 6efcdb47a4a24d3680780fb6deb13911 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:01:10.767895  8313 tablet_service.cc:1511] Processing CreateTablet for tablet b23a66e34de947348d5940084701c646 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:01:10.769100  8313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b23a66e34de947348d5940084701c646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.769100  8242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6efcdb47a4a24d3680780fb6deb13911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.770848  8446 ts_tablet_manager.cc:1434] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.001s
I20260709 13:01:10.772348  8446 tablet_bootstrap.cc:492] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:10.775508  8247 tablet_service.cc:1511] Processing CreateTablet for tablet e94bcf9755aa4f0abddcd5b9fbc2939f (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:01:10.776993  8247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e94bcf9755aa4f0abddcd5b9fbc2939f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.780898  8448 tablet_bootstrap.cc:492] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:10.781407  8448 ts_tablet_manager.cc:1403] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.172s	user 0.027s	sys 0.015s
I20260709 13:01:10.784315  8448 raft_consensus.cc:359] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:10.785028  8448 raft_consensus.cc:385] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:10.786396  8318 tablet_service.cc:1511] Processing CreateTablet for tablet e94bcf9755aa4f0abddcd5b9fbc2939f (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:01:10.788442  8318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e94bcf9755aa4f0abddcd5b9fbc2939f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.785329  8448 raft_consensus.cc:740] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:10.791467  8448 consensus_queue.cc:260] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:10.800956  8314 tablet_service.cc:1511] Processing CreateTablet for tablet 6efcdb47a4a24d3680780fb6deb13911 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:01:10.802312  8314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6efcdb47a4a24d3680780fb6deb13911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.808588  8452 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.809293  8452 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:10.812175  8452 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:10.808573  8447 raft_consensus.cc:359] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:10.813246  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:10.813755  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:01:10.815158  8295 leader_election.cc:343] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:10.815711  8297 leader_election.cc:343] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:10.816109  8297 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:10.817180  8452 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:10.813407  8447 raft_consensus.cc:385] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:10.819419  8447 raft_consensus.cc:740] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:10.820403  8447 consensus_queue.cc:260] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:10.823316  8447 ts_tablet_manager.cc:1434] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.001s
I20260709 13:01:10.824468  8447 tablet_bootstrap.cc:492] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:10.826212  8242 tablet_service.cc:1511] Processing CreateTablet for tablet 1112bcf4c4ea44e1933ee6287bc869f8 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:01:10.827502  8242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1112bcf4c4ea44e1933ee6287bc869f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.829131  8391 tablet_service.cc:1511] Processing CreateTablet for tablet b23a66e34de947348d5940084701c646 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:01:10.829391  8395 tablet_service.cc:1511] Processing CreateTablet for tablet e94bcf9755aa4f0abddcd5b9fbc2939f (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:01:10.830425  8391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b23a66e34de947348d5940084701c646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.830379  8396 tablet_service.cc:1511] Processing CreateTablet for tablet 1112bcf4c4ea44e1933ee6287bc869f8 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:01:10.831616  8390 tablet_service.cc:1511] Processing CreateTablet for tablet 6efcdb47a4a24d3680780fb6deb13911 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:01:10.832258  8388 tablet_service.cc:1511] Processing CreateTablet for tablet eeca7eb827a04189af4de94af55ffb91 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:01:10.832777  8319 tablet_service.cc:1511] Processing CreateTablet for tablet 1112bcf4c4ea44e1933ee6287bc869f8 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:01:10.833379  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:10.834342  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:01:10.835088  8369 leader_election.cc:343] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:10.835493  8446 tablet_bootstrap.cc:654] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:10.835803  8369 leader_election.cc:343] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:10.836349  8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1112bcf4c4ea44e1933ee6287bc869f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.836203  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:10.836974  8454 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:10.842048  8390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6efcdb47a4a24d3680780fb6deb13911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.850605  8448 ts_tablet_manager.cc:1434] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.069s	user 0.004s	sys 0.002s
I20260709 13:01:10.854619  8152 catalog_manager.cc:5697] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:10.834342  8319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1112bcf4c4ea44e1933ee6287bc869f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.858932  8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e94bcf9755aa4f0abddcd5b9fbc2939f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.859738  8238 tablet_service.cc:1511] Processing CreateTablet for tablet 56b5f14332d64504837ae4e2df77e409 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:01:10.861012  8238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56b5f14332d64504837ae4e2df77e409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.864849  8447 tablet_bootstrap.cc:654] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:10.865463  8448 tablet_bootstrap.cc:492] T 007e904f46ef4a1a978f97d557d2a65b P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:10.865898  8388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeca7eb827a04189af4de94af55ffb91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.882459  8318 tablet_service.cc:1511] Processing CreateTablet for tablet 56b5f14332d64504837ae4e2df77e409 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:01:10.887359  8318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56b5f14332d64504837ae4e2df77e409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.887894  8393 tablet_service.cc:1511] Processing CreateTablet for tablet 56b5f14332d64504837ae4e2df77e409 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:01:10.888870  8448 tablet_bootstrap.cc:654] T 007e904f46ef4a1a978f97d557d2a65b P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:10.889225  8393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56b5f14332d64504837ae4e2df77e409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.914556  8314 tablet_service.cc:1511] Processing CreateTablet for tablet b3d99c80602e45da9c43a49cf83b8211 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:01:10.917865  8314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3d99c80602e45da9c43a49cf83b8211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.918959  8446 tablet_bootstrap.cc:492] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:10.919550  8446 ts_tablet_manager.cc:1403] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.148s	user 0.006s	sys 0.006s
I20260709 13:01:10.922971  8238 tablet_service.cc:1511] Processing CreateTablet for tablet b3d99c80602e45da9c43a49cf83b8211 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:01:10.924741  8238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3d99c80602e45da9c43a49cf83b8211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.924993  8446 raft_consensus.cc:359] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:10.933903  8446 raft_consensus.cc:385] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:10.934262  8446 raft_consensus.cc:740] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:10.935015  8446 consensus_queue.cc:260] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:10.935377  8391 tablet_service.cc:1511] Processing CreateTablet for tablet b3d99c80602e45da9c43a49cf83b8211 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:01:10.935884  8247 tablet_service.cc:1511] Processing CreateTablet for tablet bac7f1e5192d48a98a850cb0ece17ad5 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:01:10.936695  8391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3d99c80602e45da9c43a49cf83b8211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.938031  8453 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.938745  8453 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:01:10.937260  8446 ts_tablet_manager.cc:1434] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:01:10.942523  8319 tablet_service.cc:1511] Processing CreateTablet for tablet bac7f1e5192d48a98a850cb0ece17ad5 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:01:10.943835  8319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bac7f1e5192d48a98a850cb0ece17ad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.943650  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:10.944375  8332 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:10.941888  8453 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:10.945587  8220 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:10.946533  8453 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:10.946942  8453 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:10.947335  8453 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:10.947650  8396 tablet_service.cc:1511] Processing CreateTablet for tablet bac7f1e5192d48a98a850cb0ece17ad5 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:01:10.948019  8242 tablet_service.cc:1511] Processing CreateTablet for tablet fdf12882ac9e4f7d9121a5fcb9d4efaf (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:01:10.948751  8247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bac7f1e5192d48a98a850cb0ece17ad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.949914  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:10.953380  8242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdf12882ac9e4f7d9121a5fcb9d4efaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.954429  8321 tablet_service.cc:1511] Processing CreateTablet for tablet fdf12882ac9e4f7d9121a5fcb9d4efaf (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:01:10.955585  8321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdf12882ac9e4f7d9121a5fcb9d4efaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.958420  8244 tablet_service.cc:1511] Processing CreateTablet for tablet b6e0ee9efb7d40309bc7e4d2450bb1b5 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:01:10.958726  8390 tablet_service.cc:1511] Processing CreateTablet for tablet fdf12882ac9e4f7d9121a5fcb9d4efaf (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:01:10.959695  8244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6e0ee9efb7d40309bc7e4d2450bb1b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.959403  8313 tablet_service.cc:1511] Processing CreateTablet for tablet b6e0ee9efb7d40309bc7e4d2450bb1b5 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:01:10.960639  8313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6e0ee9efb7d40309bc7e4d2450bb1b5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:01:10.961361  8222 leader_election.cc:343] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:10.952746  8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bac7f1e5192d48a98a850cb0ece17ad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.963814  8390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdf12882ac9e4f7d9121a5fcb9d4efaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.971431  8448 tablet_bootstrap.cc:492] T 007e904f46ef4a1a978f97d557d2a65b P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:10.971136  8387 tablet_service.cc:1511] Processing CreateTablet for tablet b6e0ee9efb7d40309bc7e4d2450bb1b5 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:01:10.972124  8448 ts_tablet_manager.cc:1403] T 007e904f46ef4a1a978f97d557d2a65b P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.107s	user 0.020s	sys 0.020s
I20260709 13:01:10.973009  8241 tablet_service.cc:1511] Processing CreateTablet for tablet 038cd0877a674c2996410c9f327df911 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:01:10.974241  8241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 038cd0877a674c2996410c9f327df911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.976789  8316 tablet_service.cc:1511] Processing CreateTablet for tablet 038cd0877a674c2996410c9f327df911 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:01:10.978044  8316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 038cd0877a674c2996410c9f327df911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.979179  8448 raft_consensus.cc:359] T 007e904f46ef4a1a978f97d557d2a65b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:10.981565  8448 raft_consensus.cc:385] T 007e904f46ef4a1a978f97d557d2a65b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:10.981974  8448 raft_consensus.cc:740] T 007e904f46ef4a1a978f97d557d2a65b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:10.983002  8448 consensus_queue.cc:260] T 007e904f46ef4a1a978f97d557d2a65b P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:10.985733  8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6e0ee9efb7d40309bc7e4d2450bb1b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.989034  8447 tablet_bootstrap.cc:492] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:10.989523  8447 ts_tablet_manager.cc:1403] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.165s	user 0.011s	sys 0.016s
I20260709 13:01:10.989284  8392 tablet_service.cc:1511] Processing CreateTablet for tablet 038cd0877a674c2996410c9f327df911 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:01:10.990559  8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 038cd0877a674c2996410c9f327df911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.991456  8446 tablet_bootstrap.cc:492] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:11.000412  8448 ts_tablet_manager.cc:1434] T 007e904f46ef4a1a978f97d557d2a65b P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.002s
I20260709 13:01:11.001814  8448 tablet_bootstrap.cc:492] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:11.009217  8447 raft_consensus.cc:359] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.010020  8447 raft_consensus.cc:385] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.010330  8447 raft_consensus.cc:740] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.013509  8447 consensus_queue.cc:260] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.015765  8447 ts_tablet_manager.cc:1434] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.001s
I20260709 13:01:11.017083  8448 tablet_bootstrap.cc:654] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.020794  8447 tablet_bootstrap.cc:492] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:11.024055  8241 tablet_service.cc:1511] Processing CreateTablet for tablet 98c2abe28fd449b094f4ccff4bac396e (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:01:11.025683  8241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98c2abe28fd449b094f4ccff4bac396e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.027777  8447 tablet_bootstrap.cc:654] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.058789  8446 tablet_bootstrap.cc:654] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.059715  8319 tablet_service.cc:1511] Processing CreateTablet for tablet 98c2abe28fd449b094f4ccff4bac396e (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:01:11.061154  8319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98c2abe28fd449b094f4ccff4bac396e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.063102  8453 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:01:11.070933  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:11.072937  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:11.073541  8332 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.065353  8453 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:11.082464  8238 tablet_service.cc:1511] Processing CreateTablet for tablet 9d36c3e19b8c47bc9d26b071486fdec9 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:01:11.083406  8238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d36c3e19b8c47bc9d26b071486fdec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.091893  8321 tablet_service.cc:1511] Processing CreateTablet for tablet 9d36c3e19b8c47bc9d26b071486fdec9 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
W20260709 13:01:11.088891  8222 leader_election.cc:343] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.104156  8332 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:11.104472  8395 tablet_service.cc:1511] Processing CreateTablet for tablet 98c2abe28fd449b094f4ccff4bac396e (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:01:11.105958  8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98c2abe28fd449b094f4ccff4bac396e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.106611  8393 tablet_service.cc:1511] Processing CreateTablet for tablet 9d36c3e19b8c47bc9d26b071486fdec9 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:01:11.107940  8393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d36c3e19b8c47bc9d26b071486fdec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.109604  8453 consensus_queue.cc:1048] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:11.109912  8220 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:01:11.110954  8467 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:11.111420  8467 raft_consensus.cc:697] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:11.112262  8467 consensus_queue.cc:237] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:01:11.117106  8321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d36c3e19b8c47bc9d26b071486fdec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.121034  8238 tablet_service.cc:1511] Processing CreateTablet for tablet f66b0119c0d84d8cb95b2252fcb6d390 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:01:11.124714  8238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f66b0119c0d84d8cb95b2252fcb6d390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.133864  8448 tablet_bootstrap.cc:492] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:11.134426  8448 ts_tablet_manager.cc:1403] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.133s	user 0.018s	sys 0.017s
I20260709 13:01:11.137641  8448 raft_consensus.cc:359] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.138474  8448 raft_consensus.cc:385] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.138804  8448 raft_consensus.cc:740] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.138953  8454 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.139449  8454 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.139737  8448 consensus_queue.cc:260] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.143155  8242 tablet_service.cc:1511] Processing CreateTablet for tablet f19e818631f543b28a8bec9ba4547527 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:01:11.143965  8454 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:11.144387  8242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f19e818631f543b28a8bec9ba4547527. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.145118  8313 tablet_service.cc:1511] Processing CreateTablet for tablet f66b0119c0d84d8cb95b2252fcb6d390 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:01:11.146034  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:11.146693  8332 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 0.
I20260709 13:01:11.147346  8447 tablet_bootstrap.cc:492] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:11.147786  8447 ts_tablet_manager.cc:1403] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.127s	user 0.024s	sys 0.032s
I20260709 13:01:11.147835  8369 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:11.148844  8454 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:11.149399  8454 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.149792  8454 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.150408  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:11.151136  8447 raft_consensus.cc:359] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.151881  8447 raft_consensus.cc:385] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.152235  8447 raft_consensus.cc:740] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.160303  8314 tablet_service.cc:1511] Processing CreateTablet for tablet b7c15354a2a84f868942919f011af448 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:01:11.162253  8390 tablet_service.cc:1511] Processing CreateTablet for tablet b7c15354a2a84f868942919f011af448 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:01:11.163611  8390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7c15354a2a84f868942919f011af448. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:01:11.165757  8369 leader_election.cc:343] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.166630  8448 ts_tablet_manager.cc:1434] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.000s
I20260709 13:01:11.164495  8316 tablet_service.cc:1511] Processing CreateTablet for tablet f19e818631f543b28a8bec9ba4547527 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:01:11.168388  8313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f66b0119c0d84d8cb95b2252fcb6d390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.169060  8314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7c15354a2a84f868942919f011af448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.176892  8316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f19e818631f543b28a8bec9ba4547527. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.176762  8241 tablet_service.cc:1511] Processing CreateTablet for tablet d966cf0fb9e24bd9993fb60505fdc62c (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:01:11.178048  8241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d966cf0fb9e24bd9993fb60505fdc62c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.182076  8448 tablet_bootstrap.cc:492] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:11.186789  8318 tablet_service.cc:1511] Processing CreateTablet for tablet d966cf0fb9e24bd9993fb60505fdc62c (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:01:11.188174  8318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d966cf0fb9e24bd9993fb60505fdc62c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.186823  8238 tablet_service.cc:1511] Processing CreateTablet for tablet b7c15354a2a84f868942919f011af448 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:01:11.189520  8238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7c15354a2a84f868942919f011af448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.152971  8447 consensus_queue.cc:260] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.216487  8467 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.227074  8392 tablet_service.cc:1511] Processing CreateTablet for tablet f66b0119c0d84d8cb95b2252fcb6d390 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:01:11.227578  8393 tablet_service.cc:1511] Processing CreateTablet for tablet f19e818631f543b28a8bec9ba4547527 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:01:11.228797  8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f66b0119c0d84d8cb95b2252fcb6d390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.221948  8454 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.218940  8452 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.233728  8393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f19e818631f543b28a8bec9ba4547527. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:01:11.237114  8203 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:11.252003  8390 tablet_service.cc:1511] Processing CreateTablet for tablet d966cf0fb9e24bd9993fb60505fdc62c (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:01:11.251029  8452 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.254269  8452 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:11.227144  8485 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.254994  8485 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.257037  8485 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:01:11.243870  8220 consensus_peers.cc:597] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:11.244076  8446 tablet_bootstrap.cc:492] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:11.261164  8454 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:11.261737  8446 ts_tablet_manager.cc:1403] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.293s	user 0.016s	sys 0.024s
I20260709 13:01:11.246171  8391 tablet_service.cc:1511] Processing CreateTablet for tablet a0493b0dbbc746408a5db9e698678d83 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:01:11.232661  8467 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.259791  8387 tablet_service.cc:1511] Processing CreateTablet for tablet 0c3340359c8445d8950a3e68e0070928 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:01:11.260281  8388 tablet_service.cc:1511] Processing CreateTablet for tablet 27e1a14c7c9f49e2b92fcea2a1bb191d (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:01:11.259791  8319 tablet_service.cc:1511] Processing CreateTablet for tablet a0493b0dbbc746408a5db9e698678d83 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:01:11.265043  8319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0493b0dbbc746408a5db9e698678d83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.265609  8467 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:11.266970  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:11.267561  8331 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.267751  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:01:11.269167  8220 leader_election.cc:343] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.260635  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:11.260286  8447 ts_tablet_manager.cc:1434] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.112s	user 0.008s	sys 0.001s
I20260709 13:01:11.271044  8447 tablet_bootstrap.cc:492] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
W20260709 13:01:11.271449  8369 leader_election.cc:343] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.257701  8448 tablet_bootstrap.cc:654] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.278663  8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c3340359c8445d8950a3e68e0070928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.279392  8388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27e1a14c7c9f49e2b92fcea2a1bb191d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.285594  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:11.286247  8406 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 0.
I20260709 13:01:11.288445  8297 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:11.289268  8452 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:11.289657  8452 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.289979  8452 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.295346  8447 tablet_bootstrap.cc:654] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.297691  8318 tablet_service.cc:1511] Processing CreateTablet for tablet 27e1a14c7c9f49e2b92fcea2a1bb191d (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:01:11.298147  8319 tablet_service.cc:1511] Processing CreateTablet for tablet 0c3340359c8445d8950a3e68e0070928 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
W20260709 13:01:11.298985  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 68857.2us
I20260709 13:01:11.302568  8318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27e1a14c7c9f49e2b92fcea2a1bb191d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.303507  8319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c3340359c8445d8950a3e68e0070928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.306962  8391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0493b0dbbc746408a5db9e698678d83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.307664  8446 raft_consensus.cc:359] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.308372  8446 raft_consensus.cc:385] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.308652  8446 raft_consensus.cc:740] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.309473  8446 consensus_queue.cc:260] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.312180  8390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d966cf0fb9e24bd9993fb60505fdc62c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.315914  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:11.316615  8406 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:11.302434  8331 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:11.324753  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:11.328523  8257 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 0.
I20260709 13:01:11.342170  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:11.351606  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:11.354429  8244 tablet_service.cc:1511] Processing CreateTablet for tablet 0c3340359c8445d8950a3e68e0070928 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:01:11.356333  8244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c3340359c8445d8950a3e68e0070928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.358687  8238 tablet_service.cc:1511] Processing CreateTablet for tablet 27e1a14c7c9f49e2b92fcea2a1bb191d (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:01:11.360333  8238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27e1a14c7c9f49e2b92fcea2a1bb191d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.362989  8242 tablet_service.cc:1511] Processing CreateTablet for tablet 1d3e4cc2f40346c0bc3bc739e3675685 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:01:11.341217  8369 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:11.364188  8242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d3e4cc2f40346c0bc3bc739e3675685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.364241  8454 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:11.364750  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:11.364976  8454 raft_consensus.cc:697] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:11.366158  8454 consensus_queue.cc:237] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.368646  8485 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:11.369141  8485 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.369596  8485 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:11.363000  8295 leader_election.cc:343] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.363134  8222 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:11.371953  8487 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:11.372364  8487 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.372706  8487 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.374647  8247 tablet_service.cc:1511] Processing CreateTablet for tablet a0493b0dbbc746408a5db9e698678d83 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:01:11.386467  8247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0493b0dbbc746408a5db9e698678d83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.385957  8485 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.399147  8318 tablet_service.cc:1511] Processing CreateTablet for tablet 767eba28679649519167576023555b60 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:01:11.399092  8245 tablet_service.cc:1511] Processing CreateTablet for tablet 767eba28679649519167576023555b60 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:01:11.400575  8318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 767eba28679649519167576023555b60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.403702  8245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 767eba28679649519167576023555b60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.401222  8487 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.411135  8153 catalog_manager.cc:5697] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:11.388566  8485 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:11.416849  8487 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:11.420159  8446 ts_tablet_manager.cc:1434] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.157s	user 0.005s	sys 0.003s
I20260709 13:01:11.422762  8467 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.422912  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:11.426832  8467 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.428828  8446 tablet_bootstrap.cc:492] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:11.429032  8467 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:01:11.430032  8220 leader_election.cc:343] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.430279  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:11.431079  8406 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:11.431228  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:11.432382  8222 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:11.413082  8452 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.426766  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:01:11.440439  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 75782.1us
W20260709 13:01:11.443964  8369 leader_election.cc:343] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.446023  8392 tablet_service.cc:1511] Processing CreateTablet for tablet 1d3e4cc2f40346c0bc3bc739e3675685 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:01:11.446197  8393 tablet_service.cc:1511] Processing CreateTablet for tablet 767eba28679649519167576023555b60 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:01:11.448724  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:01:11.450057  8369 leader_election.cc:343] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.450212  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:11.439275  8467 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:11.450825  8467 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.450982  8256 raft_consensus.cc:2393] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 1: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:01:11.451164  8467 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.452186  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:01:11.433595  8220 leader_election.cc:343] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.452883  8485 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:01:11.457734  8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d3e4cc2f40346c0bc3bc739e3675685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.458057  8447 tablet_bootstrap.cc:492] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:11.458396  8393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 767eba28679649519167576023555b60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.458493  8447 ts_tablet_manager.cc:1403] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.188s	user 0.016s	sys 0.003s
I20260709 13:01:11.460268  8448 tablet_bootstrap.cc:492] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:11.460718  8448 ts_tablet_manager.cc:1403] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.279s	user 0.017s	sys 0.045s
I20260709 13:01:11.461391  8447 raft_consensus.cc:359] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.462152  8447 raft_consensus.cc:385] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.462440  8447 raft_consensus.cc:740] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.463155  8447 consensus_queue.cc:260] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.465355  8447 ts_tablet_manager.cc:1434] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260709 13:01:11.467885  8448 raft_consensus.cc:359] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.468596  8448 raft_consensus.cc:385] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.468883  8448 raft_consensus.cc:740] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.478116  8405 raft_consensus.cc:2393] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 1: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:01:11.480581  8222 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:11.481606  8487 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:01:11.482611  8487 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.483029  8487 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.483158  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:11.483899  8405 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.484989  8487 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:11.486537  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:11.487510  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:01:11.488183  8220 leader_election.cc:343] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.490370  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:01:11.491143  8295 leader_election.cc:343] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.489130  8452 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:11.489472  8448 consensus_queue.cc:260] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:11.494191  8222 leader_election.cc:343] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.494591  8222 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:11.495692  8487 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:11.496070  8447 tablet_bootstrap.cc:492] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:11.493328  8467 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.496315  8154 catalog_manager.cc:5697] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 0 to 1, leader changed from <none> to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:11.497727  8448 ts_tablet_manager.cc:1434] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.002s
I20260709 13:01:11.498653  8448 tablet_bootstrap.cc:492] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:11.500094  8405 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:11.501247  8297 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:11.502332  8452 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:11.456303  8314 tablet_service.cc:1511] Processing CreateTablet for tablet 1d3e4cc2f40346c0bc3bc739e3675685 (DEFAULT_TABLE table=test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:01:11.503625  8314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d3e4cc2f40346c0bc3bc739e3675685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.505268  8452 raft_consensus.cc:697] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:11.506510  8446 tablet_bootstrap.cc:654] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.511170  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:11.511752  8331 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.511970  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:11.512526  8405 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.506707  8452 consensus_queue.cc:237] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.509884  8448 tablet_bootstrap.cc:654] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.512768  8447 tablet_bootstrap.cc:654] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.520180  8405 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:11.522286  8222 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:11.523118  8487 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:11.523598  8487 raft_consensus.cc:697] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:11.524487  8487 consensus_queue.cc:237] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.520277  8467 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:11.518625  8331 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:11.538964  8446 tablet_bootstrap.cc:492] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:11.539722  8446 ts_tablet_manager.cc:1403] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.111s	user 0.012s	sys 0.010s
I20260709 13:01:11.543028  8446 raft_consensus.cc:359] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:01:11.543804  8446 raft_consensus.cc:385] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.543895  8448 tablet_bootstrap.cc:492] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:11.544281  8446 raft_consensus.cc:740] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.544517  8448 ts_tablet_manager.cc:1403] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.046s	user 0.016s	sys 0.007s
I20260709 13:01:11.544286  8447 tablet_bootstrap.cc:492] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:11.545181  8447 ts_tablet_manager.cc:1403] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.049s	user 0.009s	sys 0.009s
I20260709 13:01:11.547531  8448 raft_consensus.cc:359] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.547248  8152 catalog_manager.cc:5697] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:11.548285  8448 raft_consensus.cc:385] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.548622  8448 raft_consensus.cc:740] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.548889  8446 consensus_queue.cc:260] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:01:11.551606  8446 ts_tablet_manager.cc:1434] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:01:11.552600  8446 tablet_bootstrap.cc:492] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:11.549321  8448 consensus_queue.cc:260] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.547983  8447 raft_consensus.cc:359] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.553344  8447 raft_consensus.cc:385] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.553681  8447 raft_consensus.cc:740] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.554467  8447 consensus_queue.cc:260] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.555441  8467 consensus_queue.cc:1048] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:11.556696  8447 ts_tablet_manager.cc:1434] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.000s
I20260709 13:01:11.557646  8447 tablet_bootstrap.cc:492] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:11.561808  8153 catalog_manager.cc:5697] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:11.562809  8448 ts_tablet_manager.cc:1434] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:01:11.563768  8448 tablet_bootstrap.cc:492] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:11.564635  8447 tablet_bootstrap.cc:654] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.568712  8446 tablet_bootstrap.cc:654] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.569839  8448 tablet_bootstrap.cc:654] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.585273  8405 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.587042  8446 tablet_bootstrap.cc:492] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:11.587576  8446 ts_tablet_manager.cc:1403] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.035s	user 0.012s	sys 0.000s
I20260709 13:01:11.590497  8446 raft_consensus.cc:359] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.591924  8446 raft_consensus.cc:385] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.592994  8446 raft_consensus.cc:740] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.593868  8446 consensus_queue.cc:260] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.594223  8453 consensus_queue.cc:1048] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:11.597735  8447 tablet_bootstrap.cc:492] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:11.598201  8447 ts_tablet_manager.cc:1403] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.041s	user 0.014s	sys 0.013s
I20260709 13:01:11.596866  8448 tablet_bootstrap.cc:492] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:11.599610  8448 ts_tablet_manager.cc:1403] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.036s	user 0.019s	sys 0.005s
I20260709 13:01:11.600994  8447 raft_consensus.cc:359] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.603793  8447 raft_consensus.cc:385] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.604247  8447 raft_consensus.cc:740] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.605288  8447 consensus_queue.cc:260] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.607795  8447 ts_tablet_manager.cc:1434] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:01:11.608690  8447 tablet_bootstrap.cc:492] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:11.602113  8448 raft_consensus.cc:359] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.609108  8448 raft_consensus.cc:385] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.609413  8448 raft_consensus.cc:740] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.603420  8446 ts_tablet_manager.cc:1434] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:01:11.610146  8448 consensus_queue.cc:260] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.610809  8446 tablet_bootstrap.cc:492] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:11.612179  8448 ts_tablet_manager.cc:1434] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:01:11.613161  8448 tablet_bootstrap.cc:492] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:11.615574  8447 tablet_bootstrap.cc:654] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.618750  8448 tablet_bootstrap.cc:654] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.633695  8332 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.635515  8446 tablet_bootstrap.cc:654] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.641360  8487 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.641949  8448 tablet_bootstrap.cc:492] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:11.641839  8487 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.642477  8448 ts_tablet_manager.cc:1403] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.010s
I20260709 13:01:11.643872  8487 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:11.645154  8448 raft_consensus.cc:359] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.645891  8448 raft_consensus.cc:385] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.645807  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:11.645998  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:11.646322  8448 raft_consensus.cc:740] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.646497  8405 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:11.646873  8331 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:11.647181  8448 consensus_queue.cc:260] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.647845  8222 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:11.648635  8487 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:11.648976  8487 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.649307  8487 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.650327  8453 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.650928  8453 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.654850  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:11.655632  8331 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:11.656097  8487 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.657158  8448 ts_tablet_manager.cc:1434] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:01:11.657156  8220 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:11.658470  8448 tablet_bootstrap.cc:492] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:11.658952  8467 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:11.659458  8446 tablet_bootstrap.cc:492] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:11.659930  8446 ts_tablet_manager.cc:1403] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.049s	user 0.009s	sys 0.004s
I20260709 13:01:11.659464  8467 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.660588  8467 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.662541  8446 raft_consensus.cc:359] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.663209  8446 raft_consensus.cc:385] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.663470  8446 raft_consensus.cc:740] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.664088  8446 consensus_queue.cc:260] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.664827  8448 tablet_bootstrap.cc:654] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.666316  8453 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:11.664516  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:11.667141  8331 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.668840  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:11.671047  8405 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:11.674346  8446 ts_tablet_manager.cc:1434] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:01:11.675285  8446 tablet_bootstrap.cc:492] T 09943d7e76a54d4e80eb181cac80bf22 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:11.676011  8448 tablet_bootstrap.cc:492] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:11.676486  8448 ts_tablet_manager.cc:1403] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.002s
I20260709 13:01:11.679261  8448 raft_consensus.cc:359] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.680037  8448 raft_consensus.cc:385] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.680337  8448 raft_consensus.cc:740] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.681321  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:11.681138  8448 consensus_queue.cc:260] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.681870  8405 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.683465  8448 ts_tablet_manager.cc:1434] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:01:11.684386  8448 tablet_bootstrap.cc:492] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:11.688395  8487 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:11.689234  8467 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.689919  8448 tablet_bootstrap.cc:654] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.694947  8331 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:11.696732  8220 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:11.697919  8487 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:11.699501  8487 raft_consensus.cc:697] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:11.700681  8487 consensus_queue.cc:237] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.701489  8453 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.701973  8453 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.703926  8453 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:11.705295  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:11.705665  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:11.705936  8330 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.706799  8446 tablet_bootstrap.cc:654] T 09943d7e76a54d4e80eb181cac80bf22 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.706445  8406 raft_consensus.cc:2393] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 1: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:01:11.714557  8405 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:11.718592  8467 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:11.720981  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:11.721796  8404 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.721383  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:11.724993  8330 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:11.735653  8220 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:11.737468  8448 tablet_bootstrap.cc:492] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:11.737643  8220 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:11.737998  8467 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.738128  8448 ts_tablet_manager.cc:1403] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.054s	user 0.019s	sys 0.011s
I20260709 13:01:11.738535  8453 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:11.739012  8453 raft_consensus.cc:697] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:11.739794  8453 consensus_queue.cc:237] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.741544  8448 raft_consensus.cc:359] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.742441  8448 raft_consensus.cc:385] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.742830  8448 raft_consensus.cc:740] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.745805  8446 tablet_bootstrap.cc:492] T 09943d7e76a54d4e80eb181cac80bf22 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:11.746145  8447 tablet_bootstrap.cc:492] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:11.746289  8446 ts_tablet_manager.cc:1403] T 09943d7e76a54d4e80eb181cac80bf22 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.071s	user 0.015s	sys 0.012s
I20260709 13:01:11.746547  8447 ts_tablet_manager.cc:1403] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.138s	user 0.047s	sys 0.072s
I20260709 13:01:11.748409  8448 consensus_queue.cc:260] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.749220  8447 raft_consensus.cc:359] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.749859  8447 raft_consensus.cc:385] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.750115  8447 raft_consensus.cc:740] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.754340  8448 ts_tablet_manager.cc:1434] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:01:11.754530  8447 consensus_queue.cc:260] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.755265  8448 tablet_bootstrap.cc:492] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:11.756489  8467 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:11.756701  8447 ts_tablet_manager.cc:1434] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:01:11.754381  8446 raft_consensus.cc:359] T 09943d7e76a54d4e80eb181cac80bf22 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.758262  8446 raft_consensus.cc:385] T 09943d7e76a54d4e80eb181cac80bf22 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.758564  8446 raft_consensus.cc:740] T 09943d7e76a54d4e80eb181cac80bf22 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.758926  8447 tablet_bootstrap.cc:492] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:11.759290  8446 consensus_queue.cc:260] T 09943d7e76a54d4e80eb181cac80bf22 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.759768  8153 catalog_manager.cc:5697] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:11.760855  8448 tablet_bootstrap.cc:654] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.765170  8447 tablet_bootstrap.cc:654] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.767234  8404 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:11.769827  8446 ts_tablet_manager.cc:1434] T 09943d7e76a54d4e80eb181cac80bf22 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.003s
I20260709 13:01:11.771076  8446 tablet_bootstrap.cc:492] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:11.778429  8446 tablet_bootstrap.cc:654] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.789172  8495 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.789750  8495 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.791625  8446 tablet_bootstrap.cc:492] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:11.792162  8446 ts_tablet_manager.cc:1403] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.004s
I20260709 13:01:11.792878  8495 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:11.794467  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:11.795542  8446 raft_consensus.cc:359] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:11.796273  8297 leader_election.cc:343] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.796430  8446 raft_consensus.cc:385] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.797012  8446 raft_consensus.cc:740] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.797942  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:11.798033  8446 consensus_queue.cc:260] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.798882  8256 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 0.
I20260709 13:01:11.800659  8295 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:01:11.802119  8495 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:11.802593  8495 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.803016  8495 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.811435  8446 ts_tablet_manager.cc:1434] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 13:01:11.822692  8446 tablet_bootstrap.cc:492] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:11.831914  8446 tablet_bootstrap.cc:654] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.831339  8495 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.839293  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:11.840183  8256 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.842104  8152 catalog_manager.cc:5697] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:11.848402  8448 tablet_bootstrap.cc:492] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:11.848899  8448 ts_tablet_manager.cc:1403] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.094s	user 0.049s	sys 0.040s
I20260709 13:01:11.849303  8447 tablet_bootstrap.cc:492] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:11.849462  8256 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:11.849751  8447 ts_tablet_manager.cc:1403] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.091s	user 0.041s	sys 0.036s
I20260709 13:01:11.851826  8448 raft_consensus.cc:359] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.852352  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:11.852576  8448 raft_consensus.cc:385] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.852418  8447 raft_consensus.cc:359] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.852979  8448 raft_consensus.cc:740] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.853298  8447 raft_consensus.cc:385] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.853609  8447 raft_consensus.cc:740] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
W20260709 13:01:11.853725  8297 leader_election.cc:343] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.853725  8448 consensus_queue.cc:260] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.854250  8447 consensus_queue.cc:260] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.856055  8448 ts_tablet_manager.cc:1434] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 13:01:11.856503  8447 ts_tablet_manager.cc:1434] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:01:11.857221  8448 tablet_bootstrap.cc:492] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:11.857359  8447 tablet_bootstrap.cc:492] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:11.860242  8495 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:11.863281  8447 tablet_bootstrap.cc:654] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.863920  8448 tablet_bootstrap.cc:654] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.861819  8295 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:01:11.869102  8495 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:11.870119  8495 raft_consensus.cc:697] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:11.871989  8495 consensus_queue.cc:237] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.876060  8446 tablet_bootstrap.cc:492] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:11.876667  8446 ts_tablet_manager.cc:1403] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.054s	user 0.017s	sys 0.000s
I20260709 13:01:11.878505  8448 tablet_bootstrap.cc:492] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:11.878961  8448 ts_tablet_manager.cc:1403] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.002s
I20260709 13:01:11.880246  8446 raft_consensus.cc:359] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.881162  8446 raft_consensus.cc:385] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.881517  8446 raft_consensus.cc:740] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.881837  8448 raft_consensus.cc:359] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.882726  8448 raft_consensus.cc:385] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.882498  8446 consensus_queue.cc:260] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.884646  8446 ts_tablet_manager.cc:1434] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.004s
I20260709 13:01:11.883088  8448 raft_consensus.cc:740] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.885361  8448 consensus_queue.cc:260] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.885807  8446 tablet_bootstrap.cc:492] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:11.886229  8447 tablet_bootstrap.cc:492] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:11.886693  8447 ts_tablet_manager.cc:1403] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.011s
I20260709 13:01:11.887408  8448 ts_tablet_manager.cc:1434] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:01:11.888305  8448 tablet_bootstrap.cc:492] T 1263b3742fd541f8bda5a6b6c10a8596 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:11.889510  8447 raft_consensus.cc:359] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.890189  8447 raft_consensus.cc:385] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.890511  8447 raft_consensus.cc:740] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.891369  8447 consensus_queue.cc:260] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.892231  8452 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.892463  8446 tablet_bootstrap.cc:654] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.892707  8452 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.894042  8448 tablet_bootstrap.cc:654] T 1263b3742fd541f8bda5a6b6c10a8596 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.894826  8452 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:11.896589  8447 ts_tablet_manager.cc:1434] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:01:11.898468  8447 tablet_bootstrap.cc:492] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:11.903512  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:11.904402  8404 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 0.
I20260709 13:01:11.906986  8447 tablet_bootstrap.cc:654] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.909761  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:11.910406  8256 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 0.
I20260709 13:01:11.911875  8297 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:11.914270  8526 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:11.914664  8526 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.914974  8526 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.920394  8447 tablet_bootstrap.cc:492] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:11.921082  8447 ts_tablet_manager.cc:1403] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.005s
W20260709 13:01:11.921391  8369 consensus_peers.cc:597] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:11.925390  8447 raft_consensus.cc:359] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.926455  8447 raft_consensus.cc:385] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.926872  8447 raft_consensus.cc:740] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.927973  8447 consensus_queue.cc:260] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.931389  8447 ts_tablet_manager.cc:1434] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:01:11.932715  8447 tablet_bootstrap.cc:492] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:11.936395  8526 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.940840  8447 tablet_bootstrap.cc:654] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.942986  8446 tablet_bootstrap.cc:492] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:11.943423  8446 ts_tablet_manager.cc:1403] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.058s	user 0.013s	sys 0.019s
I20260709 13:01:11.946403  8446 raft_consensus.cc:359] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.947053  8446 raft_consensus.cc:385] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.947309  8446 raft_consensus.cc:740] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.946691  8155 catalog_manager.cc:5697] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:11.947922  8446 consensus_queue.cc:260] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.950584  8526 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:11.951565  8446 ts_tablet_manager.cc:1434] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:01:11.951824  8492 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:11.952471  8446 tablet_bootstrap.cc:492] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:11.954265  8447 tablet_bootstrap.cc:492] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:11.954919  8447 ts_tablet_manager.cc:1403] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.011s
I20260709 13:01:11.957000  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:11.958776  8446 tablet_bootstrap.cc:654] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.961413  8256 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.958629  8447 raft_consensus.cc:359] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.961946  8447 raft_consensus.cc:385] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.960883  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:11.965827  8447 raft_consensus.cc:740] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.966017  8404 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.966768  8447 consensus_queue.cc:260] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.968806  8453 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.969266  8453 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.969596  8447 ts_tablet_manager.cc:1434] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.003s
I20260709 13:01:11.970947  8447 tablet_bootstrap.cc:492] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:11.971246  8453 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:11.973147  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:11.973784  8405 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:11.974927  8222 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:11.979820  8404 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:11.985165  8447 tablet_bootstrap.cc:654] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.986747  8297 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:11.987564  8526 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:11.988034  8526 raft_consensus.cc:697] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:11.988193  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:11.988797  8332 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:11.988880  8526 consensus_queue.cc:237] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:11.989771  8453 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:11.990146  8453 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.990490  8453 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.000439  8447 tablet_bootstrap.cc:492] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:12.001078  8447 ts_tablet_manager.cc:1403] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.000s
I20260709 13:01:12.004345  8447 raft_consensus.cc:359] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.005209  8447 raft_consensus.cc:385] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:12.007908  8492 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.011659  8447 raft_consensus.cc:740] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.011917  8492 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.012625  8447 consensus_queue.cc:260] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.014460  8492 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:12.016379  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:12.016935  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:12.017668  8332 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 0.
I20260709 13:01:12.017988  8447 ts_tablet_manager.cc:1434] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.003s
W20260709 13:01:12.018028  8369 leader_election.cc:343] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:12.018880  8256 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:12.019191  8447 tablet_bootstrap.cc:492] T 39384c6d69ad4f658d2e777dfdcbb49c P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:12.018936  8369 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:12.021025  8492 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:12.021489  8492 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.021821  8492 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.023614  8467 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:12.033924  8492 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.038571  8453 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.033245  8447 tablet_bootstrap.cc:654] T 39384c6d69ad4f658d2e777dfdcbb49c P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:12.041083  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:12.041443  8492 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:12.041679  8332 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.052896  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:12.074924  8487 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.075663  8487 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:12.104425  8369 leader_election.cc:343] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:12.105981  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:12.106911  8330 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.116937  8330 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:12.118762  8453 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:12.119817  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:12.120370  8406 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.125618  8526 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:12.126368  8495 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:12.127697  8487 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:12.128643  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:12.131384  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:01:12.132899  8222 leader_election.cc:343] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:12.134024  8220 leader_election.cc:343] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:12.134370  8220 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:12.136075  8406 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:12.137122  8222 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:12.107197  8152 catalog_manager.cc:5697] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:12.146349  8486 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:01:12.105082  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 50360.1us
I20260709 13:01:12.148332  8526 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.148813  8526 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.151329  8526 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:12.161829  8453 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:12.169323  8526 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.148789  8369 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:12.194193  8492 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:12.193177  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:12.194875  8492 raft_consensus.cc:697] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:12.195036  8257 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 0.
I20260709 13:01:12.195722  8492 consensus_queue.cc:237] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.196703  8332 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:12.198067  8295 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:12.199167  8495 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:12.199515  8495 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.199831  8495 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.194242  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:12.191308  8526 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:12.202508  8297 leader_election.cc:343] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:12.203342  8526 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:12.204329  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:12.205137  8257 raft_consensus.cc:2393] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 1: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:01:12.206214  8447 tablet_bootstrap.cc:492] T 39384c6d69ad4f658d2e777dfdcbb49c P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:12.206652  8447 ts_tablet_manager.cc:1403] T 39384c6d69ad4f658d2e777dfdcbb49c P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.188s	user 0.021s	sys 0.014s
I20260709 13:01:12.206787  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:12.207648  8403 raft_consensus.cc:2393] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 1: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:01:12.206861  8495 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.209227  8447 raft_consensus.cc:359] T 39384c6d69ad4f658d2e777dfdcbb49c P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.209992  8447 raft_consensus.cc:385] T 39384c6d69ad4f658d2e777dfdcbb49c P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:12.210259  8447 raft_consensus.cc:740] T 39384c6d69ad4f658d2e777dfdcbb49c P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.211874  8448 tablet_bootstrap.cc:492] T 1263b3742fd541f8bda5a6b6c10a8596 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:12.192018  8486 raft_consensus.cc:697] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:12.212360  8448 ts_tablet_manager.cc:1403] T 1263b3742fd541f8bda5a6b6c10a8596 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.324s	user 0.069s	sys 0.098s
I20260709 13:01:12.214088  8446 tablet_bootstrap.cc:492] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:12.214506  8446 ts_tablet_manager.cc:1403] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.262s	user 0.055s	sys 0.120s
W20260709 13:01:12.213555  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 64851.3us
I20260709 13:01:12.218389  8447 consensus_queue.cc:260] T 39384c6d69ad4f658d2e777dfdcbb49c P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.219419  8295 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:12.219817  8495 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:12.220880  8446 raft_consensus.cc:359] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.221237  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:12.221670  8452 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.221668  8446 raft_consensus.cc:385] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:12.222355  8446 raft_consensus.cc:740] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.223017  8446 consensus_queue.cc:260] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.224007  8404 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.225097  8446 ts_tablet_manager.cc:1434] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.002s
I20260709 13:01:12.225956  8446 tablet_bootstrap.cc:492] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:12.227402  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:12.231385  8257 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.233130  8537 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.233803  8537 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.236588  8537 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:12.237119  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:12.237515  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:01:12.238441  8220 leader_election.cc:343] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:12.240935  8222 leader_election.cc:343] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:12.241405  8222 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:12.232890  8487 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:12.243657  8447 ts_tablet_manager.cc:1434] T 39384c6d69ad4f658d2e777dfdcbb49c P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.037s	user 0.009s	sys 0.000s
I20260709 13:01:12.244942  8447 tablet_bootstrap.cc:492] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:12.246717  8539 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:12.249598  8448 raft_consensus.cc:359] T 1263b3742fd541f8bda5a6b6c10a8596 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.250368  8448 raft_consensus.cc:385] T 1263b3742fd541f8bda5a6b6c10a8596 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:12.250761  8448 raft_consensus.cc:740] T 1263b3742fd541f8bda5a6b6c10a8596 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.250178  8404 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:12.252496  8453 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:12.252748  8448 consensus_queue.cc:260] T 1263b3742fd541f8bda5a6b6c10a8596 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.254814  8448 ts_tablet_manager.cc:1434] T 1263b3742fd541f8bda5a6b6c10a8596 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.000s
I20260709 13:01:12.250267  8486 consensus_queue.cc:237] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.262529  8448 tablet_bootstrap.cc:492] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:12.270964  8487 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:01:12.280892  8203 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:12.282473  8297 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:12.290681  8152 catalog_manager.cc:5697] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 0 to 1, leader changed from <none> to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:12.286674  8526 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:12.296253  8452 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:12.307560  8486 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:12.309111  8446 tablet_bootstrap.cc:654] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:12.302522  8526 raft_consensus.cc:697] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:12.310869  8526 consensus_queue.cc:237] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.330466  8257 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:12.333909  8447 tablet_bootstrap.cc:654] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:12.336076  8448 tablet_bootstrap.cc:654] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:12.336711  8467 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:12.333886  8547 consensus_queue.cc:1048] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:01:12.345522  8446 tablet_bootstrap.cc:492] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:12.346098  8446 ts_tablet_manager.cc:1403] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.120s	user 0.010s	sys 0.004s
I20260709 13:01:12.361455  8486 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.362685  8492 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.363142  8492 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.365146  8492 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:12.362059  8486 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.365774  8448 tablet_bootstrap.cc:492] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:12.366837  8448 ts_tablet_manager.cc:1403] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.110s	user 0.014s	sys 0.008s
I20260709 13:01:12.366664  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:12.367357  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:12.367583  8256 raft_consensus.cc:2393] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 1: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:01:12.367792  8486 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:12.369315  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:12.370643  8369 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:12.360242  8155 catalog_manager.cc:5697] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:12.372265  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:12.372123  8448 raft_consensus.cc:359] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.372854  8448 raft_consensus.cc:385] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:12.373143  8448 raft_consensus.cc:740] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.373709  8220 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:12.373842  8448 consensus_queue.cc:260] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.374887  8486 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:12.375283  8446 raft_consensus.cc:359] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.375955  8446 raft_consensus.cc:385] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:12.376227  8446 raft_consensus.cc:740] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.376742  8448 ts_tablet_manager.cc:1434] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:01:12.376868  8446 consensus_queue.cc:260] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.377858  8485 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.385478  8485 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:12.386301  8448 tablet_bootstrap.cc:492] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:12.393024  8448 tablet_bootstrap.cc:654] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:12.393311  8446 ts_tablet_manager.cc:1434] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.047s	user 0.006s	sys 0.000s
I20260709 13:01:12.394305  8446 tablet_bootstrap.cc:492] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:12.399976  8446 tablet_bootstrap.cc:654] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:12.402658  8447 tablet_bootstrap.cc:492] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:12.403283  8447 ts_tablet_manager.cc:1403] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.158s	user 0.010s	sys 0.008s
I20260709 13:01:12.403729  8547 consensus_queue.cc:1048] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:12.404894  8448 tablet_bootstrap.cc:492] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:12.405339  8448 ts_tablet_manager.cc:1403] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.006s
I20260709 13:01:12.406189  8447 raft_consensus.cc:359] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.406875  8447 raft_consensus.cc:385] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:12.407191  8447 raft_consensus.cc:740] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.407897  8447 consensus_queue.cc:260] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.408144  8448 raft_consensus.cc:359] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.412340  8486 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.412518  8448 raft_consensus.cc:385] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:12.412842  8486 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.412364  8447 ts_tablet_manager.cc:1434] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:01:12.416026  8447 tablet_bootstrap.cc:492] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:12.416571  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:12.417833  8332 raft_consensus.cc:2393] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 1: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:01:12.419332  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:12.420410  8486 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:12.419706  8154 catalog_manager.cc:5697] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:12.420991  8526 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:12.421113  8222 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:12.422407  8486 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.423164  8547 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.423756  8547 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.426571  8547 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:12.429519  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:12.430840  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:01:12.432709  8297 leader_election.cc:343] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:12.434166  8295 leader_election.cc:343] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:12.434755  8295 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:12.412832  8448 raft_consensus.cc:740] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.435750  8526 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:12.437153  8448 consensus_queue.cc:260] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.440765  8486 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:12.443871  8448 ts_tablet_manager.cc:1434] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.000s
I20260709 13:01:12.444957  8448 tablet_bootstrap.cc:492] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:12.448681  8486 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.457180  8547 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:12.452929  8486 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.471433  8486 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:12.473886  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:01:12.475548  8222 leader_election.cc:343] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:12.476846  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:01:12.478442  8220 leader_election.cc:343] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:12.479018  8220 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:12.479944  8486 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:12.478436  8448 tablet_bootstrap.cc:654] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:12.491328  8446 tablet_bootstrap.cc:492] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:12.491811  8446 ts_tablet_manager.cc:1403] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.098s	user 0.004s	sys 0.063s
I20260709 13:01:12.495345  8446 raft_consensus.cc:359] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.495996  8446 raft_consensus.cc:385] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:12.496253  8446 raft_consensus.cc:740] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.499250  8256 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.506654  8447 tablet_bootstrap.cc:654] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:12.513320  8446 consensus_queue.cc:260] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.517025  8446 ts_tablet_manager.cc:1434] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:01:12.517983  8446 tablet_bootstrap.cc:492] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:12.519507  8448 tablet_bootstrap.cc:492] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:12.520557  8447 tablet_bootstrap.cc:492] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:12.521101  8448 ts_tablet_manager.cc:1403] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.076s	user 0.015s	sys 0.015s
I20260709 13:01:12.524305  8446 tablet_bootstrap.cc:654] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:12.528879  8447 ts_tablet_manager.cc:1403] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.113s	user 0.014s	sys 0.000s
I20260709 13:01:12.531850  8447 raft_consensus.cc:359] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.534894  8448 raft_consensus.cc:359] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.535670  8447 raft_consensus.cc:385] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:12.535808  8448 raft_consensus.cc:385] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:12.536216  8447 raft_consensus.cc:740] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.536245  8448 raft_consensus.cc:740] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.537264  8447 consensus_queue.cc:260] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.537330  8448 consensus_queue.cc:260] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.538666  8446 tablet_bootstrap.cc:492] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:12.539098  8446 ts_tablet_manager.cc:1403] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.009s
I20260709 13:01:12.539966  8447 ts_tablet_manager.cc:1434] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:01:12.540259  8448 ts_tablet_manager.cc:1434] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:01:12.540949  8447 tablet_bootstrap.cc:492] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:12.541296  8448 tablet_bootstrap.cc:492] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:12.541615  8446 raft_consensus.cc:359] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.542246  8446 raft_consensus.cc:385] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:12.542496  8446 raft_consensus.cc:740] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.543326  8446 consensus_queue.cc:260] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.547766  8447 tablet_bootstrap.cc:654] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:12.548292  8448 tablet_bootstrap.cc:654] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:12.550851  8446 ts_tablet_manager.cc:1434] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:01:12.552037  8446 tablet_bootstrap.cc:492] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:12.558928  8446 tablet_bootstrap.cc:654] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:12.559302  8447 tablet_bootstrap.cc:492] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:12.559829  8447 ts_tablet_manager.cc:1403] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.019s	user 0.017s	sys 0.000s
I20260709 13:01:12.568270  8447 raft_consensus.cc:359] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.569006  8447 raft_consensus.cc:385] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:12.569350  8447 raft_consensus.cc:740] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.570014  8447 consensus_queue.cc:260] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.580061  8447 ts_tablet_manager.cc:1434] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.000s
I20260709 13:01:12.581709  8447 tablet_bootstrap.cc:492] T 07268be9b33649e8b0e8bddf7a3bcabe P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:12.585386  8448 tablet_bootstrap.cc:492] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:12.586009  8448 ts_tablet_manager.cc:1403] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.045s	user 0.024s	sys 0.019s
I20260709 13:01:12.589888  8447 tablet_bootstrap.cc:654] T 07268be9b33649e8b0e8bddf7a3bcabe P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:12.589412  8448 raft_consensus.cc:359] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.590828  8448 raft_consensus.cc:385] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:12.591370  8448 raft_consensus.cc:740] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.592120  8448 consensus_queue.cc:260] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.592634  8446 tablet_bootstrap.cc:492] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:12.593048  8446 ts_tablet_manager.cc:1403] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.041s	user 0.015s	sys 0.019s
I20260709 13:01:12.594113  8448 ts_tablet_manager.cc:1434] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:01:12.595531  8446 raft_consensus.cc:359] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.596211  8446 raft_consensus.cc:385] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:12.596467  8446 raft_consensus.cc:740] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.596760  8448 tablet_bootstrap.cc:492] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:12.597116  8446 consensus_queue.cc:260] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.599097  8446 ts_tablet_manager.cc:1434] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:01:12.603579  8446 tablet_bootstrap.cc:492] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:12.604862  8448 tablet_bootstrap.cc:654] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:12.606087  8447 tablet_bootstrap.cc:492] T 07268be9b33649e8b0e8bddf7a3bcabe P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:12.606565  8447 ts_tablet_manager.cc:1403] T 07268be9b33649e8b0e8bddf7a3bcabe P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.003s
I20260709 13:01:12.609117  8447 raft_consensus.cc:359] T 07268be9b33649e8b0e8bddf7a3bcabe P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.609818  8447 raft_consensus.cc:385] T 07268be9b33649e8b0e8bddf7a3bcabe P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:12.610143  8447 raft_consensus.cc:740] T 07268be9b33649e8b0e8bddf7a3bcabe P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.610527  8446 tablet_bootstrap.cc:654] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:12.610922  8447 consensus_queue.cc:260] T 07268be9b33649e8b0e8bddf7a3bcabe P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.622766  8447 ts_tablet_manager.cc:1434] T 07268be9b33649e8b0e8bddf7a3bcabe P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 13:01:12.623665  8446 tablet_bootstrap.cc:492] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:12.624337  8446 ts_tablet_manager.cc:1403] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.005s
I20260709 13:01:12.628042  8446 raft_consensus.cc:359] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.628890  8446 raft_consensus.cc:385] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:12.628819  8492 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.629534  8492 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.632380  8486 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.633030  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:12.632855  8486 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:12.634366  8369 leader_election.cc:343] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:12.635278  8447 tablet_bootstrap.cc:492] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:12.635540  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:12.635839  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:12.636463  8332 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 0.
I20260709 13:01:12.637513  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:12.638116  8330 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:12.638216  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:12.639413  8485 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:12.639524  8220 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:12.640017  8485 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.640455  8467 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:12.640549  8485 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.640921  8467 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.641352  8467 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.642248  8447 tablet_bootstrap.cc:654] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:12.647166  8486 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:12.649698  8492 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:12.653162  8526 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.653870  8526 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.655145  8492 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:12.656550  8526 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:12.658754  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:12.659564  8256 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 0.
I20260709 13:01:12.658758  8547 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.661164  8547 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.662398  8295 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:12.663942  8547 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:12.664268  8452 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:12.661145  8446 raft_consensus.cc:740] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.664834  8452 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.665313  8452 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.666391  8446 consensus_queue.cc:260] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.668828  8485 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.670193  8404 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:12.672163  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:01:12.673385  8369 leader_election.cc:343] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:12.674655  8492 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.674842  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:12.675702  8406 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 0.
I20260709 13:01:12.675287  8492 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.677495  8297 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:12.678836  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:12.679256  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:12.679659  8256 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 0.
I20260709 13:01:12.680109  8255 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 0.
I20260709 13:01:12.681684  8369 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:12.685868  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:01:12.687760  8297 leader_election.cc:343] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:12.688917  8526 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:12.689447  8526 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.689667  8446 ts_tablet_manager.cc:1434] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.065s	user 0.008s	sys 0.003s
I20260709 13:01:12.689961  8526 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.686065  8492 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:12.697275  8486 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:12.702593  8452 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:12.743521  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 61926.9us
I20260709 13:01:12.745280  8492 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:12.746135  8492 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.746829  8492 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.777863  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:01:12.782464  8297 leader_election.cc:343] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:12.790715  8446 tablet_bootstrap.cc:492] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:12.780900  8467 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.795706  8452 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:12.814935  8485 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:12.826732  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:12.827425  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:12.838377  8256 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.840809  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:12.841464  8330 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.841125  8526 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.850672  8467 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:12.853514  8448 tablet_bootstrap.cc:492] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:12.853971  8448 ts_tablet_manager.cc:1403] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.257s	user 0.096s	sys 0.132s
I20260709 13:01:12.857563  8448 raft_consensus.cc:359] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.858997  8448 raft_consensus.cc:385] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:12.859232  8448 raft_consensus.cc:740] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.858515  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:12.859872  8448 consensus_queue.cc:260] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.861377  8448 ts_tablet_manager.cc:1434] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:01:12.861929  8492 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.862322  8448 tablet_bootstrap.cc:492] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:12.863694  8452 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:12.863884  8492 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:12.863916  8447 tablet_bootstrap.cc:492] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:12.865245  8447 ts_tablet_manager.cc:1403] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.230s	user 0.070s	sys 0.106s
I20260709 13:01:12.870796  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:01:12.868806  8369 leader_election.cc:343] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:12.872191  8404 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.866231  8526 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:12.877838  8576 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.874408  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:12.878381  8576 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:12.879776  8369 leader_election.cc:343] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:12.880293  8404 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:12.880451  8576 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:12.881630  8297 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:12.882122  8330 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:12.882386  8452 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:12.882969  8452 raft_consensus.cc:697] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:12.883850  8452 consensus_queue.cc:237] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.865406  8331 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.891829  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:12.893994  8220 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:12.895685  8448 tablet_bootstrap.cc:654] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:12.897871  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:12.898245  8576 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:12.899304  8576 raft_consensus.cc:697] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:12.900308  8330 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:12.900326  8485 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:12.900555  8576 consensus_queue.cc:237] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.901803  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:12.902379  8255 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.906528  8220 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:12.908227  8487 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:12.907609  8153 catalog_manager.cc:5697] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:12.908612  8487 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.908977  8487 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.922839  8255 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:12.896674  8404 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.938625  8547 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:12.951941  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:12.952687  8403 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:12.976284  8446 tablet_bootstrap.cc:654] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:12.981989  8447 raft_consensus.cc:359] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.984341  8447 raft_consensus.cc:385] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:12.984753  8447 raft_consensus.cc:740] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.976038  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:12.938668  8256 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:12.985630  8447 consensus_queue.cc:260] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.999514  8448 tablet_bootstrap.cc:492] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:13.000214  8448 ts_tablet_manager.cc:1403] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.138s	user 0.020s	sys 0.024s
I20260709 13:01:12.971124  8487 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:12.971161  8577 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 13:01:13.000936  8257 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:13.008195  8447 ts_tablet_manager.cc:1434] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.143s	user 0.007s	sys 0.001s
I20260709 13:01:13.009366  8447 tablet_bootstrap.cc:492] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:13.005774  8572 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.015479  8295 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:01:13.016309  8547 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:13.016749  8547 raft_consensus.cc:697] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:13.017570  8547 consensus_queue.cc:237] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.013586  8572 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.025192  8572 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:13.026590  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:13.027990  8403 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:13.026656  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:13.022511  8487 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:13.007853  8331 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:13.033259  8330 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:13.036922  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:13.037571  8332 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:13.037698  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:13.038436  8574 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:13.040416  8153 catalog_manager.cc:5697] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:13.038882  8574 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.044456  8574 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.044703  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:13.045885  8572 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:13.047096  8572 raft_consensus.cc:697] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:13.047964  8572 consensus_queue.cc:237] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.053777  8257 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:13.052704  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:13.058240  8256 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:13.062750  8448 raft_consensus.cc:359] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.063428  8448 raft_consensus.cc:385] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:13.065685  8448 raft_consensus.cc:740] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:13.064637  8332 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:13.066468  8448 consensus_queue.cc:260] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.073844  8220 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:13.074669  8369 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:13.075469  8485 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:13.076045  8485 raft_consensus.cc:697] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:13.075430  8155 catalog_manager.cc:5697] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:13.076481  8403 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:13.076787  8485 consensus_queue.cc:237] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.077589  8487 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:13.078040  8487 raft_consensus.cc:697] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:13.078804  8487 consensus_queue.cc:237] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.098356  8404 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:13.099910  8448 ts_tablet_manager.cc:1434] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.099s	user 0.004s	sys 0.004s
I20260709 13:01:13.101703  8448 tablet_bootstrap.cc:492] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:13.109483  8447 tablet_bootstrap.cc:654] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:13.110317  8574 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.115116  8446 tablet_bootstrap.cc:492] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:13.115583  8446 ts_tablet_manager.cc:1403] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.325s	user 0.012s	sys 0.021s
I20260709 13:01:13.117683  8153 catalog_manager.cc:5697] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 0 to 1, leader changed from <none> to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:13.120965  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:13.121119  8574 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:13.121572  8255 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.121709  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:13.122432  8332 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.119503  8446 raft_consensus.cc:359] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.124045  8446 raft_consensus.cc:385] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:13.124444  8446 raft_consensus.cc:740] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:13.131490  8446 consensus_queue.cc:260] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.134075  8446 ts_tablet_manager.cc:1434] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.003s
I20260709 13:01:13.135020  8255 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:01:13.136556  8446 tablet_bootstrap.cc:492] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:13.136976  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:13.138814  8332 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:01:13.148501  8448 tablet_bootstrap.cc:654] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:13.154165  8152 catalog_manager.cc:5697] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:13.156581  8446 tablet_bootstrap.cc:654] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:13.163941  8153 catalog_manager.cc:5697] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 0 to 1, leader changed from <none> to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:13.169596  8589 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:13.170454  8447 tablet_bootstrap.cc:492] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:13.170650  8446 tablet_bootstrap.cc:492] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:13.170969  8447 ts_tablet_manager.cc:1403] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.162s	user 0.019s	sys 0.016s
I20260709 13:01:13.171264  8446 ts_tablet_manager.cc:1403] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.011s
I20260709 13:01:13.170339  8589 raft_consensus.cc:697] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:13.174005  8589 consensus_queue.cc:237] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.174377  8446 raft_consensus.cc:359] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.175284  8446 raft_consensus.cc:385] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:13.175438  8447 raft_consensus.cc:359] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.175621  8446 raft_consensus.cc:740] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:13.176136  8447 raft_consensus.cc:385] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:13.176646  8446 consensus_queue.cc:260] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.177702  8447 raft_consensus.cc:740] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:13.185511  8446 ts_tablet_manager.cc:1434] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.014s	user 0.013s	sys 0.001s
I20260709 13:01:13.186514  8446 tablet_bootstrap.cc:492] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:13.184608  8447 consensus_queue.cc:260] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.194334  8446 tablet_bootstrap.cc:654] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:13.196580  8447 ts_tablet_manager.cc:1434] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.002s
I20260709 13:01:13.197497  8447 tablet_bootstrap.cc:492] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:13.206828  8446 tablet_bootstrap.cc:492] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:13.207510  8446 ts_tablet_manager.cc:1403] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.008s
I20260709 13:01:13.210402  8447 tablet_bootstrap.cc:654] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:13.210342  8446 raft_consensus.cc:359] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.211141  8446 raft_consensus.cc:385] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:13.211431  8446 raft_consensus.cc:740] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:13.212598  8446 consensus_queue.cc:260] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.214589  8448 tablet_bootstrap.cc:492] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:13.215102  8448 ts_tablet_manager.cc:1403] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.114s	user 0.015s	sys 0.029s
I20260709 13:01:13.215349  8446 ts_tablet_manager.cc:1434] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:01:13.216668  8446 tablet_bootstrap.cc:492] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:13.218510  8448 raft_consensus.cc:359] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.221545  8448 raft_consensus.cc:385] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:13.222008  8448 raft_consensus.cc:740] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:13.221235  8153 catalog_manager.cc:5697] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 0 to 2, leader changed from <none> to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 2 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:13.222862  8446 tablet_bootstrap.cc:654] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:13.223603  8448 consensus_queue.cc:260] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.227847  8447 tablet_bootstrap.cc:492] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:13.228307  8447 ts_tablet_manager.cc:1403] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.011s
I20260709 13:01:13.229496  8446 tablet_bootstrap.cc:492] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:13.229974  8446 ts_tablet_manager.cc:1403] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 13:01:13.230929  8447 raft_consensus.cc:359] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.231572  8447 raft_consensus.cc:385] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:13.231814  8447 raft_consensus.cc:740] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:13.232430  8447 consensus_queue.cc:260] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.232739  8446 raft_consensus.cc:359] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.233489  8446 raft_consensus.cc:385] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:13.233775  8446 raft_consensus.cc:740] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:13.234421  8447 ts_tablet_manager.cc:1434] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:01:13.234668  8446 consensus_queue.cc:260] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.235283  8447 tablet_bootstrap.cc:492] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:13.236538  8448 ts_tablet_manager.cc:1434] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.002s
I20260709 13:01:13.237031  8446 ts_tablet_manager.cc:1434] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:01:13.237929  8446 tablet_bootstrap.cc:492] T bf42cb9d00c6491eb6b0cf8b896f7c5b P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:13.241859  8447 tablet_bootstrap.cc:654] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:13.243953  8446 tablet_bootstrap.cc:654] T bf42cb9d00c6491eb6b0cf8b896f7c5b P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:13.254938  8448 tablet_bootstrap.cc:492] T 4516247b88e34a1d89ec9eb29e0ecb7f P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:13.263120  8448 tablet_bootstrap.cc:654] T 4516247b88e34a1d89ec9eb29e0ecb7f P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:13.264667  8447 tablet_bootstrap.cc:492] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:13.265213  8447 ts_tablet_manager.cc:1403] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.015s
I20260709 13:01:13.265129  8576 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.265995  8446 tablet_bootstrap.cc:492] T bf42cb9d00c6491eb6b0cf8b896f7c5b P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:13.265807  8576 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.266429  8446 ts_tablet_manager.cc:1403] T bf42cb9d00c6491eb6b0cf8b896f7c5b P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.007s
I20260709 13:01:13.268061  8576 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:13.268576  8447 raft_consensus.cc:359] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.269479  8447 raft_consensus.cc:385] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:13.269222  8446 raft_consensus.cc:359] T bf42cb9d00c6491eb6b0cf8b896f7c5b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.269899  8447 raft_consensus.cc:740] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:13.270038  8446 raft_consensus.cc:385] T bf42cb9d00c6491eb6b0cf8b896f7c5b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:13.270488  8446 raft_consensus.cc:740] T bf42cb9d00c6491eb6b0cf8b896f7c5b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:13.270670  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:13.271302  8404 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:13.270990  8447 consensus_queue.cc:260] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.271332  8446 consensus_queue.cc:260] T bf42cb9d00c6491eb6b0cf8b896f7c5b P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.272269  8487 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.272852  8487 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.274061  8446 ts_tablet_manager.cc:1434] T bf42cb9d00c6491eb6b0cf8b896f7c5b P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:01:13.273689  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:13.275038  8446 tablet_bootstrap.cc:492] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:13.275354  8487 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:13.276495  8222 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:13.277372  8577 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:13.277778  8577 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.278142  8577 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:13.282186  8220 leader_election.cc:343] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:13.282390  8447 ts_tablet_manager.cc:1434] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.017s	user 0.002s	sys 0.005s
I20260709 13:01:13.283633  8447 tablet_bootstrap.cc:492] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:13.291329  8447 tablet_bootstrap.cc:654] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:13.291814  8591 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.292213  8446 tablet_bootstrap.cc:654] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:13.292385  8591 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.294952  8487 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.295707  8487 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.296809  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:13.297660  8255 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 0.
I20260709 13:01:13.298183  8487 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:13.295018  8591 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:13.299225  8295 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:13.300143  8547 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:13.300614  8547 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.301035  8547 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:13.302325  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:13.303035  8405 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 0.
W20260709 13:01:13.303876  8203 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:13.306849  8446 tablet_bootstrap.cc:492] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:13.306715  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:13.307447  8446 ts_tablet_manager.cc:1403] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.033s	user 0.005s	sys 0.006s
I20260709 13:01:13.307946  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:13.308750  8404 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:13.309662  8332 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:13.310454  8446 raft_consensus.cc:359] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.311251  8446 raft_consensus.cc:385] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:13.311555  8446 raft_consensus.cc:740] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:13.314785  8222 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:13.312268  8446 consensus_queue.cc:260] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.316766  8591 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:13.319054  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:13.319849  8404 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:13.320627  8446 ts_tablet_manager.cc:1434] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.002s
I20260709 13:01:13.321223  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:13.321663  8446 tablet_bootstrap.cc:492] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:13.322041  8332 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:13.325359  8576 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:13.325912  8576 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.326402  8576 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:13.330168  8220 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:13.331092  8487 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:13.331487  8487 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.331830  8487 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:13.354996  8547 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.359644  8446 tablet_bootstrap.cc:654] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:13.337926  8577 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.361626  8576 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.364660  8578 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:13.365473  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:13.366117  8255 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:13.364665  8577 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:13.357283  8547 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:13.367059  8576 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:13.369773  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:13.369643  8467 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:13.370404  8330 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:13.374043  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:13.374702  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:13.375108  8405 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:13.376416  8220 leader_election.cc:343] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:13.379372  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:13.380075  8404 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:13.382711  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:13.383457  8403 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:13.407100  8330 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:13.407270  8403 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:13.352990  8487 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.414177  8220 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:13.415056  8576 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:13.415524  8576 raft_consensus.cc:697] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:13.415752  8404 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:13.416313  8576 consensus_queue.cc:237] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.446782  8452 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:13.459714  8487 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:13.462531  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:13.466014  8329 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:13.466614  8405 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:13.468408  8255 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:13.468938  8297 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:13.457980  8222 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:13.469872  8452 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:13.470475  8577 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:13.471066  8577 raft_consensus.cc:697] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:13.471979  8577 consensus_queue.cc:237] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.470407  8452 raft_consensus.cc:697] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:13.472450  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:13.474371  8403 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:13.475463  8452 consensus_queue.cc:237] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.537377  8447 tablet_bootstrap.cc:492] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:13.510084  8446 tablet_bootstrap.cc:492] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:13.539111  8446 ts_tablet_manager.cc:1403] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.218s	user 0.057s	sys 0.038s
I20260709 13:01:13.539120  8447 ts_tablet_manager.cc:1403] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.256s	user 0.057s	sys 0.055s
I20260709 13:01:13.489195  8589 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.490242  8572 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.552104  8572 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.555140  8572 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:13.556783  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:13.560220  8448 tablet_bootstrap.cc:492] T 4516247b88e34a1d89ec9eb29e0ecb7f P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:13.560820  8448 ts_tablet_manager.cc:1403] T 4516247b88e34a1d89ec9eb29e0ecb7f P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.306s	user 0.032s	sys 0.052s
I20260709 13:01:13.564810  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:13.565683  8589 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.564600  8448 raft_consensus.cc:359] T 4516247b88e34a1d89ec9eb29e0ecb7f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.567600  8448 raft_consensus.cc:385] T 4516247b88e34a1d89ec9eb29e0ecb7f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:13.568084  8448 raft_consensus.cc:740] T 4516247b88e34a1d89ec9eb29e0ecb7f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:13.571844  8448 consensus_queue.cc:260] T 4516247b88e34a1d89ec9eb29e0ecb7f P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:13.573575  8369 consensus_peers.cc:597] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:13.581292  8448 ts_tablet_manager.cc:1434] T 4516247b88e34a1d89ec9eb29e0ecb7f P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.005s
I20260709 13:01:13.582324  8448 tablet_bootstrap.cc:492] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:13.584115  8446 raft_consensus.cc:359] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.584649  8446 raft_consensus.cc:385] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:13.584867  8446 raft_consensus.cc:740] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:13.562902  8329 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:13.585475  8446 consensus_queue.cc:260] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.578817  8256 raft_consensus.cc:2393] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 1: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:01:13.588338  8485 consensus_queue.cc:1048] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:13.592175  8589 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:13.595538  8448 tablet_bootstrap.cc:654] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:13.597324  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:13.597975  8255 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 0.
I20260709 13:01:13.599710  8403 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:13.601243  8446 ts_tablet_manager.cc:1434] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.062s	user 0.006s	sys 0.000s
I20260709 13:01:13.602250  8222 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:13.602610  8446 tablet_bootstrap.cc:492] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:13.602707  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:13.603474  8329 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 0.
I20260709 13:01:13.603706  8467 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:13.604911  8576 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:13.606411  8467 raft_consensus.cc:697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:13.607288  8467 consensus_queue.cc:237] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.571640  8447 raft_consensus.cc:359] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.609290  8447 raft_consensus.cc:385] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:13.610097  8369 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:13.611129  8492 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:13.611486  8492 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.611811  8492 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:13.609781  8447 raft_consensus.cc:740] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:13.614413  8447 consensus_queue.cc:260] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.617720  8447 ts_tablet_manager.cc:1434] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.078s	user 0.007s	sys 0.000s
I20260709 13:01:13.618606  8447 tablet_bootstrap.cc:492] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:13.618790  8154 catalog_manager.cc:5697] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:13.620581  8369 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:13.627282  8492 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.628172  8589 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:13.629829  8492 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:13.632241  8154 catalog_manager.cc:5697] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:13.643188  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:13.643821  8256 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:13.644083  8448 tablet_bootstrap.cc:492] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:13.644591  8448 ts_tablet_manager.cc:1403] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.062s	user 0.008s	sys 0.013s
I20260709 13:01:13.643186  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:13.652119  8331 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:13.653795  8152 catalog_manager.cc:5697] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:13.654596  8576 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:13.657862  8448 raft_consensus.cc:359] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.658610  8448 raft_consensus.cc:385] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:13.658896  8448 raft_consensus.cc:740] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:13.659569  8448 consensus_queue.cc:260] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.645639  8589 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:13.673367  8446 tablet_bootstrap.cc:654] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:13.674996  8448 ts_tablet_manager.cc:1434] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.002s
I20260709 13:01:13.676235  8448 tablet_bootstrap.cc:492] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:13.670383  8492 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:01:13.678287  8256 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:13.680317  8331 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:13.685570  8448 tablet_bootstrap.cc:654] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:13.681224  8369 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:13.687299  8492 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:13.687740  8492 raft_consensus.cc:697] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:13.689591  8492 consensus_queue.cc:237] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.690580  8614 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 13:01:13.694741  8526 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:01:13.691104  8330 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:13.746284  8467 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.761178  8152 catalog_manager.cc:5697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:13.772959  8447 tablet_bootstrap.cc:654] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:13.773548  8257 raft_consensus.cc:1275] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:01:13.765264  8467 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.781062  8446 tablet_bootstrap.cc:492] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:13.781680  8446 ts_tablet_manager.cc:1403] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.179s	user 0.016s	sys 0.014s
I20260709 13:01:13.785903  8446 raft_consensus.cc:359] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.786588  8446 raft_consensus.cc:385] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:13.786847  8446 raft_consensus.cc:740] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:13.787462  8446 consensus_queue.cc:260] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.791183  8611 consensus_queue.cc:1048] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:01:13.793877  8467 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:13.802034  8446 ts_tablet_manager.cc:1434] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.002s
I20260709 13:01:13.795449  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:13.805290  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:13.807691  8222 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:13.808673  8331 raft_consensus.cc:1275] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:01:13.808737  8576 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:13.796671  8154 catalog_manager.cc:5697] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 0 to 1, leader changed from <none> to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:13.812218  8589 consensus_queue.cc:1048] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:13.826030  8448 tablet_bootstrap.cc:492] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:13.827128  8448 ts_tablet_manager.cc:1403] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.151s	user 0.012s	sys 0.009s
I20260709 13:01:13.829734  8611 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:13.830413  8448 raft_consensus.cc:359] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.831399  8448 raft_consensus.cc:385] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:13.831753  8448 raft_consensus.cc:740] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:13.832523  8448 consensus_queue.cc:260] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.839180  8448 ts_tablet_manager.cc:1434] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:01:13.840353  8448 tablet_bootstrap.cc:492] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:13.846019  8448 tablet_bootstrap.cc:654] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:13.848016  8446 tablet_bootstrap.cc:492] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:13.849972  8591 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.850554  8591 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.852926  8591 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:13.855904  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:13.856864  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:13.857262  8447 tablet_bootstrap.cc:492] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
W20260709 13:01:13.857339  8295 leader_election.cc:343] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:13.857860  8447 ts_tablet_manager.cc:1403] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.239s	user 0.009s	sys 0.006s
W20260709 13:01:13.858107  8297 leader_election.cc:343] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:13.860468  8297 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:13.863324  8614 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:13.872756  8447 raft_consensus.cc:359] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.873339  8447 raft_consensus.cc:385] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:13.873677  8447 raft_consensus.cc:740] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:13.872197  8492 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.874338  8447 consensus_queue.cc:260] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.874400  8492 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.876996  8447 ts_tablet_manager.cc:1434] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:01:13.877974  8447 tablet_bootstrap.cc:492] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:13.878715  8492 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:13.884357  8576 consensus_queue.cc:1048] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:13.889024  8446 tablet_bootstrap.cc:654] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:13.890401  8447 tablet_bootstrap.cc:654] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:13.893800  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:13.896287  8256 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 0.
I20260709 13:01:13.900936  8447 tablet_bootstrap.cc:492] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:13.901914  8447 ts_tablet_manager.cc:1403] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.006s
I20260709 13:01:13.902446  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:01:13.904166  8369 leader_election.cc:343] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:13.904623  8447 raft_consensus.cc:359] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.904913  8369 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:13.905328  8447 raft_consensus.cc:385] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:13.905639  8447 raft_consensus.cc:740] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:13.906801  8589 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:13.907249  8589 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.907595  8589 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:13.894284  8446 tablet_bootstrap.cc:492] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:13.908577  8447 consensus_queue.cc:260] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.909391  8446 ts_tablet_manager.cc:1403] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.062s	user 0.008s	sys 0.003s
I20260709 13:01:13.911026  8447 ts_tablet_manager.cc:1434] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:01:13.911993  8446 raft_consensus.cc:359] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.912708  8446 raft_consensus.cc:385] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:13.912955  8446 raft_consensus.cc:740] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:13.915511  8589 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.911966  8447 tablet_bootstrap.cc:492] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:13.918255  8448 tablet_bootstrap.cc:492] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:13.918776  8448 ts_tablet_manager.cc:1403] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.079s	user 0.009s	sys 0.003s
I20260709 13:01:13.921622  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:13.921407  8448 raft_consensus.cc:359] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.922276  8448 raft_consensus.cc:385] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:13.921969  8446 consensus_queue.cc:260] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.922571  8448 raft_consensus.cc:740] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:13.923187  8448 consensus_queue.cc:260] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:13.923502  8369 leader_election.cc:343] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:13.924058  8589 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:13.924369  8446 ts_tablet_manager.cc:1434] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:01:13.924708  8448 ts_tablet_manager.cc:1434] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.002s
I20260709 13:01:13.925246  8446 tablet_bootstrap.cc:492] T cb79894cecb541e3b7dd164271cadc16 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:13.925597  8448 tablet_bootstrap.cc:492] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:13.927172  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:13.927793  8255 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:13.929811  8447 tablet_bootstrap.cc:654] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:13.931707  8448 tablet_bootstrap.cc:654] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:13.938966  8446 tablet_bootstrap.cc:654] T cb79894cecb541e3b7dd164271cadc16 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:13.943406  8255 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:13.945303  8369 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:13.946429  8589 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:13.946924  8589 raft_consensus.cc:697] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:13.946987  8485 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.947444  8485 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.947795  8589 consensus_queue.cc:237] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.949536  8485 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:13.950273  8448 tablet_bootstrap.cc:492] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:13.950881  8448 ts_tablet_manager.cc:1403] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.015s
I20260709 13:01:13.951413  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:13.952324  8447 tablet_bootstrap.cc:492] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:13.952749  8447 ts_tablet_manager.cc:1403] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.041s	user 0.010s	sys 0.003s
W20260709 13:01:13.954090  8369 leader_election.cc:343] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:13.955093  8576 consensus_queue.cc:1048] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:13.955753  8447 raft_consensus.cc:359] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.956413  8447 raft_consensus.cc:385] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:13.956676  8447 raft_consensus.cc:740] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:13.959419  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:13.957652  8447 consensus_queue.cc:260] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.960556  8254 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 0.
I20260709 13:01:13.961197  8591 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:13.956719  8448 raft_consensus.cc:359] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.963686  8369 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:13.963840  8448 raft_consensus.cc:385] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:13.964274  8448 raft_consensus.cc:740] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:13.964762  8485 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:13.965126  8485 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.965452  8485 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:13.967005  8448 consensus_queue.cc:260] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.968233  8447 ts_tablet_manager.cc:1434] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:01:13.969168  8447 tablet_bootstrap.cc:492] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:13.978731  8485 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:13.981228  8448 ts_tablet_manager.cc:1434] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.003s
I20260709 13:01:13.983464  8448 tablet_bootstrap.cc:492] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:13.984695  8485 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:13.982236  8446 tablet_bootstrap.cc:492] T cb79894cecb541e3b7dd164271cadc16 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:13.985683  8446 ts_tablet_manager.cc:1403] T cb79894cecb541e3b7dd164271cadc16 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.061s	user 0.019s	sys 0.026s
I20260709 13:01:13.995477  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:13.996384  8254 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:13.999747  8255 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:14.003259  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:14.003922  8330 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:14.012708  8220 consensus_peers.cc:597] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:14.014644  8254 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:14.027761  8446 raft_consensus.cc:359] T cb79894cecb541e3b7dd164271cadc16 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.028537  8446 raft_consensus.cc:385] T cb79894cecb541e3b7dd164271cadc16 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:14.028815  8446 raft_consensus.cc:740] T cb79894cecb541e3b7dd164271cadc16 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:14.032140  8369 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:14.033174  8485 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:14.033689  8485 raft_consensus.cc:697] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:14.034703  8330 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:14.034467  8485 consensus_queue.cc:237] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.029556  8446 consensus_queue.cc:260] T cb79894cecb541e3b7dd164271cadc16 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.037137  8446 ts_tablet_manager.cc:1434] T cb79894cecb541e3b7dd164271cadc16 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.051s	user 0.003s	sys 0.003s
I20260709 13:01:14.038013  8446 tablet_bootstrap.cc:492] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:14.035072  8447 tablet_bootstrap.cc:654] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:14.042317  8152 catalog_manager.cc:5697] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 0 to 1, leader changed from <none> to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:14.047128  8452 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:14.063862  8448 tablet_bootstrap.cc:654] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:14.093612  8487 consensus_queue.cc:1048] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:14.106458  8448 tablet_bootstrap.cc:492] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:14.106616  8446 tablet_bootstrap.cc:654] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:14.106930  8448 ts_tablet_manager.cc:1403] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.124s	user 0.024s	sys 0.016s
I20260709 13:01:14.109838  8447 tablet_bootstrap.cc:492] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:14.110339  8447 ts_tablet_manager.cc:1403] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.141s	user 0.016s	sys 0.006s
I20260709 13:01:14.119472  8486 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.120210  8486 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.121872  8447 raft_consensus.cc:359] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.122725  8447 raft_consensus.cc:385] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:14.123224  8447 raft_consensus.cc:740] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:14.123946  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:14.124092  8447 consensus_queue.cc:260] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:14.125308  8220 leader_election.cc:343] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:14.125500  8448 raft_consensus.cc:359] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.125306  8486 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:14.126416  8447 ts_tablet_manager.cc:1434] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:01:14.126813  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:14.127446  8447 tablet_bootstrap.cc:492] T eeecf7303d634c0d8983b810bb97c33c P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:14.127389  8405 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:14.130131  8222 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:14.131250  8486 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:14.131593  8486 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.131906  8486 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:14.137643  8487 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:14.137825  8154 catalog_manager.cc:5697] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 0 to 1, leader changed from <none> to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:14.138764  8486 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.126152  8448 raft_consensus.cc:385] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:14.141142  8448 raft_consensus.cc:740] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:14.141806  8448 consensus_queue.cc:260] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.144435  8448 ts_tablet_manager.cc:1434] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.001s
I20260709 13:01:14.144896  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:14.145361  8448 tablet_bootstrap.cc:492] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
W20260709 13:01:14.147351  8220 leader_election.cc:343] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:14.206331  8486 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:14.207702  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:14.210166  8403 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:14.221522  8492 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:14.224702  8589 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:14.230806  8576 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:14.234404  8446 tablet_bootstrap.cc:492] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:14.235028  8446 ts_tablet_manager.cc:1403] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.197s	user 0.022s	sys 0.039s
I20260709 13:01:14.239848  8446 raft_consensus.cc:359] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.240660  8446 raft_consensus.cc:385] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:14.241034  8446 raft_consensus.cc:740] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:14.246362  8446 consensus_queue.cc:260] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.255330  8446 ts_tablet_manager.cc:1434] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 13:01:14.256368  8446 tablet_bootstrap.cc:492] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:14.259117  8492 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.261816  8492 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.264467  8492 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:14.265990  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:14.267146  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:01:14.270277  8369 leader_election.cc:343] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:14.272578  8447 tablet_bootstrap.cc:654] T eeecf7303d634c0d8983b810bb97c33c P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:14.274122  8369 leader_election.cc:343] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:14.274554  8369 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:14.275470  8589 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:14.275882  8486 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.280148  8486 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.282982  8486 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:14.283209  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:14.283854  8406 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:14.285189  8403 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:14.285316  8222 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:14.286401  8222 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:14.286875  8448 tablet_bootstrap.cc:654] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:14.286741  8467 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:14.288092  8467 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.288172  8486 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:14.288465  8467 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:14.289060  8486 raft_consensus.cc:697] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:14.289966  8486 consensus_queue.cc:237] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.297861  8446 tablet_bootstrap.cc:654] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:14.299306  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:14.299924  8332 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:14.298471  8467 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.327821  8467 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:14.329941  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:14.330761  8332 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:14.356907  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:14.358232  8405 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:14.366672  8448 tablet_bootstrap.cc:492] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:14.367125  8448 ts_tablet_manager.cc:1403] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.222s	user 0.027s	sys 0.011s
I20260709 13:01:14.382344  8330 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:14.392822  8448 raft_consensus.cc:359] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.393550  8448 raft_consensus.cc:385] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:14.393826  8448 raft_consensus.cc:740] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:14.394387  8332 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:14.397059  8220 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:14.394557  8405 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:14.397913  8486 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:14.398556  8446 tablet_bootstrap.cc:492] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:14.399119  8446 ts_tablet_manager.cc:1403] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.143s	user 0.008s	sys 0.015s
I20260709 13:01:14.398556  8486 raft_consensus.cc:697] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:14.401091  8486 consensus_queue.cc:237] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.401324  8492 consensus_queue.cc:1048] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:14.402894  8447 tablet_bootstrap.cc:492] T eeecf7303d634c0d8983b810bb97c33c P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:14.402632  8446 raft_consensus.cc:359] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.403367  8447 ts_tablet_manager.cc:1403] T eeecf7303d634c0d8983b810bb97c33c P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.276s	user 0.044s	sys 0.041s
I20260709 13:01:14.395023  8448 consensus_queue.cc:260] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.404004  8446 raft_consensus.cc:385] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:14.409636  8446 raft_consensus.cc:740] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:14.411329  8446 consensus_queue.cc:260] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.413794  8448 ts_tablet_manager.cc:1434] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.045s	user 0.003s	sys 0.003s
I20260709 13:01:14.414686  8448 tablet_bootstrap.cc:492] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:14.415326  8446 ts_tablet_manager.cc:1434] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 13:01:14.416157  8446 tablet_bootstrap.cc:492] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:14.418540  8447 raft_consensus.cc:359] T eeecf7303d634c0d8983b810bb97c33c P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.419186  8447 raft_consensus.cc:385] T eeecf7303d634c0d8983b810bb97c33c P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:14.419445  8447 raft_consensus.cc:740] T eeecf7303d634c0d8983b810bb97c33c P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:14.420626  8447 consensus_queue.cc:260] T eeecf7303d634c0d8983b810bb97c33c P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.426080  8447 ts_tablet_manager.cc:1434] T eeecf7303d634c0d8983b810bb97c33c P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 13:01:14.426985  8447 tablet_bootstrap.cc:492] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:14.435527  8153 catalog_manager.cc:5697] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:14.445030  8446 tablet_bootstrap.cc:654] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:14.450692  8448 tablet_bootstrap.cc:654] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:14.457563  8611 consensus_queue.cc:1048] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:14.461094  8447 tablet_bootstrap.cc:654] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:14.479688  8203 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:14.488757  8447 tablet_bootstrap.cc:492] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:14.489287  8447 ts_tablet_manager.cc:1403] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.062s	user 0.011s	sys 0.001s
I20260709 13:01:14.496860  8446 tablet_bootstrap.cc:492] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:14.497552  8448 tablet_bootstrap.cc:492] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:14.498008  8448 ts_tablet_manager.cc:1403] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.083s	user 0.010s	sys 0.011s
I20260709 13:01:14.498309  8447 raft_consensus.cc:359] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.498991  8447 raft_consensus.cc:385] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:14.499246  8447 raft_consensus.cc:740] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:14.499822  8447 consensus_queue.cc:260] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.500133  8448 raft_consensus.cc:359] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.500674  8448 raft_consensus.cc:385] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:14.500802  8446 ts_tablet_manager.cc:1403] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.085s	user 0.016s	sys 0.016s
I20260709 13:01:14.500922  8448 raft_consensus.cc:740] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:14.501662  8448 consensus_queue.cc:260] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.501775  8155 catalog_manager.cc:5697] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:14.503291  8448 ts_tablet_manager.cc:1434] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.003s
I20260709 13:01:14.503394  8446 raft_consensus.cc:359] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.504106  8448 tablet_bootstrap.cc:492] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:14.505776  8447 ts_tablet_manager.cc:1434] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:01:14.506664  8447 tablet_bootstrap.cc:492] T 766ac43473084aa6a9fdce530520947c P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:14.504046  8446 raft_consensus.cc:385] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:14.510098  8446 raft_consensus.cc:740] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:14.510851  8446 consensus_queue.cc:260] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.512355  8447 tablet_bootstrap.cc:654] T 766ac43473084aa6a9fdce530520947c P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:14.513124  8446 ts_tablet_manager.cc:1434] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.012s	user 0.001s	sys 0.005s
I20260709 13:01:14.513991  8446 tablet_bootstrap.cc:492] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:14.519104  8446 tablet_bootstrap.cc:654] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:14.521615  8448 tablet_bootstrap.cc:654] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:14.523061  8611 consensus_queue.cc:1048] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:14.531476  8446 tablet_bootstrap.cc:492] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:14.531849  8446 ts_tablet_manager.cc:1403] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.018s	user 0.006s	sys 0.003s
I20260709 13:01:14.541658  8446 raft_consensus.cc:359] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.543004  8446 raft_consensus.cc:385] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:14.543473  8446 raft_consensus.cc:740] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:14.544622  8446 consensus_queue.cc:260] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.542544  8447 tablet_bootstrap.cc:492] T 766ac43473084aa6a9fdce530520947c P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:14.545642  8447 ts_tablet_manager.cc:1403] T 766ac43473084aa6a9fdce530520947c P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.039s	user 0.009s	sys 0.015s
I20260709 13:01:14.547130  8446 ts_tablet_manager.cc:1434] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:01:14.548071  8446 tablet_bootstrap.cc:492] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:14.548339  8447 raft_consensus.cc:359] T 766ac43473084aa6a9fdce530520947c P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.549029  8447 raft_consensus.cc:385] T 766ac43473084aa6a9fdce530520947c P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:14.549278  8447 raft_consensus.cc:740] T 766ac43473084aa6a9fdce530520947c P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:14.549234  8589 consensus_queue.cc:1048] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:14.550272  8447 consensus_queue.cc:260] T 766ac43473084aa6a9fdce530520947c P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.558086  8447 ts_tablet_manager.cc:1434] T 766ac43473084aa6a9fdce530520947c P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:01:14.559051  8447 tablet_bootstrap.cc:492] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:14.560122  8446 tablet_bootstrap.cc:654] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:14.561326  8591 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.561870  8591 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.563352  8448 tablet_bootstrap.cc:492] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:14.563776  8448 ts_tablet_manager.cc:1403] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.060s	user 0.008s	sys 0.004s
I20260709 13:01:14.564035  8591 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:14.566634  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:01:14.568034  8297 leader_election.cc:343] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:14.568513  8448 raft_consensus.cc:359] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.568791  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:14.569201  8448 raft_consensus.cc:385] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:14.569535  8448 raft_consensus.cc:740] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
W20260709 13:01:14.570080  8295 leader_election.cc:343] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:14.572096  8295 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:14.574543  8448 consensus_queue.cc:260] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.575135  8452 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:14.575453  8487 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.575927  8487 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.576943  8448 ts_tablet_manager.cc:1434] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 13:01:14.577883  8448 tablet_bootstrap.cc:492] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:14.578609  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:14.578768  8487 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:14.579190  8330 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:14.581079  8447 tablet_bootstrap.cc:654] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:14.581562  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:14.583463  8220 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:14.585106  8487 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:14.585476  8487 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.585794  8487 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:14.587251  8448 tablet_bootstrap.cc:654] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:14.590785  8222 leader_election.cc:343] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:14.594247  8487 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.596225  8487 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:14.597012  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:14.597608  8330 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:14.602452  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:01:14.604626  8222 leader_election.cc:343] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:14.607195  8447 tablet_bootstrap.cc:492] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:14.607672  8447 ts_tablet_manager.cc:1403] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.049s	user 0.013s	sys 0.003s
I20260709 13:01:14.608265  8446 tablet_bootstrap.cc:492] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:14.608740  8446 ts_tablet_manager.cc:1403] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.061s	user 0.009s	sys 0.013s
I20260709 13:01:14.610186  8447 raft_consensus.cc:359] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.610805  8447 raft_consensus.cc:385] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:14.616393  8447 raft_consensus.cc:740] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:14.617014  8447 consensus_queue.cc:260] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.620155  8447 ts_tablet_manager.cc:1434] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:01:14.613773  8448 tablet_bootstrap.cc:492] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:14.615085  8331 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:14.621181  8448 ts_tablet_manager.cc:1403] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.043s	user 0.012s	sys 0.004s
I20260709 13:01:14.623806  8448 raft_consensus.cc:359] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.624053  8330 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:14.624456  8448 raft_consensus.cc:385] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:14.624781  8448 raft_consensus.cc:740] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:14.625298  8220 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:01:14.626528  8487 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:14.626971  8487 raft_consensus.cc:697] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:14.627775  8487 consensus_queue.cc:237] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.625353  8448 consensus_queue.cc:260] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.631454  8448 ts_tablet_manager.cc:1434] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 13:01:14.632285  8448 tablet_bootstrap.cc:492] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:14.632875  8447 tablet_bootstrap.cc:492] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:14.611361  8446 raft_consensus.cc:359] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.637272  8446 raft_consensus.cc:385] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:14.637645  8446 raft_consensus.cc:740] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:14.637856  8448 tablet_bootstrap.cc:654] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:14.638538  8446 consensus_queue.cc:260] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.654143  8447 tablet_bootstrap.cc:654] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:14.663292  8446 ts_tablet_manager.cc:1434] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.054s	user 0.008s	sys 0.001s
I20260709 13:01:14.664377  8446 tablet_bootstrap.cc:492] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:14.664527  8153 catalog_manager.cc:5697] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:14.667027  8611 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.667649  8611 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.671568  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:01:14.673414  8369 leader_election.cc:343] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:14.679615  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:01:14.681819  8369 leader_election.cc:343] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:14.682653  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:14.674503  8611 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:14.684928  8611 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:14.686014  8611 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.686573  8611 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.687151  8448 tablet_bootstrap.cc:492] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:14.687762  8448 ts_tablet_manager.cc:1403] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.056s	user 0.014s	sys 0.016s
I20260709 13:01:14.687956  8447 tablet_bootstrap.cc:492] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:14.689774  8447 ts_tablet_manager.cc:1403] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.059s	user 0.008s	sys 0.008s
I20260709 13:01:14.689036  8446 tablet_bootstrap.cc:654] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:14.690306  8448 raft_consensus.cc:359] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.690577  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:14.690999  8448 raft_consensus.cc:385] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:14.691287  8448 raft_consensus.cc:740] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:14.691810  8589 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.691897  8448 consensus_queue.cc:260] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.692343  8589 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.694109  8448 ts_tablet_manager.cc:1434] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:01:14.694538  8589 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:14.695181  8448 tablet_bootstrap.cc:492] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:14.695163  8330 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:14.692340  8447 raft_consensus.cc:359] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.697584  8447 raft_consensus.cc:385] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:14.697901  8447 raft_consensus.cc:740] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:14.698618  8487 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.698654  8447 consensus_queue.cc:260] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.699198  8487 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.701471  8448 tablet_bootstrap.cc:654] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:14.702970  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:14.704020  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:14.707587  8448 tablet_bootstrap.cc:492] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:14.705247  8447 ts_tablet_manager.cc:1434] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.001s
I20260709 13:01:14.708071  8448 ts_tablet_manager.cc:1403] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 13:01:14.701202  8487 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:14.708674  8447 tablet_bootstrap.cc:492] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
W20260709 13:01:14.708724  8222 leader_election.cc:343] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:14.709771  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:14.710736  8448 raft_consensus.cc:359] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.711112  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:14.711515  8448 raft_consensus.cc:385] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:14.711745  8329 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 0.
I20260709 13:01:14.711802  8448 raft_consensus.cc:740] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:14.712607  8448 consensus_queue.cc:260] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:14.713326  8369 leader_election.cc:343] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:14.714226  8369 leader_election.cc:343] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:14.714771  8448 ts_tablet_manager.cc:1434] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.002s
I20260709 13:01:14.714671  8369 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:14.708025  8331 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:14.715646  8448 tablet_bootstrap.cc:492] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:14.716310  8220 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:01:14.718164  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:14.718945  8254 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 0.
I20260709 13:01:14.719044  8487 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:14.719800  8487 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.720196  8487 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:14.722360  8486 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:14.725215  8447 tablet_bootstrap.cc:654] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:14.726430  8614 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.726894  8614 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.728947  8448 tablet_bootstrap.cc:654] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:14.729444  8614 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:14.729640  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:14.730338  8257 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 0.
I20260709 13:01:14.731490  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:14.732254  8403 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 0.
I20260709 13:01:14.733807  8295 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:14.735126  8447 tablet_bootstrap.cc:492] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:14.735764  8447 ts_tablet_manager.cc:1403] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.000s
I20260709 13:01:14.735879  8614 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:14.736586  8614 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.737114  8614 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:14.739240  8447 raft_consensus.cc:359] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.740191  8447 raft_consensus.cc:385] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:14.740548  8447 raft_consensus.cc:740] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:14.741130  8589 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:14.741446  8447 consensus_queue.cc:260] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.744750  8448 tablet_bootstrap.cc:492] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:14.745190  8448 ts_tablet_manager.cc:1403] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.010s
I20260709 13:01:14.689240  8611 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:14.745328  8369 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:14.747697  8448 raft_consensus.cc:359] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.752511  8448 raft_consensus.cc:385] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:14.748776  8447 ts_tablet_manager.cc:1434] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:01:14.752856  8448 raft_consensus.cc:740] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:14.753944  8447 tablet_bootstrap.cc:492] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:14.754730  8589 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:14.755767  8589 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.756747  8589 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:14.761919  8447 tablet_bootstrap.cc:654] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:14.753708  8448 consensus_queue.cc:260] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.766450  8448 ts_tablet_manager.cc:1434] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.021s	user 0.002s	sys 0.003s
I20260709 13:01:14.767742  8448 tablet_bootstrap.cc:492] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:14.775172  8589 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.776633  8447 tablet_bootstrap.cc:492] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:14.776880  8589 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:14.777233  8447 ts_tablet_manager.cc:1403] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.002s
I20260709 13:01:14.777515  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:14.778299  8257 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:14.780349  8487 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.780756  8447 raft_consensus.cc:359] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.781772  8447 raft_consensus.cc:385] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:14.782128  8447 raft_consensus.cc:740] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:14.783098  8447 consensus_queue.cc:260] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.783622  8614 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.785207  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:14.785785  8331 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:14.786878  8447 ts_tablet_manager.cc:1434] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:01:14.787863  8487 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:14.788677  8614 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:14.789326  8448 tablet_bootstrap.cc:654] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:14.789335  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:01:14.791179  8222 leader_election.cc:343] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:14.792130  8447 tablet_bootstrap.cc:492] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:14.794176  8446 tablet_bootstrap.cc:492] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:14.794617  8446 ts_tablet_manager.cc:1403] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.130s	user 0.034s	sys 0.033s
I20260709 13:01:14.797770  8447 tablet_bootstrap.cc:654] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:14.801409  8448 tablet_bootstrap.cc:492] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:14.801867  8448 ts_tablet_manager.cc:1403] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.035s	user 0.010s	sys 0.007s
I20260709 13:01:14.804410  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:14.804996  8254 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:14.810463  8467 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:14.810016  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:14.811638  8403 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:14.811764  8467 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:14.813032  8446 raft_consensus.cc:359] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.813791  8446 raft_consensus.cc:385] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:14.814152  8446 raft_consensus.cc:740] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:14.814870  8446 consensus_queue.cc:260] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.820019  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:14.820603  8330 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:14.830255  8446 ts_tablet_manager.cc:1434] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260709 13:01:14.831542  8446 tablet_bootstrap.cc:492] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:14.851773  8448 raft_consensus.cc:359] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.852716  8448 raft_consensus.cc:385] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:14.853070  8448 raft_consensus.cc:740] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:14.858093  8448 consensus_queue.cc:260] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.860787  8448 ts_tablet_manager.cc:1434] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.059s	user 0.007s	sys 0.001s
I20260709 13:01:14.862090  8448 tablet_bootstrap.cc:492] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:14.871518  8487 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.871922  8487 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.877506  8487 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:14.879995  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:14.881752  8448 tablet_bootstrap.cc:654] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:14.881744  8222 leader_election.cc:343] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:14.883240  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:14.883860  8328 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:14.884167  8331 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:14.886291  8220 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:01:14.886427  8467 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:14.887261  8486 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:14.887580  8447 tablet_bootstrap.cc:492] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:14.887610  8486 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.888134  8447 ts_tablet_manager.cc:1403] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.100s	user 0.023s	sys 0.026s
I20260709 13:01:14.886291  8369 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:14.888092  8486 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:14.889454  8611 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:14.889904  8611 raft_consensus.cc:697] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:14.890933  8611 consensus_queue.cc:237] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.891870  8403 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:14.898972  8257 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:14.899747  8297 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:14.896255  8330 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:14.901475  8254 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:14.902875  8452 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:14.903540  8452 raft_consensus.cc:697] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:14.904438  8452 consensus_queue.cc:237] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.952203  8447 raft_consensus.cc:359] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.952729  8447 raft_consensus.cc:385] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:14.952965  8447 raft_consensus.cc:740] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:14.950613  8220 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:01:14.958766  8578 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:14.967075  8446 tablet_bootstrap.cc:654] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:14.967387  8153 catalog_manager.cc:5697] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:14.971657  8447 consensus_queue.cc:260] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.953416  8486 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:14.987814  8578 raft_consensus.cc:697] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:14.991928  8447 ts_tablet_manager.cc:1434] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.103s	user 0.004s	sys 0.004s
I20260709 13:01:15.001982  8589 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.008610  8589 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.011415  8589 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:15.014722  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:01:15.016629  8369 leader_election.cc:343] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:15.017406  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:01:15.022840  8369 leader_election.cc:343] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:15.023406  8369 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:15.017606  8578 consensus_queue.cc:237] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.027845  8447 tablet_bootstrap.cc:492] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:15.030030  8486 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:15.035774  8611 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:01:15.034884  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 47349.4us
I20260709 13:01:15.056258  8153 catalog_manager.cc:5697] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 0 to 1, leader changed from <none> to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:15.058252  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:15.060933  8331 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:15.064461  8447 tablet_bootstrap.cc:654] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:15.052073  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 46811.3us
I20260709 13:01:15.069921  8331 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:15.069711  8155 catalog_manager.cc:5697] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:15.074681  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:15.075652  8446 tablet_bootstrap.cc:492] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:15.076135  8446 ts_tablet_manager.cc:1403] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.245s	user 0.008s	sys 0.007s
I20260709 13:01:15.078527  8446 raft_consensus.cc:359] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.079169  8446 raft_consensus.cc:385] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:15.079412  8446 raft_consensus.cc:740] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:15.080734  8611 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.081214  8611 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.083112  8611 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:15.083878  8220 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:01:15.085048  8222 leader_election.cc:343] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:15.085477  8689 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:15.085928  8689 raft_consensus.cc:697] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:15.086680  8689 consensus_queue.cc:237] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.087260  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:15.095822  8591 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.096506  8591 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.101554  8448 tablet_bootstrap.cc:492] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:15.102077  8448 ts_tablet_manager.cc:1403] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.241s	user 0.029s	sys 0.028s
I20260709 13:01:15.106609  8591 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:15.106695  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:15.107451  8256 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 0.
I20260709 13:01:15.080124  8446 consensus_queue.cc:260] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.107707  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:15.106961  8448 raft_consensus.cc:359] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.108323  8406 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 0.
I20260709 13:01:15.108441  8448 raft_consensus.cc:385] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:15.108899  8448 raft_consensus.cc:740] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:15.109148  8295 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:15.109637  8448 consensus_queue.cc:260] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.110486  8591 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:15.110843  8591 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.111228  8578 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.111680  8578 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.113734  8448 ts_tablet_manager.cc:1434] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:01:15.111264  8591 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:15.114768  8448 tablet_bootstrap.cc:492] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:15.114624  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:01:15.116046  8220 leader_election.cc:343] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:15.120810  8591 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.121703  8448 tablet_bootstrap.cc:654] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:15.121932  8578 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:15.122938  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:01:15.124347  8222 leader_election.cc:343] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:15.124748  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:15.124794  8222 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:15.124985  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:15.125378  8256 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:15.125562  8406 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:15.132292  8578 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:15.132114  8152 catalog_manager.cc:5697] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:15.134009  8256 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:15.135288  8295 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:15.136175  8614 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:15.132153  8406 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:15.136641  8614 raft_consensus.cc:697] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:15.124089  8591 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:15.137217  8448 tablet_bootstrap.cc:492] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:15.137715  8448 ts_tablet_manager.cc:1403] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.007s
I20260709 13:01:15.137485  8614 consensus_queue.cc:237] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.140811  8448 raft_consensus.cc:359] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.141569  8448 raft_consensus.cc:385] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:15.141841  8448 raft_consensus.cc:740] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:15.142450  8448 consensus_queue.cc:260] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.143134  8446 ts_tablet_manager.cc:1434] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.067s	user 0.005s	sys 0.001s
I20260709 13:01:15.144095  8446 tablet_bootstrap.cc:492] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:15.144629  8448 ts_tablet_manager.cc:1434] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:01:15.145469  8448 tablet_bootstrap.cc:492] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:15.148860  8447 tablet_bootstrap.cc:492] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:15.149297  8447 ts_tablet_manager.cc:1403] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.124s	user 0.025s	sys 0.005s
I20260709 13:01:15.151229  8447 raft_consensus.cc:359] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.151718  8447 raft_consensus.cc:385] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:15.151760  8448 tablet_bootstrap.cc:654] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:15.151983  8447 raft_consensus.cc:740] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:15.155658  8689 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:15.108547  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:15.158092  8330 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 1: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:01:15.159171  8369 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:15.158998  8447 consensus_queue.cc:260] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.160210  8611 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:15.161276  8446 tablet_bootstrap.cc:654] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:15.163076  8448 tablet_bootstrap.cc:492] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:15.163610  8448 ts_tablet_manager.cc:1403] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.008s
I20260709 13:01:15.165717  8447 ts_tablet_manager.cc:1434] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 13:01:15.166265  8448 raft_consensus.cc:359] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.166646  8447 tablet_bootstrap.cc:492] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:15.172096  8447 tablet_bootstrap.cc:654] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:15.174805  8448 raft_consensus.cc:385] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:15.175120  8448 raft_consensus.cc:740] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:15.175774  8448 consensus_queue.cc:260] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.177680  8448 ts_tablet_manager.cc:1434] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 13:01:15.178485  8448 tablet_bootstrap.cc:492] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:15.184897  8448 tablet_bootstrap.cc:654] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:15.193154  8447 tablet_bootstrap.cc:492] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:15.193661  8447 ts_tablet_manager.cc:1403] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.027s	user 0.007s	sys 0.003s
I20260709 13:01:15.208252  8447 raft_consensus.cc:359] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.209177  8447 raft_consensus.cc:385] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:15.209546  8447 raft_consensus.cc:740] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:15.212867  8447 consensus_queue.cc:260] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.215149  8611 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:15.223912  8452 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.224431  8452 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.225036  8446 tablet_bootstrap.cc:492] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:15.225564  8446 ts_tablet_manager.cc:1403] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.082s	user 0.035s	sys 0.029s
I20260709 13:01:15.226568  8452 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:15.227725  8685 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:15.228374  8446 raft_consensus.cc:359] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.229249  8446 raft_consensus.cc:385] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:15.229616  8446 raft_consensus.cc:740] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:15.230551  8446 consensus_queue.cc:260] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.232648  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:15.233186  8446 ts_tablet_manager.cc:1434] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:01:15.233250  8256 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 0.
I20260709 13:01:15.238062  8446 tablet_bootstrap.cc:492] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:15.239936  8447 ts_tablet_manager.cc:1434] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.000s
I20260709 13:01:15.240834  8447 tablet_bootstrap.cc:492] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:15.235189  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:15.241338  8403 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 0.
I20260709 13:01:15.246168  8446 tablet_bootstrap.cc:654] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:15.245774  8153 catalog_manager.cc:5697] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:15.247562  8295 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:15.248925  8614 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:15.249269  8614 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.249605  8614 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:15.258926  8614 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.261857  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:15.262444  8254 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:15.264711  8446 tablet_bootstrap.cc:492] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:15.265291  8446 ts_tablet_manager.cc:1403] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.011s
I20260709 13:01:15.268689  8446 raft_consensus.cc:359] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.269649  8446 raft_consensus.cc:385] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:15.269997  8446 raft_consensus.cc:740] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:15.270849  8446 consensus_queue.cc:260] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.273612  8446 ts_tablet_manager.cc:1434] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:01:15.274641  8446 tablet_bootstrap.cc:492] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:15.282226  8446 tablet_bootstrap.cc:654] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:15.269317  8254 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:15.284662  8295 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:15.285619  8452 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:15.286084  8452 raft_consensus.cc:697] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:15.286878  8452 consensus_queue.cc:237] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.295272  8446 tablet_bootstrap.cc:492] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:15.295917  8446 ts_tablet_manager.cc:1403] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.008s
I20260709 13:01:15.299451  8446 raft_consensus.cc:359] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.300333  8446 raft_consensus.cc:385] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:15.300675  8446 raft_consensus.cc:740] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:15.301589  8446 consensus_queue.cc:260] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.305158  8611 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.305806  8611 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.307777  8611 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:15.308251  8611 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.308772  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:15.308676  8611 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.309562  8328 raft_consensus.cc:2393] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 1: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:01:15.311355  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:15.313201  8369 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:15.314164  8485 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:15.311805  8446 ts_tablet_manager.cc:1434] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.001s
I20260709 13:01:15.314450  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:15.315322  8446 tablet_bootstrap.cc:492] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:15.320466  8448 tablet_bootstrap.cc:492] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:15.320942  8448 ts_tablet_manager.cc:1403] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.143s	user 0.030s	sys 0.046s
I20260709 13:01:15.321174  8446 tablet_bootstrap.cc:654] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:15.322906  8485 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:15.323756  8447 tablet_bootstrap.cc:654] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:15.324982  8614 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:15.329095  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:15.330228  8328 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 1: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:01:15.326172  8448 raft_consensus.cc:359] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.331127  8448 raft_consensus.cc:385] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:15.331398  8448 raft_consensus.cc:740] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:15.331599  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:15.332026  8448 consensus_queue.cc:260] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.332381  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:15.332747  8589 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:15.332887  8406 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:15.333599  8448 ts_tablet_manager.cc:1434] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:01:15.334487  8448 tablet_bootstrap.cc:492] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:15.341786  8687 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.342439  8687 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.343305  8448 tablet_bootstrap.cc:654] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:15.346598  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:01:15.348384  8222 leader_election.cc:343] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:15.345163  8687 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:15.358251  8492 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.358970  8492 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.362893  8492 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:15.366202  8591 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.366874  8591 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.372243  8591 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:15.372758  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:15.373646  8254 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 0.
I20260709 13:01:15.376281  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:15.377140  8328 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 0.
I20260709 13:01:15.378852  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:15.379930  8256 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 0.
I20260709 13:01:15.381232  8295 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:15.382318  8705 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:15.382792  8705 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.383208  8705 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:15.385970  8369 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:15.397922  8492 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:15.398313  8710 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:15.398453  8492 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.399062  8492 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:15.406984  8589 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:15.408654  8155 catalog_manager.cc:5697] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:15.410792  8406 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:15.412714  8611 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:15.422587  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:15.425333  8591 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:15.431051  8705 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.432682  8331 raft_consensus.cc:2393] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 1: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:01:15.442135  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:01:15.442399  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 43618.1us
I20260709 13:01:15.443897  8591 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.444456  8591 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.446575  8591 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:15.453018  8492 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:15.468940  8297 leader_election.cc:343] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:15.478482  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:15.483491  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:15.484294  8332 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:15.497568  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:15.498426  8257 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:15.501748  8295 leader_election.cc:343] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:15.511937  8332 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:15.513602  8687 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:15.513898  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:15.514849  8256 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:15.523897  8447 tablet_bootstrap.cc:492] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:15.524545  8447 ts_tablet_manager.cc:1403] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.284s	user 0.011s	sys 0.002s
I20260709 13:01:15.527025  8257 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:15.474042  8705 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:15.546468  8447 raft_consensus.cc:359] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.477633  8685 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.553808  8685 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.557260  8685 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:15.565213  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:15.566226  8403 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:15.567829  8222 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:15.568986  8687 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:15.569641  8687 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.570124  8687 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:15.476001  8492 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:15.579186  8220 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:15.581146  8689 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:15.583520  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:15.584134  8405 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 0.
I20260709 13:01:15.586050  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:15.587342  8295 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:15.589175  8614 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:15.589654  8614 raft_consensus.cc:697] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:15.590425  8614 consensus_queue.cc:237] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.597951  8492 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.600960  8448 tablet_bootstrap.cc:492] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:15.603297  8448 ts_tablet_manager.cc:1403] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.269s	user 0.085s	sys 0.161s
I20260709 13:01:15.603092  8492 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.578925  8369 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:15.604569  8710 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:15.605069  8492 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:15.605234  8710 raft_consensus.cc:697] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:15.607102  8710 consensus_queue.cc:237] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.611171  8448 raft_consensus.cc:359] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.611987  8448 raft_consensus.cc:385] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:15.611841  8297 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:15.612438  8448 raft_consensus.cc:740] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:15.613190  8448 consensus_queue.cc:260] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.613989  8685 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:15.615371  8447 raft_consensus.cc:385] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:15.615662  8447 raft_consensus.cc:740] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:15.618942  8447 consensus_queue.cc:260] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.619975  8446 tablet_bootstrap.cc:492] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:15.620433  8446 ts_tablet_manager.cc:1403] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.305s	user 0.100s	sys 0.167s
I20260709 13:01:15.620843  8447 ts_tablet_manager.cc:1434] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.096s	user 0.004s	sys 0.000s
I20260709 13:01:15.623092  8446 raft_consensus.cc:359] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.623739  8446 raft_consensus.cc:385] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:15.623822  8705 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:15.624024  8446 raft_consensus.cc:740] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:15.624272  8705 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.624457  8711 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:15.624691  8705 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:15.624715  8446 consensus_queue.cc:260] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.625469  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:15.627291  8446 ts_tablet_manager.cc:1434] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:01:15.628140  8446 tablet_bootstrap.cc:492] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
W20260709 13:01:15.630017  8297 leader_election.cc:343] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:15.631958  8447 tablet_bootstrap.cc:492] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:15.634743  8448 ts_tablet_manager.cc:1434] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.001s
I20260709 13:01:15.635649  8448 tablet_bootstrap.cc:492] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:15.644750  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:15.645443  8257 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 0.
I20260709 13:01:15.645089  8687 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.646119  8452 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:15.641324  8256 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:15.651211  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:15.651829  8330 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:15.658622  8446 tablet_bootstrap.cc:654] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:15.651010  8705 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.661387  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:15.660689  8331 raft_consensus.cc:2393] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 1: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:01:15.662357  8589 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:15.662709  8589 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.663038  8589 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:15.663743  8448 tablet_bootstrap.cc:654] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:15.663748  8705 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:15.673157  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:15.673992  8256 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:15.676995  8154 catalog_manager.cc:5697] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:15.692332  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:15.692957  8330 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:15.694736  8687 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:15.703058  8330 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:15.704906  8448 tablet_bootstrap.cc:492] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:15.707094  8447 tablet_bootstrap.cc:654] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:15.707273  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:15.709723  8448 ts_tablet_manager.cc:1403] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.074s	user 0.016s	sys 0.012s
I20260709 13:01:15.705765  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:15.709820  8153 catalog_manager.cc:5697] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 0 to 1, leader changed from <none> to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:15.701567  8589 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.714632  8578 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:15.714972  8487 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:15.720794  8446 tablet_bootstrap.cc:492] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:15.721299  8446 ts_tablet_manager.cc:1403] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.093s	user 0.010s	sys 0.002s
I20260709 13:01:15.722265  8406 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:15.732946  8589 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:15.728881  8256 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:15.736702  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:15.737914  8295 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:15.738665  8446 raft_consensus.cc:359] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.739360  8446 raft_consensus.cc:385] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:15.739626  8446 raft_consensus.cc:740] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:15.740263  8446 consensus_queue.cc:260] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.742937  8330 raft_consensus.cc:2393] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 1: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:01:15.742276  8446 ts_tablet_manager.cc:1434] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:01:15.732111  8405 raft_consensus.cc:2393] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 1: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:01:15.744460  8446 tablet_bootstrap.cc:492] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:15.743821  8705 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:15.747432  8705 raft_consensus.cc:697] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:15.748347  8705 consensus_queue.cc:237] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.751963  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:15.752753  8254 raft_consensus.cc:2393] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 1: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:01:15.755847  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:15.756588  8711 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:01:15.755091  8220 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:15.757586  8487 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:15.758059  8487 raft_consensus.cc:697] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:15.758860  8487 consensus_queue.cc:237] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.782568  8448 raft_consensus.cc:359] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.785746  8448 raft_consensus.cc:385] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:15.786495  8448 raft_consensus.cc:740] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:15.800465  8591 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:15.798664  8406 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:15.802150  8448 consensus_queue.cc:260] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.819492  8448 ts_tablet_manager.cc:1434] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.109s	user 0.006s	sys 0.002s
I20260709 13:01:15.824070  8155 catalog_manager.cc:5697] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:15.822937  8448 tablet_bootstrap.cc:492] T 1112bcf4c4ea44e1933ee6287bc869f8 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:15.827078  8447 tablet_bootstrap.cc:492] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:15.827594  8447 ts_tablet_manager.cc:1403] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.206s	user 0.016s	sys 0.021s
I20260709 13:01:15.828188  8153 catalog_manager.cc:5697] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:15.841588  8447 raft_consensus.cc:359] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.842502  8447 raft_consensus.cc:385] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:15.842877  8447 raft_consensus.cc:740] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:15.843608  8447 consensus_queue.cc:260] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.845464  8447 ts_tablet_manager.cc:1434] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.002s
I20260709 13:01:15.846431  8447 tablet_bootstrap.cc:492] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:15.875027  8446 tablet_bootstrap.cc:654] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:15.877259  8447 tablet_bootstrap.cc:654] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:15.879683  8591 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.880295  8591 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.882553  8591 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:15.878542  8452 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:15.884469  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:15.885233  8254 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 0.
I20260709 13:01:15.887043  8295 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:15.896209  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:15.897010  8403 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 0.
I20260709 13:01:15.897289  8705 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:15.903813  8705 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.904322  8705 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:15.909575  8203 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:15.928644  8591 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:15.934139  8728 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.936944  8728 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.939021  8728 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:15.939116  8452 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:15.940172  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:01:15.941787  8295 leader_election.cc:343] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:15.947932  8448 tablet_bootstrap.cc:654] T 1112bcf4c4ea44e1933ee6287bc869f8 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:15.959250  8705 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.970535  8446 tablet_bootstrap.cc:492] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:15.962307  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:15.971127  8446 ts_tablet_manager.cc:1403] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.227s	user 0.016s	sys 0.004s
I20260709 13:01:15.971144  8257 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:15.966641  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:15.973502  8403 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:15.974252  8446 raft_consensus.cc:359] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.974951  8446 raft_consensus.cc:385] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:15.975239  8446 raft_consensus.cc:740] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:15.954416  8447 tablet_bootstrap.cc:492] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:15.976362  8447 ts_tablet_manager.cc:1403] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.130s	user 0.027s	sys 0.019s
I20260709 13:01:15.958042  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:15.978940  8447 raft_consensus.cc:359] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.979387  8257 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:15.979602  8447 raft_consensus.cc:385] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:15.979905  8447 raft_consensus.cc:740] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:15.980626  8295 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:15.980341  8446 consensus_queue.cc:260] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.980510  8447 consensus_queue.cc:260] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.981370  8728 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:15.981916  8728 raft_consensus.cc:697] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:15.983274  8448 tablet_bootstrap.cc:492] T 1112bcf4c4ea44e1933ee6287bc869f8 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:15.982771  8728 consensus_queue.cc:237] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.983738  8448 ts_tablet_manager.cc:1403] T 1112bcf4c4ea44e1933ee6287bc869f8 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.161s	user 0.010s	sys 0.002s
I20260709 13:01:15.984660  8447 ts_tablet_manager.cc:1434] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:01:15.985591  8447 tablet_bootstrap.cc:492] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:15.961370  8705 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:15.990072  8448 raft_consensus.cc:359] T 1112bcf4c4ea44e1933ee6287bc869f8 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.991750  8448 raft_consensus.cc:385] T 1112bcf4c4ea44e1933ee6287bc869f8 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:15.992053  8448 raft_consensus.cc:740] T 1112bcf4c4ea44e1933ee6287bc869f8 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:15.992589  8446 ts_tablet_manager.cc:1434] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.002s
I20260709 13:01:15.992822  8448 consensus_queue.cc:260] T 1112bcf4c4ea44e1933ee6287bc869f8 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:15.994673  8446 tablet_bootstrap.cc:492] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:15.995235  8448 ts_tablet_manager.cc:1434] T 1112bcf4c4ea44e1933ee6287bc869f8 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:01:15.996137  8448 tablet_bootstrap.cc:492] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:16.001639  8448 tablet_bootstrap.cc:654] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:15.997448  8403 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:16.005602  8446 tablet_bootstrap.cc:654] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:16.005246  8297 leader_election.cc:343] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:16.014801  8297 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:16.017745  8728 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:16.019954  8446 tablet_bootstrap.cc:492] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:16.020464  8446 ts_tablet_manager.cc:1403] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.004s
I20260709 13:01:16.023609  8446 raft_consensus.cc:359] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.024358  8446 raft_consensus.cc:385] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:16.024667  8446 raft_consensus.cc:740] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:16.025599  8446 consensus_queue.cc:260] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.024560  8155 catalog_manager.cc:5697] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:16.028620  8446 ts_tablet_manager.cc:1434] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:01:16.029676  8446 tablet_bootstrap.cc:492] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:16.039500  8446 tablet_bootstrap.cc:654] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:16.041467  8447 tablet_bootstrap.cc:654] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:16.051128  8448 tablet_bootstrap.cc:492] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:16.051643  8448 ts_tablet_manager.cc:1403] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.056s	user 0.023s	sys 0.020s
I20260709 13:01:16.054081  8448 raft_consensus.cc:359] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.054749  8448 raft_consensus.cc:385] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:16.055006  8448 raft_consensus.cc:740] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:16.055629  8448 consensus_queue.cc:260] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.058642  8448 ts_tablet_manager.cc:1434] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:01:16.061623  8448 tablet_bootstrap.cc:492] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:16.071328  8448 tablet_bootstrap.cc:654] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:16.080497  8689 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.081290  8689 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.083551  8448 tablet_bootstrap.cc:492] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:16.085049  8448 ts_tablet_manager.cc:1403] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.001s
I20260709 13:01:16.085021  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:16.086947  8331 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:16.087455  8711 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.088665  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:16.088595  8711 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.084335  8689 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:16.095494  8403 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:16.096211  8711 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:16.091161  8448 raft_consensus.cc:359] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.092176  8220 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:16.097317  8448 raft_consensus.cc:385] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:16.097657  8448 raft_consensus.cc:740] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:16.098289  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:16.098448  8689 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:16.098346  8448 consensus_queue.cc:260] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.099135  8689 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.099431  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:16.100028  8689 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:16.100054  8331 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 0.
I20260709 13:01:16.101866  8448 ts_tablet_manager.cc:1434] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:01:16.103303  8448 tablet_bootstrap.cc:492] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:16.104867  8711 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.105309  8711 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.107524  8711 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:16.109093  8448 tablet_bootstrap.cc:654] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:16.110649  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:16.111526  8254 raft_consensus.cc:2393] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 1: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:01:16.112510  8369 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:16.113052  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:16.115242  8369 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:16.115968  8492 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:16.116384  8492 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.116729  8492 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:16.118928  8591 consensus_queue.cc:1048] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:16.120397  8710 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:16.121888  8710 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:16.127625  8447 tablet_bootstrap.cc:492] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:16.128156  8447 ts_tablet_manager.cc:1403] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.143s	user 0.030s	sys 0.038s
I20260709 13:01:16.128603  8492 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.130852  8446 tablet_bootstrap.cc:492] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:16.136574  8446 ts_tablet_manager.cc:1403] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.107s	user 0.014s	sys 0.013s
I20260709 13:01:16.145915  8447 raft_consensus.cc:359] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.146893  8447 raft_consensus.cc:385] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:16.147173  8447 raft_consensus.cc:740] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:16.147895  8447 consensus_queue.cc:260] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.150127  8447 ts_tablet_manager.cc:1434] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.004s
I20260709 13:01:16.150942  8447 tablet_bootstrap.cc:492] T 56b5f14332d64504837ae4e2df77e409 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:16.152295  8689 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.154171  8710 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:16.156358  8492 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:16.159855  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:16.161407  8330 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:16.162729  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:16.164623  8257 raft_consensus.cc:2393] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 1: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:01:16.165043  8689 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:16.165130  8614 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:16.159713  8446 raft_consensus.cc:359] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.165933  8446 raft_consensus.cc:385] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:16.166203  8446 raft_consensus.cc:740] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:16.166818  8446 consensus_queue.cc:260] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.167420  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:16.169608  8452 consensus_queue.cc:1048] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:16.170917  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:16.171696  8403 raft_consensus.cc:2393] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 1: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:01:16.176009  8448 tablet_bootstrap.cc:492] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:16.176455  8448 ts_tablet_manager.cc:1403] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.073s	user 0.018s	sys 0.007s
I20260709 13:01:16.177505  8330 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:16.181661  8222 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:16.184056  8446 ts_tablet_manager.cc:1434] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.047s	user 0.003s	sys 0.003s
I20260709 13:01:16.185585  8369 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:16.186890  8446 tablet_bootstrap.cc:492] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:16.191550  8589 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:16.185247  8689 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:01:16.193876  8448 raft_consensus.cc:359] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.198052  8448 raft_consensus.cc:385] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:16.198472  8448 raft_consensus.cc:740] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:16.213997  8448 consensus_queue.cc:260] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.218628  8447 tablet_bootstrap.cc:654] T 56b5f14332d64504837ae4e2df77e409 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:16.229779  8448 ts_tablet_manager.cc:1434] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.001s
I20260709 13:01:16.232198  8685 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:16.218528  8492 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:01:16.236255  8446 tablet_bootstrap.cc:654] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:16.245721  8448 tablet_bootstrap.cc:492] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:16.249125  8589 raft_consensus.cc:697] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:16.252875  8589 consensus_queue.cc:237] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.282418  8448 tablet_bootstrap.cc:654] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:16.289251  8591 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:16.293388  8155 catalog_manager.cc:5697] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 0 to 1, leader changed from <none> to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:16.304229  8447 tablet_bootstrap.cc:492] T 56b5f14332d64504837ae4e2df77e409 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:16.304706  8447 ts_tablet_manager.cc:1403] T 56b5f14332d64504837ae4e2df77e409 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.154s	user 0.033s	sys 0.013s
I20260709 13:01:16.307863  8447 raft_consensus.cc:359] T 56b5f14332d64504837ae4e2df77e409 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.308703  8447 raft_consensus.cc:385] T 56b5f14332d64504837ae4e2df77e409 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:16.309032  8447 raft_consensus.cc:740] T 56b5f14332d64504837ae4e2df77e409 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:16.309082  8591 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.311005  8591 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.313903  8591 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:16.378715  8446 tablet_bootstrap.cc:492] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:16.382450  8446 ts_tablet_manager.cc:1403] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.198s	user 0.014s	sys 0.016s
I20260709 13:01:16.384091  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:16.380030  8448 tablet_bootstrap.cc:492] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:16.380359  8705 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.384617  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:16.385224  8448 ts_tablet_manager.cc:1403] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.140s	user 0.017s	sys 0.012s
I20260709 13:01:16.385322  8705 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.387385  8705 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:16.387850  8448 raft_consensus.cc:359] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:16.388393  8297 leader_election.cc:343] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:16.322034  8447 consensus_queue.cc:260] T 56b5f14332d64504837ae4e2df77e409 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.384702  8446 raft_consensus.cc:359] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.389276  8446 raft_consensus.cc:385] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:16.389590  8446 raft_consensus.cc:740] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:16.390103  8748 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.390166  8446 consensus_queue.cc:260] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.390730  8748 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.388507  8448 raft_consensus.cc:385] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:16.391968  8448 raft_consensus.cc:740] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:16.392643  8448 consensus_queue.cc:260] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.393850  8748 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:16.394166  8448 ts_tablet_manager.cc:1434] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:01:16.395110  8448 tablet_bootstrap.cc:492] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
W20260709 13:01:16.381512  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 66129.3us
I20260709 13:01:16.399006  8446 ts_tablet_manager.cc:1434] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.002s
I20260709 13:01:16.399976  8446 tablet_bootstrap.cc:492] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
W20260709 13:01:16.401626  8295 leader_election.cc:343] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:16.402165  8295 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:16.401767  8447 ts_tablet_manager.cc:1434] T 56b5f14332d64504837ae4e2df77e409 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.097s	user 0.006s	sys 0.008s
I20260709 13:01:16.403335  8591 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:16.404523  8447 tablet_bootstrap.cc:492] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:16.407425  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:16.407795  8448 tablet_bootstrap.cc:654] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:16.408576  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:16.409294  8405 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 0.
I20260709 13:01:16.409865  8446 tablet_bootstrap.cc:654] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:16.412317  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:16.412732  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:16.412998  8255 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 0.
I20260709 13:01:16.415259  8447 tablet_bootstrap.cc:654] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:16.416189  8297 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:16.417527  8705 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:16.417930  8705 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.418431  8705 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:16.425494  8446 tablet_bootstrap.cc:492] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:16.426523  8446 ts_tablet_manager.cc:1403] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.002s
I20260709 13:01:16.428496  8447 tablet_bootstrap.cc:492] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:16.428999  8447 ts_tablet_manager.cc:1403] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.007s
I20260709 13:01:16.429824  8448 tablet_bootstrap.cc:492] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:16.430254  8448 ts_tablet_manager.cc:1403] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.035s	user 0.010s	sys 0.004s
I20260709 13:01:16.431705  8447 raft_consensus.cc:359] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.431993  8705 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.432511  8447 raft_consensus.cc:385] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:16.432862  8447 raft_consensus.cc:740] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:16.432675  8448 raft_consensus.cc:359] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.433399  8448 raft_consensus.cc:385] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:16.438887  8448 raft_consensus.cc:740] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:16.439566  8448 consensus_queue.cc:260] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.441110  8448 ts_tablet_manager.cc:1434] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.011s	user 0.001s	sys 0.003s
W20260709 13:01:16.434130  8369 leader_election.cc:343] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:16.442070  8448 tablet_bootstrap.cc:492] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
W20260709 13:01:16.443707  8369 leader_election.cc:343] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:16.444298  8369 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:16.446084  8589 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:16.433665  8447 consensus_queue.cc:260] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.448040  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:16.448592  8256 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:16.436264  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:16.450182  8405 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:16.438578  8446 raft_consensus.cc:359] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.454180  8446 raft_consensus.cc:385] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:16.454421  8446 raft_consensus.cc:740] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:16.455061  8446 consensus_queue.cc:260] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.456439  8446 ts_tablet_manager.cc:1434] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.030s	user 0.003s	sys 0.000s
I20260709 13:01:16.457216  8446 tablet_bootstrap.cc:492] T fdf12882ac9e4f7d9121a5fcb9d4efaf P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:16.441116  8487 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 13:01:16.459898  8447 ts_tablet_manager.cc:1434] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260709 13:01:16.460812  8447 tablet_bootstrap.cc:492] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:16.434620  8591 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.463479  8256 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:16.463609  8591 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.435670  8705 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:16.465548  8405 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:16.466892  8297 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:16.467214  8591 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:16.468201  8591 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:16.468604  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:16.469485  8591 raft_consensus.cc:697] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:16.471010  8591 consensus_queue.cc:237] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.471942  8446 tablet_bootstrap.cc:654] T fdf12882ac9e4f7d9121a5fcb9d4efaf P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:16.485231  8448 tablet_bootstrap.cc:654] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:16.469889  8297 leader_election.cc:343] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:16.533030  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 63019.7us
W20260709 13:01:16.526469  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 74362.1us
I20260709 13:01:16.551664  8447 tablet_bootstrap.cc:654] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:16.555576  8728 consensus_queue.cc:1048] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:16.562088  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:16.581934  8614 consensus_queue.cc:1048] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20260709 13:01:16.586110  8295 leader_election.cc:343] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:16.587132  8295 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:16.589335  8591 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:16.590008  8487 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.590998  8487 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.593127  8689 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.605772  8614 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.594051  8487 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:16.607298  8747 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.606492  8614 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.608188  8747 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.593968  8689 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.634055  8689 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:16.637979  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:16.638397  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:16.638800  8331 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:16.639439  8332 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:16.654845  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:01:16.657189  8295 leader_election.cc:343] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:16.644194  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:16.653909  8614 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:16.668810  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:16.669507  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:16.670050  8401 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 0.
I20260709 13:01:16.670248  8220 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:16.670400  8402 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:16.674997  8220 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:16.676488  8297 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:16.679165  8762 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:16.681627  8762 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.682060  8762 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:16.683702  8487 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:16.684077  8487 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.684520  8487 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:16.688840  8747 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:16.692365  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:16.685949  8728 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:16.693364  8728 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.694396  8728 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:16.696419  8448 tablet_bootstrap.cc:492] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:16.697126  8448 ts_tablet_manager.cc:1403] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.255s	user 0.022s	sys 0.010s
W20260709 13:01:16.697714  8222 leader_election.cc:343] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:16.701326  8448 raft_consensus.cc:359] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.728830  8448 raft_consensus.cc:385] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:16.729161  8448 raft_consensus.cc:740] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:16.729773  8448 consensus_queue.cc:260] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.701897  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:16.724354  8446 tablet_bootstrap.cc:492] T fdf12882ac9e4f7d9121a5fcb9d4efaf P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:16.706229  8447 tablet_bootstrap.cc:492] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:16.741063  8748 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.741761  8748 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.744267  8748 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:16.747145  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:16.747786  8330 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 0.
I20260709 13:01:16.741326  8447 ts_tablet_manager.cc:1403] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.281s	user 0.010s	sys 0.027s
I20260709 13:01:16.749828  8369 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:16.750099  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:16.751667  8448 ts_tablet_manager.cc:1434] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.054s	user 0.008s	sys 0.000s
I20260709 13:01:16.757019  8487 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.758097  8446 ts_tablet_manager.cc:1403] T fdf12882ac9e4f7d9121a5fcb9d4efaf P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.301s	user 0.015s	sys 0.012s
I20260709 13:01:16.751668  8710 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:16.761274  8710 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.761749  8710 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:16.761957  8155 catalog_manager.cc:5697] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:16.735570  8332 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 0.
I20260709 13:01:16.767644  8728 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.775122  8710 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.775763  8446 raft_consensus.cc:359] T fdf12882ac9e4f7d9121a5fcb9d4efaf P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.776619  8446 raft_consensus.cc:385] T fdf12882ac9e4f7d9121a5fcb9d4efaf P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:16.776965  8446 raft_consensus.cc:740] T fdf12882ac9e4f7d9121a5fcb9d4efaf P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:16.753094  8447 raft_consensus.cc:359] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.781404  8447 raft_consensus.cc:385] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:16.781770  8447 raft_consensus.cc:740] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:16.784312  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:16.784971  8332 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:16.788295  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:01:16.792249  8295 leader_election.cc:343] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:16.794080  8487 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:16.786947  8446 consensus_queue.cc:260] T fdf12882ac9e4f7d9121a5fcb9d4efaf P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.795578  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:16.797495  8728 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:16.798447  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:16.799948  8448 tablet_bootstrap.cc:492] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:16.802418  8369 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:16.777498  8762 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.805838  8401 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:16.809001  8614 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.807763  8406 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:16.810252  8747 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:16.809626  8614 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.810663  8747 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.811132  8747 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:16.814299  8614 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:16.812994  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:16.818034  8330 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:16.819832  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:16.820523  8328 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:16.821141  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:16.810618  8251 raft_consensus.cc:2393] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 1: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:01:16.805864  8710 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:16.824097  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:16.824616  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:16.824581  8492 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:01:16.826174  8297 leader_election.cc:343] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:16.827514  8762 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:01:16.828280  8295 leader_election.cc:343] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:16.828743  8295 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:16.829487  8614 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:16.835942  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:16.840368  8328 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:16.842716  8401 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:16.843998  8222 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:16.844734  8762 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:16.845206  8762 raft_consensus.cc:697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:16.846329  8762 consensus_queue.cc:237] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.850301  8332 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:16.837051  8447 consensus_queue.cc:260] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.851310  8255 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 0.
I20260709 13:01:16.851802  8406 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:16.847864  8369 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:16.852967  8748 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:16.853487  8748 raft_consensus.cc:697] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:16.852977  8330 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:16.853849  8297 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:16.853672  8446 ts_tablet_manager.cc:1434] T fdf12882ac9e4f7d9121a5fcb9d4efaf P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.095s	user 0.005s	sys 0.004s
I20260709 13:01:16.854317  8748 consensus_queue.cc:237] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.855151  8220 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:16.855690  8614 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:16.856642  8447 ts_tablet_manager.cc:1434] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.107s	user 0.006s	sys 0.005s
I20260709 13:01:16.857067  8487 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:16.858987  8487 raft_consensus.cc:697] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:16.860886  8614 raft_consensus.cc:697] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:16.861378  8446 tablet_bootstrap.cc:492] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:16.848433  8747 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.865926  8404 raft_consensus.cc:2393] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 1: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:01:16.873574  8448 tablet_bootstrap.cc:654] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:16.868208  8747 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:16.864454  8487 consensus_queue.cc:237] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.879487  8447 tablet_bootstrap.cc:492] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:16.882611  8589 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:16.881135  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:16.884624  8614 consensus_queue.cc:237] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:16.882203  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:16.896211  8330 raft_consensus.cc:2393] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 1: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:01:16.983690  8369 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:16.992092  8748 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:01:17.004181  8448 tablet_bootstrap.cc:492] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:17.004892  8448 ts_tablet_manager.cc:1403] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.218s	user 0.019s	sys 0.000s
I20260709 13:01:17.024024  8447 tablet_bootstrap.cc:654] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:17.048192  8448 raft_consensus.cc:359] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.072115  8154 catalog_manager.cc:5697] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:17.084647  8153 catalog_manager.cc:5697] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 0 to 1, leader changed from <none> to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:17.087412  8446 tablet_bootstrap.cc:654] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:17.049211  8448 raft_consensus.cc:385] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:17.089468  8448 raft_consensus.cc:740] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:17.095278  8705 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:17.105182  8448 consensus_queue.cc:260] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:17.110682  8203 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:17.111398  8448 ts_tablet_manager.cc:1434] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.106s	user 0.010s	sys 0.000s
I20260709 13:01:17.112574  8448 tablet_bootstrap.cc:492] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:17.121474  8446 tablet_bootstrap.cc:492] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:17.121995  8446 ts_tablet_manager.cc:1403] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.261s	user 0.016s	sys 0.013s
I20260709 13:01:17.126196  8446 raft_consensus.cc:359] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.126874  8446 raft_consensus.cc:385] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:17.127140  8446 raft_consensus.cc:740] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:17.127988  8446 consensus_queue.cc:260] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.130352  8446 ts_tablet_manager.cc:1434] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:01:17.132328  8446 tablet_bootstrap.cc:492] T b3d99c80602e45da9c43a49cf83b8211 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:17.143007  8447 tablet_bootstrap.cc:492] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:17.145095  8447 ts_tablet_manager.cc:1403] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.266s	user 0.014s	sys 0.003s
I20260709 13:01:17.152211  8446 tablet_bootstrap.cc:654] T b3d99c80602e45da9c43a49cf83b8211 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:17.151988  8447 raft_consensus.cc:359] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.152711  8447 raft_consensus.cc:385] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:17.152997  8447 raft_consensus.cc:740] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:17.153877  8447 consensus_queue.cc:260] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.167812  8748 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.168612  8748 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.171713  8748 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:17.174134  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:17.175287  8447 ts_tablet_manager.cc:1434] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 13:01:17.176314  8447 tablet_bootstrap.cc:492] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
W20260709 13:01:17.176139  8369 leader_election.cc:343] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:17.182744  8447 tablet_bootstrap.cc:654] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:17.185603  8452 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:17.173341  8448 tablet_bootstrap.cc:654] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:17.192757  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:17.202765  8155 catalog_manager.cc:5697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:17.205148  8155 catalog_manager.cc:5697] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:17.207044  8369 leader_election.cc:343] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:17.214048  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:17.214811  8748 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:17.208542  8447 tablet_bootstrap.cc:492] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:17.217103  8447 ts_tablet_manager.cc:1403] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.041s	user 0.013s	sys 0.004s
I20260709 13:01:17.220098  8446 tablet_bootstrap.cc:492] T b3d99c80602e45da9c43a49cf83b8211 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:17.220543  8446 ts_tablet_manager.cc:1403] T b3d99c80602e45da9c43a49cf83b8211 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.088s	user 0.013s	sys 0.035s
I20260709 13:01:17.226857  8447 raft_consensus.cc:359] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.227594  8447 raft_consensus.cc:385] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:17.227873  8447 raft_consensus.cc:740] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:17.228598  8447 consensus_queue.cc:260] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.230496  8447 ts_tablet_manager.cc:1434] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:01:17.231366  8447 tablet_bootstrap.cc:492] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:17.234404  8446 raft_consensus.cc:359] T b3d99c80602e45da9c43a49cf83b8211 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.235103  8446 raft_consensus.cc:385] T b3d99c80602e45da9c43a49cf83b8211 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:17.235379  8446 raft_consensus.cc:740] T b3d99c80602e45da9c43a49cf83b8211 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:17.236049  8448 tablet_bootstrap.cc:492] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:17.236033  8446 consensus_queue.cc:260] T b3d99c80602e45da9c43a49cf83b8211 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.236598  8448 ts_tablet_manager.cc:1403] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.124s	user 0.012s	sys 0.002s
I20260709 13:01:17.238325  8446 ts_tablet_manager.cc:1434] T b3d99c80602e45da9c43a49cf83b8211 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:01:17.239357  8446 tablet_bootstrap.cc:492] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:17.239452  8448 raft_consensus.cc:359] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.240370  8448 raft_consensus.cc:385] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:17.240690  8448 raft_consensus.cc:740] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:17.242033  8448 consensus_queue.cc:260] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.244580  8448 ts_tablet_manager.cc:1434] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.002s
I20260709 13:01:17.247687  8447 tablet_bootstrap.cc:654] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:17.248878  8448 tablet_bootstrap.cc:492] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:17.255756  8446 tablet_bootstrap.cc:654] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:17.262094  8447 tablet_bootstrap.cc:492] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:17.263559  8447 ts_tablet_manager.cc:1403] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.032s	user 0.018s	sys 0.003s
I20260709 13:01:17.262964  8448 tablet_bootstrap.cc:654] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:17.266654  8447 raft_consensus.cc:359] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.267381  8447 raft_consensus.cc:385] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:17.267655  8447 raft_consensus.cc:740] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:17.268338  8447 consensus_queue.cc:260] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.269502  8446 tablet_bootstrap.cc:492] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:17.269935  8446 ts_tablet_manager.cc:1403] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.002s
I20260709 13:01:17.270401  8447 ts_tablet_manager.cc:1434] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:01:17.272253  8447 tablet_bootstrap.cc:492] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:17.273146  8446 raft_consensus.cc:359] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.273844  8446 raft_consensus.cc:385] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:17.273972  8448 tablet_bootstrap.cc:492] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:17.274109  8446 raft_consensus.cc:740] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:17.274999  8448 ts_tablet_manager.cc:1403] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.000s
I20260709 13:01:17.275414  8446 consensus_queue.cc:260] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.277612  8446 ts_tablet_manager.cc:1434] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:01:17.278469  8447 tablet_bootstrap.cc:654] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:17.278591  8446 tablet_bootstrap.cc:492] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:17.286617  8446 tablet_bootstrap.cc:654] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:17.277784  8448 raft_consensus.cc:359] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.288303  8448 raft_consensus.cc:385] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:17.288622  8448 raft_consensus.cc:740] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:17.289410  8448 consensus_queue.cc:260] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.294935  8448 ts_tablet_manager.cc:1434] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.000s
I20260709 13:01:17.295930  8448 tablet_bootstrap.cc:492] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:17.297106  8446 tablet_bootstrap.cc:492] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:17.297629  8446 ts_tablet_manager.cc:1403] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.005s
I20260709 13:01:17.300257  8446 raft_consensus.cc:359] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.300980  8446 raft_consensus.cc:385] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:17.301287  8446 raft_consensus.cc:740] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:17.301975  8446 consensus_queue.cc:260] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.304726  8446 ts_tablet_manager.cc:1434] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:01:17.306243  8448 tablet_bootstrap.cc:654] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:17.307297  8447 tablet_bootstrap.cc:492] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:17.307843  8447 ts_tablet_manager.cc:1403] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.036s	user 0.018s	sys 0.000s
I20260709 13:01:17.311007  8447 raft_consensus.cc:359] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.311749  8447 raft_consensus.cc:385] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:17.312060  8447 raft_consensus.cc:740] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:17.312728  8447 consensus_queue.cc:260] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.315389  8447 ts_tablet_manager.cc:1434] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:01:17.316382  8447 tablet_bootstrap.cc:492] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:17.319234  8762 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.319772  8762 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.322531  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:17.322764  8447 tablet_bootstrap.cc:654] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:17.323205  8448 tablet_bootstrap.cc:492] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:17.323096  8332 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:17.323823  8448 ts_tablet_manager.cc:1403] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.001s
I20260709 13:01:17.324515  8220 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:17.325402  8761 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:17.325770  8761 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.326085  8761 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:17.326860  8448 raft_consensus.cc:359] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.327553  8448 raft_consensus.cc:385] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:17.327818  8448 raft_consensus.cc:740] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:17.329792  8448 consensus_queue.cc:260] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.331911  8448 ts_tablet_manager.cc:1434] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 13:01:17.334120  8748 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.334769  8748 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.333352  8448 tablet_bootstrap.cc:492] T f19e818631f543b28a8bec9ba4547527 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
W20260709 13:01:17.339277  8295 consensus_peers.cc:597] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:17.340441  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:17.341771  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:01:17.343168  8369 leader_election.cc:343] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:17.343988  8762 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:01:17.345038  8369 leader_election.cc:343] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:17.337358  8748 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:17.345570  8369 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:17.346735  8747 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:17.356612  8761 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.363796  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:17.364573  8405 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:17.365278  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:17.370048  8332 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:17.384680  8685 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:17.366084  8446 tablet_bootstrap.cc:492] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:17.367103  8762 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:17.394639  8762 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:17.395342  8728 consensus_queue.cc:1048] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:17.402634  8761 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:17.407891  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:17.408555  8402 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:17.416251  8446 tablet_bootstrap.cc:654] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:17.416832  8332 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:17.424649  8255 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:17.426254  8747 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:17.443987  8402 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:17.435325  8614 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.445518  8222 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:17.445343  8614 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.447038  8689 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:17.447937  8614 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:17.448380  8689 raft_consensus.cc:697] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:17.449496  8689 consensus_queue.cc:237] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.452423  8447 tablet_bootstrap.cc:492] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:17.452935  8447 ts_tablet_manager.cc:1403] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.137s	user 0.028s	sys 0.045s
I20260709 13:01:17.455040  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:17.455808  8447 raft_consensus.cc:359] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.456426  8447 raft_consensus.cc:385] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:17.456653  8447 raft_consensus.cc:740] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:17.457224  8447 consensus_queue.cc:260] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:17.457893  8295 leader_election.cc:343] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:17.465960  8446 tablet_bootstrap.cc:492] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:17.468050  8446 ts_tablet_manager.cc:1403] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.102s	user 0.014s	sys 0.000s
I20260709 13:01:17.471143  8446 raft_consensus.cc:359] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.471993  8446 raft_consensus.cc:385] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:17.472360  8446 raft_consensus.cc:740] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:17.473232  8446 consensus_queue.cc:260] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.476142  8446 ts_tablet_manager.cc:1434] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:01:17.477172  8446 tablet_bootstrap.cc:492] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:17.480412  8747 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:17.486650  8447 ts_tablet_manager.cc:1434] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.033s	user 0.004s	sys 0.004s
I20260709 13:01:17.487607  8447 tablet_bootstrap.cc:492] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:17.496929  8447 tablet_bootstrap.cc:654] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:17.498157  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:17.504307  8446 tablet_bootstrap.cc:654] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:17.506910  8448 tablet_bootstrap.cc:654] T f19e818631f543b28a8bec9ba4547527 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:17.516896  8446 tablet_bootstrap.cc:492] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:17.514578  8447 tablet_bootstrap.cc:492] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:17.524144  8446 ts_tablet_manager.cc:1403] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.047s	user 0.018s	sys 0.000s
I20260709 13:01:17.524539  8447 ts_tablet_manager.cc:1403] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.008s
W20260709 13:01:17.514304  8297 leader_election.cc:343] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:17.525089  8297 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:17.527190  8614 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:17.528784  8446 raft_consensus.cc:359] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.529695  8446 raft_consensus.cc:385] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:17.530045  8446 raft_consensus.cc:740] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:17.531026  8446 consensus_queue.cc:260] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.532183  8728 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.532106  8155 catalog_manager.cc:5697] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:17.532737  8728 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.534700  8446 ts_tablet_manager.cc:1434] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:01:17.534981  8728 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:17.535800  8446 tablet_bootstrap.cc:492] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:17.539448  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:17.541874  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:17.548305  8255 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 0.
I20260709 13:01:17.545059  8769 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:01:17.545686  8747 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.550819  8747 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:17.551324  8297 leader_election.cc:343] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:17.553864  8747 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:17.557530  8446 tablet_bootstrap.cc:654] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:17.558768  8710 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.559290  8710 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.559478  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:17.562270  8710 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:17.563364  8769 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.563866  8769 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.567252  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:17.567858  8402 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:17.569597  8222 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:17.570395  8487 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:17.570735  8487 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.571044  8487 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:17.575908  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:17.576625  8328 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 0.
I20260709 13:01:17.536803  8447 raft_consensus.cc:359] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.577137  8447 raft_consensus.cc:385] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:17.577474  8447 raft_consensus.cc:740] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
W20260709 13:01:17.578035  8369 leader_election.cc:343] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:17.578171  8447 consensus_queue.cc:260] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.578799  8369 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:17.579859  8747 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:17.580245  8747 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.580529  8447 ts_tablet_manager.cc:1434] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.056s	user 0.003s	sys 0.002s
I20260709 13:01:17.580595  8747 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:17.585333  8447 tablet_bootstrap.cc:492] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:17.586593  8295 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:01:17.589231  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:17.591871  8332 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:17.593618  8448 tablet_bootstrap.cc:492] T f19e818631f543b28a8bec9ba4547527 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:17.595132  8614 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:17.595667  8614 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.594097  8448 ts_tablet_manager.cc:1403] T f19e818631f543b28a8bec9ba4547527 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.261s	user 0.011s	sys 0.001s
I20260709 13:01:17.596102  8614 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:17.596060  8487 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.597898  8685 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.598416  8685 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.600575  8685 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:17.601513  8448 raft_consensus.cc:359] T f19e818631f543b28a8bec9ba4547527 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.602214  8448 raft_consensus.cc:385] T f19e818631f543b28a8bec9ba4547527 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:17.602502  8448 raft_consensus.cc:740] T f19e818631f543b28a8bec9ba4547527 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:17.602663  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:17.603183  8448 consensus_queue.cc:260] T f19e818631f543b28a8bec9ba4547527 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:17.606474  8222 leader_election.cc:343] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:17.607560  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:17.608381  8447 tablet_bootstrap.cc:654] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:17.609208  8685 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.609781  8685 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.612995  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:17.613660  8402 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:17.614822  8222 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:17.615237  8448 ts_tablet_manager.cc:1434] T f19e818631f543b28a8bec9ba4547527 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:01:17.615612  8689 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:17.616011  8689 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.616262  8448 tablet_bootstrap.cc:492] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:17.616405  8689 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:17.621071  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:17.621713  8332 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:17.625129  8614 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.626794  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:17.642727  8747 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.644675  8685 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:17.647459  8614 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:17.633159  8448 tablet_bootstrap.cc:654] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:17.635550  8769 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:01:17.636260  8369 leader_election.cc:343] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:17.636646  8220 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:01:17.656563  8220 leader_election.cc:343] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:17.656797  8446 tablet_bootstrap.cc:492] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:17.657258  8446 ts_tablet_manager.cc:1403] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.122s	user 0.041s	sys 0.041s
I20260709 13:01:17.657984  8685 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:17.658579  8685 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.659076  8685 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:17.659832  8446 raft_consensus.cc:359] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.660506  8446 raft_consensus.cc:385] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:17.660795  8446 raft_consensus.cc:740] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:17.638197  8761 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.661393  8446 consensus_queue.cc:260] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.661744  8761 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.663381  8446 ts_tablet_manager.cc:1434] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:01:17.664090  8761 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:17.647384  8689 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.665176  8446 tablet_bootstrap.cc:492] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:17.666375  8689 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:17.648813  8487 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:17.650722  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:17.670058  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:17.670668  8406 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:17.671710  8446 tablet_bootstrap.cc:654] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:17.672940  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:17.673597  8403 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:17.651114  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:17.676364  8405 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:17.650297  8747 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:17.653940  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:17.637866  8447 tablet_bootstrap.cc:492] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:17.677951  8255 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:17.678119  8447 ts_tablet_manager.cc:1403] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.097s	user 0.008s	sys 0.007s
I20260709 13:01:17.680387  8685 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.627041  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:17.681890  8330 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 0.
I20260709 13:01:17.682351  8685 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:17.683842  8405 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:17.684118  8446 tablet_bootstrap.cc:492] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:17.681031  8447 raft_consensus.cc:359] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.684468  8447 raft_consensus.cc:385] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:17.684737  8447 raft_consensus.cc:740] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:17.673141  8448 tablet_bootstrap.cc:492] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:17.685181  8446 ts_tablet_manager.cc:1403] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.000s
I20260709 13:01:17.685547  8448 ts_tablet_manager.cc:1403] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.070s	user 0.010s	sys 0.008s
I20260709 13:01:17.685375  8447 consensus_queue.cc:260] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.687479  8447 ts_tablet_manager.cc:1434] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:01:17.688352  8447 tablet_bootstrap.cc:492] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:17.688306  8446 raft_consensus.cc:359] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.688493  8448 raft_consensus.cc:359] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.689280  8446 raft_consensus.cc:385] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:17.689306  8448 raft_consensus.cc:385] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:17.689842  8448 raft_consensus.cc:740] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:17.690177  8446 raft_consensus.cc:740] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:17.690536  8448 consensus_queue.cc:260] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.691020  8446 consensus_queue.cc:260] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.693042  8448 ts_tablet_manager.cc:1434] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:01:17.693336  8446 ts_tablet_manager.cc:1434] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
W20260709 13:01:17.694042  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 57836.3us
I20260709 13:01:17.694483  8446 tablet_bootstrap.cc:492] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:17.698649  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:17.700881  8446 tablet_bootstrap.cc:654] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:17.703275  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:17.677281  8406 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:17.704636  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:17.704838  8492 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:17.705214  8492 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.705778  8492 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:17.705245  8405 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:17.709678  8255 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:17.710882  8222 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:17.711449  8295 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:17.711767  8222 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:17.712595  8761 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:17.713001  8761 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.713158  8762 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:17.713487  8761 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:17.713742  8762 raft_consensus.cc:697] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:17.714690  8762 consensus_queue.cc:237] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.712620  8614 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:17.719072  8614 raft_consensus.cc:697] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:17.720221  8614 consensus_queue.cc:237] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.722152  8448 tablet_bootstrap.cc:492] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:17.722049  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:17.723799  8328 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:17.726858  8222 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:17.729689  8487 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:17.730245  8487 raft_consensus.cc:697] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:17.731002  8487 consensus_queue.cc:237] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.735728  8447 tablet_bootstrap.cc:654] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:17.720690  8297 leader_election.cc:343] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:17.749548  8405 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
W20260709 13:01:17.750973  8369 leader_election.cc:343] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:17.764480  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 43834.8us
I20260709 13:01:17.754848  8328 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:17.756661  8761 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.766100  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:17.774828  8330 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:17.765672  8492 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.767506  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:17.779487  8492 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:17.787699  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:17.785969  8222 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:17.788532  8331 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:17.789945  8369 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:17.791101  8492 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.791808  8492 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.781666  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:17.800395  8446 tablet_bootstrap.cc:492] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:17.800838  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:17.800894  8446 ts_tablet_manager.cc:1403] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.107s	user 0.022s	sys 0.054s
I20260709 13:01:17.837652  8492 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:01:17.815061  8220 leader_election.cc:343] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:17.791018  8689 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:17.852344  8330 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:17.859465  8332 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:17.838462  8728 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.838694  8329 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:17.850270  8747 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:01:17.858672  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 68839.8us
I20260709 13:01:17.877020  8331 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:17.856706  8689 raft_consensus.cc:697] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:17.797086  8253 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:17.891289  8747 raft_consensus.cc:697] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:17.894244  8747 consensus_queue.cc:237] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.896370  8448 tablet_bootstrap.cc:654] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:17.897719  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:17.898589  8256 raft_consensus.cc:2393] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 1: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:01:17.914121  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 73676.9us
I20260709 13:01:17.865715  8728 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.931677  8689 consensus_queue.cc:237] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.938340  8329 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:17.936877  8446 raft_consensus.cc:359] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.953027  8446 raft_consensus.cc:385] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:17.953536  8446 raft_consensus.cc:740] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:17.954427  8446 consensus_queue.cc:260] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.964907  8728 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:17.967489  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:17.972942  8447 tablet_bootstrap.cc:492] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:17.973596  8447 ts_tablet_manager.cc:1403] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.285s	user 0.042s	sys 0.058s
I20260709 13:01:17.978972  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:17.979763  8405 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:17.984694  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:17.988852  8447 raft_consensus.cc:359] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.989774  8447 raft_consensus.cc:385] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:01:17.991072  8295 leader_election.cc:343] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:17.990104  8447 raft_consensus.cc:740] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:17.993860  8447 consensus_queue.cc:260] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:17.995553  8446 ts_tablet_manager.cc:1434] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.158s	user 0.008s	sys 0.000s
I20260709 13:01:17.996651  8446 tablet_bootstrap.cc:492] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:17.997578  8447 ts_tablet_manager.cc:1434] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:01:17.998755  8447 tablet_bootstrap.cc:492] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:18.003716  8448 tablet_bootstrap.cc:492] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:18.004359  8448 ts_tablet_manager.cc:1403] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.282s	user 0.013s	sys 0.004s
I20260709 13:01:18.006613  8405 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
W20260709 13:01:17.977888  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 62828.1us
I20260709 13:01:18.010830  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:18.028059  8332 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:18.018152  8761 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:18.035804  8155 catalog_manager.cc:5697] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:18.038048  8446 tablet_bootstrap.cc:654] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:18.040962  8222 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:18.052132  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:18.053849  8448 raft_consensus.cc:359] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:18.054738  8448 raft_consensus.cc:385] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:18.055013  8448 raft_consensus.cc:740] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:18.051168  8817 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:18.056649  8492 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:18.055667  8448 consensus_queue.cc:260] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:18.057930  8369 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:18.059068  8448 ts_tablet_manager.cc:1434] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.054s	user 0.007s	sys 0.002s
I20260709 13:01:18.061760  8817 raft_consensus.cc:697] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:18.062204  8827 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:18.062745  8817 consensus_queue.cc:237] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:18.073099  8447 tablet_bootstrap.cc:654] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:18.076956  8492 raft_consensus.cc:697] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:18.079514  8492 consensus_queue.cc:237] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:18.084408  8448 tablet_bootstrap.cc:492] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:18.097911  8452 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.098441  8452 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:18.125197  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:18.127669  8452 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:01:18.140602  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 55758.1us
I20260709 13:01:18.104014  8762 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.040s
I20260709 13:01:18.228945  8448 tablet_bootstrap.cc:654] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:18.267290  8827 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:01:18.306133  8203 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:18.340564  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:18.341279  8447 tablet_bootstrap.cc:492] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:18.341768  8447 ts_tablet_manager.cc:1403] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.343s	user 0.028s	sys 0.003s
W20260709 13:01:18.343454  8831 log.cc:927] Time spent T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:01:18.345122  8818 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:01:18.346086  8829 log.cc:927] Time spent T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
W20260709 13:01:18.346241  8832 log.cc:927] Time spent T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:01:18.346015  8818 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:18.350138  8828 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.350447  8446 tablet_bootstrap.cc:492] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:18.350919  8446 ts_tablet_manager.cc:1403] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.355s	user 0.010s	sys 0.021s
I20260709 13:01:18.352443  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:18.352938  8447 raft_consensus.cc:359] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:18.346122  8819 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.353632  8447 raft_consensus.cc:385] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:18.353945  8447 raft_consensus.cc:740] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:18.353916  8819 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:18.350658  8828 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:18.359830  8685 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:01:18.360927  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 253401us
I20260709 13:01:18.355765  8447 consensus_queue.cc:260] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:18.371788  8828 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:18.374532  8448 tablet_bootstrap.cc:492] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:18.375039  8448 ts_tablet_manager.cc:1403] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.306s	user 0.012s	sys 0.003s
I20260709 13:01:18.377476  8819 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:18.379513  8448 raft_consensus.cc:359] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:18.381333  8448 raft_consensus.cc:385] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:18.381630  8448 raft_consensus.cc:740] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:18.383724  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:18.382980  8448 consensus_queue.cc:260] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:18.390372  8326 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:18.406687  8448 ts_tablet_manager.cc:1434] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260709 13:01:18.409518  8447 ts_tablet_manager.cc:1434] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.067s	user 0.007s	sys 0.000s
I20260709 13:01:18.419071  8447 tablet_bootstrap.cc:492] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:18.418906  8835 consensus_queue.cc:1048] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:01:18.432564  8448 tablet_bootstrap.cc:492] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:18.443315  8446 raft_consensus.cc:359] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:18.444362  8446 raft_consensus.cc:385] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:18.444722  8446 raft_consensus.cc:740] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:18.445529  8446 consensus_queue.cc:260] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:18.482057  8446 ts_tablet_manager.cc:1434] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.131s	user 0.005s	sys 0.012s
I20260709 13:01:18.483355  8446 tablet_bootstrap.cc:492] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:18.502079  8728 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:18.504315  8326 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:18.509773  8154 catalog_manager.cc:5697] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 0 to 1, leader changed from <none> to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:18.518600  8818 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:01:18.554673  8369 leader_election.cc:343] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:18.557080  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:18.563959  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:18.564558  8331 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 0.
I20260709 13:01:18.583639  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:18.584528  8400 raft_consensus.cc:2393] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 1: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:01:18.576526  8254 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:18.591595  8153 catalog_manager.cc:5697] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:18.593335  8153 catalog_manager.cc:5697] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:18.600823  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:01:18.605268  8369 leader_election.cc:343] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:18.616246  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:18.617352  8328 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 1: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:01:18.636750  8447 tablet_bootstrap.cc:654] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:18.649492  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:18.651259  8685 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:01:18.659039  8448 tablet_bootstrap.cc:654] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:18.651144  8819 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:01:18.657003  8827 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:18.670687  8830 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:18.690218  8446 tablet_bootstrap.cc:654] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:18.664211  8827 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.662840  8710 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20260709 13:01:18.699890  8827 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:18.705811  8850 log.cc:927] Time spent T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
W20260709 13:01:18.720356  8369 consensus_peers.cc:597] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:18.734781  8830 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
W20260709 13:01:18.762190  8297 leader_election.cc:343] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:18.762823  8297 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:18.754516  8155 catalog_manager.cc:5697] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 0 to 1, leader changed from <none> to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:18.750257  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:18.766855  8614 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:01:18.826191  8852 log.cc:927] Time spent T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:01:18.832003  8447 tablet_bootstrap.cc:492] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:18.835093  8836 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:18.865494  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 90383.2us
W20260709 13:01:18.872604  8853 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:01:18.912057  8836 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:18.938971  8836 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:18.984488  8827 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:19.018754  8447 ts_tablet_manager.cc:1403] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.603s	user 0.040s	sys 0.071s
I20260709 13:01:19.020294  8448 tablet_bootstrap.cc:492] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:19.021492  8448 ts_tablet_manager.cc:1403] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.603s	user 0.017s	sys 0.016s
W20260709 13:01:19.026257  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 154675us
I20260709 13:01:19.028877  8833 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.041950  8845 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:01:19.085377  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 46282.5us
I20260709 13:01:19.086665  8297 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:19.041160  8833 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:19.090899  8827 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:19.095646  8866 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:19.096110  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:19.099987  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:19.100175  8448 raft_consensus.cc:359] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:19.103169  8448 raft_consensus.cc:385] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:19.103569  8448 raft_consensus.cc:740] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:19.104588  8448 consensus_queue.cc:260] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:19.131934  8446 tablet_bootstrap.cc:492] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:19.132920  8446 ts_tablet_manager.cc:1403] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.650s	user 0.013s	sys 0.023s
I20260709 13:01:19.145574  8448 ts_tablet_manager.cc:1434] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.123s	user 0.007s	sys 0.001s
I20260709 13:01:19.145051  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:19.150043  8814 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:01:19.159369  8448 tablet_bootstrap.cc:492] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
I20260709 13:01:19.159538  8826 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:19.160087  8826 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:19.160916  8833 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:19.166006  8447 raft_consensus.cc:359] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:19.166893  8447 raft_consensus.cc:385] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:19.167259  8447 raft_consensus.cc:740] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:19.167824  8826 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:19.169663  8839 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 13:01:19.189348  8447 consensus_queue.cc:260] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:19.189949  8761 consensus_queue.cc:1048] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:01:19.189251  8220 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:19.209595  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:19.205886  8819 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:01:19.270797  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 52172.1us
I20260709 13:01:19.271224  8447 ts_tablet_manager.cc:1434] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.252s	user 0.008s	sys 0.000s
I20260709 13:01:19.272039  8446 raft_consensus.cc:359] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:19.272729  8446 raft_consensus.cc:385] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:19.273010  8446 raft_consensus.cc:740] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:19.273622  8446 consensus_queue.cc:260] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:19.185484  8769 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:01:19.277138  8329 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:19.300727  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 179421us
I20260709 13:01:19.303788  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:19.304534  8328 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 0.
I20260709 13:01:19.306288  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:19.307583  8446 ts_tablet_manager.cc:1434] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.174s	user 0.004s	sys 0.004s
I20260709 13:01:19.310201  8446 tablet_bootstrap.cc:492] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:19.316790  8153 catalog_manager.cc:5697] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:19.318602  8153 catalog_manager.cc:5697] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:19.321962  8447 tablet_bootstrap.cc:492] T a0493b0dbbc746408a5db9e698678d83 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:19.327850  8448 tablet_bootstrap.cc:654] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:19.333037  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 55206.1us
I20260709 13:01:19.338809  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:01:19.347800  8222 leader_election.cc:343] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:19.338147  8432 tablet.cc:2406] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:01:19.364889  8871 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:19.365348  8871 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.365711  8871 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:19.395995  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 52102us
W20260709 13:01:19.450402  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 101491us
W20260709 13:01:19.462769  8203 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:19.490983  8836 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.491636  8836 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:19.494220  8836 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:01:19.536253  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 262234us
I20260709 13:01:19.537334  8220 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:19.716795  8846 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.717550  8846 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:19.720173  8846 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:01:19.742478  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 205262us
I20260709 13:01:19.742739  8329 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:19.743388  8836 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:19.749982  8446 tablet_bootstrap.cc:654] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:19.751302  8871 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:19.753328  8871 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:19.763640  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:19.764623  8397 raft_consensus.cc:2393] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 1: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:01:19.764717  8447 tablet_bootstrap.cc:654] T a0493b0dbbc746408a5db9e698678d83 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:19.769711  8369 leader_election.cc:343] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:19.773078  8220 leader_election.cc:343] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:19.773665  8220 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:19.780356  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:19.768782  8876 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.782687  8876 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:19.787045  8875 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:19.787325  8876 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:19.787715  8875 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:19.799261  8848 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.801537  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:19.803654  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:19.804343  8331 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 0.
I20260709 13:01:19.800189  8848 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:19.809284  8821 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:19.829823  8487 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:19.830464  8487 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:19.837500  8446 tablet_bootstrap.cc:492] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:19.838315  8446 ts_tablet_manager.cc:1403] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.528s	user 0.015s	sys 0.014s
I20260709 13:01:19.867568  8220 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:01:19.836126  8821 raft_consensus.cc:697] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:19.872341  8814 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:19.879855  8814 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.907086  8448 tablet_bootstrap.cc:492] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:19.907757  8448 ts_tablet_manager.cc:1403] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.750s	user 0.014s	sys 0.024s
I20260709 13:01:19.904209  8446 raft_consensus.cc:359] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:19.910450  8446 raft_consensus.cc:385] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:19.911286  8879 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:19.912563  8446 raft_consensus.cc:740] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
W20260709 13:01:19.919013  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 49693.9us
I20260709 13:01:19.909821  8814 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:19.912444  8818 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:19.911881  8879 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:19.948465  8487 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:01:19.971338  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 43527.4us
I20260709 13:01:19.932731  8818 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:19.975703  8448 raft_consensus.cc:359] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:19.976442  8448 raft_consensus.cc:385] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:19.976723  8448 raft_consensus.cc:740] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:19.981101  8448 consensus_queue.cc:260] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:19.999706  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:01:20.006310  8896 log.cc:927] Time spent T f19e818631f543b28a8bec9ba4547527 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:01:20.014433  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:20.021507  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:20.022370  8325 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 0.
I20260709 13:01:20.025233  8448 ts_tablet_manager.cc:1434] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.117s	user 0.011s	sys 0.003s
I20260709 13:01:20.026744  8448 tablet_bootstrap.cc:492] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728: Bootstrap starting.
W20260709 13:01:20.029717  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 57735.6us
I20260709 13:01:20.030673  8818 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:19.920892  8821 consensus_queue.cc:237] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.032752  8369 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:20.008407  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:20.055341  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:20.056267  8877 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:20.033697  8899 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:20.036854  8816 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:20.058312  8257 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:20.068840  8836 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:20.069473  8836 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.070308  8446 consensus_queue.cc:260] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.048697  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:20.044204  8814 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.067581  8816 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.092995  8447 tablet_bootstrap.cc:492] T a0493b0dbbc746408a5db9e698678d83 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:20.093704  8447 ts_tablet_manager.cc:1403] T a0493b0dbbc746408a5db9e698678d83 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.772s	user 0.009s	sys 0.018s
I20260709 13:01:20.089715  8836 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:20.104657  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:20.136684  8446 ts_tablet_manager.cc:1434] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.298s	user 0.017s	sys 0.010s
W20260709 13:01:20.137882  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 58361.7us
I20260709 13:01:20.056898  8877 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:20.228822  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 90028.9us
I20260709 13:01:20.229445  8255 raft_consensus.cc:2393] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 1: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:01:20.083716  8325 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:20.254356  8814 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:20.254814  8849 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:20.259380  8446 tablet_bootstrap.cc:492] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f: Bootstrap starting.
I20260709 13:01:20.094726  8816 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:20.260650  8867 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:20.260586  8824 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:20.072779  8899 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:20.269644  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 179999us
I20260709 13:01:20.259138  8821 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.257491  8845 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:01:20.327783  8900 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:01:20.328635  8452 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:20.329123  8452 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
W20260709 13:01:20.336531  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 80950.5us
I20260709 13:01:20.336875  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:20.255398  8849 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:20.340071  8903 log.cc:927] Time spent T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:01:20.326431  8824 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.332662  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:20.335074  8848 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:20.351050  8452 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:20.279665  8821 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.332468  8897 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:20.340878  8614 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:20.346787  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:20.354192  8846 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:20.272473  8867 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.350355  8877 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:20.282232  8845 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.361464  8897 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.365254  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:20.352252  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:20.366773  8402 raft_consensus.cc:2393] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 1: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:01:20.369257  8824 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:20.453184  8446 tablet_bootstrap.cc:654] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:20.447939  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 74674.9us
W20260709 13:01:20.453923  8369 leader_election.cc:343] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:20.361505  8846 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:20.411572  8905 log.cc:927] Time spent T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:01:20.450129  8448 tablet_bootstrap.cc:654] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:20.450196  8814 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:20.450736  8867 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:20.450598  8897 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:20.449018  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:20.356930  8614 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.415402  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:20.456856  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:20.465157  8871 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:20.460606  8447 raft_consensus.cc:359] T a0493b0dbbc746408a5db9e698678d83 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.468221  8447 raft_consensus.cc:385] T a0493b0dbbc746408a5db9e698678d83 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:20.468557  8447 raft_consensus.cc:740] T a0493b0dbbc746408a5db9e698678d83 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:20.466132  8295 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:20.472090  8452 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:20.474212  8845 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:20.476091  8447 consensus_queue.cc:260] T a0493b0dbbc746408a5db9e698678d83 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.484782  8899 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:20.471946  8871 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.488405  8871 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:20.485903  8849 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:20.479067  8908 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:20.489480  8908 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:20.489876  8908 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:20.491901  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:20.493103  8447 ts_tablet_manager.cc:1434] T a0493b0dbbc746408a5db9e698678d83 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.399s	user 0.010s	sys 0.002s
I20260709 13:01:20.492220  8614 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:20.494184  8821 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:20.489225  8153 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45434 (request call id 96) took 1741 ms. Trace:
I20260709 13:01:20.478211  8845 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.476938  8325 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:20.490466  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:20.466123  8814 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.519928  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:20.520413  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:20.520576  8331 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:20.465369  8816 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.518138  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:20.521540  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:20.529376  8816 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:20.530575  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:20.518288  8845 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.527380  8369 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:20.534564  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:20.536773  8826 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:20.537138  8826 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:20.537545  8826 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:20.537742  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:20.538324  8255 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:20.545631  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:20.548264  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:20.559440  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:20.560607  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:20.560824  8879 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:20.565752  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:20.551028  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:20.551282  8816 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:20.569721  8220 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:20.574373  8487 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:20.583372  8255 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:20.585547  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:01:20.585704  8369 leader_election.cc:343] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:20.566849  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:20.586186  8816 raft_consensus.cc:697] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:20.565258  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:20.590659  8249 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 0.
I20260709 13:01:20.593207  8814 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:20.587535  8761 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:20.593611  8816 consensus_queue.cc:237] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.594305  8761 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.596235  8761 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:20.606580  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:20.607764  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:20.609678  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:20.609735  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:20.610222  8325 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:20.613159  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:20.604024  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:20.614104  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:20.603461  8908 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.618240  8845 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:20.622310  8446 tablet_bootstrap.cc:492] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f: No bootstrap required, opened a new log
I20260709 13:01:20.623168  8446 ts_tablet_manager.cc:1403] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f: Time spent bootstrapping tablet: real 0.365s	user 0.019s	sys 0.020s
I20260709 13:01:20.626559  8446 raft_consensus.cc:359] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.628258  8447 tablet_bootstrap.cc:492] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:20.627837  8826 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.627202  8446 raft_consensus.cc:385] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:20.632227  8446 raft_consensus.cc:740] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Initialized, Role: FOLLOWER
I20260709 13:01:20.632788  8446 consensus_queue.cc:260] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.635298  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:20.636551  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:20.637190  8405 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:20.497982  8153 rpcz_store.cc:276] 0709 13:01:18.750161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.754180 (+  4019us) service_pool.cc:224] Handling call
0709 13:01:20.489104 (+1734924us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:19.743085 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:19.744359 (+  1274us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:19.744398 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:01:19.744415 (+    17us) tablet.cc:662] Decoding operations
0709 13:01:19.744838 (+   423us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:19.744899 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:19.744933 (+    34us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:01:19.745288 (+   355us) tablet.cc:701] Row locks acquired
0709 13:01:19.745310 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:01:19.745458 (+   148us) write_op.cc:270] Start()
0709 13:01:19.745599 (+   141us) write_op.cc:276] Timestamp: P: 1783602079745399 usec, L: 0
0709 13:01:19.745641 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 13:01:19.785602 (+ 39961us) log.cc:844] Serialized 916 byte log entry
0709 13:01:19.834020 (+ 48418us) op_driver.cc:464] REPLICATION: finished
0709 13:01:19.962716 (+128696us) write_op.cc:301] APPLY: starting
0709 13:01:19.967507 (+  4791us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:19.978660 (+ 11153us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:20.017695 (+ 39035us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:20.018026 (+   331us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:20.018194 (+   168us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:01:20.018248 (+    54us) write_op.cc:312] APPLY: finished
0709 13:01:20.018460 (+   212us) log.cc:844] Serialized 28 byte log entry
0709 13:01:20.018648 (+   188us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:20.018733 (+    85us) write_op.cc:454] Released schema lock
0709 13:01:20.018772 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:20.019126 (+   354us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:20.019158 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4979,"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":102185,"apply.run_cpu_time_us":2195,"apply.run_wall_time_us":70183,"num_ops":2,"prepare.queue_time_us":118,"prepare.run_cpu_time_us":3240,"prepare.run_wall_time_us":42792,"replication_time_us":73114,"thread_start_us":48359,"threads_started":1}]]}
I20260709 13:01:20.634404  8908 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:20.640563  8448 tablet_bootstrap.cc:492] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728: No bootstrap required, opened a new log
I20260709 13:01:20.640658  8447 tablet_bootstrap.cc:654] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:20.641331  8448 ts_tablet_manager.cc:1403] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728: Time spent bootstrapping tablet: real 0.615s	user 0.014s	sys 0.017s
I20260709 13:01:20.643391  8446 ts_tablet_manager.cc:1434] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:01:20.643895  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:20.647444  8846 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:01:20.652798  8203 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:20.656311  8297 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:20.657141  8846 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:20.657788  8325 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:20.659850  8826 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:20.661255  8297 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:20.663291  8369 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:20.666251  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:20.670346  8876 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:20.670677  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:20.670331  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:20.672272  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:20.670899  8846 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:20.675683  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:20.676469  8329 raft_consensus.cc:2393] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 1: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:01:20.677212  8404 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:20.678333  8748 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:20.678767  8748 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.679100  8748 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:20.685808  8297 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:20.688442  8847 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:20.691556  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:20.693536  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:20.702168  8154 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45448 (request call id 108) took 1106 ms. Trace:
I20260709 13:01:20.702553  8154 rpcz_store.cc:276] 0709 13:01:19.595638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.595765 (+   127us) service_pool.cc:224] Handling call
0709 13:01:20.702110 (+1106345us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":236700,"reported_tablets":1}
I20260709 13:01:20.704035  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:20.701181  8155 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45456 (request call id 97) took 1290 ms. Trace:
I20260709 13:01:20.705524  8155 rpcz_store.cc:276] 0709 13:01:19.447903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.450471 (+  2568us) service_pool.cc:224] Handling call
0709 13:01:20.701118 (+1250647us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":441875,"reported_tablets":5}
I20260709 13:01:20.711267  8297 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:20.712808  8297 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:20.708806  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:20.713757  8846 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:20.715764  8297 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:20.717190  8848 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:20.718261  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:20.719918  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:20.721457  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:20.714445  8614 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:20.713990  8830 consensus_queue.cc:1048] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:01:20.723646  8816 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.724117  8816 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.726024  8816 leader_election.cc:290] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:20.710709  8876 raft_consensus.cc:697] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:20.727501  8876 consensus_queue.cc:237] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.717386  8297 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:20.734068  8297 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:20.733448  8848 consensus_queue.cc:1059] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:20.734768  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:20.735378  8848 raft_consensus.cc:3055] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:20.735734  8848 raft_consensus.cc:740] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:20.733952  8614 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:20.736546  8848 consensus_queue.cc:260] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.737154  8614 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:20.737638  8614 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:20.737973  8908 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:20.739558  8448 raft_consensus.cc:359] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.740279  8448 raft_consensus.cc:385] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:20.740566  8448 raft_consensus.cc:740] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Initialized, Role: FOLLOWER
I20260709 13:01:20.741199  8748 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.742930  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:20.745092  8369 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:20.741235  8448 consensus_queue.cc:260] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.750963  8614 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.753170  8908 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:20.756328  8614 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:20.758860  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:20.759445  8403 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 0.
I20260709 13:01:20.759577  8220 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:20.760921  8220 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:20.764403  8897 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:20.764490  8814 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:20.766323  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:20.767525  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:20.769022  8448 ts_tablet_manager.cc:1434] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728: Time spent starting tablet: real 0.127s	user 0.003s	sys 0.004s
I20260709 13:01:20.745064  8849 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:20.775579  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:20.770427  8297 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:20.771610  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:20.773589  8222 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:20.778445  8222 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:20.779109  8761 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:20.779696  8835 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:20.776273  8332 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 0.
I20260709 13:01:20.780995  8849 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:20.780925  8848 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:20.781759  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 106966us
W20260709 13:01:20.782554  8295 leader_election.cc:343] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:20.782776  8849 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.783198  8849 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:20.783802  8295 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:20.784608  8614 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:20.786765  8222 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:20.787649  8405 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:20.787829  8761 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:20.790817  8222 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:20.791540  8761 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:20.794051  8220 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:20.794778  8899 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:01:20.795715  8849 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.797389  8222 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:20.797659  8849 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:20.798238  8899 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:20.804095  8748 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:20.806545  8369 leader_election.cc:304] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:20.807509  8826 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:20.807861  8826 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.808171  8826 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:20.810623  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:20.811388  8255 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:20.815241  8899 consensus_queue.cc:1059] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:20.815464  8826 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.816988  8899 raft_consensus.cc:3055] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:20.817333  8899 raft_consensus.cc:740] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:01:20.818890  8369 leader_election.cc:343] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:20.820078  8369 consensus_peers.cc:597] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:20.818940  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:20.818471  8899 consensus_queue.cc:260] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.820837  8332 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:20.821748  8899 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:20.824174  8255 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:20.828125  8332 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:20.828356  8447 tablet_bootstrap.cc:492] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:20.829028  8447 ts_tablet_manager.cc:1403] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.201s	user 0.013s	sys 0.002s
I20260709 13:01:20.830825  8826 leader_election.cc:290] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:20.832558  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:20.833276  8405 raft_consensus.cc:2393] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 1: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:01:20.836892  8447 raft_consensus.cc:359] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.837651  8447 raft_consensus.cc:385] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:20.838312  8447 raft_consensus.cc:740] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:20.838989  8447 consensus_queue.cc:260] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.840978  8447 ts_tablet_manager.cc:1434] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:01:20.842633  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:20.844579  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:20.842867  8447 tablet_bootstrap.cc:492] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:20.855655  8447 tablet_bootstrap.cc:654] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:20.856983  8295 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:20.857787  8849 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:01:20.858141  8295 leader_election.cc:343] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:20.858728  8295 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:20.859586  8849 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:01:20.863615  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:20.866317  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:20.867328  8332 raft_consensus.cc:2393] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 1: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:01:20.867774  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:20.868474  8876 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:20.868947  8876 raft_consensus.cc:697] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:20.869763  8876 consensus_queue.cc:237] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:20.881605  8369 leader_election.cc:343] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:20.889231  8154 catalog_manager.cc:5697] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 0 to 1, leader changed from <none> to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:01:20.892241  8369 leader_election.cc:304] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:20.893244  8447 tablet_bootstrap.cc:492] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:20.893860  8447 ts_tablet_manager.cc:1403] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.051s	user 0.016s	sys 0.002s
I20260709 13:01:20.894109  8748 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:01:20.899250  8447 raft_consensus.cc:359] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.899894  8447 raft_consensus.cc:385] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:20.900148  8447 raft_consensus.cc:740] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:20.901347  8447 consensus_queue.cc:260] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.911727  8447 ts_tablet_manager.cc:1434] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 13:01:20.912680  8447 tablet_bootstrap.cc:492] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b: Bootstrap starting.
I20260709 13:01:20.920835  8447 tablet_bootstrap.cc:654] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:20.926004  8255 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:20.951768  8153 catalog_manager.cc:5697] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 0 to 1, leader changed from <none> to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:20.957508  8328 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:20.958992  8897 consensus_queue.cc:1059] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:20.959908  8899 raft_consensus.cc:3055] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:20.960284  8899 raft_consensus.cc:740] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:20.961097  8899 consensus_queue.cc:260] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:20.962706  8899 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:20.964148  8153 catalog_manager.cc:5697] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 0 to 1, leader changed from <none> to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:20.978871  8447 tablet_bootstrap.cc:492] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b: No bootstrap required, opened a new log
I20260709 13:01:20.979452  8447 ts_tablet_manager.cc:1403] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b: Time spent bootstrapping tablet: real 0.067s	user 0.011s	sys 0.027s
I20260709 13:01:20.999451  8447 raft_consensus.cc:359] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:21.000119  8447 raft_consensus.cc:385] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:21.000365  8447 raft_consensus.cc:740] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Initialized, Role: FOLLOWER
I20260709 13:01:21.002692  8447 consensus_queue.cc:260] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:21.005085  8447 ts_tablet_manager.cc:1434] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.002s
I20260709 13:01:21.010272  8153 catalog_manager.cc:5697] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 0 to 1, leader changed from <none> to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:21.023767  8876 consensus_queue.cc:1048] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:21.035553  8827 consensus_queue.cc:1048] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:21.163966  8876 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:21.173146  8871 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:21.298980  8899 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.299471  8899 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:21.301321  8899 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:21.301847  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:21.302250  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:21.303629  8222 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:21.304273  8899 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:21.317231  8748 consensus_queue.cc:1048] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:21.328428  8827 consensus_queue.cc:1048] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:21.380803  8255 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:21.403820  8761 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.404297  8761 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:21.408161  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:21.408432  8761 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:21.408845  8405 raft_consensus.cc:2393] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 1: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:01:21.410180  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:21.410763  8332 raft_consensus.cc:2393] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 1: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:01:21.411559  8220 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:21.412259  8761 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:21.418236  8761 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:01:21.690117  8203 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:21.999207  8899 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.999783  8899 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:22.001556  8899 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:22.002164  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:22.002425  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:22.003835  8222 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:22.004528  8899 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:22.006623  8848 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.007176  8848 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:22.008786  8848 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:22.009393  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:22.009716  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:22.010370  8399 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:22.011406  8297 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:22.012009  8848 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:22.012276  8848 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:22.012482  8848 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:22.017824  8848 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:22.019934  8848 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:22.020539  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:22.020961  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:22.021518  8399 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:22.026307  8399 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:01:22.027370  8297 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:22.028338  8614 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:22.028828  8614 raft_consensus.cc:697] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:22.029964  8614 consensus_queue.cc:237] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:22.042807  8153 catalog_manager.cc:5697] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:22.091527  8899 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.092296  8899 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:22.094733  8899 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:22.095446  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:22.095577  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:22.096972  8220 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:22.097608  8899 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:22.154768  8827 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:22.155189  8827 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:22.156679  8827 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:22.157392  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:22.157622  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:22.157979  8255 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:01:22.159091  8369 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:22.159650  8827 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:22.159943  8827 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:22.160290  8827 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:22.166965  8827 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:22.168756  8827 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:22.169282  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:22.169633  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:22.174302  8255 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:01:22.175094  8369 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:22.175786  8827 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:22.176292  8827 raft_consensus.cc:697] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:22.177158  8827 consensus_queue.cc:237] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:22.186759  8153 catalog_manager.cc:5697] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 1 to 2, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 2 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:22.437379  8848 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:22.438098  8848 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:22.439808  8848 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:22.440713  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:22.440871  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:22.441543  8255 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:22.441704  8399 raft_consensus.cc:2393] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 2: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:01:22.442598  8295 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:22.443300  8848 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:22.443663  8848 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:22.444010  8848 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:22.450402  8848 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:22.453056  8848 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:22.454809  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:22.455595  8405 raft_consensus.cc:2393] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 2: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:01:22.455675  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:22.462656  8255 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:22.466295  8295 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:01:22.467281  8848 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:22.467779  8848 raft_consensus.cc:697] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:22.468636  8848 consensus_queue.cc:237] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:22.483037  8153 catalog_manager.cc:5697] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 1 to 2, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:22.562593  8328 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:22.563728  8871 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:22.591141  8404 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:22.598459  8614 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:22.602155  8255 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:22.605242  8871 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:22.615355  8252 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:01:22.623607  8252 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:22.625905  8848 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:01:22.720623  8203 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:22.792332  8848 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.792793  8848 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:22.794517  8848 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:22.795019  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:22.795627  8255 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:22.795549  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:22.796303  8399 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:22.796675  8295 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:22.797358  8848 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:22.797684  8848 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:22.797875  8848 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:22.801365  8947 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.801842  8947 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:22.802711  8848 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:22.803925  8947 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:22.804544  8848 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:22.804801  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:22.804466  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:22.805208  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:22.805723  8255 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:01:22.805727  8252 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:22.805816  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:22.806614  8399 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:22.806780  8369 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:22.807521  8947 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:22.807862  8947 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:22.808183  8947 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:22.811383  8252 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:22.812633  8295 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:22.813454  8848 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:22.813409  8399 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:22.814337  8848 raft_consensus.cc:697] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:22.815414  8947 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:22.816242  8848 consensus_queue.cc:237] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:22.817337  8947 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:22.818398  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:22.818398  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:22.819072  8252 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:22.827888  8152 catalog_manager.cc:5697] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 0 to 2, leader changed from <none> to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:22.830816  8252 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:01:22.832072  8369 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:22.832875  8947 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:22.833412  8947 raft_consensus.cc:697] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:22.834224  8947 consensus_queue.cc:237] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:22.843734  8152 catalog_manager.cc:5697] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 2 to 3, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 3 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:22.877045  8255 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:22.878229  8848 consensus_queue.cc:1059] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:22.878913  8943 raft_consensus.cc:3055] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:22.879170  8943 raft_consensus.cc:740] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:22.879796  8943 consensus_queue.cc:260] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:22.880637  8943 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:23.197880  8399 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:01:23.199213  8848 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:23.209106  8252 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:01:23.210546  8614 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:23.359167  8332 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:01:23.359553  8332 pending_rounds.cc:85] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 1
I20260709 13:01:23.359814  8332 pending_rounds.cc:107] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:01:23.360056  8332 raft_consensus.cc:2889] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:23.361289  8947 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:23.371412  8252 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:23.372990  8871 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:23.699795  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:23.739696  8963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:23.739989  8966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:23.741758  8964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:23.745113  7164 server_base.cc:1061] running on GCE node
I20260709 13:01:23.746456  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:23.746750  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:23.746969  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602083746953 us; error 0 us; skew 500 ppm
I20260709 13:01:23.747800  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:23.753191  7164 webserver.cc:533] Webserver started at http://127.6.255.4:33273/ using document root <none> and password file <none>
I20260709 13:01:23.754403  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:23.754750  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:23.755208  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:23.756889  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-3-root/instance:
uuid: "f865c4d761f5446eb12b3a029234772e"
format_stamp: "Formatted at 2026-07-09 13:01:23 on dist-test-slave-rnv4"
I20260709 13:01:23.763989  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:01:23.767724  8971 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:23.768599  7164 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:01:23.770860  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-3-root
uuid: "f865c4d761f5446eb12b3a029234772e"
format_stamp: "Formatted at 2026-07-09 13:01:23 on dist-test-slave-rnv4"
I20260709 13:01:23.771286  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:23.818984  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:23.820307  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:23.821862  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:23.824563  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:23.824751  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:23.825057  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:23.825230  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:23.890333  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.4:33825
I20260709 13:01:23.890436  9034 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.4:33825 every 8 connection(s)
I20260709 13:01:23.910468  7164 auto_leader_rebalancer-test.cc:384] add a tserver: f865c4d761f5446eb12b3a029234772e
I20260709 13:01:23.927945  9035 heartbeater.cc:344] Connected to a master server at 127.6.255.62:40337
I20260709 13:01:23.928452  9035 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:23.929318  9035 heartbeater.cc:507] Master 127.6.255.62:40337 requested a full tablet report, sending...
I20260709 13:01:23.931213  8153 ts_manager.cc:194] Registered new tserver with Master: f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:23.932865  8153 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55754
I20260709 13:01:24.879810  8252 consensus_queue.cc:237] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:24.886536  8332 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:24.886528  8399 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:24.888012  9040 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:24.889279  8962 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:24.908610  9040 raft_consensus.cc:2955] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:24.910574  8399 raft_consensus.cc:2955] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:24.912611  8332 raft_consensus.cc:2955] T 1263b3742fd541f8bda5a6b6c10a8596 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
W20260709 13:01:24.929185  8223 consensus_peers.cc:597] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1263b3742fd541f8bda5a6b6c10a8596. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:24.931406  8153 catalog_manager.cc:5697] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:24.936425  9035 heartbeater.cc:499] Master 127.6.255.62:40337 was elected leader, sending a full tablet report...
I20260709 13:01:24.948369  8405 consensus_queue.cc:237] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:24.975826  8332 raft_consensus.cc:1275] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:24.975837  8252 raft_consensus.cc:1275] T 766ac43473084aa6a9fdce530520947c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:24.977455  8947 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:24.978737  8871 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:01:24.993242  8372 consensus_peers.cc:597] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 766ac43473084aa6a9fdce530520947c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:24.998342  8871 raft_consensus.cc:2955] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:25.000324  8328 raft_consensus.cc:2955] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:25.001596  8252 raft_consensus.cc:2955] T 766ac43473084aa6a9fdce530520947c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:25.013813  8255 consensus_queue.cc:237] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:25.025767  8154 catalog_manager.cc:5697] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
W20260709 13:01:25.035060  8223 consensus_peers.cc:597] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bf42cb9d00c6491eb6b0cf8b896f7c5b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:25.036320  8328 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:25.038333  8405 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:25.040409  9040 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:25.041455  8899 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:25.060534  8899 raft_consensus.cc:2955] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:25.062445  8328 raft_consensus.cc:2955] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:25.067006  8405 raft_consensus.cc:2955] T bf42cb9d00c6491eb6b0cf8b896f7c5b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:25.073348  8399 consensus_queue.cc:237] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:25.079418  8154 catalog_manager.cc:5697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:25.090867  8328 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:25.092669  9058 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:01:25.094630  8372 consensus_peers.cc:597] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4516247b88e34a1d89ec9eb29e0ecb7f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:25.097039  8255 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:25.098532  9058 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:25.109328  9057 raft_consensus.cc:2955] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:25.114588  8255 raft_consensus.cc:2955] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:25.120677  8328 raft_consensus.cc:2955] T 4516247b88e34a1d89ec9eb29e0ecb7f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:25.132411  8154 catalog_manager.cc:5697] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:25.372464  9067 ts_tablet_manager.cc:933] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:25.375439  9067 tablet_copy_client.cc:323] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.3:37367
I20260709 13:01:25.390929  8416 tablet_copy_service.cc:140] P e64761f43e4240319c41117cf886d22f: Received BeginTabletCopySession request for tablet 766ac43473084aa6a9fdce530520947c from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:34462)
I20260709 13:01:25.391547  8416 tablet_copy_service.cc:161] P e64761f43e4240319c41117cf886d22f: Beginning new tablet copy session on tablet 766ac43473084aa6a9fdce530520947c from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:34462: session id = f865c4d761f5446eb12b3a029234772e-766ac43473084aa6a9fdce530520947c
I20260709 13:01:25.398783  8416 tablet_copy_source_session.cc:215] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:25.402521  9067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 766ac43473084aa6a9fdce530520947c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:25.413159  9070 ts_tablet_manager.cc:933] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679)
I20260709 13:01:25.424943  9067 tablet_copy_client.cc:806] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:25.425778  9067 tablet_copy_client.cc:670] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:25.429183  9070 tablet_copy_client.cc:323] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.1:44679
I20260709 13:01:25.430284  9067 tablet_copy_client.cc:538] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:25.440402  9067 tablet_bootstrap.cc:492] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:25.445591  8267 tablet_copy_service.cc:140] P 8c42259d7c4b463a9f485f42c320c81b: Received BeginTabletCopySession request for tablet bf42cb9d00c6491eb6b0cf8b896f7c5b from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:49220)
I20260709 13:01:25.446280  8267 tablet_copy_service.cc:161] P 8c42259d7c4b463a9f485f42c320c81b: Beginning new tablet copy session on tablet bf42cb9d00c6491eb6b0cf8b896f7c5b from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:49220: session id = f865c4d761f5446eb12b3a029234772e-bf42cb9d00c6491eb6b0cf8b896f7c5b
I20260709 13:01:25.454573  8267 tablet_copy_source_session.cc:215] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:25.457607  9070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf42cb9d00c6491eb6b0cf8b896f7c5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:25.471155  9070 tablet_copy_client.cc:806] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:25.471717  9070 tablet_copy_client.cc:670] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:25.474650  9067 tablet_bootstrap.cc:492] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:25.475716  9067 tablet_bootstrap.cc:492] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:25.476397  9067 ts_tablet_manager.cc:1403] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.011s
I20260709 13:01:25.483361  9070 tablet_copy_client.cc:538] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:25.485644  9074 ts_tablet_manager.cc:933] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679)
I20260709 13:01:25.495260  9074 tablet_copy_client.cc:323] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.1:44679
I20260709 13:01:25.496805  8267 tablet_copy_service.cc:140] P 8c42259d7c4b463a9f485f42c320c81b: Received BeginTabletCopySession request for tablet 1263b3742fd541f8bda5a6b6c10a8596 from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:49220)
I20260709 13:01:25.497331  8267 tablet_copy_service.cc:161] P 8c42259d7c4b463a9f485f42c320c81b: Beginning new tablet copy session on tablet 1263b3742fd541f8bda5a6b6c10a8596 from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:49220: session id = f865c4d761f5446eb12b3a029234772e-1263b3742fd541f8bda5a6b6c10a8596
I20260709 13:01:25.505829  8267 tablet_copy_source_session.cc:215] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:25.509392  9070 tablet_bootstrap.cc:492] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:25.510028  9067 raft_consensus.cc:359] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:25.513440  9067 raft_consensus.cc:740] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:25.514184  9067 consensus_queue.cc:260] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:25.520143  9074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1263b3742fd541f8bda5a6b6c10a8596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:25.535527  9067 ts_tablet_manager.cc:1434] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.049s	user 0.009s	sys 0.000s
I20260709 13:01:25.538678  8416 tablet_copy_service.cc:342] P e64761f43e4240319c41117cf886d22f: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-766ac43473084aa6a9fdce530520947c received from {username='slave'} at 127.0.0.1:34462
I20260709 13:01:25.539209  8416 tablet_copy_service.cc:434] P e64761f43e4240319c41117cf886d22f: ending tablet copy session f865c4d761f5446eb12b3a029234772e-766ac43473084aa6a9fdce530520947c on tablet 766ac43473084aa6a9fdce530520947c with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:25.544989  9074 tablet_copy_client.cc:806] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:25.545701  9074 tablet_copy_client.cc:670] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:25.555387  9074 tablet_copy_client.cc:538] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:25.566659  9070 tablet_bootstrap.cc:492] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:25.567690  9074 tablet_bootstrap.cc:492] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:25.569761  9070 tablet_bootstrap.cc:492] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:25.570533  9070 ts_tablet_manager.cc:1403] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.061s	user 0.031s	sys 0.026s
I20260709 13:01:25.574324  9070 raft_consensus.cc:359] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:25.575232  9070 raft_consensus.cc:740] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:25.575945  9070 consensus_queue.cc:260] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:25.578504  9070 ts_tablet_manager.cc:1434] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:01:25.580662  8267 tablet_copy_service.cc:342] P 8c42259d7c4b463a9f485f42c320c81b: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-bf42cb9d00c6491eb6b0cf8b896f7c5b received from {username='slave'} at 127.0.0.1:49220
I20260709 13:01:25.581127  8267 tablet_copy_service.cc:434] P 8c42259d7c4b463a9f485f42c320c81b: ending tablet copy session f865c4d761f5446eb12b3a029234772e-bf42cb9d00c6491eb6b0cf8b896f7c5b on tablet bf42cb9d00c6491eb6b0cf8b896f7c5b with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:25.598062  9074 tablet_bootstrap.cc:492] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:25.599016  9074 tablet_bootstrap.cc:492] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:25.599753  9074 ts_tablet_manager.cc:1403] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.032s	user 0.026s	sys 0.005s
I20260709 13:01:25.602836  9074 raft_consensus.cc:359] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:25.603740  9074 raft_consensus.cc:740] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:25.604395  9074 consensus_queue.cc:260] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:25.606817  9074 ts_tablet_manager.cc:1434] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:01:25.608976  8267 tablet_copy_service.cc:342] P 8c42259d7c4b463a9f485f42c320c81b: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-1263b3742fd541f8bda5a6b6c10a8596 received from {username='slave'} at 127.0.0.1:49220
I20260709 13:01:25.609329  8267 tablet_copy_service.cc:434] P 8c42259d7c4b463a9f485f42c320c81b: ending tablet copy session f865c4d761f5446eb12b3a029234772e-1263b3742fd541f8bda5a6b6c10a8596 on tablet 1263b3742fd541f8bda5a6b6c10a8596 with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:25.653903  9074 ts_tablet_manager.cc:933] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:25.655751  9074 tablet_copy_client.cc:323] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.3:37367
I20260709 13:01:25.657281  8416 tablet_copy_service.cc:140] P e64761f43e4240319c41117cf886d22f: Received BeginTabletCopySession request for tablet 4516247b88e34a1d89ec9eb29e0ecb7f from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:34462)
I20260709 13:01:25.657837  8416 tablet_copy_service.cc:161] P e64761f43e4240319c41117cf886d22f: Beginning new tablet copy session on tablet 4516247b88e34a1d89ec9eb29e0ecb7f from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:34462: session id = f865c4d761f5446eb12b3a029234772e-4516247b88e34a1d89ec9eb29e0ecb7f
I20260709 13:01:25.664733  8416 tablet_copy_source_session.cc:215] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:25.667476  9074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4516247b88e34a1d89ec9eb29e0ecb7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:25.680099  9074 tablet_copy_client.cc:806] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:25.680677  9074 tablet_copy_client.cc:670] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:25.684211  9074 tablet_copy_client.cc:538] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:25.692224  9074 tablet_bootstrap.cc:492] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:25.713270  9074 tablet_bootstrap.cc:492] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:25.714213  9074 tablet_bootstrap.cc:492] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:25.714857  9074 ts_tablet_manager.cc:1403] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.023s	user 0.019s	sys 0.004s
I20260709 13:01:25.717892  9074 raft_consensus.cc:359] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:25.718799  9074 raft_consensus.cc:740] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:25.719549  9074 consensus_queue.cc:260] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:25.722368  9074 ts_tablet_manager.cc:1434] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:01:25.724064  8416 tablet_copy_service.cc:342] P e64761f43e4240319c41117cf886d22f: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-4516247b88e34a1d89ec9eb29e0ecb7f received from {username='slave'} at 127.0.0.1:34462
I20260709 13:01:25.724493  8416 tablet_copy_service.cc:434] P e64761f43e4240319c41117cf886d22f: ending tablet copy session f865c4d761f5446eb12b3a029234772e-4516247b88e34a1d89ec9eb29e0ecb7f on tablet 4516247b88e34a1d89ec9eb29e0ecb7f with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:25.819947  9009 raft_consensus.cc:1217] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:25.970268  9009 raft_consensus.cc:1217] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:25.992393  9009 raft_consensus.cc:1217] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:26.180922  9009 raft_consensus.cc:1217] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:26.382501  9058 raft_consensus.cc:1064] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:26.384909  9058 consensus_queue.cc:237] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:26.394178  9009 raft_consensus.cc:1275] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:26.396230  8328 raft_consensus.cc:1275] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:26.396718  9057 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:26.397415  8254 raft_consensus.cc:1275] T 766ac43473084aa6a9fdce530520947c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:26.398428  9058 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:26.404138  9059 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:26.415732  9058 raft_consensus.cc:2955] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:26.433781  9008 raft_consensus.cc:2955] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:26.434629  8254 raft_consensus.cc:2955] T 766ac43473084aa6a9fdce530520947c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:26.437052  9082 raft_consensus.cc:1064] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:26.449625  9082 consensus_queue.cc:237] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:26.436746  8328 raft_consensus.cc:2955] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:26.459419  8153 catalog_manager.cc:5697] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:26.465771  9008 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:26.470593  8405 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:26.471495  9081 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:26.472380  9040 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:26.475127  8329 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:26.483527  9040 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:26.511312  9081 raft_consensus.cc:2955] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:26.513000  8399 raft_consensus.cc:2955] T bf42cb9d00c6491eb6b0cf8b896f7c5b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:26.522806  9008 raft_consensus.cc:2955] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:26.534106  9040 raft_consensus.cc:1064] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:26.536536  9040 consensus_queue.cc:237] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:26.548563  8328 raft_consensus.cc:2955] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:26.555205  8154 catalog_manager.cc:5697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:26.556463  8332 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:26.559882  9082 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:26.574422  9008 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:26.593072  9082 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:26.602789  8399 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:26.607077  8399 consensus_queue.cc:237] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:26.619311  9008 raft_consensus.cc:1275] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:26.621654  9058 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:26.622489  8329 raft_consensus.cc:1275] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:26.624253  9057 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:26.633854  9081 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:26.634182  9058 raft_consensus.cc:2955] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:26.638339  8329 raft_consensus.cc:2955] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:26.646245  9008 raft_consensus.cc:2955] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:01:26.653576  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 75492.9us
I20260709 13:01:26.655320  8141 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 766ac43473084aa6a9fdce530520947c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:26.661489  8151 catalog_manager.cc:5697] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:26.663757  8254 consensus_queue.cc:237] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:26.682869  8328 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:26.685392  9007 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:26.686636  8245 tablet_service.cc:1558] Processing DeleteTablet for tablet 766ac43473084aa6a9fdce530520947c with delete_type TABLET_DATA_TOMBSTONED (TS 8c42259d7c4b463a9f485f42c320c81b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55248
I20260709 13:01:26.687970  9040 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:26.694013  9106 tablet_replica.cc:333] T 766ac43473084aa6a9fdce530520947c P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:01:26.696693  9106 raft_consensus.cc:2243] T 766ac43473084aa6a9fdce530520947c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:01:26.696792  9030 diagnostics_log.cc:230] Unable to collect stacks to diagnostics log: Incomplete: not collecting stack trace since debugger or strace is attached
I20260709 13:01:26.698545  9106 raft_consensus.cc:2272] T 766ac43473084aa6a9fdce530520947c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:26.705565  9106 ts_tablet_manager.cc:1916] T 766ac43473084aa6a9fdce530520947c P 8c42259d7c4b463a9f485f42c320c81b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:26.709391  9040 raft_consensus.cc:2955] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:26.712682  9009 raft_consensus.cc:2955] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:26.721138  9082 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:26.721836  9097 raft_consensus.cc:2955] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:26.734478  9008 raft_consensus.cc:2955] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:26.735740  9106 ts_tablet_manager.cc:1929] T 766ac43473084aa6a9fdce530520947c P 8c42259d7c4b463a9f485f42c320c81b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:26.736253  9106 log.cc:1199] T 766ac43473084aa6a9fdce530520947c P 8c42259d7c4b463a9f485f42c320c81b: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-0-root/wals/766ac43473084aa6a9fdce530520947c
W20260709 13:01:26.737946  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 60303.7us
W20260709 13:01:26.771103  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 60355.2us
I20260709 13:01:26.772686  8139 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bf42cb9d00c6491eb6b0cf8b896f7c5b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:26.779838  8139 catalog_manager.cc:5028] TS 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): tablet 766ac43473084aa6a9fdce530520947c (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:26.776098  8154 catalog_manager.cc:5697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:26.789973  8328 raft_consensus.cc:2955] T 1263b3742fd541f8bda5a6b6c10a8596 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:26.862519  9084 raft_consensus.cc:1064] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:26.910146  8324 raft_consensus.cc:2955] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:01:26.955484  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 86543us
I20260709 13:01:26.969903  8152 catalog_manager.cc:5697] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:26.980172  8404 raft_consensus.cc:2955] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:26.968149  9084 consensus_queue.cc:237] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:27.025527  8393 tablet_service.cc:1558] Processing DeleteTablet for tablet bf42cb9d00c6491eb6b0cf8b896f7c5b with delete_type TABLET_DATA_TOMBSTONED (TS e64761f43e4240319c41117cf886d22f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39294
I20260709 13:01:27.075070  9117 tablet_replica.cc:333] T bf42cb9d00c6491eb6b0cf8b896f7c5b P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:01:27.076608  9117 raft_consensus.cc:2243] T bf42cb9d00c6491eb6b0cf8b896f7c5b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:27.096684  9117 raft_consensus.cc:2272] T bf42cb9d00c6491eb6b0cf8b896f7c5b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:27.101682  8256 consensus_queue.cc:237] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:27.113629  9008 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:27.123235  9008 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:27.125141  9057 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:27.133122  9117 ts_tablet_manager.cc:1916] T bf42cb9d00c6491eb6b0cf8b896f7c5b P e64761f43e4240319c41117cf886d22f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:27.135294  8323 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:27.137566  9040 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:27.139318  8252 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:27.145408  8398 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:27.146733  9115 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:27.159875  9058 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:27.182180  9107 raft_consensus.cc:2955] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:27.191036  9117 ts_tablet_manager.cc:1929] T bf42cb9d00c6491eb6b0cf8b896f7c5b P e64761f43e4240319c41117cf886d22f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:27.192754  9117 log.cc:1199] T bf42cb9d00c6491eb6b0cf8b896f7c5b P e64761f43e4240319c41117cf886d22f: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-2-root/wals/bf42cb9d00c6491eb6b0cf8b896f7c5b
I20260709 13:01:27.203899  8139 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1263b3742fd541f8bda5a6b6c10a8596 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:27.205262  9008 raft_consensus.cc:2955] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:27.209376  8141 catalog_manager.cc:5028] TS e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): tablet bf42cb9d00c6491eb6b0cf8b896f7c5b (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:27.218629  9116 raft_consensus.cc:2955] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:27.218636  8152 catalog_manager.cc:5697] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:27.230645  9040 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:27.238255  8253 raft_consensus.cc:2955] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:27.246927  8314 tablet_service.cc:1558] Processing DeleteTablet for tablet 1263b3742fd541f8bda5a6b6c10a8596 with delete_type TABLET_DATA_TOMBSTONED (TS ff7a6fe533e94c1bae555b4cae427728 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57170
I20260709 13:01:27.250559  8398 raft_consensus.cc:2955] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:27.263316  9009 raft_consensus.cc:2955] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:27.268733  8324 raft_consensus.cc:2955] T 4516247b88e34a1d89ec9eb29e0ecb7f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:27.272292  8151 catalog_manager.cc:5697] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:27.287758  9123 tablet_replica.cc:333] T 1263b3742fd541f8bda5a6b6c10a8596 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:01:27.288959  9123 raft_consensus.cc:2243] T 1263b3742fd541f8bda5a6b6c10a8596 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:27.289809  9123 raft_consensus.cc:2272] T 1263b3742fd541f8bda5a6b6c10a8596 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:27.295429  9123 ts_tablet_manager.cc:1916] T 1263b3742fd541f8bda5a6b6c10a8596 P ff7a6fe533e94c1bae555b4cae427728: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:27.303627  8398 consensus_queue.cc:237] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:27.310819  9123 ts_tablet_manager.cc:1929] T 1263b3742fd541f8bda5a6b6c10a8596 P ff7a6fe533e94c1bae555b4cae427728: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:27.311794  9123 log.cc:1199] T 1263b3742fd541f8bda5a6b6c10a8596 P ff7a6fe533e94c1bae555b4cae427728: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-1-root/wals/1263b3742fd541f8bda5a6b6c10a8596
I20260709 13:01:27.314455  8252 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:27.317550  9057 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:27.317276  9009 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:27.324644  9114 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:27.326476  9114 raft_consensus.cc:2955] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:27.337236  8252 raft_consensus.cc:2955] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:27.345520  9008 raft_consensus.cc:2955] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:27.351099  8141 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4516247b88e34a1d89ec9eb29e0ecb7f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:27.358503  8154 catalog_manager.cc:5697] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:27.364068  8139 catalog_manager.cc:5028] TS ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): tablet 1263b3742fd541f8bda5a6b6c10a8596 (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:27.370843  8314 tablet_service.cc:1558] Processing DeleteTablet for tablet 4516247b88e34a1d89ec9eb29e0ecb7f with delete_type TABLET_DATA_TOMBSTONED (TS ff7a6fe533e94c1bae555b4cae427728 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57170
I20260709 13:01:27.371821  9123 tablet_replica.cc:333] T 4516247b88e34a1d89ec9eb29e0ecb7f P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:01:27.380697  9123 raft_consensus.cc:2243] T 4516247b88e34a1d89ec9eb29e0ecb7f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:27.381621  9123 raft_consensus.cc:2272] T 4516247b88e34a1d89ec9eb29e0ecb7f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:27.385541  9123 ts_tablet_manager.cc:1916] T 4516247b88e34a1d89ec9eb29e0ecb7f P ff7a6fe533e94c1bae555b4cae427728: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:27.401224  9123 ts_tablet_manager.cc:1929] T 4516247b88e34a1d89ec9eb29e0ecb7f P ff7a6fe533e94c1bae555b4cae427728: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:27.401664  9123 log.cc:1199] T 4516247b88e34a1d89ec9eb29e0ecb7f P ff7a6fe533e94c1bae555b4cae427728: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-1-root/wals/4516247b88e34a1d89ec9eb29e0ecb7f
I20260709 13:01:27.403045  8139 catalog_manager.cc:5028] TS ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): tablet 4516247b88e34a1d89ec9eb29e0ecb7f (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:28.482508  8324 consensus_queue.cc:237] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:28.496744  8253 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:28.498162  8398 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:28.499516  9105 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:28.500684  9127 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:28.510291  9127 raft_consensus.cc:2955] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
W20260709 13:01:28.512435  8298 consensus_peers.cc:597] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 767eba28679649519167576023555b60. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:28.512737  8253 raft_consensus.cc:2955] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:28.513695  8398 raft_consensus.cc:2955] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:28.529341  8398 consensus_queue.cc:237] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:28.530488  8153 catalog_manager.cc:5697] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 2 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
W20260709 13:01:28.537356  8372 consensus_peers.cc:597] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 07268be9b33649e8b0e8bddf7a3bcabe. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:28.537544  8324 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:28.538944  9057 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:28.539207  8253 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:28.540643  9114 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:28.549330  9057 raft_consensus.cc:2955] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:28.568423  8324 raft_consensus.cc:2955] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:28.570956  8253 raft_consensus.cc:2955] T 07268be9b33649e8b0e8bddf7a3bcabe P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:28.577515  8256 consensus_queue.cc:237] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:28.584313  8154 catalog_manager.cc:5697] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:28.604652  8398 raft_consensus.cc:1275] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:28.605502  8323 raft_consensus.cc:1275] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:28.607060  9040 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:01:28.610456  8223 consensus_peers.cc:597] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d32240ced15f4c36a9ce65ca5a13634a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:28.617779  9040 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:28.619670  9110 raft_consensus.cc:2955] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:28.621055  8323 raft_consensus.cc:2955] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:28.627861  8398 raft_consensus.cc:2955] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:28.635505  8323 consensus_queue.cc:237] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:28.641736  8151 catalog_manager.cc:5697] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:28.654791  8398 raft_consensus.cc:1275] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:28.655566  8298 consensus_peers.cc:597] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b6e0ee9efb7d40309bc7e4d2450bb1b5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:28.656047  9127 consensus_queue.cc:1048] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:28.656827  8256 raft_consensus.cc:1275] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:28.658362  9127 consensus_queue.cc:1048] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:28.670153  9108 raft_consensus.cc:2955] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:28.676110  8404 raft_consensus.cc:2955] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:28.678087  8253 raft_consensus.cc:2955] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:28.693135  8154 catalog_manager.cc:5697] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:28.957310  9149 ts_tablet_manager.cc:933] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:28.959900  9149 tablet_copy_client.cc:323] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.3:37367
I20260709 13:01:28.962162  8416 tablet_copy_service.cc:140] P e64761f43e4240319c41117cf886d22f: Received BeginTabletCopySession request for tablet 07268be9b33649e8b0e8bddf7a3bcabe from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:34462)
I20260709 13:01:28.962774  8416 tablet_copy_service.cc:161] P e64761f43e4240319c41117cf886d22f: Beginning new tablet copy session on tablet 07268be9b33649e8b0e8bddf7a3bcabe from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:34462: session id = f865c4d761f5446eb12b3a029234772e-07268be9b33649e8b0e8bddf7a3bcabe
I20260709 13:01:28.969225  8416 tablet_copy_source_session.cc:215] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:28.972695  9149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07268be9b33649e8b0e8bddf7a3bcabe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:28.996898  9149 tablet_copy_client.cc:806] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:28.997642  9149 tablet_copy_client.cc:670] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:29.001979  9149 tablet_copy_client.cc:538] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:29.014799  9149 tablet_bootstrap.cc:492] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:29.039176  9149 tablet_bootstrap.cc:492] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:29.040112  9149 tablet_bootstrap.cc:492] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:29.040839  9149 ts_tablet_manager.cc:1403] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.008s
I20260709 13:01:29.043984  9149 raft_consensus.cc:359] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:29.044785  9149 raft_consensus.cc:740] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:29.045449  9149 consensus_queue.cc:260] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:29.048686  9149 ts_tablet_manager.cc:1434] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.003s
I20260709 13:01:29.050649  8416 tablet_copy_service.cc:342] P e64761f43e4240319c41117cf886d22f: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-07268be9b33649e8b0e8bddf7a3bcabe received from {username='slave'} at 127.0.0.1:34462
I20260709 13:01:29.051102  8416 tablet_copy_service.cc:434] P e64761f43e4240319c41117cf886d22f: ending tablet copy session f865c4d761f5446eb12b3a029234772e-07268be9b33649e8b0e8bddf7a3bcabe on tablet 07268be9b33649e8b0e8bddf7a3bcabe with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:29.065534  9149 ts_tablet_manager.cc:933] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679)
I20260709 13:01:29.067288  9149 tablet_copy_client.cc:323] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.1:44679
I20260709 13:01:29.069228  8267 tablet_copy_service.cc:140] P 8c42259d7c4b463a9f485f42c320c81b: Received BeginTabletCopySession request for tablet d32240ced15f4c36a9ce65ca5a13634a from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:49220)
I20260709 13:01:29.069811  8267 tablet_copy_service.cc:161] P 8c42259d7c4b463a9f485f42c320c81b: Beginning new tablet copy session on tablet d32240ced15f4c36a9ce65ca5a13634a from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:49220: session id = f865c4d761f5446eb12b3a029234772e-d32240ced15f4c36a9ce65ca5a13634a
I20260709 13:01:29.076059  8267 tablet_copy_source_session.cc:215] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:29.079684  9149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d32240ced15f4c36a9ce65ca5a13634a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.096696  9149 tablet_copy_client.cc:806] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:29.097311  9149 tablet_copy_client.cc:670] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:29.101892  9149 tablet_copy_client.cc:538] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:29.109023  9154 ts_tablet_manager.cc:933] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:29.109927  9149 tablet_bootstrap.cc:492] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:29.115187  9154 tablet_copy_client.cc:323] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.2:38599
I20260709 13:01:29.139226  9158 ts_tablet_manager.cc:933] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:29.141880  9158 tablet_copy_client.cc:323] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.2:38599
I20260709 13:01:29.154546  8342 tablet_copy_service.cc:140] P ff7a6fe533e94c1bae555b4cae427728: Received BeginTabletCopySession request for tablet 767eba28679649519167576023555b60 from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:41664)
I20260709 13:01:29.154847  8341 tablet_copy_service.cc:140] P ff7a6fe533e94c1bae555b4cae427728: Received BeginTabletCopySession request for tablet b6e0ee9efb7d40309bc7e4d2450bb1b5 from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:41664)
I20260709 13:01:29.155230  8342 tablet_copy_service.cc:161] P ff7a6fe533e94c1bae555b4cae427728: Beginning new tablet copy session on tablet 767eba28679649519167576023555b60 from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:41664: session id = f865c4d761f5446eb12b3a029234772e-767eba28679649519167576023555b60
I20260709 13:01:29.161755  8341 tablet_copy_service.cc:161] P ff7a6fe533e94c1bae555b4cae427728: Beginning new tablet copy session on tablet b6e0ee9efb7d40309bc7e4d2450bb1b5 from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:41664: session id = f865c4d761f5446eb12b3a029234772e-b6e0ee9efb7d40309bc7e4d2450bb1b5
I20260709 13:01:29.161748  8342 tablet_copy_source_session.cc:215] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:29.163332  9149 tablet_bootstrap.cc:492] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:29.164316  9149 tablet_bootstrap.cc:492] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:29.165369  9149 ts_tablet_manager.cc:1403] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.056s	user 0.017s	sys 0.014s
I20260709 13:01:29.166532  9154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 767eba28679649519167576023555b60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.168489  9149 raft_consensus.cc:359] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:29.169394  9149 raft_consensus.cc:740] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:29.170094  9149 consensus_queue.cc:260] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:29.173004  9149 ts_tablet_manager.cc:1434] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:01:29.176182  8267 tablet_copy_service.cc:342] P 8c42259d7c4b463a9f485f42c320c81b: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-d32240ced15f4c36a9ce65ca5a13634a received from {username='slave'} at 127.0.0.1:49220
I20260709 13:01:29.176671  8267 tablet_copy_service.cc:434] P 8c42259d7c4b463a9f485f42c320c81b: ending tablet copy session f865c4d761f5446eb12b3a029234772e-d32240ced15f4c36a9ce65ca5a13634a on tablet d32240ced15f4c36a9ce65ca5a13634a with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:29.169767  8341 tablet_copy_source_session.cc:215] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:29.196100  9154 tablet_copy_client.cc:806] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:29.196786  9154 tablet_copy_client.cc:670] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:29.204243  9158 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6e0ee9efb7d40309bc7e4d2450bb1b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.212155  9154 tablet_copy_client.cc:538] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:29.223351  9158 tablet_copy_client.cc:806] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:29.223942  9158 tablet_copy_client.cc:670] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:29.227510  9158 tablet_copy_client.cc:538] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:29.227784  9154 tablet_bootstrap.cc:492] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:29.236783  9158 tablet_bootstrap.cc:492] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:29.262514  9154 tablet_bootstrap.cc:492] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:29.263432  9154 tablet_bootstrap.cc:492] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:29.264060  9154 ts_tablet_manager.cc:1403] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.036s	user 0.023s	sys 0.009s
I20260709 13:01:29.266907  9154 raft_consensus.cc:359] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:29.267695  9154 raft_consensus.cc:740] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:29.268729  9158 tablet_bootstrap.cc:492] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:29.268324  9154 consensus_queue.cc:260] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:29.269645  9158 tablet_bootstrap.cc:492] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:29.270315  9158 ts_tablet_manager.cc:1403] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.034s	user 0.021s	sys 0.006s
I20260709 13:01:29.276856  9158 raft_consensus.cc:359] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:29.277586  9154 ts_tablet_manager.cc:1434] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:01:29.277848  9158 raft_consensus.cc:740] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:29.278630  9158 consensus_queue.cc:260] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:29.281189  9158 ts_tablet_manager.cc:1434] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:01:29.282847  8341 tablet_copy_service.cc:342] P ff7a6fe533e94c1bae555b4cae427728: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-b6e0ee9efb7d40309bc7e4d2450bb1b5 received from {username='slave'} at 127.0.0.1:41664
I20260709 13:01:29.283288  8341 tablet_copy_service.cc:434] P ff7a6fe533e94c1bae555b4cae427728: ending tablet copy session f865c4d761f5446eb12b3a029234772e-b6e0ee9efb7d40309bc7e4d2450bb1b5 on tablet b6e0ee9efb7d40309bc7e4d2450bb1b5 with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:29.294143  8341 tablet_copy_service.cc:342] P ff7a6fe533e94c1bae555b4cae427728: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-767eba28679649519167576023555b60 received from {username='slave'} at 127.0.0.1:41664
I20260709 13:01:29.294629  8341 tablet_copy_service.cc:434] P ff7a6fe533e94c1bae555b4cae427728: ending tablet copy session f865c4d761f5446eb12b3a029234772e-767eba28679649519167576023555b60 on tablet 767eba28679649519167576023555b60 with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:29.371692  9008 raft_consensus.cc:1217] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:29.549539  9008 raft_consensus.cc:1217] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:29.610754  9008 raft_consensus.cc:1217] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 13:01:29.617522  9008 raft_consensus.cc:1217] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:29.830380  9114 raft_consensus.cc:1064] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:29.832664  9114 consensus_queue.cc:237] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:29.850066  9008 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:29.850656  8324 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:29.851648  9057 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:29.852451  9140 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:29.856375  8256 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:29.860682  9140 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:29.918052  9009 raft_consensus.cc:2955] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:29.928591  9057 raft_consensus.cc:2955] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:29.934687  8151 catalog_manager.cc:5697] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:29.941308  8256 raft_consensus.cc:2955] T 07268be9b33649e8b0e8bddf7a3bcabe P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:29.945001  8332 raft_consensus.cc:2955] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:29.973389  8399 consensus_queue.cc:237] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:29.978304  9009 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:29.979665  8332 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:29.980715  9165 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:29.987434  9057 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:29.990527  9057 raft_consensus.cc:2955] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:29.993345  9008 raft_consensus.cc:2955] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:29.999583  9148 raft_consensus.cc:1064] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:29.999866  8332 raft_consensus.cc:2955] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.002120  9148 consensus_queue.cc:237] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:30.004506  8141 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 07268be9b33649e8b0e8bddf7a3bcabe with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:30.007756  9147 raft_consensus.cc:1064] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:30.010150  9147 consensus_queue.cc:237] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:30.014701  8154 catalog_manager.cc:5697] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:30.023267  8256 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:30.023609  8399 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:30.025484  9163 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:30.026535  9108 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:30.033857  8245 tablet_service.cc:1558] Processing DeleteTablet for tablet 07268be9b33649e8b0e8bddf7a3bcabe with delete_type TABLET_DATA_TOMBSTONED (TS 8c42259d7c4b463a9f485f42c320c81b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55248
I20260709 13:01:30.023238  9009 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 2 LEARNER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:30.036271  9108 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:30.044664  9178 tablet_replica.cc:333] T 07268be9b33649e8b0e8bddf7a3bcabe P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:01:30.045890  9178 raft_consensus.cc:2243] T 07268be9b33649e8b0e8bddf7a3bcabe P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:30.046697  9178 raft_consensus.cc:2272] T 07268be9b33649e8b0e8bddf7a3bcabe P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:30.048713  9008 raft_consensus.cc:1275] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:30.050354  9166 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:30.053175  9178 ts_tablet_manager.cc:1916] T 07268be9b33649e8b0e8bddf7a3bcabe P 8c42259d7c4b463a9f485f42c320c81b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:30.056550  8399 raft_consensus.cc:1275] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:30.048280  9163 raft_consensus.cc:2955] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.058509  9166 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:30.058040  8256 raft_consensus.cc:2955] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.073807  9147 raft_consensus.cc:1064] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:30.076354  8323 raft_consensus.cc:1275] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:30.077384  9147 consensus_queue.cc:237] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:30.078011  9166 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:30.088495  9178 ts_tablet_manager.cc:1929] T 07268be9b33649e8b0e8bddf7a3bcabe P 8c42259d7c4b463a9f485f42c320c81b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:30.090973  9178 log.cc:1199] T 07268be9b33649e8b0e8bddf7a3bcabe P 8c42259d7c4b463a9f485f42c320c81b: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-0-root/wals/07268be9b33649e8b0e8bddf7a3bcabe
I20260709 13:01:30.099179  8139 catalog_manager.cc:5028] TS 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): tablet 07268be9b33649e8b0e8bddf7a3bcabe (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:30.104522  9166 raft_consensus.cc:2955] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.110282  8323 raft_consensus.cc:2955] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.116887  8399 raft_consensus.cc:2955] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.123446  8153 catalog_manager.cc:5697] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:01:30.136948  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 54772.7us
I20260709 13:01:30.141568  8154 catalog_manager.cc:5697] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.177551  9009 raft_consensus.cc:2955] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.183122  9007 raft_consensus.cc:1275] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:30.197634  8256 raft_consensus.cc:1275] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:30.200979  9147 consensus_queue.cc:1048] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:30.204121  9186 consensus_queue.cc:1048] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:01:30.223919  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 66378.5us
W20260709 13:01:30.225641  9184 log.cc:927] Time spent T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:01:30.238108  8404 raft_consensus.cc:1275] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:30.241072  8398 raft_consensus.cc:2955] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.247444  8324 consensus_queue.cc:237] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:30.252485  9163 consensus_queue.cc:1048] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:30.262885  9009 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:30.265151  9189 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:30.270774  9163 raft_consensus.cc:2955] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.279563  9188 raft_consensus.cc:2955] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.281294  9009 raft_consensus.cc:2955] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.282074  8404 raft_consensus.cc:2955] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.291929  8256 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:30.293720  8256 raft_consensus.cc:2955] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.305243  8153 catalog_manager.cc:5697] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.300315  9007 raft_consensus.cc:2955] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.313985  9186 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:30.313845  9008 raft_consensus.cc:2955] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.317708  8139 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 767eba28679649519167576023555b60 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:30.325959  8151 catalog_manager.cc:5697] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New cstate: current_term: 2 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.332442  8254 raft_consensus.cc:2955] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.341403  8324 consensus_queue.cc:237] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:30.345407  8393 tablet_service.cc:1558] Processing DeleteTablet for tablet 767eba28679649519167576023555b60 with delete_type TABLET_DATA_TOMBSTONED (TS e64761f43e4240319c41117cf886d22f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39294
I20260709 13:01:30.344825  8253 consensus_queue.cc:237] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:30.355904  9007 raft_consensus.cc:1275] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:30.357371  8399 raft_consensus.cc:1275] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:30.358485  9194 tablet_replica.cc:333] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:01:30.357388  9188 consensus_queue.cc:1048] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:30.359761  9194 raft_consensus.cc:2243] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:30.360500  9194 raft_consensus.cc:2272] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:30.364708  9188 consensus_queue.cc:1048] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:30.366119  9163 raft_consensus.cc:2955] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.371939  8404 raft_consensus.cc:1275] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:30.373658  9110 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:30.375012  9007 raft_consensus.cc:2955] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.384372  8139 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b6e0ee9efb7d40309bc7e4d2450bb1b5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:30.389895  9194 ts_tablet_manager.cc:1916] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:30.396937  8399 raft_consensus.cc:2955] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.402747  9009 raft_consensus.cc:1275] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:30.408258  9166 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:30.411110  9166 raft_consensus.cc:2955] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.418138  8403 raft_consensus.cc:2955] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.416805  8153 catalog_manager.cc:5697] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.443008  9007 raft_consensus.cc:2955] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.445313  8245 tablet_service.cc:1558] Processing DeleteTablet for tablet b6e0ee9efb7d40309bc7e4d2450bb1b5 with delete_type TABLET_DATA_TOMBSTONED (TS 8c42259d7c4b463a9f485f42c320c81b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55248
I20260709 13:01:30.446377  9178 tablet_replica.cc:333] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:01:30.447583  9178 raft_consensus.cc:2243] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:30.448403  9178 raft_consensus.cc:2272] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:30.454355  9178 ts_tablet_manager.cc:1916] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P 8c42259d7c4b463a9f485f42c320c81b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:30.464314  8153 catalog_manager.cc:5697] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:30.469967  9194 ts_tablet_manager.cc:1929] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 13:01:30.472307  9194 log.cc:1199] T 767eba28679649519167576023555b60 P e64761f43e4240319c41117cf886d22f: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-2-root/wals/767eba28679649519167576023555b60
I20260709 13:01:30.482084  8141 catalog_manager.cc:5028] TS e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): tablet 767eba28679649519167576023555b60 (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:30.483028  8139 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d32240ced15f4c36a9ce65ca5a13634a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:30.485149  8314 tablet_service.cc:1558] Processing DeleteTablet for tablet d32240ced15f4c36a9ce65ca5a13634a with delete_type TABLET_DATA_TOMBSTONED (TS ff7a6fe533e94c1bae555b4cae427728 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57170
I20260709 13:01:30.502871  9195 tablet_replica.cc:333] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:01:30.505839  9195 raft_consensus.cc:2243] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:30.506404  9195 raft_consensus.cc:2272] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:30.504781  9178 ts_tablet_manager.cc:1929] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P 8c42259d7c4b463a9f485f42c320c81b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:30.507164  9178 log.cc:1199] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P 8c42259d7c4b463a9f485f42c320c81b: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-0-root/wals/b6e0ee9efb7d40309bc7e4d2450bb1b5
I20260709 13:01:30.508535  8139 catalog_manager.cc:5028] TS 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): tablet b6e0ee9efb7d40309bc7e4d2450bb1b5 (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:30.509727  9195 ts_tablet_manager.cc:1916] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:30.520936  9195 ts_tablet_manager.cc:1929] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:30.521268  9195 log.cc:1199] T d32240ced15f4c36a9ce65ca5a13634a P ff7a6fe533e94c1bae555b4cae427728: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-1-root/wals/d32240ced15f4c36a9ce65ca5a13634a
I20260709 13:01:30.522683  8139 catalog_manager.cc:5028] TS ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): tablet d32240ced15f4c36a9ce65ca5a13634a (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:31.559607  8254 consensus_queue.cc:237] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:31.565824  8403 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:31.565907  8323 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:31.566819  8223 consensus_peers.cc:597] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fdf12882ac9e4f7d9121a5fcb9d4efaf. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:31.567420  9198 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:31.568352  9196 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:31.575589  9196 raft_consensus.cc:2955] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:31.578207  8324 raft_consensus.cc:2955] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:31.578114  8403 raft_consensus.cc:2955] T fdf12882ac9e4f7d9121a5fcb9d4efaf P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:31.594825  8152 catalog_manager.cc:5697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:31.602595  8403 consensus_queue.cc:237] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:31.611286  8332 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:31.611428  8253 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:31.612915  9165 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:01:31.614073  8372 consensus_peers.cc:597] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 038cd0877a674c2996410c9f327df911. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:31.614637  9165 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:31.637878  8253 raft_consensus.cc:2955] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:31.636832  9165 raft_consensus.cc:2955] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:31.642397  8324 raft_consensus.cc:2955] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:31.659343  8253 consensus_queue.cc:237] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:31.662678  8154 catalog_manager.cc:5697] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:31.666131  8403 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:31.666973  8223 consensus_peers.cc:597] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b3d99c80602e45da9c43a49cf83b8211. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:31.667514  9198 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:31.673229  8324 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:31.674911  9196 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:31.677848  9198 raft_consensus.cc:2955] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:31.686879  8403 raft_consensus.cc:2955] T b3d99c80602e45da9c43a49cf83b8211 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:31.692410  8399 consensus_queue.cc:237] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:31.692497  8324 raft_consensus.cc:2955] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:31.698297  8152 catalog_manager.cc:5697] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:31.721406  8324 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:31.722797  8372 consensus_peers.cc:597] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4b9b125a55cf4a5d9db580313045edde. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:31.723480  9140 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:31.726787  8256 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:31.728085  9057 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:31.739485  9057 raft_consensus.cc:2955] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:31.740587  8324 raft_consensus.cc:2955] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:31.765529  8152 catalog_manager.cc:5697] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:31.782529  8256 raft_consensus.cc:2955] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:32.119801  9215 ts_tablet_manager.cc:933] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679)
I20260709 13:01:32.121827  9215 tablet_copy_client.cc:323] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.1:44679
I20260709 13:01:32.123514  8267 tablet_copy_service.cc:140] P 8c42259d7c4b463a9f485f42c320c81b: Received BeginTabletCopySession request for tablet b3d99c80602e45da9c43a49cf83b8211 from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:49220)
I20260709 13:01:32.123994  8267 tablet_copy_service.cc:161] P 8c42259d7c4b463a9f485f42c320c81b: Beginning new tablet copy session on tablet b3d99c80602e45da9c43a49cf83b8211 from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:49220: session id = f865c4d761f5446eb12b3a029234772e-b3d99c80602e45da9c43a49cf83b8211
I20260709 13:01:32.129282  9217 ts_tablet_manager.cc:933] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679)
I20260709 13:01:32.133917  8267 tablet_copy_source_session.cc:215] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:32.136183  9217 tablet_copy_client.cc:323] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.1:44679
I20260709 13:01:32.137354  9215 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3d99c80602e45da9c43a49cf83b8211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:32.137914  8267 tablet_copy_service.cc:140] P 8c42259d7c4b463a9f485f42c320c81b: Received BeginTabletCopySession request for tablet fdf12882ac9e4f7d9121a5fcb9d4efaf from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:49220)
I20260709 13:01:32.138437  8267 tablet_copy_service.cc:161] P 8c42259d7c4b463a9f485f42c320c81b: Beginning new tablet copy session on tablet fdf12882ac9e4f7d9121a5fcb9d4efaf from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:49220: session id = f865c4d761f5446eb12b3a029234772e-fdf12882ac9e4f7d9121a5fcb9d4efaf
I20260709 13:01:32.147370  8267 tablet_copy_source_session.cc:215] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:32.155656  9215 tablet_copy_client.cc:806] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:32.156355  9215 tablet_copy_client.cc:670] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:32.165037  9217 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdf12882ac9e4f7d9121a5fcb9d4efaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:32.167618  9215 tablet_copy_client.cc:538] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:32.179453  9215 tablet_bootstrap.cc:492] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:32.209062  9219 ts_tablet_manager.cc:933] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:32.210088  9215 tablet_bootstrap.cc:492] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:32.211349  9215 tablet_bootstrap.cc:492] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:32.212283  9215 ts_tablet_manager.cc:1403] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.033s	user 0.013s	sys 0.016s
I20260709 13:01:32.213501  9217 tablet_copy_client.cc:806] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:32.214166  9217 tablet_copy_client.cc:670] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:32.215910  9215 raft_consensus.cc:359] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:32.216830  9215 raft_consensus.cc:740] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:32.217525  9215 consensus_queue.cc:260] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:32.219763  9217 tablet_copy_client.cc:538] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:32.220950  9219 tablet_copy_client.cc:323] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.3:37367
I20260709 13:01:32.222887  8416 tablet_copy_service.cc:140] P e64761f43e4240319c41117cf886d22f: Received BeginTabletCopySession request for tablet 038cd0877a674c2996410c9f327df911 from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:34462)
I20260709 13:01:32.223587  8416 tablet_copy_service.cc:161] P e64761f43e4240319c41117cf886d22f: Beginning new tablet copy session on tablet 038cd0877a674c2996410c9f327df911 from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:34462: session id = f865c4d761f5446eb12b3a029234772e-038cd0877a674c2996410c9f327df911
I20260709 13:01:32.234222  8416 tablet_copy_source_session.cc:215] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:32.234606  9217 tablet_bootstrap.cc:492] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:32.237498  9219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 038cd0877a674c2996410c9f327df911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:32.246992  9215 ts_tablet_manager.cc:1434] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.034s	user 0.010s	sys 0.011s
I20260709 13:01:32.262595  8267 tablet_copy_service.cc:342] P 8c42259d7c4b463a9f485f42c320c81b: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-b3d99c80602e45da9c43a49cf83b8211 received from {username='slave'} at 127.0.0.1:49220
I20260709 13:01:32.263054  8267 tablet_copy_service.cc:434] P 8c42259d7c4b463a9f485f42c320c81b: ending tablet copy session f865c4d761f5446eb12b3a029234772e-b3d99c80602e45da9c43a49cf83b8211 on tablet b3d99c80602e45da9c43a49cf83b8211 with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:32.267266  9219 tablet_copy_client.cc:806] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:32.267937  9219 tablet_copy_client.cc:670] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:32.271600  9219 tablet_copy_client.cc:538] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:32.274601  9217 tablet_bootstrap.cc:492] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:32.275763  9217 tablet_bootstrap.cc:492] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:32.276467  9217 ts_tablet_manager.cc:1403] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.042s	user 0.015s	sys 0.007s
I20260709 13:01:32.280755  9215 ts_tablet_manager.cc:933] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:32.280380  9217 raft_consensus.cc:359] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:32.281224  9217 raft_consensus.cc:740] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:32.281982  9217 consensus_queue.cc:260] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:32.289322  9215 tablet_copy_client.cc:323] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.3:37367
I20260709 13:01:32.291147  8416 tablet_copy_service.cc:140] P e64761f43e4240319c41117cf886d22f: Received BeginTabletCopySession request for tablet 4b9b125a55cf4a5d9db580313045edde from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:34462)
I20260709 13:01:32.291386  9217 ts_tablet_manager.cc:1434] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.003s
I20260709 13:01:32.291733  8416 tablet_copy_service.cc:161] P e64761f43e4240319c41117cf886d22f: Beginning new tablet copy session on tablet 4b9b125a55cf4a5d9db580313045edde from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:34462: session id = f865c4d761f5446eb12b3a029234772e-4b9b125a55cf4a5d9db580313045edde
I20260709 13:01:32.292901  9219 tablet_bootstrap.cc:492] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:32.299394  8416 tablet_copy_source_session.cc:215] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:32.301297  8267 tablet_copy_service.cc:342] P 8c42259d7c4b463a9f485f42c320c81b: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-fdf12882ac9e4f7d9121a5fcb9d4efaf received from {username='slave'} at 127.0.0.1:49220
I20260709 13:01:32.305706  8267 tablet_copy_service.cc:434] P 8c42259d7c4b463a9f485f42c320c81b: ending tablet copy session f865c4d761f5446eb12b3a029234772e-fdf12882ac9e4f7d9121a5fcb9d4efaf on tablet fdf12882ac9e4f7d9121a5fcb9d4efaf with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:32.312074  9215 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b9b125a55cf4a5d9db580313045edde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:32.328086  9215 tablet_copy_client.cc:806] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:32.328613  9215 tablet_copy_client.cc:670] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:32.345530  9215 tablet_copy_client.cc:538] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:32.357571  9215 tablet_bootstrap.cc:492] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:32.362546  9219 tablet_bootstrap.cc:492] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:32.363591  9219 tablet_bootstrap.cc:492] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:32.364276  9219 ts_tablet_manager.cc:1403] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.072s	user 0.028s	sys 0.019s
I20260709 13:01:32.367642  9219 raft_consensus.cc:359] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:32.368454  9219 raft_consensus.cc:740] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:32.369084  9219 consensus_queue.cc:260] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:32.371515  9219 ts_tablet_manager.cc:1434] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:01:32.373713  8416 tablet_copy_service.cc:342] P e64761f43e4240319c41117cf886d22f: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-038cd0877a674c2996410c9f327df911 received from {username='slave'} at 127.0.0.1:34462
I20260709 13:01:32.374101  8416 tablet_copy_service.cc:434] P e64761f43e4240319c41117cf886d22f: ending tablet copy session f865c4d761f5446eb12b3a029234772e-038cd0877a674c2996410c9f327df911 on tablet 038cd0877a674c2996410c9f327df911 with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:32.380261  9215 tablet_bootstrap.cc:492] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:32.381294  9215 tablet_bootstrap.cc:492] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:32.382047  9215 ts_tablet_manager.cc:1403] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.007s
I20260709 13:01:32.384871  9215 raft_consensus.cc:359] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:32.385663  9215 raft_consensus.cc:740] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:32.386262  9215 consensus_queue.cc:260] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:32.388542  9215 ts_tablet_manager.cc:1434] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:01:32.390130  8416 tablet_copy_service.cc:342] P e64761f43e4240319c41117cf886d22f: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-4b9b125a55cf4a5d9db580313045edde received from {username='slave'} at 127.0.0.1:34462
I20260709 13:01:32.390535  8416 tablet_copy_service.cc:434] P e64761f43e4240319c41117cf886d22f: ending tablet copy session f865c4d761f5446eb12b3a029234772e-4b9b125a55cf4a5d9db580313045edde on tablet 4b9b125a55cf4a5d9db580313045edde with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:32.555545  9007 raft_consensus.cc:1217] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:32.732060  9007 raft_consensus.cc:1217] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:32.764344  9007 raft_consensus.cc:1217] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:32.812340  9007 raft_consensus.cc:1217] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:32.984907  9196 raft_consensus.cc:1064] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:32.987612  9196 consensus_queue.cc:237] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:32.999735  8399 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:33.000963  9007 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:33.002712  9148 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:33.003667  9196 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:33.009244  8323 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:33.011700  9198 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:33.025144  9229 raft_consensus.cc:2955] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.026767  8399 raft_consensus.cc:2955] T b3d99c80602e45da9c43a49cf83b8211 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.038582  8324 raft_consensus.cc:2955] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.038389  9007 raft_consensus.cc:2955] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.043844  8152 catalog_manager.cc:5697] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:33.086622  8254 consensus_queue.cc:237] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:33.096989  9007 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:33.097644  8332 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:33.099046  9198 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:33.099943  9148 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:33.108726  9196 raft_consensus.cc:2955] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.110139  9007 raft_consensus.cc:2955] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.119545  8332 raft_consensus.cc:2955] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.122822  8139 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b3d99c80602e45da9c43a49cf83b8211 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:33.128134  8152 catalog_manager.cc:5697] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.151356  8393 tablet_service.cc:1558] Processing DeleteTablet for tablet b3d99c80602e45da9c43a49cf83b8211 with delete_type TABLET_DATA_TOMBSTONED (TS e64761f43e4240319c41117cf886d22f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39294
I20260709 13:01:33.161201  9238 tablet_replica.cc:333] T b3d99c80602e45da9c43a49cf83b8211 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:01:33.166502  9238 raft_consensus.cc:2243] T b3d99c80602e45da9c43a49cf83b8211 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:33.167294  9238 raft_consensus.cc:2272] T b3d99c80602e45da9c43a49cf83b8211 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:33.170126  9238 ts_tablet_manager.cc:1916] T b3d99c80602e45da9c43a49cf83b8211 P e64761f43e4240319c41117cf886d22f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:33.184351  9238 ts_tablet_manager.cc:1929] T b3d99c80602e45da9c43a49cf83b8211 P e64761f43e4240319c41117cf886d22f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:33.184830  9238 log.cc:1199] T b3d99c80602e45da9c43a49cf83b8211 P e64761f43e4240319c41117cf886d22f: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-2-root/wals/b3d99c80602e45da9c43a49cf83b8211
I20260709 13:01:33.186200  8141 catalog_manager.cc:5028] TS e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): tablet b3d99c80602e45da9c43a49cf83b8211 (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:33.202692  9057 raft_consensus.cc:1064] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:33.204758  9057 consensus_queue.cc:237] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:33.210176  8324 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:33.211457  9165 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:33.212630  8254 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:33.218308  9007 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:33.220038  9057 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:33.221794  9140 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:33.231240  9057 raft_consensus.cc:2955] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.233548  8254 raft_consensus.cc:2955] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.233353  9007 raft_consensus.cc:2955] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.249552  8152 catalog_manager.cc:5697] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.258423  8324 raft_consensus.cc:2955] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.278859  8398 consensus_queue.cc:237] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:33.282910  9196 raft_consensus.cc:1064] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:33.285912  9007 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:33.285501  9196 consensus_queue.cc:237] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:33.286814  8254 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:33.290419  9057 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:33.292043  9140 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:33.297775  9140 raft_consensus.cc:2955] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.300989  9007 raft_consensus.cc:2955] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.307842  9009 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:33.310657  8141 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4b9b125a55cf4a5d9db580313045edde with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:33.310034  8254 raft_consensus.cc:2955] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.317786  8151 catalog_manager.cc:5697] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.309325  9229 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 13:01:33.325246  8398 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:33.327267  8324 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:33.328014  9196 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:33.329878  9198 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:33.336810  9240 raft_consensus.cc:1064] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:33.340492  8314 tablet_service.cc:1558] Processing DeleteTablet for tablet 4b9b125a55cf4a5d9db580313045edde with delete_type TABLET_DATA_TOMBSTONED (TS ff7a6fe533e94c1bae555b4cae427728 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57170
I20260709 13:01:33.343226  9240 consensus_queue.cc:237] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:33.352895  9247 tablet_replica.cc:333] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:01:33.354030  9247 raft_consensus.cc:2243] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:33.354543  9198 raft_consensus.cc:2955] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.368595  9247 raft_consensus.cc:2272] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:33.367690  8398 raft_consensus.cc:2955] T fdf12882ac9e4f7d9121a5fcb9d4efaf P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.372347  9247 ts_tablet_manager.cc:1916] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:33.379195  8154 catalog_manager.cc:5697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:33.373209  8324 raft_consensus.cc:2955] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.377506  9007 raft_consensus.cc:2955] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.394555  9247 ts_tablet_manager.cc:1929] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:33.395107  9247 log.cc:1199] T 4b9b125a55cf4a5d9db580313045edde P ff7a6fe533e94c1bae555b4cae427728: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-1-root/wals/4b9b125a55cf4a5d9db580313045edde
I20260709 13:01:33.396742  8139 catalog_manager.cc:5028] TS ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): tablet 4b9b125a55cf4a5d9db580313045edde (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:33.401695  8254 consensus_queue.cc:237] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:33.407542  9007 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:33.408318  9009 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:33.410858  8323 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:33.412046  9198 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:33.416749  9140 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:33.417140  8324 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:33.420069  9196 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:33.422832  9140 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:33.424511  8254 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:33.432495  9240 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:33.435779  9198 raft_consensus.cc:2955] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.438853  8324 raft_consensus.cc:2955] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.448354  9007 raft_consensus.cc:2955] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.473109  8139 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fdf12882ac9e4f7d9121a5fcb9d4efaf with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:33.485742  8154 catalog_manager.cc:5697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.505505  9240 raft_consensus.cc:2955] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.516269  8393 tablet_service.cc:1558] Processing DeleteTablet for tablet fdf12882ac9e4f7d9121a5fcb9d4efaf with delete_type TABLET_DATA_TOMBSTONED (TS e64761f43e4240319c41117cf886d22f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39294
I20260709 13:01:33.518384  9238 tablet_replica.cc:333] T fdf12882ac9e4f7d9121a5fcb9d4efaf P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:01:33.517395  8332 raft_consensus.cc:2955] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.519424  9238 raft_consensus.cc:2243] T fdf12882ac9e4f7d9121a5fcb9d4efaf P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:33.520077  9238 raft_consensus.cc:2272] T fdf12882ac9e4f7d9121a5fcb9d4efaf P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:33.524389  9238 ts_tablet_manager.cc:1916] T fdf12882ac9e4f7d9121a5fcb9d4efaf P e64761f43e4240319c41117cf886d22f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:33.544145  8252 raft_consensus.cc:2955] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.551386  8154 catalog_manager.cc:5697] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.565539  9238 ts_tablet_manager.cc:1929] T fdf12882ac9e4f7d9121a5fcb9d4efaf P e64761f43e4240319c41117cf886d22f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:33.566102  9238 log.cc:1199] T fdf12882ac9e4f7d9121a5fcb9d4efaf P e64761f43e4240319c41117cf886d22f: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-2-root/wals/fdf12882ac9e4f7d9121a5fcb9d4efaf
I20260709 13:01:33.572003  9007 raft_consensus.cc:2955] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.586916  8141 catalog_manager.cc:5028] TS e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): tablet fdf12882ac9e4f7d9121a5fcb9d4efaf (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:33.595847  8398 consensus_queue.cc:237] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:33.603363  8324 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:33.605005  9240 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:33.609197  9007 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:33.616309  9257 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:33.624074  9240 raft_consensus.cc:2955] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.625149  9009 raft_consensus.cc:2955] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.639300  8324 raft_consensus.cc:2955] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.651125  8141 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 038cd0877a674c2996410c9f327df911 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:33.652127  8151 catalog_manager.cc:5697] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:33.665854  8245 tablet_service.cc:1558] Processing DeleteTablet for tablet 038cd0877a674c2996410c9f327df911 with delete_type TABLET_DATA_TOMBSTONED (TS 8c42259d7c4b463a9f485f42c320c81b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55248
I20260709 13:01:33.667740  9260 tablet_replica.cc:333] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:01:33.668779  9260 raft_consensus.cc:2243] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:33.669291  9260 raft_consensus.cc:2272] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:33.671929  9260 ts_tablet_manager.cc:1916] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:33.684231  9260 ts_tablet_manager.cc:1929] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:33.684566  9260 log.cc:1199] T 038cd0877a674c2996410c9f327df911 P 8c42259d7c4b463a9f485f42c320c81b: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-0-root/wals/038cd0877a674c2996410c9f327df911
I20260709 13:01:33.685987  8139 catalog_manager.cc:5028] TS 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): tablet 038cd0877a674c2996410c9f327df911 (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:34.741556  8252 consensus_queue.cc:237] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:34.749926  8405 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:34.751845  9230 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:01:34.752796  8223 consensus_peers.cc:597] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cb79894cecb541e3b7dd164271cadc16. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:34.753304  8332 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:34.754882  9229 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:34.761591  9230 raft_consensus.cc:2955] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:34.763226  8405 raft_consensus.cc:2955] T cb79894cecb541e3b7dd164271cadc16 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:34.770871  8332 raft_consensus.cc:2955] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:34.784396  8252 consensus_queue.cc:237] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
W20260709 13:01:34.794137  8223 consensus_peers.cc:597] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f19e818631f543b28a8bec9ba4547527. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:34.797528  8398 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:34.798892  9229 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:34.796545  8151 catalog_manager.cc:5697] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:34.803998  8324 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:34.810314  9265 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:34.818138  9263 raft_consensus.cc:2955] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:34.837934  8332 consensus_queue.cc:237] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:34.844717  8405 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:34.846118  9259 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:01:34.853287  8298 consensus_peers.cc:597] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: eeecf7303d634c0d8983b810bb97c33c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:34.859642  8332 raft_consensus.cc:2955] T f19e818631f543b28a8bec9ba4547527 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:34.860800  8252 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:34.862903  9262 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:34.869103  8398 raft_consensus.cc:2955] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:34.873767  9259 raft_consensus.cc:2955] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:34.878186  8252 raft_consensus.cc:2955] T eeecf7303d634c0d8983b810bb97c33c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:34.893312  8403 consensus_queue.cc:237] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:34.901788  8154 catalog_manager.cc:5697] T f19e818631f543b28a8bec9ba4547527 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:34.904726  8332 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:34.905651  8372 consensus_peers.cc:597] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39384c6d69ad4f658d2e777dfdcbb49c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:34.906374  9256 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:34.906816  8252 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:34.903326  8154 catalog_manager.cc:5697] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:34.911856  9256 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:34.908813  8403 raft_consensus.cc:2955] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:34.931878  9261 raft_consensus.cc:2955] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:34.957979  8332 raft_consensus.cc:2955] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:34.969489  8254 raft_consensus.cc:2955] T 39384c6d69ad4f658d2e777dfdcbb49c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:34.979036  8151 catalog_manager.cc:5697] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:35.215010  9288 ts_tablet_manager.cc:933] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679)
I20260709 13:01:35.217172  9288 tablet_copy_client.cc:323] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.1:44679
I20260709 13:01:35.218889  8267 tablet_copy_service.cc:140] P 8c42259d7c4b463a9f485f42c320c81b: Received BeginTabletCopySession request for tablet cb79894cecb541e3b7dd164271cadc16 from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:49220)
I20260709 13:01:35.219473  8267 tablet_copy_service.cc:161] P 8c42259d7c4b463a9f485f42c320c81b: Beginning new tablet copy session on tablet cb79894cecb541e3b7dd164271cadc16 from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:49220: session id = f865c4d761f5446eb12b3a029234772e-cb79894cecb541e3b7dd164271cadc16
I20260709 13:01:35.227293  8267 tablet_copy_source_session.cc:215] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:35.230388  9288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb79894cecb541e3b7dd164271cadc16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.243932  9288 tablet_copy_client.cc:806] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:35.244511  9288 tablet_copy_client.cc:670] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:35.248271  9288 tablet_copy_client.cc:538] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:35.255256  9288 tablet_bootstrap.cc:492] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:35.275246  9288 tablet_bootstrap.cc:492] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:35.277339  9288 tablet_bootstrap.cc:492] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:35.278424  9288 ts_tablet_manager.cc:1403] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.012s
I20260709 13:01:35.300767  9288 raft_consensus.cc:359] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:35.301762  9288 raft_consensus.cc:740] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:35.304119  9288 consensus_queue.cc:260] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:35.307488  9288 ts_tablet_manager.cc:1434] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.005s
I20260709 13:01:35.310436  8267 tablet_copy_service.cc:342] P 8c42259d7c4b463a9f485f42c320c81b: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-cb79894cecb541e3b7dd164271cadc16 received from {username='slave'} at 127.0.0.1:49220
I20260709 13:01:35.310863  8267 tablet_copy_service.cc:434] P 8c42259d7c4b463a9f485f42c320c81b: ending tablet copy session f865c4d761f5446eb12b3a029234772e-cb79894cecb541e3b7dd164271cadc16 on tablet cb79894cecb541e3b7dd164271cadc16 with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:35.361474  9288 ts_tablet_manager.cc:933] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:35.363126  9288 tablet_copy_client.cc:323] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.3:37367
I20260709 13:01:35.364567  8416 tablet_copy_service.cc:140] P e64761f43e4240319c41117cf886d22f: Received BeginTabletCopySession request for tablet 39384c6d69ad4f658d2e777dfdcbb49c from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:34462)
I20260709 13:01:35.365056  8416 tablet_copy_service.cc:161] P e64761f43e4240319c41117cf886d22f: Beginning new tablet copy session on tablet 39384c6d69ad4f658d2e777dfdcbb49c from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:34462: session id = f865c4d761f5446eb12b3a029234772e-39384c6d69ad4f658d2e777dfdcbb49c
I20260709 13:01:35.371780  8416 tablet_copy_source_session.cc:215] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:35.383260  9288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39384c6d69ad4f658d2e777dfdcbb49c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.391140  9293 ts_tablet_manager.cc:933] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679)
I20260709 13:01:35.397207  9293 tablet_copy_client.cc:323] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.1:44679
I20260709 13:01:35.399221  8267 tablet_copy_service.cc:140] P 8c42259d7c4b463a9f485f42c320c81b: Received BeginTabletCopySession request for tablet f19e818631f543b28a8bec9ba4547527 from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:49220)
I20260709 13:01:35.399783  8267 tablet_copy_service.cc:161] P 8c42259d7c4b463a9f485f42c320c81b: Beginning new tablet copy session on tablet f19e818631f543b28a8bec9ba4547527 from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:49220: session id = f865c4d761f5446eb12b3a029234772e-f19e818631f543b28a8bec9ba4547527
I20260709 13:01:35.413640  9295 ts_tablet_manager.cc:933] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:35.415181  9288 tablet_copy_client.cc:806] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:35.415773  9288 tablet_copy_client.cc:670] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:35.419978  9288 tablet_copy_client.cc:538] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:35.426185  8267 tablet_copy_source_session.cc:215] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:35.428687  9295 tablet_copy_client.cc:323] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.2:38599
I20260709 13:01:35.429984  9293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f19e818631f543b28a8bec9ba4547527. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.430409  8341 tablet_copy_service.cc:140] P ff7a6fe533e94c1bae555b4cae427728: Received BeginTabletCopySession request for tablet eeecf7303d634c0d8983b810bb97c33c from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:41664)
I20260709 13:01:35.430992  8341 tablet_copy_service.cc:161] P ff7a6fe533e94c1bae555b4cae427728: Beginning new tablet copy session on tablet eeecf7303d634c0d8983b810bb97c33c from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:41664: session id = f865c4d761f5446eb12b3a029234772e-eeecf7303d634c0d8983b810bb97c33c
I20260709 13:01:35.436918  9288 tablet_bootstrap.cc:492] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:35.451274  9293 tablet_copy_client.cc:806] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:35.451821  9293 tablet_copy_client.cc:670] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:35.452994  8341 tablet_copy_source_session.cc:215] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:35.456074  9295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeecf7303d634c0d8983b810bb97c33c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.462291  9293 tablet_copy_client.cc:538] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:35.468330  9288 tablet_bootstrap.cc:492] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:35.469223  9288 tablet_bootstrap.cc:492] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:35.471372  9295 tablet_copy_client.cc:806] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:35.471669  9293 tablet_bootstrap.cc:492] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:35.472337  9295 tablet_copy_client.cc:670] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:35.473522  9288 ts_tablet_manager.cc:1403] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.036s	user 0.020s	sys 0.012s
I20260709 13:01:35.476936  9295 tablet_copy_client.cc:538] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:35.476694  9288 raft_consensus.cc:359] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:35.477506  9288 raft_consensus.cc:740] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:35.478024  9288 consensus_queue.cc:260] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:35.480341  9288 ts_tablet_manager.cc:1434] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:01:35.482347  8416 tablet_copy_service.cc:342] P e64761f43e4240319c41117cf886d22f: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-39384c6d69ad4f658d2e777dfdcbb49c received from {username='slave'} at 127.0.0.1:34462
I20260709 13:01:35.482745  8416 tablet_copy_service.cc:434] P e64761f43e4240319c41117cf886d22f: ending tablet copy session f865c4d761f5446eb12b3a029234772e-39384c6d69ad4f658d2e777dfdcbb49c on tablet 39384c6d69ad4f658d2e777dfdcbb49c with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:35.489907  9295 tablet_bootstrap.cc:492] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:35.514238  9293 tablet_bootstrap.cc:492] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:35.515211  9293 tablet_bootstrap.cc:492] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:35.515982  9293 ts_tablet_manager.cc:1403] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.045s	user 0.010s	sys 0.017s
I20260709 13:01:35.519193  9293 raft_consensus.cc:359] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:35.521471  9293 raft_consensus.cc:740] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:35.522178  9293 consensus_queue.cc:260] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:35.526880  9295 tablet_bootstrap.cc:492] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:35.528698  9295 tablet_bootstrap.cc:492] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:35.529333  9295 ts_tablet_manager.cc:1403] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.040s	user 0.014s	sys 0.018s
I20260709 13:01:35.529673  9293 ts_tablet_manager.cc:1434] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.001s
I20260709 13:01:35.531531  8267 tablet_copy_service.cc:342] P 8c42259d7c4b463a9f485f42c320c81b: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-f19e818631f543b28a8bec9ba4547527 received from {username='slave'} at 127.0.0.1:49220
I20260709 13:01:35.531949  8267 tablet_copy_service.cc:434] P 8c42259d7c4b463a9f485f42c320c81b: ending tablet copy session f865c4d761f5446eb12b3a029234772e-f19e818631f543b28a8bec9ba4547527 on tablet f19e818631f543b28a8bec9ba4547527 with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:35.532455  9295 raft_consensus.cc:359] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:35.533800  9295 raft_consensus.cc:740] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:35.534479  9295 consensus_queue.cc:260] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:35.536793  9295 ts_tablet_manager.cc:1434] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:01:35.538462  8341 tablet_copy_service.cc:342] P ff7a6fe533e94c1bae555b4cae427728: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-eeecf7303d634c0d8983b810bb97c33c received from {username='slave'} at 127.0.0.1:41664
I20260709 13:01:35.538909  8341 tablet_copy_service.cc:434] P ff7a6fe533e94c1bae555b4cae427728: ending tablet copy session f865c4d761f5446eb12b3a029234772e-eeecf7303d634c0d8983b810bb97c33c on tablet eeecf7303d634c0d8983b810bb97c33c with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:35.837894  9007 raft_consensus.cc:1217] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:35.883215  9007 raft_consensus.cc:1217] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:35.957492  9007 raft_consensus.cc:1217] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:35.974673  9007 raft_consensus.cc:1217] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:36.349848  9265 raft_consensus.cc:1064] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:36.351653  9265 consensus_queue.cc:237] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:36.357878  8403 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:36.359223  8332 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:36.359891  9302 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:36.365453  9007 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:36.367684  9229 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:36.368427  9230 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:36.397305  9230 raft_consensus.cc:2955] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:36.399536  8398 raft_consensus.cc:2955] T cb79894cecb541e3b7dd164271cadc16 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:36.404318  8324 raft_consensus.cc:2955] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:36.409502  9007 raft_consensus.cc:2955] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:36.419178  8154 catalog_manager.cc:5697] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:36.440280  9302 raft_consensus.cc:1064] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:36.443004  9302 consensus_queue.cc:237] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:36.460345  9007 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:36.461585  8398 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:36.463075  8332 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:36.463271  9229 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:36.464949  9265 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:36.465744  9230 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:36.479967  8252 consensus_queue.cc:237] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:36.482717  9286 raft_consensus.cc:1064] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:36.485204  9286 consensus_queue.cc:237] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:36.498695  9009 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:36.500428  9230 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:36.501333  8332 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:36.502931  9230 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:36.510690  9302 raft_consensus.cc:2955] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:36.512283  9009 raft_consensus.cc:2955] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:36.520406  9229 raft_consensus.cc:2955] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:36.521219  8332 raft_consensus.cc:2955] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:36.537778  8139 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cb79894cecb541e3b7dd164271cadc16 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:36.540540  9007 raft_consensus.cc:2955] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:36.557888  8324 raft_consensus.cc:2955] T f19e818631f543b28a8bec9ba4547527 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:36.558842  8152 catalog_manager.cc:5697] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:36.567102  8323 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:36.568663  9300 raft_consensus.cc:1064] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:36.569589  9008 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:36.573854  8252 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:36.571151  9300 consensus_queue.cc:237] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:36.580869  9287 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:36.596704  8403 raft_consensus.cc:2955] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:36.602615  8151 catalog_manager.cc:5697] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:36.608723  9320 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 13:01:36.621456  9283 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:36.622172  9009 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:36.624303  8254 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:36.624866  8393 tablet_service.cc:1558] Processing DeleteTablet for tablet cb79894cecb541e3b7dd164271cadc16 with delete_type TABLET_DATA_TOMBSTONED (TS e64761f43e4240319c41117cf886d22f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39294
I20260709 13:01:36.658192  8398 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:36.659152  9326 tablet_replica.cc:333] T cb79894cecb541e3b7dd164271cadc16 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:01:36.665685  8254 consensus_queue.cc:237] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:36.625114  9300 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:01:36.693063  8398 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:36.695976  9008 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:36.698016  9319 raft_consensus.cc:2955] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:36.700918  9230 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:01:36.704524  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 67024.4us
W20260709 13:01:36.712246  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 56613.1us
I20260709 13:01:36.712194  9229 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:36.726758  9282 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 13:01:36.735917  8332 raft_consensus.cc:2955] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:36.736446  9229 raft_consensus.cc:2955] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:36.737610  9332 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:36.757031  9326 raft_consensus.cc:2243] T cb79894cecb541e3b7dd164271cadc16 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:36.757882  9326 raft_consensus.cc:2272] T cb79894cecb541e3b7dd164271cadc16 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:36.757068  9006 raft_consensus.cc:2955] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:36.738509  8398 raft_consensus.cc:2955] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:36.777197  9009 raft_consensus.cc:2955] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:36.781562  9326 ts_tablet_manager.cc:1916] T cb79894cecb541e3b7dd164271cadc16 P e64761f43e4240319c41117cf886d22f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:36.811468  8139 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f19e818631f543b28a8bec9ba4547527 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:01:36.820641  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 72074.7us
I20260709 13:01:36.820076  8152 catalog_manager.cc:5697] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:01:36.833657  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 113104us
I20260709 13:01:36.909387  9326 ts_tablet_manager.cc:1929] T cb79894cecb541e3b7dd164271cadc16 P e64761f43e4240319c41117cf886d22f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:36.909883  9326 log.cc:1199] T cb79894cecb541e3b7dd164271cadc16 P e64761f43e4240319c41117cf886d22f: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-2-root/wals/cb79894cecb541e3b7dd164271cadc16
I20260709 13:01:36.945158  9334 raft_consensus.cc:2955] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:36.959759  8403 raft_consensus.cc:2955] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:36.984764  8151 catalog_manager.cc:5697] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:37.026921  8252 raft_consensus.cc:2955] T eeecf7303d634c0d8983b810bb97c33c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:37.032764  8314 tablet_service.cc:1558] Processing DeleteTablet for tablet f19e818631f543b28a8bec9ba4547527 with delete_type TABLET_DATA_TOMBSTONED (TS ff7a6fe533e94c1bae555b4cae427728 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57170
I20260709 13:01:37.041774  8254 raft_consensus.cc:2955] T 39384c6d69ad4f658d2e777dfdcbb49c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:37.067373  8141 catalog_manager.cc:5028] TS e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): tablet cb79894cecb541e3b7dd164271cadc16 (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:37.088542  8398 consensus_queue.cc:237] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:37.099402  9341 tablet_replica.cc:333] T f19e818631f543b28a8bec9ba4547527 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:01:37.100565  9007 raft_consensus.cc:2955] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:37.105409  8151 catalog_manager.cc:5697] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:37.146185  9006 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:37.151656  8332 consensus_queue.cc:237] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:37.147970  9319 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:01:37.177659  9320 raft_consensus.cc:2955] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:37.178833  9007 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:37.180502  9262 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:37.183099  9006 raft_consensus.cc:2955] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:37.187759  9334 raft_consensus.cc:2955] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:37.189527  9009 raft_consensus.cc:2955] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:37.190804  8332 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:37.194959  8403 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 13:01:37.266090  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 58668.8us
I20260709 13:01:37.269930  9319 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:01:37.289363  9328 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:37.290707  9341 raft_consensus.cc:2243] T f19e818631f543b28a8bec9ba4547527 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:37.291834  9341 raft_consensus.cc:2272] T f19e818631f543b28a8bec9ba4547527 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:37.290867  8332 raft_consensus.cc:2955] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:37.318127  8139 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet eeecf7303d634c0d8983b810bb97c33c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:37.329295  9341 ts_tablet_manager.cc:1916] T f19e818631f543b28a8bec9ba4547527 P ff7a6fe533e94c1bae555b4cae427728: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:37.346736  9341 ts_tablet_manager.cc:1929] T f19e818631f543b28a8bec9ba4547527 P ff7a6fe533e94c1bae555b4cae427728: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:37.347395  9341 log.cc:1199] T f19e818631f543b28a8bec9ba4547527 P ff7a6fe533e94c1bae555b4cae427728: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-1-root/wals/f19e818631f543b28a8bec9ba4547527
I20260709 13:01:37.773816  8141 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 39384c6d69ad4f658d2e777dfdcbb49c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:37.825345  8139 catalog_manager.cc:5028] TS ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): tablet f19e818631f543b28a8bec9ba4547527 (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:37.945542  8405 raft_consensus.cc:2955] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:38.013747  8152 catalog_manager.cc:5697] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:38.038918  8245 tablet_service.cc:1558] Processing DeleteTablet for tablet 39384c6d69ad4f658d2e777dfdcbb49c with delete_type TABLET_DATA_TOMBSTONED (TS 8c42259d7c4b463a9f485f42c320c81b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55248
I20260709 13:01:38.039651  8151 catalog_manager.cc:5697] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:38.040896  9351 tablet_replica.cc:333] T 39384c6d69ad4f658d2e777dfdcbb49c P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:01:38.042301  9351 raft_consensus.cc:2243] T 39384c6d69ad4f658d2e777dfdcbb49c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:38.043092  9351 raft_consensus.cc:2272] T 39384c6d69ad4f658d2e777dfdcbb49c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:38.046883  9351 ts_tablet_manager.cc:1916] T 39384c6d69ad4f658d2e777dfdcbb49c P 8c42259d7c4b463a9f485f42c320c81b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:38.054160  8245 tablet_service.cc:1558] Processing DeleteTablet for tablet eeecf7303d634c0d8983b810bb97c33c with delete_type TABLET_DATA_TOMBSTONED (TS 8c42259d7c4b463a9f485f42c320c81b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55248
I20260709 13:01:38.062913  9351 ts_tablet_manager.cc:1929] T 39384c6d69ad4f658d2e777dfdcbb49c P 8c42259d7c4b463a9f485f42c320c81b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:38.063318  9351 log.cc:1199] T 39384c6d69ad4f658d2e777dfdcbb49c P 8c42259d7c4b463a9f485f42c320c81b: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-0-root/wals/39384c6d69ad4f658d2e777dfdcbb49c
I20260709 13:01:38.064677  9351 tablet_replica.cc:333] T eeecf7303d634c0d8983b810bb97c33c P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:01:38.064689  8139 catalog_manager.cc:5028] TS 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): tablet 39384c6d69ad4f658d2e777dfdcbb49c (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:38.065584  9351 raft_consensus.cc:2243] T eeecf7303d634c0d8983b810bb97c33c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:38.066176  9351 raft_consensus.cc:2272] T eeecf7303d634c0d8983b810bb97c33c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:38.068116  9351 ts_tablet_manager.cc:1916] T eeecf7303d634c0d8983b810bb97c33c P 8c42259d7c4b463a9f485f42c320c81b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:38.081151  9351 ts_tablet_manager.cc:1929] T eeecf7303d634c0d8983b810bb97c33c P 8c42259d7c4b463a9f485f42c320c81b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:38.081516  9351 log.cc:1199] T eeecf7303d634c0d8983b810bb97c33c P 8c42259d7c4b463a9f485f42c320c81b: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-0-root/wals/eeecf7303d634c0d8983b810bb97c33c
I20260709 13:01:38.082657  8139 catalog_manager.cc:5028] TS 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): tablet eeecf7303d634c0d8983b810bb97c33c (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
W20260709 13:01:38.824991  8353 debug-util.cc:398] Leaking SignalData structure 0x7b0800162400 after lost signal to thread 7167
W20260709 13:01:38.825855  8353 debug-util.cc:398] Leaking SignalData structure 0x7b08001c2380 after lost signal to thread 8189
W20260709 13:01:38.826805  8353 debug-util.cc:398] Leaking SignalData structure 0x7b08002f76e0 after lost signal to thread 8281
W20260709 13:01:38.827785  8353 debug-util.cc:398] Leaking SignalData structure 0x7b08003104a0 after lost signal to thread 8356
W20260709 13:01:38.828811  8353 debug-util.cc:398] Leaking SignalData structure 0x7b08003314e0 after lost signal to thread 8430
W20260709 13:01:38.829852  8353 debug-util.cc:398] Leaking SignalData structure 0x7b0800330ce0 after lost signal to thread 9034
I20260709 13:01:39.135476  8329 consensus_queue.cc:237] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:39.141741  8397 raft_consensus.cc:1275] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:39.143316  8249 raft_consensus.cc:1275] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:39.143324  8298 consensus_peers.cc:597] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2323b22f124b407c95caf5525949dd6c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:39.143311  9346 consensus_queue.cc:1048] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:39.145144  9262 consensus_queue.cc:1048] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:39.154773  9262 raft_consensus.cc:2955] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:39.156262  8249 raft_consensus.cc:2955] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:39.161392  8397 raft_consensus.cc:2955] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:39.176285  8152 catalog_manager.cc:5697] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:39.184470  8249 consensus_queue.cc:237] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
W20260709 13:01:39.206434  8223 consensus_peers.cc:597] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 10cffa7c6411440ca77cee6731faca67. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:39.218196  8397 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:39.220731  8329 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:39.220782  9353 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:39.222931  9359 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:39.244323  9318 raft_consensus.cc:2955] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:39.245493  8329 raft_consensus.cc:2955] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:39.253489  8397 raft_consensus.cc:2955] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:39.283607  8153 catalog_manager.cc:5697] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:39.317164  8404 consensus_queue.cc:237] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:39.329572  8249 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:39.329641  8372 consensus_peers.cc:597] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a0493b0dbbc746408a5db9e698678d83. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:39.331000  8325 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:39.332538  9287 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:39.334614  9287 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:39.396701  9287 raft_consensus.cc:2955] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:39.400331  8254 raft_consensus.cc:2955] T a0493b0dbbc746408a5db9e698678d83 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:39.402839  8325 raft_consensus.cc:2955] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:39.423067  8152 catalog_manager.cc:5697] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:39.440104  8249 consensus_queue.cc:237] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:39.450112  8405 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:39.451562  9318 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:39.451799  8325 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:39.456313  9318 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:01:39.461941  8223 consensus_peers.cc:597] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 83839ff3ce80447bb72b3c8d15b630ff. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:39.480244  9265 raft_consensus.cc:2955] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:39.481608  8404 raft_consensus.cc:2955] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:39.488616  8325 raft_consensus.cc:2955] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:39.497206  8153 catalog_manager.cc:5697] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:39.677304  9374 ts_tablet_manager.cc:933] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679)
I20260709 13:01:39.679425  9374 tablet_copy_client.cc:323] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.1:44679
I20260709 13:01:39.681409  8267 tablet_copy_service.cc:140] P 8c42259d7c4b463a9f485f42c320c81b: Received BeginTabletCopySession request for tablet 10cffa7c6411440ca77cee6731faca67 from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:49220)
I20260709 13:01:39.681972  8267 tablet_copy_service.cc:161] P 8c42259d7c4b463a9f485f42c320c81b: Beginning new tablet copy session on tablet 10cffa7c6411440ca77cee6731faca67 from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:49220: session id = f865c4d761f5446eb12b3a029234772e-10cffa7c6411440ca77cee6731faca67
I20260709 13:01:39.691869  8267 tablet_copy_source_session.cc:215] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:39.696098  9374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10cffa7c6411440ca77cee6731faca67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:39.713172  9374 tablet_copy_client.cc:806] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:39.714986  9374 tablet_copy_client.cc:670] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:39.719123  9374 tablet_copy_client.cc:538] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:39.731171  9374 tablet_bootstrap.cc:492] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:39.774469  9376 ts_tablet_manager.cc:933] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:39.783142  9374 tablet_bootstrap.cc:492] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:39.784524  9374 tablet_bootstrap.cc:492] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:39.785635  9374 ts_tablet_manager.cc:1403] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.055s	user 0.026s	sys 0.016s
I20260709 13:01:39.786271  9376 tablet_copy_client.cc:323] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.2:38599
I20260709 13:01:39.788321  8341 tablet_copy_service.cc:140] P ff7a6fe533e94c1bae555b4cae427728: Received BeginTabletCopySession request for tablet 2323b22f124b407c95caf5525949dd6c from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:41664)
I20260709 13:01:39.788880  8341 tablet_copy_service.cc:161] P ff7a6fe533e94c1bae555b4cae427728: Beginning new tablet copy session on tablet 2323b22f124b407c95caf5525949dd6c from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:41664: session id = f865c4d761f5446eb12b3a029234772e-2323b22f124b407c95caf5525949dd6c
I20260709 13:01:39.790376  9374 raft_consensus.cc:359] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:39.791517  9374 raft_consensus.cc:740] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:39.792315  9374 consensus_queue.cc:260] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:39.799594  8341 tablet_copy_source_session.cc:215] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:39.804566  9376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2323b22f124b407c95caf5525949dd6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:39.846179  9380 ts_tablet_manager.cc:933] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:39.859436  9376 tablet_copy_client.cc:806] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:39.860209  9376 tablet_copy_client.cc:670] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:39.861933  9374 ts_tablet_manager.cc:1434] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.076s	user 0.014s	sys 0.000s
I20260709 13:01:39.871064  8267 tablet_copy_service.cc:342] P 8c42259d7c4b463a9f485f42c320c81b: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-10cffa7c6411440ca77cee6731faca67 received from {username='slave'} at 127.0.0.1:49220
I20260709 13:01:39.871140  9376 tablet_copy_client.cc:538] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:39.871622  8267 tablet_copy_service.cc:434] P 8c42259d7c4b463a9f485f42c320c81b: ending tablet copy session f865c4d761f5446eb12b3a029234772e-10cffa7c6411440ca77cee6731faca67 on tablet 10cffa7c6411440ca77cee6731faca67 with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:39.897579  9380 tablet_copy_client.cc:323] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.3:37367
I20260709 13:01:39.903128  8416 tablet_copy_service.cc:140] P e64761f43e4240319c41117cf886d22f: Received BeginTabletCopySession request for tablet a0493b0dbbc746408a5db9e698678d83 from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:34462)
I20260709 13:01:39.903810  8416 tablet_copy_service.cc:161] P e64761f43e4240319c41117cf886d22f: Beginning new tablet copy session on tablet a0493b0dbbc746408a5db9e698678d83 from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:34462: session id = f865c4d761f5446eb12b3a029234772e-a0493b0dbbc746408a5db9e698678d83
I20260709 13:01:39.906818  9376 tablet_bootstrap.cc:492] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:39.940173  8416 tablet_copy_source_session.cc:215] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:39.980064  9376 tablet_bootstrap.cc:492] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:39.983314  9376 tablet_bootstrap.cc:492] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:39.984089  9376 ts_tablet_manager.cc:1403] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.078s	user 0.021s	sys 0.005s
I20260709 13:01:39.988088  9376 raft_consensus.cc:359] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:39.990054  9376 raft_consensus.cc:740] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:39.990252  9380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0493b0dbbc746408a5db9e698678d83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:39.990857  9376 consensus_queue.cc:260] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:39.993690  9376 ts_tablet_manager.cc:1434] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 13:01:39.995548  8341 tablet_copy_service.cc:342] P ff7a6fe533e94c1bae555b4cae427728: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-2323b22f124b407c95caf5525949dd6c received from {username='slave'} at 127.0.0.1:41664
I20260709 13:01:39.995985  8341 tablet_copy_service.cc:434] P ff7a6fe533e94c1bae555b4cae427728: ending tablet copy session f865c4d761f5446eb12b3a029234772e-2323b22f124b407c95caf5525949dd6c on tablet 2323b22f124b407c95caf5525949dd6c with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:40.033793  9380 tablet_copy_client.cc:806] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:40.036576  9380 tablet_copy_client.cc:670] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:40.040768  9380 tablet_copy_client.cc:538] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:40.050129  9380 tablet_bootstrap.cc:492] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:40.056792  9006 raft_consensus.cc:1217] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:40.060072  9376 ts_tablet_manager.cc:933] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679)
I20260709 13:01:40.080042  9376 tablet_copy_client.cc:323] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.1:44679
I20260709 13:01:40.082598  8267 tablet_copy_service.cc:140] P 8c42259d7c4b463a9f485f42c320c81b: Received BeginTabletCopySession request for tablet 83839ff3ce80447bb72b3c8d15b630ff from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:49220)
I20260709 13:01:40.083143  8267 tablet_copy_service.cc:161] P 8c42259d7c4b463a9f485f42c320c81b: Beginning new tablet copy session on tablet 83839ff3ce80447bb72b3c8d15b630ff from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:49220: session id = f865c4d761f5446eb12b3a029234772e-83839ff3ce80447bb72b3c8d15b630ff
I20260709 13:01:40.094270  9380 tablet_bootstrap.cc:492] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:40.095646  9380 tablet_bootstrap.cc:492] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:40.096580  9380 ts_tablet_manager.cc:1403] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.047s	user 0.014s	sys 0.017s
I20260709 13:01:40.099788  9380 raft_consensus.cc:359] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:40.100764  9380 raft_consensus.cc:740] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:40.101498  9380 consensus_queue.cc:260] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:40.104640  9380 ts_tablet_manager.cc:1434] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:01:40.110002  8416 tablet_copy_service.cc:342] P e64761f43e4240319c41117cf886d22f: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-a0493b0dbbc746408a5db9e698678d83 received from {username='slave'} at 127.0.0.1:34462
I20260709 13:01:40.110430  8416 tablet_copy_service.cc:434] P e64761f43e4240319c41117cf886d22f: ending tablet copy session f865c4d761f5446eb12b3a029234772e-a0493b0dbbc746408a5db9e698678d83 on tablet a0493b0dbbc746408a5db9e698678d83 with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:40.119541  8267 tablet_copy_source_session.cc:215] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:40.122833  9376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83839ff3ce80447bb72b3c8d15b630ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:40.151363  9376 tablet_copy_client.cc:806] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:40.152070  9376 tablet_copy_client.cc:670] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:40.155717  9376 tablet_copy_client.cc:538] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:40.165910  9376 tablet_bootstrap.cc:492] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:40.193980  9376 tablet_bootstrap.cc:492] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:40.195307  9376 tablet_bootstrap.cc:492] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:40.196048  9376 ts_tablet_manager.cc:1403] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.007s
I20260709 13:01:40.199156  9376 raft_consensus.cc:359] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:40.200122  9376 raft_consensus.cc:740] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:40.200804  9376 consensus_queue.cc:260] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:40.211089  9376 ts_tablet_manager.cc:1434] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:01:40.226679  8267 tablet_copy_service.cc:342] P 8c42259d7c4b463a9f485f42c320c81b: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-83839ff3ce80447bb72b3c8d15b630ff received from {username='slave'} at 127.0.0.1:49220
I20260709 13:01:40.227111  8267 tablet_copy_service.cc:434] P 8c42259d7c4b463a9f485f42c320c81b: ending tablet copy session f865c4d761f5446eb12b3a029234772e-83839ff3ce80447bb72b3c8d15b630ff on tablet 83839ff3ce80447bb72b3c8d15b630ff with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:40.235342  9006 raft_consensus.cc:1217] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:40.339313  9006 raft_consensus.cc:1217] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:40.519204  9006 raft_consensus.cc:1217] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:40.593189  9353 raft_consensus.cc:1064] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:40.595624  9353 consensus_queue.cc:237] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:40.602989  9006 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:40.603138  8404 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:40.604568  9265 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:40.606233  9359 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:40.610131  8325 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:40.617308  9359 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:40.637878  9265 raft_consensus.cc:2955] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:40.646552  9006 raft_consensus.cc:2955] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:40.656273  8404 raft_consensus.cc:2955] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:40.666342  8153 catalog_manager.cc:5697] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:40.722219  8325 raft_consensus.cc:2955] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:40.733497  8249 consensus_queue.cc:237] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:40.735265  9262 raft_consensus.cc:1064] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:40.737550  9262 consensus_queue.cc:237] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:40.739475  9009 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:40.740759  9359 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:40.742810  8404 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:40.747169  9353 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:40.759398  9353 raft_consensus.cc:2955] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:40.762688  9009 raft_consensus.cc:2955] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:40.766527  8404 raft_consensus.cc:2955] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:40.795388  8152 catalog_manager.cc:5697] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:40.797638  8139 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 10cffa7c6411440ca77cee6731faca67 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:40.813517  8314 tablet_service.cc:1558] Processing DeleteTablet for tablet 10cffa7c6411440ca77cee6731faca67 with delete_type TABLET_DATA_TOMBSTONED (TS ff7a6fe533e94c1bae555b4cae427728 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57170
I20260709 13:01:40.816052  8405 raft_consensus.cc:1275] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:40.817723  9346 consensus_queue.cc:1048] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:40.820773  9397 tablet_replica.cc:333] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:01:40.821859  9397 raft_consensus.cc:2243] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:40.834038  9397 raft_consensus.cc:2272] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:40.840770  8249 raft_consensus.cc:1275] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:40.846407  9262 consensus_queue.cc:1048] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:40.913810  9006 raft_consensus.cc:1275] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:40.925895  9397 ts_tablet_manager.cc:1916] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:40.980424  9287 raft_consensus.cc:1064] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:41.002202  9262 consensus_queue.cc:1048] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:41.016372  9287 consensus_queue.cc:237] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:41.030656  9359 raft_consensus.cc:1064] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:41.033175  9359 consensus_queue.cc:237] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:41.078083  9346 raft_consensus.cc:2955] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.075801  8405 raft_consensus.cc:2955] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.099627  9397 ts_tablet_manager.cc:1929] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:41.107430  9397 log.cc:1199] T 10cffa7c6411440ca77cee6731faca67 P ff7a6fe533e94c1bae555b4cae427728: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-1-root/wals/10cffa7c6411440ca77cee6731faca67
I20260709 13:01:41.138732  8152 catalog_manager.cc:5697] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.142828  8256 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:41.141110  8325 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:41.141551  9006 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:41.144935  9287 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:41.158210  9401 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:01:41.174407  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 164110us
I20260709 13:01:41.175319  8139 catalog_manager.cc:5028] TS ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): tablet 10cffa7c6411440ca77cee6731faca67 (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:41.178066  8256 raft_consensus.cc:2955] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.188115  9287 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:41.208809  9009 raft_consensus.cc:2955] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.220820  8404 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:41.223340  9007 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:41.223387  9353 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:41.225713  9415 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:41.232952  8325 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:41.235008  9415 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:41.235858  9401 raft_consensus.cc:2955] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.237771  9006 raft_consensus.cc:2955] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.253228  8152 catalog_manager.cc:5697] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.271329  8248 raft_consensus.cc:2955] T a0493b0dbbc746408a5db9e698678d83 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.273145  8329 raft_consensus.cc:2955] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.301677  8397 consensus_queue.cc:237] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:41.320991  9007 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:41.322810  9287 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:41.323648  8324 consensus_queue.cc:237] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:41.337855  9007 raft_consensus.cc:1275] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:41.342407  9401 raft_consensus.cc:2955] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.346364  9006 raft_consensus.cc:2955] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.351195  9359 raft_consensus.cc:2955] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.356509  9008 raft_consensus.cc:2955] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.362437  8141 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a0493b0dbbc746408a5db9e698678d83 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:41.370752  8324 raft_consensus.cc:2955] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.374410  9346 consensus_queue.cc:1048] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:41.376909  8151 catalog_manager.cc:5697] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:41.384963  8397 raft_consensus.cc:2955] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.391866  9262 raft_consensus.cc:2955] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.402302  9009 raft_consensus.cc:2955] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.406188  8254 raft_consensus.cc:1275] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:41.408592  8325 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:41.412966  8139 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2323b22f124b407c95caf5525949dd6c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:41.413703  9340 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:41.418206  8325 raft_consensus.cc:2955] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.432399  8152 catalog_manager.cc:5697] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.436049  9346 consensus_queue.cc:1048] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:41.441864  8245 tablet_service.cc:1558] Processing DeleteTablet for tablet a0493b0dbbc746408a5db9e698678d83 with delete_type TABLET_DATA_TOMBSTONED (TS 8c42259d7c4b463a9f485f42c320c81b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55248
I20260709 13:01:41.445364  8248 raft_consensus.cc:2955] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.453926  8254 consensus_queue.cc:237] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:41.459749  9006 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:41.460335  8329 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:41.463459  9359 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:41.463958  8151 catalog_manager.cc:5697] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.470754  9415 raft_consensus.cc:2955] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.477489  9387 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:41.481801  9006 raft_consensus.cc:2955] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.486483  8325 raft_consensus.cc:2955] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.495931  8393 tablet_service.cc:1558] Processing DeleteTablet for tablet 2323b22f124b407c95caf5525949dd6c with delete_type TABLET_DATA_TOMBSTONED (TS e64761f43e4240319c41117cf886d22f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39294
I20260709 13:01:41.510311  8151 catalog_manager.cc:5697] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 3 to 4, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:41.521718  8139 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 83839ff3ce80447bb72b3c8d15b630ff with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:41.535182  8393 tablet_service.cc:1558] Processing DeleteTablet for tablet 83839ff3ce80447bb72b3c8d15b630ff with delete_type TABLET_DATA_TOMBSTONED (TS e64761f43e4240319c41117cf886d22f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39294
I20260709 13:01:41.536991  9426 tablet_replica.cc:333] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:01:41.538223  9426 raft_consensus.cc:2243] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:41.539075  9426 raft_consensus.cc:2272] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:41.540544  9425 tablet_replica.cc:333] T a0493b0dbbc746408a5db9e698678d83 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:01:41.546185  9425 raft_consensus.cc:2243] T a0493b0dbbc746408a5db9e698678d83 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:41.555195  9425 raft_consensus.cc:2272] T a0493b0dbbc746408a5db9e698678d83 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:41.559182  9426 ts_tablet_manager.cc:1916] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:41.559006  9425 ts_tablet_manager.cc:1916] T a0493b0dbbc746408a5db9e698678d83 P 8c42259d7c4b463a9f485f42c320c81b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:41.585173  9425 ts_tablet_manager.cc:1929] T a0493b0dbbc746408a5db9e698678d83 P 8c42259d7c4b463a9f485f42c320c81b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:41.585696  9425 log.cc:1199] T a0493b0dbbc746408a5db9e698678d83 P 8c42259d7c4b463a9f485f42c320c81b: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-0-root/wals/a0493b0dbbc746408a5db9e698678d83
I20260709 13:01:41.588105  8139 catalog_manager.cc:5028] TS 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): tablet a0493b0dbbc746408a5db9e698678d83 (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:41.591388  9426 ts_tablet_manager.cc:1929] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:41.591919  9426 log.cc:1199] T 2323b22f124b407c95caf5525949dd6c P e64761f43e4240319c41117cf886d22f: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-2-root/wals/2323b22f124b407c95caf5525949dd6c
I20260709 13:01:41.594014  8141 catalog_manager.cc:5028] TS e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): tablet 2323b22f124b407c95caf5525949dd6c (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:41.597180  9426 tablet_replica.cc:333] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:01:41.606256  9426 raft_consensus.cc:2243] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:41.606940  9426 raft_consensus.cc:2272] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:41.612102  9426 ts_tablet_manager.cc:1916] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:41.629653  9426 ts_tablet_manager.cc:1929] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:41.630072  9426 log.cc:1199] T 83839ff3ce80447bb72b3c8d15b630ff P e64761f43e4240319c41117cf886d22f: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-2-root/wals/83839ff3ce80447bb72b3c8d15b630ff
I20260709 13:01:41.631721  8141 catalog_manager.cc:5028] TS e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): tablet 83839ff3ce80447bb72b3c8d15b630ff (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:42.618501  8397 consensus_queue.cc:237] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:42.626693  8329 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:42.627950  9340 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:42.628015  8254 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:42.628750  8372 consensus_peers.cc:597] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 007e904f46ef4a1a978f97d557d2a65b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:42.632251  9340 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:42.662564  8325 raft_consensus.cc:2955] T 007e904f46ef4a1a978f97d557d2a65b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:42.648798  9431 raft_consensus.cc:2955] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:42.679184  8254 raft_consensus.cc:2955] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:42.681378  8153 catalog_manager.cc:5697] T 007e904f46ef4a1a978f97d557d2a65b P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:42.696072  8325 consensus_queue.cc:237] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:42.711655  8397 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:42.712291  8298 consensus_peers.cc:597] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 954d2ad09dcb4697adccd661625a841b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:42.713590  9409 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:42.719187  8254 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:42.721980  9427 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:42.756989  9440 raft_consensus.cc:2955] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:42.760699  8404 raft_consensus.cc:2955] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:42.780237  8152 catalog_manager.cc:5697] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:42.780905  8248 consensus_queue.cc:237] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
W20260709 13:01:42.799746  8223 consensus_peers.cc:597] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7c15354a2a84f868942919f011af448. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:42.797253  8254 raft_consensus.cc:2955] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:42.803856  8397 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:42.804653  8324 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:42.805258  9415 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:42.810559  9353 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:42.819186  9387 raft_consensus.cc:2955] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:42.830049  8332 raft_consensus.cc:2955] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:42.844168  8152 catalog_manager.cc:5697] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:42.848667  8248 consensus_queue.cc:237] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
W20260709 13:01:42.870048  8223 consensus_peers.cc:597] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60c80a6b33dc431eac0fe35b37d4ea23. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:42.870388  8404 raft_consensus.cc:2955] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:42.879186  8397 raft_consensus.cc:1275] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:42.881150  9353 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:42.890667  8332 raft_consensus.cc:1275] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:42.893517  9353 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:42.906158  9353 raft_consensus.cc:2955] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:42.928285  8152 catalog_manager.cc:5697] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:42.932662  8325 raft_consensus.cc:2955] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:42.939406  8404 raft_consensus.cc:2955] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:43.346349  9454 ts_tablet_manager.cc:933] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:43.354326  9456 ts_tablet_manager.cc:933] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679)
I20260709 13:01:43.358775  9456 tablet_copy_client.cc:323] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.1:44679
I20260709 13:01:43.361109  9454 tablet_copy_client.cc:323] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.2:38599
I20260709 13:01:43.361552  9457 ts_tablet_manager.cc:933] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:43.362036  8267 tablet_copy_service.cc:140] P 8c42259d7c4b463a9f485f42c320c81b: Received BeginTabletCopySession request for tablet b7c15354a2a84f868942919f011af448 from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:49220)
I20260709 13:01:43.362768  8267 tablet_copy_service.cc:161] P 8c42259d7c4b463a9f485f42c320c81b: Beginning new tablet copy session on tablet b7c15354a2a84f868942919f011af448 from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:49220: session id = f865c4d761f5446eb12b3a029234772e-b7c15354a2a84f868942919f011af448
I20260709 13:01:43.363591  8341 tablet_copy_service.cc:140] P ff7a6fe533e94c1bae555b4cae427728: Received BeginTabletCopySession request for tablet 954d2ad09dcb4697adccd661625a841b from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:41664)
I20260709 13:01:43.364095  8341 tablet_copy_service.cc:161] P ff7a6fe533e94c1bae555b4cae427728: Beginning new tablet copy session on tablet 954d2ad09dcb4697adccd661625a841b from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:41664: session id = f865c4d761f5446eb12b3a029234772e-954d2ad09dcb4697adccd661625a841b
I20260709 13:01:43.370111  8267 tablet_copy_source_session.cc:215] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:43.374570  8341 tablet_copy_source_session.cc:215] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:43.377075  9456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7c15354a2a84f868942919f011af448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:43.385995  9454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 954d2ad09dcb4697adccd661625a841b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:43.412580  9456 tablet_copy_client.cc:806] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:43.413208  9456 tablet_copy_client.cc:670] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:43.430168  9461 ts_tablet_manager.cc:933] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679)
I20260709 13:01:43.440368  9456 tablet_copy_client.cc:538] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:43.441799  9461 tablet_copy_client.cc:323] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.1:44679
I20260709 13:01:43.445452  8267 tablet_copy_service.cc:140] P 8c42259d7c4b463a9f485f42c320c81b: Received BeginTabletCopySession request for tablet 60c80a6b33dc431eac0fe35b37d4ea23 from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:49220)
I20260709 13:01:43.446048  8267 tablet_copy_service.cc:161] P 8c42259d7c4b463a9f485f42c320c81b: Beginning new tablet copy session on tablet 60c80a6b33dc431eac0fe35b37d4ea23 from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:49220: session id = f865c4d761f5446eb12b3a029234772e-60c80a6b33dc431eac0fe35b37d4ea23
I20260709 13:01:43.447453  9457 tablet_copy_client.cc:323] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.3:37367
I20260709 13:01:43.454852  9454 tablet_copy_client.cc:806] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:43.455616  9454 tablet_copy_client.cc:670] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:43.461576  8267 tablet_copy_source_session.cc:215] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:43.462536  9456 tablet_bootstrap.cc:492] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:43.470350  9461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60c80a6b33dc431eac0fe35b37d4ea23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:43.469519  9454 tablet_copy_client.cc:538] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:43.481386  8416 tablet_copy_service.cc:140] P e64761f43e4240319c41117cf886d22f: Received BeginTabletCopySession request for tablet 007e904f46ef4a1a978f97d557d2a65b from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:34462)
I20260709 13:01:43.482096  8416 tablet_copy_service.cc:161] P e64761f43e4240319c41117cf886d22f: Beginning new tablet copy session on tablet 007e904f46ef4a1a978f97d557d2a65b from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:34462: session id = f865c4d761f5446eb12b3a029234772e-007e904f46ef4a1a978f97d557d2a65b
I20260709 13:01:43.495311  9454 tablet_bootstrap.cc:492] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:43.509626  8416 tablet_copy_source_session.cc:215] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:43.531463  9457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 007e904f46ef4a1a978f97d557d2a65b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:43.568713  9454 tablet_bootstrap.cc:492] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:43.569890  9454 tablet_bootstrap.cc:492] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:43.570753  9454 ts_tablet_manager.cc:1403] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.076s	user 0.021s	sys 0.011s
I20260709 13:01:43.574265  9454 raft_consensus.cc:359] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:43.575165  9454 raft_consensus.cc:740] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:43.575846  9454 consensus_queue.cc:260] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:43.581528  9461 tablet_copy_client.cc:806] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:43.582226  9461 tablet_copy_client.cc:670] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:43.587468  9456 tablet_bootstrap.cc:492] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:43.591871  9454 ts_tablet_manager.cc:1434] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.005s
I20260709 13:01:43.597203  9456 tablet_bootstrap.cc:492] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:43.597968  9456 ts_tablet_manager.cc:1403] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.137s	user 0.040s	sys 0.040s
I20260709 13:01:43.598134  8341 tablet_copy_service.cc:342] P ff7a6fe533e94c1bae555b4cae427728: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-954d2ad09dcb4697adccd661625a841b received from {username='slave'} at 127.0.0.1:41664
I20260709 13:01:43.598616  8341 tablet_copy_service.cc:434] P ff7a6fe533e94c1bae555b4cae427728: ending tablet copy session f865c4d761f5446eb12b3a029234772e-954d2ad09dcb4697adccd661625a841b on tablet 954d2ad09dcb4697adccd661625a841b with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:43.604220  9456 raft_consensus.cc:359] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:43.605171  9456 raft_consensus.cc:740] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:43.605907  9456 consensus_queue.cc:260] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:43.608480  9456 ts_tablet_manager.cc:1434] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:01:43.613809  9461 tablet_copy_client.cc:538] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:43.614625  8267 tablet_copy_service.cc:342] P 8c42259d7c4b463a9f485f42c320c81b: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-b7c15354a2a84f868942919f011af448 received from {username='slave'} at 127.0.0.1:49220
I20260709 13:01:43.615126  8267 tablet_copy_service.cc:434] P 8c42259d7c4b463a9f485f42c320c81b: ending tablet copy session f865c4d761f5446eb12b3a029234772e-b7c15354a2a84f868942919f011af448 on tablet b7c15354a2a84f868942919f011af448 with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:43.625422  9457 tablet_copy_client.cc:806] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:43.626112  9457 tablet_copy_client.cc:670] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:43.632402  9461 tablet_bootstrap.cc:492] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:43.635008  9457 tablet_copy_client.cc:538] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:43.661628  9457 tablet_bootstrap.cc:492] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:43.679777  9461 tablet_bootstrap.cc:492] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:43.680820  9461 tablet_bootstrap.cc:492] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:43.686893  9006 raft_consensus.cc:1217] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:43.692737  9461 ts_tablet_manager.cc:1403] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.061s	user 0.028s	sys 0.004s
I20260709 13:01:43.697499  9461 raft_consensus.cc:359] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:43.698483  9461 raft_consensus.cc:740] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:43.699257  9461 consensus_queue.cc:260] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:43.705619  9461 ts_tablet_manager.cc:1434] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.004s
I20260709 13:01:43.707708  8267 tablet_copy_service.cc:342] P 8c42259d7c4b463a9f485f42c320c81b: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-60c80a6b33dc431eac0fe35b37d4ea23 received from {username='slave'} at 127.0.0.1:49220
I20260709 13:01:43.708269  8267 tablet_copy_service.cc:434] P 8c42259d7c4b463a9f485f42c320c81b: ending tablet copy session f865c4d761f5446eb12b3a029234772e-60c80a6b33dc431eac0fe35b37d4ea23 on tablet 60c80a6b33dc431eac0fe35b37d4ea23 with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:43.720448  9457 tablet_bootstrap.cc:492] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:43.721673  9457 tablet_bootstrap.cc:492] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:43.722498  9457 ts_tablet_manager.cc:1403] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.062s	user 0.020s	sys 0.007s
I20260709 13:01:43.728157  9457 raft_consensus.cc:359] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:43.729193  9457 raft_consensus.cc:740] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:43.729986  9457 consensus_queue.cc:260] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:43.733042  9457 ts_tablet_manager.cc:1434] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.003s
I20260709 13:01:43.735389  8416 tablet_copy_service.cc:342] P e64761f43e4240319c41117cf886d22f: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-007e904f46ef4a1a978f97d557d2a65b received from {username='slave'} at 127.0.0.1:34462
I20260709 13:01:43.735842  8416 tablet_copy_service.cc:434] P e64761f43e4240319c41117cf886d22f: ending tablet copy session f865c4d761f5446eb12b3a029234772e-007e904f46ef4a1a978f97d557d2a65b on tablet 007e904f46ef4a1a978f97d557d2a65b with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:43.885900  9006 raft_consensus.cc:1217] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 13:01:43.943894  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 49544.9us
I20260709 13:01:43.951988  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:01:44.061092  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 165078us
I20260709 13:01:44.065573  9009 raft_consensus.cc:1217] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 13:01:44.094756  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 53940.6us
W20260709 13:01:44.137048  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 49478.7us
W20260709 13:01:44.139107  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 57032.4us
I20260709 13:01:44.168933  9007 raft_consensus.cc:1217] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:44.179471  9473 raft_consensus.cc:1064] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:44.182368  9473 consensus_queue.cc:237] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:44.243553  8404 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:44.244302  9009 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:44.244900  8254 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:44.245795  9473 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:44.247274  9409 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:44.248044  9481 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:44.339231  9473 raft_consensus.cc:2955] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:44.352620  8404 raft_consensus.cc:2955] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:44.354126  9486 raft_consensus.cc:1064] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:44.357234  9486 consensus_queue.cc:237] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:44.371986  9009 raft_consensus.cc:2955] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:44.375742  8152 catalog_manager.cc:5697] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:44.380944  8254 raft_consensus.cc:2955] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:44.384197  9007 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:44.385766  9430 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:44.408392  8332 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:44.413012  8251 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:44.421272  9449 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:44.423157  9449 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:44.458755  9449 raft_consensus.cc:2955] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:44.476146  8249 raft_consensus.cc:2955] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:44.491564  8404 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:60896
I20260709 13:01:44.492255  8404 raft_consensus.cc:606] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:44.494611  9009 raft_consensus.cc:2955] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:44.493603  8332 raft_consensus.cc:2955] T 007e904f46ef4a1a978f97d557d2a65b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:44.533376  8249 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "ff7a6fe533e94c1bae555b4cae427728"
 from {username='slave'} at 127.0.0.1:50420
I20260709 13:01:44.533968  8249 raft_consensus.cc:606] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Received request to transfer leadership to TS ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:44.548699  8151 catalog_manager.cc:5697] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:44.553901  8249 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "ff7a6fe533e94c1bae555b4cae427728"
 from {username='slave'} at 127.0.0.1:50420
I20260709 13:01:44.554523  8249 raft_consensus.cc:606] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Received request to transfer leadership to TS ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:44.558347  8249 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7c15354a2a84f868942919f011af448"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "ff7a6fe533e94c1bae555b4cae427728"
 from {username='slave'} at 127.0.0.1:50420
I20260709 13:01:44.558847  8249 raft_consensus.cc:606] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Received request to transfer leadership to TS ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:44.564338  8249 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "f865c4d761f5446eb12b3a029234772e"
 from {username='slave'} at 127.0.0.1:50420
I20260709 13:01:44.564870  8249 raft_consensus.cc:606] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Received request to transfer leadership to TS f865c4d761f5446eb12b3a029234772e
I20260709 13:01:44.568449  8249 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "f865c4d761f5446eb12b3a029234772e"
 from {username='slave'} at 127.0.0.1:50420
I20260709 13:01:44.569058  8249 raft_consensus.cc:606] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Received request to transfer leadership to TS f865c4d761f5446eb12b3a029234772e
I20260709 13:01:44.570361  8332 consensus_queue.cc:237] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:44.578691  9484 raft_consensus.cc:1064] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:44.598341  9484 consensus_queue.cc:237] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:44.595039  8404 consensus_queue.cc:237] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:44.588296  9415 raft_consensus.cc:1064] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
W20260709 13:01:44.601702  9415 raft_consensus.cc:1068] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b: Unable to promote non-voter f865c4d761f5446eb12b3a029234772e: Service unavailable: leader transfer in progress
I20260709 13:01:44.607642  9007 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:44.609417  9487 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:44.624754  9009 raft_consensus.cc:1275] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:44.626917  8404 raft_consensus.cc:1275] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:44.627791  9415 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:44.633349  9415 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:44.636305  9006 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:44.642131  8329 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:01:44.646785  8329 raft_consensus.cc:606] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:44.647125  9401 raft_consensus.cc:2955] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:44.648238  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 13:01:44.649900  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:44.650462  9409 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:44.662626  9008 raft_consensus.cc:2955] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:44.670163  9485 raft_consensus.cc:993] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b: : Instructing follower ff7a6fe533e94c1bae555b4cae427728 to start an election
I20260709 13:01:44.670680  9485 raft_consensus.cc:1081] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Signalling peer ff7a6fe533e94c1bae555b4cae427728 to start an election
I20260709 13:01:44.674854  9006 raft_consensus.cc:2955] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:44.672271  9477 raft_consensus.cc:2955] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:44.691741  8249 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:44.694727  9440 raft_consensus.cc:993] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728: : Instructing follower 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:01:44.695153  9440 raft_consensus.cc:1081] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Signalling peer 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:01:44.702549  9475 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:44.708081  8323 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
 from {username='slave'} at 127.0.0.1:57186
I20260709 13:01:44.708750  8323 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:44.709098  8323 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:44.709514  8325 raft_consensus.cc:1275] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:44.716436  8323 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:44.720019  9415 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:44.724521  8152 catalog_manager.cc:5697] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:44.744693  8251 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:55262
I20260709 13:01:44.745271  8251 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:44.745647  8251 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:44.747536  8314 tablet_service.cc:1558] Processing DeleteTablet for tablet 007e904f46ef4a1a978f97d557d2a65b with delete_type TABLET_DATA_TOMBSTONED (TS ff7a6fe533e94c1bae555b4cae427728 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57170
I20260709 13:01:44.748482  8256 raft_consensus.cc:2955] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:44.754026  8323 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:44.755255  9415 raft_consensus.cc:2955] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:44.758627  8404 raft_consensus.cc:2955] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:44.762178  8397 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:44.760773  8152 catalog_manager.cc:5697] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:44.765717  8141 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 007e904f46ef4a1a978f97d557d2a65b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:44.775226  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:44.775878  8397 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:44.783341  8397 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:44.768520  8251 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:44.786726  9481 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:44.792040  8251 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:44.793025  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:44.793615  8397 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:44.795487  9508 tablet_replica.cc:333] T 007e904f46ef4a1a978f97d557d2a65b P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:01:44.800208  9508 raft_consensus.cc:2243] T 007e904f46ef4a1a978f97d557d2a65b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:44.801007  9508 raft_consensus.cc:2272] T 007e904f46ef4a1a978f97d557d2a65b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:44.796775  8297 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:44.806747  9477 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:44.801975  8139 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 954d2ad09dcb4697adccd661625a841b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:44.807181  9477 raft_consensus.cc:697] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:44.808063  9477 consensus_queue.cc:237] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:44.803960  8397 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:01:44.815099  8404 raft_consensus.cc:2955] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:44.817342  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:44.817950  8332 raft_consensus.cc:3055] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:44.818279  8332 raft_consensus.cc:740] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:44.819021  8332 consensus_queue.cc:260] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:44.820206  8332 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:44.823989  8222 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:44.824791  9484 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:44.825217  9484 raft_consensus.cc:697] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:44.826086  9353 raft_consensus.cc:993] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b: : Instructing follower f865c4d761f5446eb12b3a029234772e to start an election
I20260709 13:01:44.826563  9353 raft_consensus.cc:1081] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Signalling peer f865c4d761f5446eb12b3a029234772e to start an election
I20260709 13:01:44.826035  9484 consensus_queue.cc:237] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:44.829772  8325 raft_consensus.cc:2955] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:44.835584  8153 catalog_manager.cc:5697] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:44.840377  9006 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a"
dest_uuid: "f865c4d761f5446eb12b3a029234772e"
 from {username='slave'} at 127.0.0.1:56940
I20260709 13:01:44.841063  9006 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:44.841451  9006 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:44.847888  8245 tablet_service.cc:1558] Processing DeleteTablet for tablet 954d2ad09dcb4697adccd661625a841b with delete_type TABLET_DATA_TOMBSTONED (TS 8c42259d7c4b463a9f485f42c320c81b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55248
I20260709 13:01:44.848418  9006 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:44.852810  9006 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:44.858254  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:44.886946  8249 raft_consensus.cc:3055] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:44.887302  8249 raft_consensus.cc:740] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:44.888060  8249 consensus_queue.cc:260] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:44.889153  8249 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:44.896377  8249 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:44.875123  9430 raft_consensus.cc:993] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f: : Instructing follower 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:01:44.897770  9401 raft_consensus.cc:1081] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Signalling peer 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:01:44.853924  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:44.899729  8404 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:44.876549  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:44.876744  8397 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:44.901731  8256 raft_consensus.cc:3055] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:44.878618  9484 raft_consensus.cc:993] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b: : Instructing follower f865c4d761f5446eb12b3a029234772e to start an election
I20260709 13:01:44.902064  8256 raft_consensus.cc:740] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:44.880046  8153 catalog_manager.cc:5697] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 1 to 2, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 2 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:44.866900  9007 raft_consensus.cc:2955] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:44.903067  9484 raft_consensus.cc:1081] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Signalling peer f865c4d761f5446eb12b3a029234772e to start an election
I20260709 13:01:44.902940  8256 consensus_queue.cc:260] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:44.904071  8256 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:44.909219  9511 tablet_replica.cc:333] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:01:44.915908  9006 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527"
dest_uuid: "f865c4d761f5446eb12b3a029234772e"
 from {username='slave'} at 127.0.0.1:56940
I20260709 13:01:44.916311  9415 raft_consensus.cc:993] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b: : Instructing follower ff7a6fe533e94c1bae555b4cae427728 to start an election
I20260709 13:01:44.916615  9006 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:44.917048  9006 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:44.917923  9508 ts_tablet_manager.cc:1916] T 007e904f46ef4a1a978f97d557d2a65b P ff7a6fe533e94c1bae555b4cae427728: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:44.916811  9415 raft_consensus.cc:1081] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Signalling peer ff7a6fe533e94c1bae555b4cae427728 to start an election
I20260709 13:01:44.923651  8256 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:01:44.924813  8332 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:01:44.928815  9511 raft_consensus.cc:2243] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:44.941790  9511 raft_consensus.cc:2272] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:44.941119  8153 catalog_manager.cc:5697] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 1 to 2, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:44.928483  9006 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:44.930824  8972 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:01:44.948423  9465 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:44.932210  8324 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b7c15354a2a84f868942919f011af448"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
 from {username='slave'} at 127.0.0.1:57186
I20260709 13:01:44.936383  8249 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:55264
I20260709 13:01:44.940930  8404 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:01:44.949924  8324 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:44.961736  8249 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:44.961859  8324 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:44.962172  8249 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:44.970359  8324 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:44.987581  9006 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:44.991852  8324 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:44.984906  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:44.993268  8404 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:44.997867  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:44.998471  8405 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:44.998693  8404 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:01:44.973342  9511 ts_tablet_manager.cc:1916] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:45.004863  8974 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:01:45.006956  9513 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:45.010241  9513 raft_consensus.cc:697] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:01:45.007416  8405 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:45.012912  8297 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:45.013949  9409 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:45.014443  9409 raft_consensus.cc:697] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:45.012418  9513 consensus_queue.cc:237] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:45.015775  9511 ts_tablet_manager.cc:1929] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:45.016216  9511 log.cc:1199] T 954d2ad09dcb4697adccd661625a841b P 8c42259d7c4b463a9f485f42c320c81b: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-0-root/wals/954d2ad09dcb4697adccd661625a841b
I20260709 13:01:45.015331  9409 consensus_queue.cc:237] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:44.975839  8249 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:45.019511  8249 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:45.021538  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:45.021819  9484 raft_consensus.cc:993] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b: : Instructing follower ff7a6fe533e94c1bae555b4cae427728 to start an election
I20260709 13:01:45.021670  8139 catalog_manager.cc:5028] TS 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): tablet 954d2ad09dcb4697adccd661625a841b (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:45.023324  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:45.034817  9484 raft_consensus.cc:1081] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Signalling peer ff7a6fe533e94c1bae555b4cae427728 to start an election
I20260709 13:01:45.035132  8255 raft_consensus.cc:3055] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:45.035503  8255 raft_consensus.cc:740] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:45.036316  8255 consensus_queue.cc:260] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:45.037837  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:45.024822  8254 consensus_queue.cc:237] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:45.038420  8324 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:45.026346  9465 raft_consensus.cc:697] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:01:45.034664  8256 raft_consensus.cc:3055] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:45.040953  8256 raft_consensus.cc:740] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:45.041270  9465 consensus_queue.cc:237] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:45.041774  8256 consensus_queue.cc:260] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:45.043033  8256 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:45.043219  8403 raft_consensus.cc:1275] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:45.022173  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:45.043867  8405 raft_consensus.cc:3055] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:45.044164  8405 raft_consensus.cc:740] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:45.044579  9484 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:45.044871  8405 consensus_queue.cc:260] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:45.038334  8323 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
 from {username='slave'} at 127.0.0.1:57186
I20260709 13:01:45.038007  8255 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:45.046219  8323 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:45.046617  8323 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:45.049862  9006 raft_consensus.cc:1275] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:45.054194  9415 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:45.060456  8256 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:01:45.063084  9415 raft_consensus.cc:2955] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:45.066880  8324 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:01:45.064091  8255 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:45.068887  8403 raft_consensus.cc:2955] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:45.072258  8405 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:45.075392  8220 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:45.076254  9484 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:45.056892  8323 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:45.080281  8323 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:45.030498  9508 ts_tablet_manager.cc:1929] T 007e904f46ef4a1a978f97d557d2a65b P ff7a6fe533e94c1bae555b4cae427728: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:45.082785  9508 log.cc:1199] T 007e904f46ef4a1a978f97d557d2a65b P ff7a6fe533e94c1bae555b4cae427728: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-1-root/wals/007e904f46ef4a1a978f97d557d2a65b
I20260709 13:01:45.081890  8405 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:01:45.086548  8139 catalog_manager.cc:5028] TS ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): tablet 007e904f46ef4a1a978f97d557d2a65b (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:45.077122  9484 raft_consensus.cc:697] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:45.092697  9484 consensus_queue.cc:237] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:45.092438  8152 catalog_manager.cc:5697] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 1 to 2, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 2 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:45.095907  8153 catalog_manager.cc:5697] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:45.114393  8154 catalog_manager.cc:5697] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 1 to 2, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:45.115340  8151 catalog_manager.cc:5697] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 1 to 2, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 2 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:45.128273  8314 tablet_service.cc:1558] Processing DeleteTablet for tablet 60c80a6b33dc431eac0fe35b37d4ea23 with delete_type TABLET_DATA_TOMBSTONED (TS ff7a6fe533e94c1bae555b4cae427728 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57170
I20260709 13:01:45.131070  9508 tablet_replica.cc:333] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:01:45.131811  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:45.132051  9508 raft_consensus.cc:2243] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:45.132437  8405 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:45.132786  9508 raft_consensus.cc:2272] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:45.138892  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:45.139526  8255 raft_consensus.cc:3055] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:45.139827  8255 raft_consensus.cc:740] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:45.140271  8139 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 60c80a6b33dc431eac0fe35b37d4ea23 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:45.140499  8255 consensus_queue.cc:260] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:45.148919  8255 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:45.150939  8153 catalog_manager.cc:5697] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 1 to 2, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 2 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:45.154959  8405 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:45.155779  9508 ts_tablet_manager.cc:1916] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:45.156831  9006 raft_consensus.cc:2955] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:45.162884  8297 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:45.164264  8255 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:45.165693  9481 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:45.166785  9481 raft_consensus.cc:697] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:45.167625  9481 consensus_queue.cc:237] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:45.195500  8151 catalog_manager.cc:5697] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 1 to 2, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:45.227327  8405 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:45.229578  9481 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:45.234292  9508 ts_tablet_manager.cc:1929] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:45.234817  9508 log.cc:1199] T 60c80a6b33dc431eac0fe35b37d4ea23 P ff7a6fe533e94c1bae555b4cae427728: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-1-root/wals/60c80a6b33dc431eac0fe35b37d4ea23
I20260709 13:01:45.236304  8139 catalog_manager.cc:5028] TS ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): tablet 60c80a6b33dc431eac0fe35b37d4ea23 (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:45.269405  8256 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:45.277926  9477 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:45.322224  8323 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:45.323770  9445 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:45.337944  8404 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:45.340215  9485 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:45.517241  8324 raft_consensus.cc:1275] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:45.518850  9415 consensus_queue.cc:1048] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:45.521663  8248 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:45.523025  9465 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:45.551807  8405 raft_consensus.cc:1275] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:45.555387  9006 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Advancing to term 2
I20260709 13:01:45.562558  9006 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 2 LEARNER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:45.563949  9440 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:45.572937  9484 consensus_queue.cc:1048] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:45.578102  8255 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:45.585496  9481 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:45.600239  8404 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:45.608932  8254 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:45.612931  9465 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:45.616875  9440 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:45.618603  9440 raft_consensus.cc:1064] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
W20260709 13:01:45.618957  9440 raft_consensus.cc:1068] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: Unable to promote non-voter f865c4d761f5446eb12b3a029234772e: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:01:45.614149  8404 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:45.635542  9477 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:45.655728  8403 raft_consensus.cc:1275] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:45.658087  9532 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:01:45.672015  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:45.688798  8397 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:45.693413  9409 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:45.697397  8255 raft_consensus.cc:1275] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:01:45.757506  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 70370.6us
W20260709 13:01:45.759217  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 57807.9us
I20260709 13:01:45.763692  9531 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:45.777063  9409 raft_consensus.cc:1064] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:45.785058  9409 consensus_queue.cc:237] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:45.816309  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:45.834510  8256 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:45.837035  9440 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:45.838268  8404 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:45.844544  9440 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:01:45.848641  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 60653.2us
I20260709 13:01:45.855463  9007 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 2 LEARNER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:45.858417  9440 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:45.866948  8404 raft_consensus.cc:2955] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:45.865875  9542 raft_consensus.cc:2955] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:45.883085  9007 raft_consensus.cc:2955] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:45.907888  8254 raft_consensus.cc:2955] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:45.911265  8153 catalog_manager.cc:5697] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index 2 to 4, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:45.964516  8323 consensus_queue.cc:237] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:45.970929  8254 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:45.970978  9007 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:45.972622  9544 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:45.974663  9409 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:45.982596  9542 raft_consensus.cc:2955] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:45.985342  8404 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "f865c4d761f5446eb12b3a029234772e"
 from {username='slave'} at 127.0.0.1:60896
I20260709 13:01:45.985942  8404 raft_consensus.cc:606] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Received request to transfer leadership to TS f865c4d761f5446eb12b3a029234772e
I20260709 13:01:45.990329  8256 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "f865c4d761f5446eb12b3a029234772e"
 from {username='slave'} at 127.0.0.1:50420
I20260709 13:01:45.990840  8256 raft_consensus.cc:606] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Received request to transfer leadership to TS f865c4d761f5446eb12b3a029234772e
I20260709 13:01:45.992974  9007 raft_consensus.cc:2955] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:45.994521  8256 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "f865c4d761f5446eb12b3a029234772e"
 from {username='slave'} at 127.0.0.1:50420
I20260709 13:01:45.995501  8256 raft_consensus.cc:606] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Received request to transfer leadership to TS f865c4d761f5446eb12b3a029234772e
I20260709 13:01:45.999444  8323 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:01:45.998910  8256 raft_consensus.cc:2955] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:46.000070  8323 raft_consensus.cc:606] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:46.003087  8254 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "f865c4d761f5446eb12b3a029234772e"
 from {username='slave'} at 127.0.0.1:50420
I20260709 13:01:46.003597  8254 raft_consensus.cc:606] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Received request to transfer leadership to TS f865c4d761f5446eb12b3a029234772e
I20260709 13:01:46.005937  8254 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "f865c4d761f5446eb12b3a029234772e"
 from {username='slave'} at 127.0.0.1:50420
I20260709 13:01:46.006440  8254 raft_consensus.cc:606] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Received request to transfer leadership to TS f865c4d761f5446eb12b3a029234772e
I20260709 13:01:46.008251  8139 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b7c15354a2a84f868942919f011af448 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:46.010418  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:01:46.011901  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:46.015872  8152 catalog_manager.cc:5697] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 4 to 5, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New cstate: current_term: 2 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:46.034345  8393 tablet_service.cc:1558] Processing DeleteTablet for tablet b7c15354a2a84f868942919f011af448 with delete_type TABLET_DATA_TOMBSTONED (TS e64761f43e4240319c41117cf886d22f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:39294
I20260709 13:01:46.036650  9548 tablet_replica.cc:333] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:01:46.037781  9548 raft_consensus.cc:2243] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:46.038568  9548 raft_consensus.cc:2272] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:46.041848  9548 ts_tablet_manager.cc:1916] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:46.056155  9548 ts_tablet_manager.cc:1929] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:01:46.056588  9548 log.cc:1199] T b7c15354a2a84f868942919f011af448 P e64761f43e4240319c41117cf886d22f: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-2-root/wals/b7c15354a2a84f868942919f011af448
I20260709 13:01:46.058053  9409 raft_consensus.cc:993] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728: : Instructing follower 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:01:46.058431  8141 catalog_manager.cc:5028] TS e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): tablet b7c15354a2a84f868942919f011af448 (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:46.058684  9542 raft_consensus.cc:1081] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Signalling peer 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:01:46.061358  8256 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:55262
I20260709 13:01:46.061928  8256 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:46.062287  8256 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:46.067878  8256 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:46.069411  8256 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:46.070230  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:46.070781  8323 raft_consensus.cc:3055] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:46.070595  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:46.071079  8323 raft_consensus.cc:740] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:46.071290  8404 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:46.071859  8323 consensus_queue.cc:260] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:46.072747  8323 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:46.077960  8404 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:01:46.079314  8222 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:46.079433  8323 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:01:46.080153  9485 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:46.080663  9485 raft_consensus.cc:697] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:46.081560  9485 consensus_queue.cc:237] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:46.091764  8151 catalog_manager.cc:5697] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 1 to 2, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 2 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:46.106027  9415 raft_consensus.cc:993] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b: : Instructing follower f865c4d761f5446eb12b3a029234772e to start an election
I20260709 13:01:46.106532  9484 raft_consensus.cc:1081] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Signalling peer f865c4d761f5446eb12b3a029234772e to start an election
I20260709 13:01:46.107893  9006 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67"
dest_uuid: "f865c4d761f5446eb12b3a029234772e"
 from {username='slave'} at 127.0.0.1:56940
I20260709 13:01:46.108371  9006 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:46.108633  9006 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:46.112854  9006 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:46.114293  9006 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:46.115101  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:46.115270  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:46.115680  8248 raft_consensus.cc:3055] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:46.115703  8404 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:46.116130  8248 raft_consensus.cc:740] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:46.116874  8248 consensus_queue.cc:260] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:46.118029  8248 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:46.122207  8404 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:01:46.123199  8974 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:01:46.123893  9532 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:46.124361  9532 raft_consensus.cc:697] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:01:46.124908  8248 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:01:46.125180  9532 consensus_queue.cc:237] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:46.132472  8151 catalog_manager.cc:5697] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 1 to 2, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 2 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:46.287770  9484 raft_consensus.cc:993] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b: : Instructing follower f865c4d761f5446eb12b3a029234772e to start an election
I20260709 13:01:46.288285  9485 raft_consensus.cc:1081] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Signalling peer f865c4d761f5446eb12b3a029234772e to start an election
I20260709 13:01:46.289841  9007 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596"
dest_uuid: "f865c4d761f5446eb12b3a029234772e"
 from {username='slave'} at 127.0.0.1:56940
I20260709 13:01:46.290422  9007 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:46.290794  9007 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:46.295486  9007 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:46.297160  9007 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:46.298308  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:46.298974  8248 raft_consensus.cc:3055] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:46.300274  8248 raft_consensus.cc:740] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:46.301012  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:46.301684  8397 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:46.301188  8248 consensus_queue.cc:260] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:46.302771  8248 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:46.308187  8397 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:01:46.309299  8974 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:01:46.309454  9485 raft_consensus.cc:993] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b: : Instructing follower f865c4d761f5446eb12b3a029234772e to start an election
I20260709 13:01:46.310052  9484 raft_consensus.cc:1081] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Signalling peer f865c4d761f5446eb12b3a029234772e to start an election
I20260709 13:01:46.310199  9532 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:46.310701  9532 raft_consensus.cc:697] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:01:46.311619  8248 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:01:46.311515  9532 consensus_queue.cc:237] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:46.311805  9007 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16"
dest_uuid: "f865c4d761f5446eb12b3a029234772e"
 from {username='slave'} at 127.0.0.1:56940
I20260709 13:01:46.312458  9007 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:46.312819  9007 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:46.320928  9007 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:46.321200  8151 catalog_manager.cc:5697] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 1 to 2, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 2 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:46.324994  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:46.326277  8248 raft_consensus.cc:3055] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:46.325573  9007 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:46.326622  8248 raft_consensus.cc:740] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:46.328436  8248 consensus_queue.cc:260] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:46.329502  8248 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:46.332273  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:46.333129  8324 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:46.340936  8248 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:01:46.342156  8972 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:01:46.342993  9532 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:46.343482  9532 raft_consensus.cc:697] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:01:46.344400  9532 consensus_queue.cc:237] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:46.345749  8324 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:01:46.357345  8151 catalog_manager.cc:5697] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 1 to 2, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 2 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:46.383356  9401 raft_consensus.cc:993] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f: : Instructing follower f865c4d761f5446eb12b3a029234772e to start an election
I20260709 13:01:46.383834  9430 raft_consensus.cc:1081] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Signalling peer f865c4d761f5446eb12b3a029234772e to start an election
I20260709 13:01:46.385515  9006 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe"
dest_uuid: "f865c4d761f5446eb12b3a029234772e"
 from {username='slave'} at 127.0.0.1:56948
I20260709 13:01:46.386056  9006 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:46.386363  9006 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:46.392745  9006 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:46.394850  9006 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:46.394975  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:46.395619  8323 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:46.395886  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:46.396409  8404 raft_consensus.cc:3055] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:46.396693  8404 raft_consensus.cc:740] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:46.397495  8404 consensus_queue.cc:260] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:46.398314  8404 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:46.400115  8323 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:01:46.401153  8972 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:46.401870  9532 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:46.402302  9532 raft_consensus.cc:697] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:01:46.402781  8404 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:01:46.403121  9532 consensus_queue.cc:237] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:46.412369  8151 catalog_manager.cc:5697] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 1 to 2, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 2 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:46.423220  9484 raft_consensus.cc:993] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b: : Instructing follower f865c4d761f5446eb12b3a029234772e to start an election
I20260709 13:01:46.423691  9484 raft_consensus.cc:1081] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Signalling peer f865c4d761f5446eb12b3a029234772e to start an election
I20260709 13:01:46.424928  9006 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf"
dest_uuid: "f865c4d761f5446eb12b3a029234772e"
 from {username='slave'} at 127.0.0.1:56940
I20260709 13:01:46.425412  9006 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:46.425698  9006 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:46.431705  9006 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:46.433562  9006 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:46.434126  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:46.434676  8248 raft_consensus.cc:3055] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:46.434708  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:46.435084  8248 raft_consensus.cc:740] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:46.435356  8323 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:46.435830  8248 consensus_queue.cc:260] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:46.436719  8248 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:46.440099  8323 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:01:46.441134  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:46.442056  9528 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:46.442134  8248 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:01:46.442576  9528 raft_consensus.cc:697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:01:46.443396  9528 consensus_queue.cc:237] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:46.450742  8151 catalog_manager.cc:5697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 1 to 2, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 2 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:46.542142  8404 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:46.543273  9528 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:46.556485  8248 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:46.558552  9528 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:46.627820  8404 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:46.628865  9415 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:46.636869  8323 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:46.638576  9415 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:46.725513  8397 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:46.726642  9552 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:46.738422  8248 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:46.740751  9532 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:46.743860  8324 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:46.745283  9532 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:46.772468  8254 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:46.776832  9532 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:46.855568  8397 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:46.856936  9532 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:46.868162  8324 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:46.869668  9532 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:46.885764  8254 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:46.888798  8324 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:46.891295  9532 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:46.892004  9552 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:47.047021  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:47.191174  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:47.220377  8397 consensus_queue.cc:237] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:47.227842  8255 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:01:47.228299  8372 consensus_peers.cc:597] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e94bcf9755aa4f0abddcd5b9fbc2939f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:47.229403  8324 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:47.230055  9475 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:47.231112  9401 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:01:47.246628  9430 raft_consensus.cc:2955] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:47.248497  8324 raft_consensus.cc:2955] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:47.254109  8255 raft_consensus.cc:2955] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:47.256994  8397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:60896
I20260709 13:01:47.257916  8397 raft_consensus.cc:606] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:47.261967  8254 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "f865c4d761f5446eb12b3a029234772e"
 from {username='slave'} at 127.0.0.1:50420
I20260709 13:01:47.262590  8254 raft_consensus.cc:606] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Received request to transfer leadership to TS f865c4d761f5446eb12b3a029234772e
I20260709 13:01:47.267284  8397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "766ac43473084aa6a9fdce530520947c"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "f865c4d761f5446eb12b3a029234772e"
 from {username='slave'} at 127.0.0.1:60896
I20260709 13:01:47.267941  8397 raft_consensus.cc:606] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Received request to transfer leadership to TS f865c4d761f5446eb12b3a029234772e
I20260709 13:01:47.275300  8324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7c15354a2a84f868942919f011af448"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:01:47.279102  8152 catalog_manager.cc:5697] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: config changed from index -1 to 3, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 3 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:47.281525  8324 raft_consensus.cc:606] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:47.288498  8397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:60896
I20260709 13:01:47.289402  8397 raft_consensus.cc:606] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:47.291057  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:01:47.292865  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:47.302479  8324 consensus_queue.cc:237] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:47.329162  8405 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:47.331899  9430 raft_consensus.cc:993] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f: : Instructing follower 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:01:47.332383  9475 raft_consensus.cc:1081] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Signalling peer 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:01:47.334308  8254 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:55264
I20260709 13:01:47.334610  8255 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:47.334915  8254 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:47.335256  8254 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:47.335870  9477 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:47.342945  9542 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:01:47.349026  8298 consensus_peers.cc:597] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 56b5f14332d64504837ae4e2df77e409. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:47.349023  8254 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:47.373983  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:47.375767  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:47.376737  8397 raft_consensus.cc:3055] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:47.377023  8397 raft_consensus.cc:740] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:47.378100  8397 consensus_queue.cc:260] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:47.379544  8397 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:47.376212  8254 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:47.386283  8324 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:47.423537  9575 raft_consensus.cc:993] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b: : Instructing follower f865c4d761f5446eb12b3a029234772e to start an election
I20260709 13:01:47.424281  9575 raft_consensus.cc:1081] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Signalling peer f865c4d761f5446eb12b3a029234772e to start an election
I20260709 13:01:47.430665  9006 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23"
dest_uuid: "f865c4d761f5446eb12b3a029234772e"
 from {username='slave'} at 127.0.0.1:56940
I20260709 13:01:47.431209  9006 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:47.431514  9006 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:47.432338  8397 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:01:47.433820  8222 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:47.435382  9415 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:47.435774  9415 raft_consensus.cc:697] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:47.436460  9415 consensus_queue.cc:237] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:47.447480  9409 raft_consensus.cc:2955] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:47.458355  9475 raft_consensus.cc:993] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f: : Instructing follower 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:01:47.458079  9006 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:47.458806  9475 raft_consensus.cc:1081] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Signalling peer 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:01:47.456811  8249 raft_consensus.cc:2955] T 56b5f14332d64504837ae4e2df77e409 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:47.461828  8251 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:55264
I20260709 13:01:47.462450  8251 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:47.462893  8251 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:47.463647  8324 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:01:47.466220  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:47.466864  8254 raft_consensus.cc:3055] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:47.467175  8254 raft_consensus.cc:740] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:47.468021  8254 consensus_queue.cc:260] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:47.469122  8254 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:47.470098  9006 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:47.471120  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:47.471665  8397 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:47.477240  8397 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:01:47.481207  8254 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:01:47.481554  8974 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:01:47.482787  9552 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:47.483270  9552 raft_consensus.cc:697] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:01:47.484280  9552 consensus_queue.cc:237] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:47.489719  8405 raft_consensus.cc:2955] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:47.492671  8151 catalog_manager.cc:5697] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 1 to 2, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 2 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:47.494511  8324 consensus_queue.cc:237] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:47.497711  8152 catalog_manager.cc:5697] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:47.515308  8251 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:47.517751  8251 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:47.519106  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:47.520076  8397 raft_consensus.cc:3055] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:47.520401  8397 raft_consensus.cc:740] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:47.521332  8397 consensus_queue.cc:260] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:47.522306  8405 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:47.522477  8397 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:47.523778  9477 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:01:47.525487  8298 consensus_peers.cc:597] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 09943d7e76a54d4e80eb181cac80bf22. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:47.526515  8153 catalog_manager.cc:5697] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 1 to 2, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 2 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.529458  8397 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:01:47.530757  8222 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:47.531548  9415 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:47.532018  9415 raft_consensus.cc:697] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:47.532927  9415 consensus_queue.cc:237] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:47.535693  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:01:47.536439  9006 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:47.544786  9542 raft_consensus.cc:2955] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:47.546247  8251 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:47.548053  9409 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:47.556969  9006 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:01:47.562974  8404 raft_consensus.cc:2955] T 09943d7e76a54d4e80eb181cac80bf22 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:47.578090  8151 catalog_manager.cc:5697] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 1 to 2, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 2 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:47.591212  8397 consensus_queue.cc:237] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:47.597735  8153 catalog_manager.cc:5697] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:47.606099  8251 raft_consensus.cc:2955] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:47.617486  8332 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:47.619055  9475 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:01:47.629668  8372 consensus_peers.cc:597] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1112bcf4c4ea44e1933ee6287bc869f8. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:47.631999  8251 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:47.634359  9475 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:47.637979  9475 raft_consensus.cc:2955] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:47.647531  8332 raft_consensus.cc:2955] T 1112bcf4c4ea44e1933ee6287bc869f8 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:47.657460  8153 catalog_manager.cc:5697] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:47.659008  8251 raft_consensus.cc:2955] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:47.664778  9409 raft_consensus.cc:993] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: : Instructing follower 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:01:47.665319  9409 raft_consensus.cc:1081] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Signalling peer 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:01:47.667539  8249 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b7c15354a2a84f868942919f011af448"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:55262
I20260709 13:01:47.668988  8249 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:47.669420  8249 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:47.683025  8249 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:47.685552  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:47.686947  8324 raft_consensus.cc:3055] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:47.686887  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:01:47.687392  8324 raft_consensus.cc:740] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:47.687574  9006 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:47.688097  8324 consensus_queue.cc:260] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:47.688853  8324 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:47.686452  8249 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:47.693691  8324 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:01:47.694011  9006 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:01:47.694917  8220 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:47.695858  9484 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:47.696331  9484 raft_consensus.cc:697] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:47.697265  9484 consensus_queue.cc:237] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:47.718714  8153 catalog_manager.cc:5697] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 2 to 3, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 3 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:47.774834  9430 raft_consensus.cc:993] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f: : Instructing follower f865c4d761f5446eb12b3a029234772e to start an election
I20260709 13:01:47.775318  9430 raft_consensus.cc:1081] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Signalling peer f865c4d761f5446eb12b3a029234772e to start an election
I20260709 13:01:47.777037  9006 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "766ac43473084aa6a9fdce530520947c"
dest_uuid: "f865c4d761f5446eb12b3a029234772e"
 from {username='slave'} at 127.0.0.1:56948
I20260709 13:01:47.777624  9006 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:47.777980  9006 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:47.784730  9006 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:47.786525  9006 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:47.787449  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:47.787469  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:47.788112  8397 raft_consensus.cc:3055] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:47.788188  8332 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:47.788447  8397 raft_consensus.cc:740] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:47.789247  8397 consensus_queue.cc:260] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:47.790258  8397 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:47.794723  8332 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:01:47.795432  8397 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:01:47.795986  8972 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:47.797869  9552 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:47.798347  9552 raft_consensus.cc:697] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:01:47.799211  9552 consensus_queue.cc:237] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:47.815402  9597 ts_tablet_manager.cc:933] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:47.817306  9597 tablet_copy_client.cc:323] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.3:37367
I20260709 13:01:47.819077  8416 tablet_copy_service.cc:140] P e64761f43e4240319c41117cf886d22f: Received BeginTabletCopySession request for tablet e94bcf9755aa4f0abddcd5b9fbc2939f from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:34462)
I20260709 13:01:47.818539  8153 catalog_manager.cc:5697] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 1 to 2, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 2 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.819577  8416 tablet_copy_service.cc:161] P e64761f43e4240319c41117cf886d22f: Beginning new tablet copy session on tablet e94bcf9755aa4f0abddcd5b9fbc2939f from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:34462: session id = f865c4d761f5446eb12b3a029234772e-e94bcf9755aa4f0abddcd5b9fbc2939f
I20260709 13:01:47.828882  8416 tablet_copy_source_session.cc:215] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:47.834005  9597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e94bcf9755aa4f0abddcd5b9fbc2939f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:47.859043  9600 ts_tablet_manager.cc:933] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:47.861259  9597 tablet_copy_client.cc:806] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:47.861969  9597 tablet_copy_client.cc:670] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:47.863502  9600 tablet_copy_client.cc:323] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.2:38599
I20260709 13:01:47.866202  9597 tablet_copy_client.cc:538] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:47.866483  8341 tablet_copy_service.cc:140] P ff7a6fe533e94c1bae555b4cae427728: Received BeginTabletCopySession request for tablet 56b5f14332d64504837ae4e2df77e409 from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:41664)
I20260709 13:01:47.867079  8341 tablet_copy_service.cc:161] P ff7a6fe533e94c1bae555b4cae427728: Beginning new tablet copy session on tablet 56b5f14332d64504837ae4e2df77e409 from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:41664: session id = f865c4d761f5446eb12b3a029234772e-56b5f14332d64504837ae4e2df77e409
I20260709 13:01:47.873857  8341 tablet_copy_source_session.cc:215] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:47.876001  9597 tablet_bootstrap.cc:492] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:47.877259  9600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56b5f14332d64504837ae4e2df77e409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:47.889845  9600 tablet_copy_client.cc:806] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:47.890508  9600 tablet_copy_client.cc:670] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:47.894766  9600 tablet_copy_client.cc:538] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:47.904367  9600 tablet_bootstrap.cc:492] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:47.910064  9597 tablet_bootstrap.cc:492] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:47.911163  9597 tablet_bootstrap.cc:492] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:47.911864  9597 ts_tablet_manager.cc:1403] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.036s	user 0.024s	sys 0.002s
I20260709 13:01:47.915127  9597 raft_consensus.cc:359] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:47.917382  8249 raft_consensus.cc:1275] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:47.918987  9569 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:47.915978  9597 raft_consensus.cc:740] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:47.926831  9597 consensus_queue.cc:260] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:47.950314  8397 raft_consensus.cc:1275] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:47.952677  9569 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:47.947577  9597 ts_tablet_manager.cc:1434] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.035s	user 0.009s	sys 0.000s
I20260709 13:01:47.981985  8416 tablet_copy_service.cc:342] P e64761f43e4240319c41117cf886d22f: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-e94bcf9755aa4f0abddcd5b9fbc2939f received from {username='slave'} at 127.0.0.1:34462
I20260709 13:01:47.982483  8416 tablet_copy_service.cc:434] P e64761f43e4240319c41117cf886d22f: ending tablet copy session f865c4d761f5446eb12b3a029234772e-e94bcf9755aa4f0abddcd5b9fbc2939f on tablet e94bcf9755aa4f0abddcd5b9fbc2939f with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:47.995653  9597 ts_tablet_manager.cc:933] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:48.013582  8404 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:48.015345  9415 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:48.020998  9597 tablet_copy_client.cc:323] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.2:38599
I20260709 13:01:48.024808  8341 tablet_copy_service.cc:140] P ff7a6fe533e94c1bae555b4cae427728: Received BeginTabletCopySession request for tablet 09943d7e76a54d4e80eb181cac80bf22 from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:41664)
I20260709 13:01:48.025411  8341 tablet_copy_service.cc:161] P ff7a6fe533e94c1bae555b4cae427728: Beginning new tablet copy session on tablet 09943d7e76a54d4e80eb181cac80bf22 from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:41664: session id = f865c4d761f5446eb12b3a029234772e-09943d7e76a54d4e80eb181cac80bf22
I20260709 13:01:48.033970  8341 tablet_copy_source_session.cc:215] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:48.035871  8332 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:48.045583  9597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09943d7e76a54d4e80eb181cac80bf22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:48.090240  9008 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:48.093204  9597 tablet_copy_client.cc:806] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:48.094000  9597 tablet_copy_client.cc:670] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:48.094705  8403 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:48.096289  9484 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:48.099675  9597 tablet_copy_client.cc:538] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:48.104781  9415 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:48.107883  9484 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:48.119802  9597 tablet_bootstrap.cc:492] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:48.148712  9605 ts_tablet_manager.cc:933] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:01:48.164783  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 47004.3us
I20260709 13:01:48.191012  9605 tablet_copy_client.cc:323] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.3:37367
I20260709 13:01:48.197950  8416 tablet_copy_service.cc:140] P e64761f43e4240319c41117cf886d22f: Received BeginTabletCopySession request for tablet 1112bcf4c4ea44e1933ee6287bc869f8 from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:34462)
I20260709 13:01:48.198551  8416 tablet_copy_service.cc:161] P e64761f43e4240319c41117cf886d22f: Beginning new tablet copy session on tablet 1112bcf4c4ea44e1933ee6287bc869f8 from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:34462: session id = f865c4d761f5446eb12b3a029234772e-1112bcf4c4ea44e1933ee6287bc869f8
I20260709 13:01:48.206219  8416 tablet_copy_source_session.cc:215] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:01:48.213004  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 60866.3us
I20260709 13:01:48.215520  8404 raft_consensus.cc:1275] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:48.216883  9552 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:48.219435  9605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1112bcf4c4ea44e1933ee6287bc869f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:48.293963  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:48.299853  9600 tablet_bootstrap.cc:492] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:48.323158  9600 tablet_bootstrap.cc:492] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:48.323890  9600 ts_tablet_manager.cc:1403] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.420s	user 0.109s	sys 0.127s
I20260709 13:01:48.333923  9600 raft_consensus.cc:359] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:48.334791  9600 raft_consensus.cc:740] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:48.335433  9600 consensus_queue.cc:260] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:48.346834  9008 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:01:48.405401  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 100152us
W20260709 13:01:48.406155  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 101802us
W20260709 13:01:48.449826  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 107802us
I20260709 13:01:48.453900  9600 ts_tablet_manager.cc:1434] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.130s	user 0.007s	sys 0.001s
W20260709 13:01:48.457827  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 50814.6us
W20260709 13:01:48.458312  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 52149.1us
W20260709 13:01:48.494665  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 215154us
I20260709 13:01:48.495937  9007 raft_consensus.cc:1217] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:48.512018  8324 raft_consensus.cc:1275] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:01:48.519258  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 162600us
I20260709 13:01:48.537323  8324 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:01:48.548203  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 50271us
I20260709 13:01:48.529744  9616 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:01:48.575429  8341 tablet_copy_service.cc:342] P ff7a6fe533e94c1bae555b4cae427728: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-56b5f14332d64504837ae4e2df77e409 received from {username='slave'} at 127.0.0.1:41664
I20260709 13:01:48.597743  8341 tablet_copy_service.cc:434] P ff7a6fe533e94c1bae555b4cae427728: ending tablet copy session f865c4d761f5446eb12b3a029234772e-56b5f14332d64504837ae4e2df77e409 on tablet 56b5f14332d64504837ae4e2df77e409 with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:48.615361  9597 tablet_bootstrap.cc:492] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:48.616571  9597 tablet_bootstrap.cc:492] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:48.620920  9597 ts_tablet_manager.cc:1403] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.498s	user 0.028s	sys 0.102s
I20260709 13:01:48.654722  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:01:48.654706  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 98131.8us
I20260709 13:01:48.686522  9619 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:48.696259  9597 raft_consensus.cc:359] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:48.698176  9597 raft_consensus.cc:740] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:48.713994  9597 consensus_queue.cc:260] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
W20260709 13:01:48.730340  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 266047us
W20260709 13:01:48.733820  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 176375us
I20260709 13:01:48.742022  9605 tablet_copy_client.cc:806] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:48.764914  9605 tablet_copy_client.cc:670] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:48.772149  9007 raft_consensus.cc:1217] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
W20260709 13:01:48.800670  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 264557us
W20260709 13:01:48.804507  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 265036us
W20260709 13:01:48.819061  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 268854us
I20260709 13:01:48.819183  8329 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:01:48.838829  8329 raft_consensus.cc:606] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:48.717221  9597 ts_tablet_manager.cc:1434] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.063s	user 0.005s	sys 0.000s
I20260709 13:01:48.853168  9644 raft_consensus.cc:993] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728: : Instructing follower 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:01:48.857559  9640 raft_consensus.cc:1081] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Signalling peer 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:01:48.862360  9532 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:01:48.864086  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 60078.2us
I20260709 13:01:48.873592  9009 raft_consensus.cc:1217] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:48.878499  8256 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:55262
I20260709 13:01:48.879096  8256 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:48.879436  8256 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:48.881620  8341 tablet_copy_service.cc:342] P ff7a6fe533e94c1bae555b4cae427728: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-09943d7e76a54d4e80eb181cac80bf22 received from {username='slave'} at 127.0.0.1:41664
I20260709 13:01:48.882097  8341 tablet_copy_service.cc:434] P ff7a6fe533e94c1bae555b4cae427728: ending tablet copy session f865c4d761f5446eb12b3a029234772e-09943d7e76a54d4e80eb181cac80bf22 on tablet 09943d7e76a54d4e80eb181cac80bf22 with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:48.885636  8251 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:50420
I20260709 13:01:48.886261  8251 raft_consensus.cc:606] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:01:48.903816  8251 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:50420
I20260709 13:01:48.904472  8251 raft_consensus.cc:606] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:01:48.905730  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:01:48.907207  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:48.892051  8256 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:48.942099  9605 tablet_copy_client.cc:538] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:48.958040  9639 raft_consensus.cc:993] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b: : Instructing follower e64761f43e4240319c41117cf886d22f to start an election
I20260709 13:01:48.958631  9639 raft_consensus.cc:1081] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Signalling peer e64761f43e4240319c41117cf886d22f to start an election
I20260709 13:01:48.959306  9644 raft_consensus.cc:1064] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:48.964491  8256 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:48.967173  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:48.967974  8397 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:48.980046  8405 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:39308
I20260709 13:01:48.980607  8405 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:48.980912  8405 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:48.985857  9605 tablet_bootstrap.cc:492] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:48.988732  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:48.991652  8325 raft_consensus.cc:3055] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:48.992085  8325 raft_consensus.cc:740] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:48.998715  9644 consensus_queue.cc:237] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:49.001785  8397 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:01:49.002199  9575 raft_consensus.cc:993] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b: : Instructing follower e64761f43e4240319c41117cf886d22f to start an election
I20260709 13:01:49.002667  9575 raft_consensus.cc:1081] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Signalling peer e64761f43e4240319c41117cf886d22f to start an election
I20260709 13:01:49.006124  8222 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:49.008047  9639 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:49.008293  8402 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:39308
I20260709 13:01:49.008517  9639 raft_consensus.cc:697] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:49.009130  8402 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:49.009709  8402 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:49.009630  9639 consensus_queue.cc:237] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:49.000183  8325 consensus_queue.cc:260] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:49.015926  8325 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:49.016214  8405 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:49.019599  8405 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:49.022099  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:49.022727  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:49.022903  8256 raft_consensus.cc:3055] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:49.023283  8324 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:49.023386  8256 raft_consensus.cc:740] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:49.025197  8256 consensus_queue.cc:260] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:49.026852  8256 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:49.032794  8324 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:01:49.046679  8256 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:01:49.060686  8325 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:01:49.087365  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:49.088546  9595 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:49.089334  9595 raft_consensus.cc:697] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:49.090361  9595 consensus_queue.cc:237] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:49.100193  8402 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:49.119997  8151 catalog_manager.cc:5697] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 1 to 2, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 2 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:49.122292  8153 catalog_manager.cc:5697] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 2 to 3, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 3 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:49.131481  8402 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:49.132691  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:49.134305  8329 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:49.132035  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:49.138154  8256 raft_consensus.cc:3055] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:49.138496  8256 raft_consensus.cc:740] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:49.139456  8251 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:49.139330  8256 consensus_queue.cc:260] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:49.140691  8256 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:49.141888  8402 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:49.144546  9008 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:49.147078  9623 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:49.148160  9640 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:49.150003  9640 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:49.150449  9636 raft_consensus.cc:1064] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:49.151821  8329 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:01:49.153026  9636 consensus_queue.cc:237] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:49.154866  8256 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:01:49.164552  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:49.165578  9649 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:49.166090  9649 raft_consensus.cc:697] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:49.166896  9649 consensus_queue.cc:237] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:49.170930  9623 raft_consensus.cc:2955] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:49.172358  8402 raft_consensus.cc:2955] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:49.175740  9007 raft_consensus.cc:2955] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:49.185896  9605 tablet_bootstrap.cc:492] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:49.187048  9605 tablet_bootstrap.cc:492] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:49.187726  9605 ts_tablet_manager.cc:1403] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.202s	user 0.034s	sys 0.004s
I20260709 13:01:49.191478  9605 raft_consensus.cc:359] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:49.192327  9605 raft_consensus.cc:740] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:49.193235  9605 consensus_queue.cc:260] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:49.195032  9605 ts_tablet_manager.cc:1434] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:01:49.217195  8329 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:01:49.220268  9007 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 3 LEARNER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:01:49.222503  9637 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:49.224072  8416 tablet_copy_service.cc:342] P e64761f43e4240319c41117cf886d22f: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-1112bcf4c4ea44e1933ee6287bc869f8 received from {username='slave'} at 127.0.0.1:34462
I20260709 13:01:49.224174  8251 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:01:49.224603  8416 tablet_copy_service.cc:434] P e64761f43e4240319c41117cf886d22f: ending tablet copy session f865c4d761f5446eb12b3a029234772e-1112bcf4c4ea44e1933ee6287bc869f8 on tablet 1112bcf4c4ea44e1933ee6287bc869f8 with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:49.223855  9630 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:49.230957  9637 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:49.269726  8256 raft_consensus.cc:2955] T 56b5f14332d64504837ae4e2df77e409 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:49.276191  8152 catalog_manager.cc:5697] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:49.290982  9008 raft_consensus.cc:1217] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:49.305743  9595 raft_consensus.cc:2955] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:49.321451  8329 raft_consensus.cc:2955] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:49.335896  9008 raft_consensus.cc:2955] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:49.344101  8153 catalog_manager.cc:5697] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 2 to 3, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 3 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:49.347643  8248 raft_consensus.cc:2955] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:49.382491  8151 catalog_manager.cc:5697] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: config changed from index 3 to 4, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:49.372890  8324 consensus_queue.cc:237] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:49.393402  8402 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:49.395975  9623 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:49.397493  9008 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:49.402667  9623 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:49.406697  9640 raft_consensus.cc:2955] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:49.427268  8397 consensus_queue.cc:237] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:49.436980  8405 raft_consensus.cc:2955] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:49.440524  9008 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:49.441998  8248 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:49.442377  9595 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:49.443538  9637 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:49.456459  9007 raft_consensus.cc:2955] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:49.477468  9649 raft_consensus.cc:2955] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:49.479918  8154 catalog_manager.cc:5697] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 3 to 4, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New cstate: current_term: 1 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:49.483707  8402 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:49.483335  8248 raft_consensus.cc:2955] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:49.494751  9008 raft_consensus.cc:2955] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:49.506225  8139 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 56b5f14332d64504837ae4e2df77e409 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:49.510519  9575 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:49.502174  8329 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:49.516081  9595 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:49.545288  8245 tablet_service.cc:1558] Processing DeleteTablet for tablet 56b5f14332d64504837ae4e2df77e409 with delete_type TABLET_DATA_TOMBSTONED (TS 8c42259d7c4b463a9f485f42c320c81b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55248
I20260709 13:01:49.545212  8152 catalog_manager.cc:5697] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 4 to 5, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New cstate: current_term: 3 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:49.563999  8314 tablet_service.cc:1558] Processing DeleteTablet for tablet e94bcf9755aa4f0abddcd5b9fbc2939f with delete_type TABLET_DATA_TOMBSTONED (TS ff7a6fe533e94c1bae555b4cae427728 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:57170
I20260709 13:01:49.574548  8256 raft_consensus.cc:1275] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:49.575939  9595 consensus_queue.cc:1048] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:49.582145  8325 raft_consensus.cc:1275] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:49.584479  9595 consensus_queue.cc:1048] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:49.585973  9008 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Advancing to term 2
I20260709 13:01:49.587800  9678 tablet_replica.cc:333] T 56b5f14332d64504837ae4e2df77e409 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:01:49.589871  9678 raft_consensus.cc:2243] T 56b5f14332d64504837ae4e2df77e409 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:49.593834  9678 raft_consensus.cc:2272] T 56b5f14332d64504837ae4e2df77e409 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:49.606976  8332 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:49.605849  9678 ts_tablet_manager.cc:1916] T 56b5f14332d64504837ae4e2df77e409 P 8c42259d7c4b463a9f485f42c320c81b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:49.613500  8251 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:49.615646  9630 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:49.623524  8141 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e94bcf9755aa4f0abddcd5b9fbc2939f with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:49.646932  9678 ts_tablet_manager.cc:1929] T 56b5f14332d64504837ae4e2df77e409 P 8c42259d7c4b463a9f485f42c320c81b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:49.652704  9678 log.cc:1199] T 56b5f14332d64504837ae4e2df77e409 P 8c42259d7c4b463a9f485f42c320c81b: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-0-root/wals/56b5f14332d64504837ae4e2df77e409
I20260709 13:01:49.652930  9679 tablet_replica.cc:333] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:01:49.656432  9575 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:49.662016  9679 raft_consensus.cc:2243] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:49.662901  9679 raft_consensus.cc:2272] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:49.665925  9008 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 2 LEARNER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:49.666900  9679 ts_tablet_manager.cc:1916] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:49.683698  9575 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:49.696943  9637 raft_consensus.cc:1064] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:49.699437  9637 consensus_queue.cc:237] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:49.681864  8139 catalog_manager.cc:5028] TS 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): tablet 56b5f14332d64504837ae4e2df77e409 (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:49.859755  9679 ts_tablet_manager.cc:1929] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 13:01:49.860283  9679 log.cc:1199] T e94bcf9755aa4f0abddcd5b9fbc2939f P ff7a6fe533e94c1bae555b4cae427728: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-1-root/wals/e94bcf9755aa4f0abddcd5b9fbc2939f
I20260709 13:01:49.870283  8139 catalog_manager.cc:5028] TS ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): tablet e94bcf9755aa4f0abddcd5b9fbc2939f (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:49.871366  9006 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:49.872969  9649 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:49.881600  8248 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:49.896951  9649 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:49.908087  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:49.926558  8325 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:49.928751  9649 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:49.944994  9639 raft_consensus.cc:1064] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:49.951471  9639 consensus_queue.cc:237] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:49.968196  8324 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:49.969859  9007 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 2 LEARNER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:49.972530  9639 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:49.974838  8405 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:49.978032  9649 raft_consensus.cc:2955] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:49.986510  8329 raft_consensus.cc:2955] T 1112bcf4c4ea44e1933ee6287bc869f8 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:49.994959  9643 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:49.996831  8255 raft_consensus.cc:2955] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:49.999744  9643 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:50.005246  8151 catalog_manager.cc:5697] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:50.013535  9008 raft_consensus.cc:2955] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:50.022876  9691 raft_consensus.cc:2955] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:50.038188  8405 raft_consensus.cc:2955] T 09943d7e76a54d4e80eb181cac80bf22 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:50.044032  9008 raft_consensus.cc:2955] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:50.063760  8153 catalog_manager.cc:5697] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: config changed from index 2 to 4, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:50.075788  8329 raft_consensus.cc:2955] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:50.077610  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:50.084656  8405 consensus_queue.cc:237] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:50.090234  9008 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:50.090337  8255 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:50.091677  9628 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:50.093541  9649 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:50.099290  8255 consensus_queue.cc:237] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:50.101886  9649 raft_consensus.cc:2955] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:50.108417  8248 raft_consensus.cc:2955] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:50.124236  9008 raft_consensus.cc:2955] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:50.128559  8141 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1112bcf4c4ea44e1933ee6287bc869f8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:50.129948  9007 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:50.129946  8329 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:50.132225  9639 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:50.138463  9643 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:50.141458  9575 raft_consensus.cc:2955] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:50.142743  8151 catalog_manager.cc:5697] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 3 to 4, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New cstate: current_term: 1 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:50.142953  8324 raft_consensus.cc:2955] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:50.156206  8314 tablet_service.cc:1558] Processing DeleteTablet for tablet 1112bcf4c4ea44e1933ee6287bc869f8 with delete_type TABLET_DATA_TOMBSTONED (TS ff7a6fe533e94c1bae555b4cae427728 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57170
I20260709 13:01:50.157159  9679 tablet_replica.cc:333] T 1112bcf4c4ea44e1933ee6287bc869f8 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:01:50.157922  8139 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 09943d7e76a54d4e80eb181cac80bf22 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:50.158989  9008 raft_consensus.cc:2955] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:50.168983  9679 raft_consensus.cc:2243] T 1112bcf4c4ea44e1933ee6287bc869f8 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:50.169864  9679 raft_consensus.cc:2272] T 1112bcf4c4ea44e1933ee6287bc869f8 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:50.169068  8152 catalog_manager.cc:5697] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: config changed from index 4 to 5, VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3) evicted. New cstate: current_term: 2 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:50.173450  9679 ts_tablet_manager.cc:1916] T 1112bcf4c4ea44e1933ee6287bc869f8 P ff7a6fe533e94c1bae555b4cae427728: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:50.232314  8393 tablet_service.cc:1558] Processing DeleteTablet for tablet 09943d7e76a54d4e80eb181cac80bf22 with delete_type TABLET_DATA_TOMBSTONED (TS e64761f43e4240319c41117cf886d22f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:39294
I20260709 13:01:50.235836  9695 tablet_replica.cc:333] T 09943d7e76a54d4e80eb181cac80bf22 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:01:50.236971  9695 raft_consensus.cc:2243] T 09943d7e76a54d4e80eb181cac80bf22 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:50.237776  9695 raft_consensus.cc:2272] T 09943d7e76a54d4e80eb181cac80bf22 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:50.241540  9695 ts_tablet_manager.cc:1916] T 09943d7e76a54d4e80eb181cac80bf22 P e64761f43e4240319c41117cf886d22f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:50.248266  9679 ts_tablet_manager.cc:1929] T 1112bcf4c4ea44e1933ee6287bc869f8 P ff7a6fe533e94c1bae555b4cae427728: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:50.248796  9679 log.cc:1199] T 1112bcf4c4ea44e1933ee6287bc869f8 P ff7a6fe533e94c1bae555b4cae427728: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-1-root/wals/1112bcf4c4ea44e1933ee6287bc869f8
I20260709 13:01:50.251458  8139 catalog_manager.cc:5028] TS ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): tablet 1112bcf4c4ea44e1933ee6287bc869f8 (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:50.252226  8405 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:60896
I20260709 13:01:50.252774  8405 raft_consensus.cc:606] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:50.255398  8324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:01:50.255924  8324 raft_consensus.cc:606] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:01:50.258555  8255 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "f865c4d761f5446eb12b3a029234772e"
 from {username='slave'} at 127.0.0.1:50420
I20260709 13:01:50.258991  8255 raft_consensus.cc:606] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Received request to transfer leadership to TS f865c4d761f5446eb12b3a029234772e
I20260709 13:01:50.260025  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:01:50.261629  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:50.262595  9695 ts_tablet_manager.cc:1929] T 09943d7e76a54d4e80eb181cac80bf22 P e64761f43e4240319c41117cf886d22f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:01:50.262921  9695 log.cc:1199] T 09943d7e76a54d4e80eb181cac80bf22 P e64761f43e4240319c41117cf886d22f: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-2-root/wals/09943d7e76a54d4e80eb181cac80bf22
I20260709 13:01:50.264154  8141 catalog_manager.cc:5028] TS e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): tablet 09943d7e76a54d4e80eb181cac80bf22 (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:01:50.271314  9575 raft_consensus.cc:993] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: : Instructing follower f865c4d761f5446eb12b3a029234772e to start an election
I20260709 13:01:50.271806  9639 raft_consensus.cc:1081] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Signalling peer f865c4d761f5446eb12b3a029234772e to start an election
I20260709 13:01:50.273486  9008 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211"
dest_uuid: "f865c4d761f5446eb12b3a029234772e"
 from {username='slave'} at 127.0.0.1:56940
I20260709 13:01:50.274039  9008 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:50.274400  9008 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.279106  9008 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:50.280651  9008 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:50.281260  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:50.281726  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:50.281901  8255 raft_consensus.cc:3055] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:50.282280  8255 raft_consensus.cc:740] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:50.282341  8324 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.283114  8255 consensus_queue.cc:260] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:50.284556  8255 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.289450  8324 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:01:50.290612  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:50.291517  9647 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:50.292023  9647 raft_consensus.cc:697] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:01:50.292831  9647 consensus_queue.cc:237] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:50.290992  8255 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:01:50.301788  8153 catalog_manager.cc:5697] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 1 to 2, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 2 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:50.382647  9628 raft_consensus.cc:993] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f: : Instructing follower 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:01:50.383138  9628 raft_consensus.cc:1081] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Signalling peer 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:01:50.384547  8255 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:55264
I20260709 13:01:50.385066  8255 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:50.385366  8255 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.390946  8255 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:50.392530  8255 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:50.393020  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:50.393393  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:01:50.393639  8405 raft_consensus.cc:3055] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:50.393944  9008 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.394001  8405 raft_consensus.cc:740] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:50.394923  8405 consensus_queue.cc:260] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:50.396199  8405 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.400511  9008 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:01:50.402354  8405 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:01:50.402448  8223 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:01:50.403264  9639 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:50.403710  9639 raft_consensus.cc:697] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:50.404415  9639 consensus_queue.cc:237] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:50.414434  8153 catalog_manager.cc:5697] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 1 to 2, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 2 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:50.680487  8324 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:50.681790  9645 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:50.691723  9689 raft_consensus.cc:993] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728: : Instructing follower e64761f43e4240319c41117cf886d22f to start an election
I20260709 13:01:50.692288  9644 raft_consensus.cc:1081] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Signalling peer e64761f43e4240319c41117cf886d22f to start an election
I20260709 13:01:50.695380  8402 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:39306
I20260709 13:01:50.695885  8402 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:50.696184  8402 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:50.702932  8248 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:50.704396  9647 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:50.707676  8402 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:50.710330  8402 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:50.711788  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:50.712361  8324 raft_consensus.cc:3055] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:50.712836  8324 raft_consensus.cc:740] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:50.713644  8324 consensus_queue.cc:260] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:50.714751  8324 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:50.712303  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:50.718233  8255 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:50.727525  8255 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:01:50.732658  8324 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:01:50.734396  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:50.736408  9649 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:50.736912  9649 raft_consensus.cc:697] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:50.737944  9649 consensus_queue.cc:237] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:50.748805  8151 catalog_manager.cc:5697] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 2 to 3, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 3 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:50.937752  9008 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:50.938846  9639 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:50.949719  8402 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:50.952009  9575 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:51.122784  8324 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:51.123888  9649 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:51.133806  8248 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:51.135351  9649 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:51.262534  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:51.318285  8402 consensus_queue.cc:237] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:51.324451  8329 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 13:01:51.326375  8372 consensus_peers.cc:597] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ae82ced6ff014453bbcfd5f9f14b1f8d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:51.330471  9714 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:51.331204  8248 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:51.332060  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:51.333639  9649 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:51.349386  9714 raft_consensus.cc:2955] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:51.353469  8329 raft_consensus.cc:2955] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:51.356735  8251 raft_consensus.cc:2955] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:51.385090  8402 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "038cd0877a674c2996410c9f327df911"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "ff7a6fe533e94c1bae555b4cae427728"
 from {username='slave'} at 127.0.0.1:60896
I20260709 13:01:51.384421  8151 catalog_manager.cc:5697] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index -1 to 3, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 2 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:51.385634  8402 raft_consensus.cc:606] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Received request to transfer leadership to TS ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:51.388430  8248 consensus_queue.cc:237] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:51.389652  8251 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "ff7a6fe533e94c1bae555b4cae427728"
 from {username='slave'} at 127.0.0.1:50420
I20260709 13:01:51.390866  8251 raft_consensus.cc:606] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Received request to transfer leadership to TS ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:51.393404  8402 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:60896
I20260709 13:01:51.393893  8402 raft_consensus.cc:606] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:51.394901  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:01:51.396821  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:01:51.406955  8223 consensus_peers.cc:597] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bac7f1e5192d48a98a850cb0ece17ad5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:51.409752  8402 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:51.410912  8324 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:51.411398  9639 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:51.412596  9707 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:51.441267  9707 raft_consensus.cc:2955] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:51.450438  8324 raft_consensus.cc:2955] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:51.479868  8248 consensus_queue.cc:237] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:51.487210  8405 raft_consensus.cc:2955] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:51.491060  8153 catalog_manager.cc:5697] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: config changed from index -1 to 3, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 2 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:51.516644  8403 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:51.524068  8223 consensus_peers.cc:597] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 430fa172910841a49c0c80f2cbf76ae5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:51.525228  8324 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:51.526757  9707 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:51.531724  9639 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:51.539040  9707 raft_consensus.cc:2955] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:51.553501  8329 raft_consensus.cc:2955] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:51.556421  8324 consensus_queue.cc:237] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:51.587275  8152 catalog_manager.cc:5697] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 1 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:51.590601  8405 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:51.591413  8402 raft_consensus.cc:2955] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:51.592196  9644 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:51.593645  8248 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:01:51.596839  8298 consensus_peers.cc:597] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 -> Peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Couldn't send request to peer f865c4d761f5446eb12b3a029234772e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 81804f59e3124d27998ea80fc1c36663. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:51.597649  9689 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:51.603847  9644 raft_consensus.cc:2955] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:51.610597  8403 raft_consensus.cc:2955] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:51.643231  8152 catalog_manager.cc:5697] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: config changed from index -1 to 3, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New cstate: current_term: 3 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:51.649365  8248 raft_consensus.cc:2955] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:01:51.683930  9639 raft_consensus.cc:993] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b: : Instructing follower ff7a6fe533e94c1bae555b4cae427728 to start an election
I20260709 13:01:51.684482  9707 raft_consensus.cc:1081] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Signalling peer ff7a6fe533e94c1bae555b4cae427728 to start an election
I20260709 13:01:51.685985  8324 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
 from {username='slave'} at 127.0.0.1:57186
I20260709 13:01:51.686450  8324 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:51.686830  8324 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:51.689817  9714 raft_consensus.cc:993] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f: : Instructing follower ff7a6fe533e94c1bae555b4cae427728 to start an election
I20260709 13:01:51.690433  9711 raft_consensus.cc:1081] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Signalling peer ff7a6fe533e94c1bae555b4cae427728 to start an election
I20260709 13:01:51.693540  8324 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:51.696116  8324 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:51.696033  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:51.697141  8248 raft_consensus.cc:3055] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:51.697767  8248 raft_consensus.cc:740] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:51.698357  8324 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "038cd0877a674c2996410c9f327df911"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
 from {username='slave'} at 127.0.0.1:57202
I20260709 13:01:51.699021  8324 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:51.699426  8324 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:51.699267  8248 consensus_queue.cc:260] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:51.700553  8248 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:51.701543  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:51.702124  8403 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:51.706141  8324 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:51.708856  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:51.709368  8402 raft_consensus.cc:3055] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:51.709662  8402 raft_consensus.cc:740] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:51.710323  8402 consensus_queue.cc:260] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:51.714303  8402 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:51.718345  8248 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:51.712978  8403 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:51.719727  8295 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:51.720731  9730 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:51.721271  9730 raft_consensus.cc:697] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:51.722313  9730 consensus_queue.cc:237] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:51.723116  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:01:51.723667  9007 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:51.730018  9007 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:51.731156  8298 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:51.732136  9731 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:51.733073  9731 raft_consensus.cc:697] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:51.734018  9731 consensus_queue.cc:237] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:51.735086  8324 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:51.732090  8402 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:51.741796  8152 catalog_manager.cc:5697] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 1 to 2, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:51.761127  8152 catalog_manager.cc:5697] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 1 to 2, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:51.943482  9734 ts_tablet_manager.cc:933] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:51.945122  9734 tablet_copy_client.cc:323] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.3:37367
I20260709 13:01:51.946888  8416 tablet_copy_service.cc:140] P e64761f43e4240319c41117cf886d22f: Received BeginTabletCopySession request for tablet ae82ced6ff014453bbcfd5f9f14b1f8d from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:34462)
I20260709 13:01:51.947484  8416 tablet_copy_service.cc:161] P e64761f43e4240319c41117cf886d22f: Beginning new tablet copy session on tablet ae82ced6ff014453bbcfd5f9f14b1f8d from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:34462: session id = f865c4d761f5446eb12b3a029234772e-ae82ced6ff014453bbcfd5f9f14b1f8d
I20260709 13:01:51.950769  9714 raft_consensus.cc:993] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f: : Instructing follower 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:01:51.951215  9714 raft_consensus.cc:1081] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Signalling peer 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:01:51.953143  8248 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:55264
I20260709 13:01:51.953737  8248 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:51.954126  8248 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:51.958580  8416 tablet_copy_source_session.cc:215] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:51.962172  9734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae82ced6ff014453bbcfd5f9f14b1f8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:51.962513  8248 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:51.965899  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:51.967315  8324 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:51.975966  9736 ts_tablet_manager.cc:933] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679)
I20260709 13:01:52.002543  9734 tablet_copy_client.cc:806] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:52.002995  8324 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:01:52.005326  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:52.005970  8403 raft_consensus.cc:3055] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:52.006460  8403 raft_consensus.cc:740] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:52.007387  8403 consensus_queue.cc:260] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:52.008690  8403 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:52.006121  8220 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:52.010486  9639 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:52.011152  9639 raft_consensus.cc:697] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:52.011970  9639 consensus_queue.cc:237] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:52.003547  9734 tablet_copy_client.cc:670] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:52.004351  8248 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:52.023746  9734 tablet_copy_client.cc:538] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:52.017217  9736 tablet_copy_client.cc:323] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.1:44679
I20260709 13:01:52.028959  8267 tablet_copy_service.cc:140] P 8c42259d7c4b463a9f485f42c320c81b: Received BeginTabletCopySession request for tablet bac7f1e5192d48a98a850cb0ece17ad5 from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:49220)
I20260709 13:01:52.029527  8267 tablet_copy_service.cc:161] P 8c42259d7c4b463a9f485f42c320c81b: Beginning new tablet copy session on tablet bac7f1e5192d48a98a850cb0ece17ad5 from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:49220: session id = f865c4d761f5446eb12b3a029234772e-bac7f1e5192d48a98a850cb0ece17ad5
I20260709 13:01:52.029907  8152 catalog_manager.cc:5697] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 2 to 3, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 3 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:52.035566  8403 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:01:52.042920  8267 tablet_copy_source_session.cc:215] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:52.043838  9734 tablet_bootstrap.cc:492] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:52.045687  9736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bac7f1e5192d48a98a850cb0ece17ad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:52.063236  9736 tablet_copy_client.cc:806] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:52.064180  9736 tablet_copy_client.cc:670] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:52.067952  9736 tablet_copy_client.cc:538] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:52.079824  9736 tablet_bootstrap.cc:492] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:52.084290  9734 tablet_bootstrap.cc:492] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:52.085418  9734 tablet_bootstrap.cc:492] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:52.086208  9734 ts_tablet_manager.cc:1403] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.043s	user 0.023s	sys 0.004s
I20260709 13:01:52.089488  9734 raft_consensus.cc:359] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:52.090531  9734 raft_consensus.cc:740] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:52.091240  9734 consensus_queue.cc:260] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:52.094344  9734 ts_tablet_manager.cc:1434] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:01:52.096660  8416 tablet_copy_service.cc:342] P e64761f43e4240319c41117cf886d22f: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-ae82ced6ff014453bbcfd5f9f14b1f8d received from {username='slave'} at 127.0.0.1:34462
I20260709 13:01:52.097101  8416 tablet_copy_service.cc:434] P e64761f43e4240319c41117cf886d22f: ending tablet copy session f865c4d761f5446eb12b3a029234772e-ae82ced6ff014453bbcfd5f9f14b1f8d on tablet ae82ced6ff014453bbcfd5f9f14b1f8d with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:52.127301  9734 ts_tablet_manager.cc:933] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679)
I20260709 13:01:52.128067  9736 tablet_bootstrap.cc:492] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:52.129024  9736 tablet_bootstrap.cc:492] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:52.129928  9736 ts_tablet_manager.cc:1403] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.050s	user 0.020s	sys 0.016s
I20260709 13:01:52.163599  9734 tablet_copy_client.cc:323] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.1:44679
I20260709 13:01:52.164186  9736 raft_consensus.cc:359] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:52.165293  9736 raft_consensus.cc:740] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:52.166327  8267 tablet_copy_service.cc:140] P 8c42259d7c4b463a9f485f42c320c81b: Received BeginTabletCopySession request for tablet 430fa172910841a49c0c80f2cbf76ae5 from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:49220)
I20260709 13:01:52.166154  9736 consensus_queue.cc:260] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:52.166851  8267 tablet_copy_service.cc:161] P 8c42259d7c4b463a9f485f42c320c81b: Beginning new tablet copy session on tablet 430fa172910841a49c0c80f2cbf76ae5 from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:49220: session id = f865c4d761f5446eb12b3a029234772e-430fa172910841a49c0c80f2cbf76ae5
I20260709 13:01:52.169668  9736 ts_tablet_manager.cc:1434] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.039s	user 0.009s	sys 0.006s
I20260709 13:01:52.173404  8266 tablet_copy_service.cc:342] P 8c42259d7c4b463a9f485f42c320c81b: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-bac7f1e5192d48a98a850cb0ece17ad5 received from {username='slave'} at 127.0.0.1:49220
I20260709 13:01:52.173918  8266 tablet_copy_service.cc:434] P 8c42259d7c4b463a9f485f42c320c81b: ending tablet copy session f865c4d761f5446eb12b3a029234772e-bac7f1e5192d48a98a850cb0ece17ad5 on tablet bac7f1e5192d48a98a850cb0ece17ad5 with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:52.182996  8267 tablet_copy_source_session.cc:215] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:52.186529  9736 ts_tablet_manager.cc:933] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: Initiating tablet copy from peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:52.201273  9734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 430fa172910841a49c0c80f2cbf76ae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:52.203313  9008 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:52.205078  9723 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:52.212688  8403 raft_consensus.cc:1275] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:52.214351  9723 consensus_queue.cc:1048] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:52.242081  9734 tablet_copy_client.cc:806] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:52.250542  9734 tablet_copy_client.cc:670] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:52.245374  9736 tablet_copy_client.cc:323] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.255.2:38599
I20260709 13:01:52.252772  8341 tablet_copy_service.cc:140] P ff7a6fe533e94c1bae555b4cae427728: Received BeginTabletCopySession request for tablet 81804f59e3124d27998ea80fc1c36663 from peer f865c4d761f5446eb12b3a029234772e ({username='slave'} at 127.0.0.1:41664)
I20260709 13:01:52.253293  8341 tablet_copy_service.cc:161] P ff7a6fe533e94c1bae555b4cae427728: Beginning new tablet copy session on tablet 81804f59e3124d27998ea80fc1c36663 from peer f865c4d761f5446eb12b3a029234772e at {username='slave'} at 127.0.0.1:41664: session id = f865c4d761f5446eb12b3a029234772e-81804f59e3124d27998ea80fc1c36663
I20260709 13:01:52.260830  8341 tablet_copy_source_session.cc:215] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:52.261224  9734 tablet_copy_client.cc:538] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:52.263589  8248 raft_consensus.cc:1275] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:52.264277  8403 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:52.265815  9723 consensus_queue.cc:1048] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:52.267251  9723 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:52.271165  9734 tablet_bootstrap.cc:492] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:52.273518  9736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81804f59e3124d27998ea80fc1c36663. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:52.334199  9734 tablet_bootstrap.cc:492] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:52.340092  9734 tablet_bootstrap.cc:492] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:52.341823  9734 ts_tablet_manager.cc:1403] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.071s	user 0.024s	sys 0.001s
I20260709 13:01:52.345413  9734 raft_consensus.cc:359] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:52.355949  9734 raft_consensus.cc:740] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:52.357064  9734 consensus_queue.cc:260] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
W20260709 13:01:52.375111  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 73630us
I20260709 13:01:52.388371  9734 ts_tablet_manager.cc:1434] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.000s
I20260709 13:01:52.390331  8267 tablet_copy_service.cc:342] P 8c42259d7c4b463a9f485f42c320c81b: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-430fa172910841a49c0c80f2cbf76ae5 received from {username='slave'} at 127.0.0.1:49220
I20260709 13:01:52.390877  8267 tablet_copy_service.cc:434] P 8c42259d7c4b463a9f485f42c320c81b: ending tablet copy session f865c4d761f5446eb12b3a029234772e-430fa172910841a49c0c80f2cbf76ae5 on tablet 430fa172910841a49c0c80f2cbf76ae5 with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:52.398258  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:52.418016  9008 raft_consensus.cc:1217] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:01:52.451480  8403 raft_consensus.cc:1275] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:52.452979  9707 consensus_queue.cc:1048] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:52.457212  9736 tablet_copy_client.cc:806] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:52.457904  9736 tablet_copy_client.cc:670] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:52.463901  9008 raft_consensus.cc:1217] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:01:52.476079  9736 tablet_copy_client.cc:538] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:52.501833  8324 raft_consensus.cc:1275] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:52.506315  9639 consensus_queue.cc:1048] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:01:52.565933  9008 raft_consensus.cc:1217] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:52.567013  9736 tablet_bootstrap.cc:492] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: Bootstrap starting.
I20260709 13:01:52.580768  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:52.623517  9736 tablet_bootstrap.cc:492] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:52.626760  9736 tablet_bootstrap.cc:492] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: Bootstrap complete.
I20260709 13:01:52.627441  9736 ts_tablet_manager.cc:1403] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: Time spent bootstrapping tablet: real 0.061s	user 0.033s	sys 0.005s
I20260709 13:01:52.630195  9736 raft_consensus.cc:359] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:52.631008  9736 raft_consensus.cc:740] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Initialized, Role: LEARNER
I20260709 13:01:52.636539  9736 consensus_queue.cc:260] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:52.643114  9736 ts_tablet_manager.cc:1434] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:01:52.649258  8341 tablet_copy_service.cc:342] P ff7a6fe533e94c1bae555b4cae427728: Request end of tablet copy session f865c4d761f5446eb12b3a029234772e-81804f59e3124d27998ea80fc1c36663 received from {username='slave'} at 127.0.0.1:41664
I20260709 13:01:52.649753  8341 tablet_copy_service.cc:434] P ff7a6fe533e94c1bae555b4cae427728: ending tablet copy session f865c4d761f5446eb12b3a029234772e-81804f59e3124d27998ea80fc1c36663 on tablet 81804f59e3124d27998ea80fc1c36663 with peer f865c4d761f5446eb12b3a029234772e
I20260709 13:01:52.661521  9008 raft_consensus.cc:1217] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 13:01:52.688118  8324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:01:52.688766  8324 raft_consensus.cc:606] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:52.720769  8256 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:50420
I20260709 13:01:52.721499  8256 raft_consensus.cc:606] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
W20260709 13:01:52.894691  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 173942us
I20260709 13:01:52.940510  9751 raft_consensus.cc:993] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b: : Instructing follower e64761f43e4240319c41117cf886d22f to start an election
I20260709 13:01:52.941205  9751 raft_consensus.cc:1081] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Signalling peer e64761f43e4240319c41117cf886d22f to start an election
I20260709 13:01:52.948184  8332 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:01:52.948835  8332 raft_consensus.cc:606] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:52.965699  9761 raft_consensus.cc:993] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728: : Instructing follower 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:01:52.966158  9761 raft_consensus.cc:1081] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Signalling peer 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:01:52.971092  8248 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:50420
I20260709 13:01:52.971772  8248 raft_consensus.cc:606] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:01:52.975322  9714 raft_consensus.cc:1064] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:52.977792  9714 consensus_queue.cc:237] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:52.979982  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:01:52.981137  8248 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:55262
I20260709 13:01:52.981698  8248 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:52.982048  8248 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:52.987025  8402 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:39308
I20260709 13:01:52.987607  8402 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:52.987937  8402 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:52.994128  8248 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:53.017514  8248 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:53.024278  8402 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:53.024547  9761 raft_consensus.cc:993] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: : Instructing follower 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:01:53.025532  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:53.029284  9761 raft_consensus.cc:1081] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Signalling peer 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:01:53.031849  8403 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:53.045697  8248 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:55262
I20260709 13:01:53.046398  8248 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:53.046435  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:53.047475  8329 raft_consensus.cc:3055] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:53.047760  8329 raft_consensus.cc:740] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:53.048460  8329 consensus_queue.cc:260] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:53.050081  8329 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:53.052624  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 90601.3us
I20260709 13:01:53.054497  8402 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:53.056200  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:53.056083  8403 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:01:53.063077  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:53.063608  8325 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:53.068432  8325 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:01:53.072060  8329 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:01:53.074384  8369 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:53.075150  9711 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:53.075621  9711 raft_consensus.cc:697] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:53.075568  8248 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:53.078995  8329 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:53.078938  9711 consensus_queue.cc:237] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:53.080651  9649 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:53.081007  8220 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:53.084204  9639 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:53.084686  9639 raft_consensus.cc:697] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 4 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:53.091981  9639 consensus_queue.cc:237] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:53.095881  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:53.099607  8255 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
W20260709 13:01:53.101670  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 68144.1us
I20260709 13:01:53.114106  9008 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 2 LEARNER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:53.125936  8248 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:53.129606  9649 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:53.139015  8248 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:53.141341  9649 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:53.310249  9649 raft_consensus.cc:2955] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:53.312774  8251 raft_consensus.cc:3055] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:53.313242  8251 raft_consensus.cc:740] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:53.315289  9754 raft_consensus.cc:1064] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:53.315053  8251 consensus_queue.cc:260] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
W20260709 13:01:53.329604  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 248709us
I20260709 13:01:53.334342  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:53.335129  8325 raft_consensus.cc:3055] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:53.335486  8325 raft_consensus.cc:740] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:53.336483  8325 consensus_queue.cc:260] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:53.347657  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 242721us
W20260709 13:01:53.369416  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.390739  8152 catalog_manager.cc:5697] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 1 to 2, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 2 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:54.054858  9030 debug-util.cc:398] Leaking SignalData structure 0x7b0800257e00 after lost signal to thread 7167
I20260709 13:01:54.184440  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:01:54.262832  9030 debug-util.cc:398] Leaking SignalData structure 0x7b0800231120 after lost signal to thread 8189
I20260709 13:01:54.263127  8251 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:54.264242  9030 debug-util.cc:398] Leaking SignalData structure 0x7b080028eda0 after lost signal to thread 8281
W20260709 13:01:54.265411  9030 debug-util.cc:398] Leaking SignalData structure 0x7b080031a620 after lost signal to thread 8356
W20260709 13:01:54.270537  9030 debug-util.cc:398] Leaking SignalData structure 0x7b080035d0a0 after lost signal to thread 8430
I20260709 13:01:54.267362  8325 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:54.278383  9730 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:54.278952  9777 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:54.279045  9730 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:54.279894  9777 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:01:54.278442  9030 debug-util.cc:398] Leaking SignalData structure 0x7b0800328ea0 after lost signal to thread 9034
W20260709 13:01:54.266867  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.287636  8251 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:01:54.288549  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1331) took 1193 ms. Trace:
I20260709 13:01:54.288739  9754 consensus_queue.cc:237] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:54.267611  9008 raft_consensus.cc:2955] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:54.289383  8251 rpcz_store.cc:276] 0709 13:01:53.094970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.095674 (+   704us) service_pool.cc:224] Handling call
0709 13:01:53.312652 (+216978us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b54003c21a0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f8828735dc0 00007f88287c1e18 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6
0709 13:01:54.288462 (+975810us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":487993600}
I20260709 13:01:54.282256  9777 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:54.281313  9730 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:54.293674  9730 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:54.295282  9730 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:54.285234  8325 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:01:54.297623  9730 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:54.300899  8256 raft_consensus.cc:2955] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:01:54.320720  8152 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.891s	user 0.006s	sys 0.008s
W20260709 13:01:54.321384  8152 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.892s	user 0.007s	sys 0.008s
I20260709 13:01:54.326371  8220 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:54.329207  9779 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:01:54.329298  8332 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.241s	user 0.020s	sys 0.153s
W20260709 13:01:54.343391  8332 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.255s	user 0.023s	sys 0.153s
I20260709 13:01:54.344515  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1413) took 1261 ms. Trace:
W20260709 13:01:54.345463  8973 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.004s	sys 0.004s
W20260709 13:01:54.345849  8973 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.006s	sys 0.004s
I20260709 13:01:54.345011  8332 rpcz_store.cc:276] 0709 13:01:53.083450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.085533 (+  2083us) service_pool.cc:224] Handling call
0709 13:01:53.086229 (+   696us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:53.086737 (+   508us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:53.086778 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:53.087974 (+  1196us) log.cc:844] Serialized 548 byte log entry
0709 13:01:54.343980 (+1256006us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:54.344044 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.344149 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:54.344214 (+    65us) raft_consensus.cc:1685] finished
0709 13:01:54.344234 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.344400 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1255615,"threads_started":1,"wal-append.queue_time_us":1175981}
I20260709 13:01:54.343452  9779 raft_consensus.cc:697] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:54.359946  9779 consensus_queue.cc:237] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:54.364068  9793 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:01:54.384126  7169 debug-util.cc:398] Leaking SignalData structure 0x7b080000fc80 after lost signal to thread 8332
I20260709 13:01:54.431191  9786 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:54.611231  9786 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:54.611830  9793 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:54.458958  9779 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:54.470327  9789 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:54.471431  9777 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:01:54.613839  9777 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:54.616653  9777 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:54.613933  9789 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:54.620368  9789 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:01:54.493809  9796 log.cc:927] Time spent T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:01:54.520301  9788 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:54.622572  9788 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:54.625985  9788 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:54.626639  9788 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:01:54.627249  9788 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:54.614053  9779 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:54.632231  9779 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:54.539564  9784 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:54.633898  9784 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:54.548576  9792 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:54.552011  9783 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:54.620925  9789 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:54.593029  9800 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:54.593273  9798 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:54.604709  9801 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:01:54.651758  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.571897  9799 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:54.662443  9799 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:54.663250  9806 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:54.664004  9806 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:54.657902  9809 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:54.617370  9777 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:54.657094  9808 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:54.669226  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:54.670224  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:54.845947  9777 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:54.847265  9797 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:54.847961  9797 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:54.848865  9777 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:54.672272  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:54.845957  9808 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:54.778227  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:54.849830  9809 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:54.658526  9784 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:54.856297  9809 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:54.659943  9811 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:54.675630  9788 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:01:54.869848  9787 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.029s	sys 0.035s
I20260709 13:01:54.870241  9795 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:01:54.870476  9802 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:54.658735  9789 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:54.891512  9803 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:54.893930  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:01:54.870257  9787 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.029s	sys 0.035s
I20260709 13:01:54.862195  9784 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:54.871213  9786 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:54.871737  9788 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.870853  9795 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:54.892373  9803 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:54.912220  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:54.871093  9802 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:54.915916  9805 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:01:54.653128  9783 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:54.937472  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:54.938351  8256 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:01:54.924356  8254 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:54.678262  9800 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:54.677876  9798 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:54.869388  9809 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:54.660115  9792 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:54.814579  9801 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:54.911005  9786 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:54.911172  9808 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:54.974179  9812 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:54.869874  9811 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:54.973097  9809 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:54.916604  9805 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:54.989287  9810 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:54.992213  9807 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:54.994091  9813 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:01:54.996479  9807 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:54.908573  9784 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:54.974926  9812 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:54.990782  9808 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:01:55.010346  9813 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:54.990003  9810 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:55.021795  9804 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:01:55.028435  8151 catalog_manager.cc:5697] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 3 to 4, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 4 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:54.984908  9786 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:55.061937  8975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.211s	user 0.004s	sys 0.011s
I20260709 13:01:55.080464  9779 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:01:55.133350  8975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.292s	user 0.004s	sys 0.011s
I20260709 13:01:55.152199  9758 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:55.152908  9758 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:55.162719  9711 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:01:55.163436  9711 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:55.169981  9761 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:55.176115  9649 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:55.176754  9649 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:01:55.218954  8972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.516s	user 0.008s	sys 0.004s
W20260709 13:01:55.219439  8972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.517s	user 0.009s	sys 0.005s
W20260709 13:01:55.222394  9008 consensus_meta.cc:287] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.927s	user 0.012s	sys 0.000s
I20260709 13:01:55.161001  9799 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:55.026160  9808 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:55.039137  9804 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:55.295058  9766 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:01:55.295928  9731 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:01:55.301295  8353 debug-util.cc:398] Leaking SignalData structure 0x7b0800331080 after lost signal to thread 7167
W20260709 13:01:55.302313  8353 debug-util.cc:398] Leaking SignalData structure 0x7b0800326b00 after lost signal to thread 8189
I20260709 13:01:55.247725  9799 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:55.305253  9731 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:55.295946  9766 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:55.327787  9781 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:55.349759  9785 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:55.350651  9785 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:55.328588  9781 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:55.369398  9794 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:55.151511  9779 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:55.370275  9794 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:55.389814  9003 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 3 LEARNER]: Advancing to term 4
I20260709 13:01:55.418987  9730 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:55.419868  9730 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:55.485777  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:55.486928  9004 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:55.487664  9004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56954 (request call id 256) took 1132 ms. Trace:
I20260709 13:01:55.488113  9004 rpcz_store.cc:276] 0709 13:01:54.355711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.450339 (+1094628us) service_pool.cc:224] Handling call
0709 13:01:55.487561 (+ 37222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:55.494742  9751 raft_consensus.cc:1064] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:55.495810  9002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 258) took 1094 ms. Trace:
I20260709 13:01:55.500525  9002 rpcz_store.cc:276] 0709 13:01:54.356521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.320288 (+963767us) service_pool.cc:224] Handling call
0709 13:01:55.404597 (+ 84309us) spinlock_profiling.cc:243] Waited 5.56 ms on lock 0x7b5800175898. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699eee 0000000000375939 00007f882d70d1e0 00007f882d7d60c4 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:01:55.404842 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:55.405116 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:55.405158 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:55.405202 (+    44us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:55.405239 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.441557 (+ 36318us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.450365 (+  8808us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12783488}
I20260709 13:01:55.492904  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:55.511838  9005 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:55.520812  9001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 259) took 1163 ms. Trace:
I20260709 13:01:55.521447  9001 rpcz_store.cc:276] 0709 13:01:54.356920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.519571 (+1162651us) service_pool.cc:224] Handling call
0709 13:01:55.520065 (+   494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:55.520312 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:55.520352 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:55.520420 (+    68us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:55.520476 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.520555 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.520738 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:55.323231  9799 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:55.595113  9813 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:55.595780  9783 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:55.596045  9813 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:01:55.596931  8353 debug-util.cc:398] Leaking SignalData structure 0x7b08000fc1a0 after lost signal to thread 8281
I20260709 13:01:55.596887  9813 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:55.597937  9797 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:55.598208  9789 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:55.599062  9804 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:55.599318  9810 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:55.600561  9786 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:01:55.601213  8222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.000s	sys 0.004s
W20260709 13:01:55.601598  8221 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.013s	sys 0.023s
W20260709 13:01:55.601650  8222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.001s	sys 0.004s
I20260709 13:01:55.601917  9818 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:01:55.602093  8221 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.013s	sys 0.023s
I20260709 13:01:55.602150  9811 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:54.917892  9788 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:55.602504  9818 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:55.603758  9803 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:01:55.605278  8371 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.952s	user 0.009s	sys 0.028s
W20260709 13:01:55.605620  8371 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.953s	user 0.010s	sys 0.028s
I20260709 13:01:55.596340  9783 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:01:55.598062  8353 debug-util.cc:398] Leaking SignalData structure 0x7b0800162820 after lost signal to thread 8356
W20260709 13:01:55.606207  8220 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.833s	user 0.027s	sys 0.025s
W20260709 13:01:55.608248  8220 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.835s	user 0.027s	sys 0.025s
W20260709 13:01:55.608902  8353 debug-util.cc:398] Leaking SignalData structure 0x7b08002d1b40 after lost signal to thread 8430
W20260709 13:01:55.609213  8353 debug-util.cc:398] Leaking SignalData structure 0x7b08000be5c0 after lost signal to thread 8431
W20260709 13:01:55.610213  8353 debug-util.cc:398] Leaking SignalData structure 0x7b0800070140 after lost signal to thread 9034
W20260709 13:01:55.610471  8353 debug-util.cc:398] Leaking SignalData structure 0x7b08001fb260 after lost signal to thread 9035
W20260709 13:01:55.606750  8298 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.791s	user 0.018s	sys 0.044s
W20260709 13:01:55.611040  8298 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.018s	sys 0.044s
W20260709 13:01:55.615584  9751 raft_consensus.cc:1068] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b: Unable to promote non-voter f865c4d761f5446eb12b3a029234772e: Illegal state: Replica 8c42259d7c4b463a9f485f42c320c81b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 2 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
W20260709 13:01:55.617411  8297 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.003s	sys 0.006s
W20260709 13:01:55.617731  8297 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.003s	sys 0.006s
W20260709 13:01:55.618969  8372 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.959s	user 0.025s	sys 0.122s
W20260709 13:01:55.595525  8369 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.037s	sys 0.104s
W20260709 13:01:55.619596  8369 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.037s	sys 0.104s
W20260709 13:01:55.619251  8372 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.959s	user 0.025s	sys 0.122s
I20260709 13:01:55.621025  9006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 253) took 1253 ms. Trace:
I20260709 13:01:55.644165  9649 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:55.621626  9006 rpcz_store.cc:276] 0709 13:01:54.354638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.281753 (+927115us) service_pool.cc:224] Handling call
0709 13:01:55.282106 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:55.607296 (+325190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:55.607325 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:55.607364 (+    39us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:55.607392 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.607444 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.607555 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:55.649013  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:55.649302  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:55.650063  9006 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:55.652686  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:55.653833  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:55.659017  9821 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:55.659750  9821 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:55.662860  9821 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:55.595214  9817 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:55.607703  9731 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:55.674070  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:55.595173  9802 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:55.605985  9766 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:55.618906  9809 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:55.692240  8329 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:01:55.728547  9008 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.017s	sys 0.059s
I20260709 13:01:55.619271  9783 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:55.734131  9008 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.017s	sys 0.059s
W20260709 13:01:55.730849  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 108160us
I20260709 13:01:55.731877  9009 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 1 LEARNER]: Advancing to term 2
I20260709 13:01:55.732748  9818 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:55.683765  9810 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:55.738056  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:55.738238  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:55.742664  8329 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:55.672850  9795 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:55.685987  9788 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:55.740309  9766 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:55.170665  9761 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:55.760577  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:55.762656  9781 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:55.751868  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:55.765170  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:55.765889  8325 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:55.750291  8298 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:55.773422  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:55.741811  9797 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:55.750125  9730 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:55.775616  9799 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:01:55.802875  8296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.011s	sys 0.064s
I20260709 13:01:55.802971  8331 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:55.824098  9009 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 2 LEARNER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:55.765805  9806 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:55.773319  9788 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:55.823357  9781 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:55.826185  9835 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:55.802237  9820 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:01:55.826998  9795 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:55.827214  9801 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:01:55.827272  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 49641.9us
I20260709 13:01:55.827438  9799 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:55.827858  9812 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:01:55.779124  8222 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.3:37367 timed out after 1.618s (SENT) [suppressed 2 similar messages]
I20260709 13:01:55.770695  9808 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:55.832883  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:55.833961  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:55.835315  9008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 373) took 2492 ms. Trace:
I20260709 13:01:55.835193  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:55.773013  9792 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:55.697233  9817 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:55.835158  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:01:55.836366  8296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.859s	user 0.012s	sys 0.064s
I20260709 13:01:55.750788  9810 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:01:55.768117  9766 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:55.802098  9762 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:55.778633  9805 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:01:55.906849  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 72416.7us
I20260709 13:01:55.907354  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:55.909869  8402 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:01:55.910761  8329 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:01:55.833321  9788 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:55.914779  9649 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:55.824097  9758 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:55.919052  8298 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:55.920769  9649 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:55.895491  9008 rpcz_store.cc:276] 0709 13:01:53.329556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.329743 (+   187us) service_pool.cc:224] Handling call
0709 13:01:54.267229 (+937486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.267466 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:55.223023 (+955557us) log.cc:844] Serialized 14 byte log entry
0709 13:01:55.803365 (+580342us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:55.821965 (+ 18600us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:55.822026 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.822109 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.822260 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":29695,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":5941,"thread_start_us":579929,"threads_started":2,"wal-append.queue_time_us":536277}
I20260709 13:01:55.978780  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:55.847030  9785 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:55.894304  9730 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:55.984525  8403 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:55.833498  9793 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:55.910068  9711 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:55.986526  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:55.999747  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:55.802676  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:55.907359  9811 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:55.994899  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.004354  8402 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:01:55.916072  9792 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.802573  9825 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:55.801736  9797 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:55.897107  9751 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:56.005692  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:55.930408  9788 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:55.915927  9806 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:01:55.907904  9799 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:55.977842  9812 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:55.916875  9795 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:55.928102  9789 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:55.980960  9794 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.006666  9649 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:56.011404  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:55.982337  9818 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:01:56.014518  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:55.997294  9817 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:56.002911  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.003268  9731 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.855254  9781 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:56.011027  9792 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
W20260709 13:01:56.028610  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 109640us
I20260709 13:01:56.054940  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.056077  8397 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:55.917336  9820 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:01:56.030359  8223 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.093s	user 0.036s	sys 0.091s
W20260709 13:01:56.058133  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 1.12127e+06us
I20260709 13:01:56.058463  9807 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.059000  9820 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:56.059429  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:56.038627  8403 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:01:55.998829  9730 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.062695  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:55.985893  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:56.080024  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.080907  8248 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 1.
I20260709 13:01:56.014518  9815 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:56.013815  9821 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:56.084911  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.083355  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.063907  9731 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:56.022138  9808 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:56.089018  9007 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:56.008601  9811 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.022759  9784 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:55.891728  9835 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.015156  9751 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.130749  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.133103  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.127128  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:56.127848  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.020234  9819 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:56.087218  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.129999  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.136250  8249 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:56.017771  9806 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:56.139894  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.085965  9783 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:55.991472  9762 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:56.144155  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.144753  8255 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:56.019960  9812 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.051779  9789 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.126673  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:56.143078  8401 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:56.143291  9731 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.055860  9818 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:56.145932  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.146575  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.146765  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.052505  9828 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:56.094620  9808 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:56.158059  8249 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:56.151764  8223 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:01:56.154711  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.160702  8256 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:01:56.160821  8254 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:56.094748  9821 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.148128  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:56.139796  9761 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.017537  9825 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:56.179740  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.180323  9825 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.180585  8257 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:56.159657  9808 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:56.182163  9797 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:56.186326  9730 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.185791  9799 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:56.164744  9777 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:56.095877  9815 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.190749  9821 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.148900  9826 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:56.215005  9777 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:01:56.235554  9003 consensus_meta.cc:287] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.843s	user 0.004s	sys 0.016s
I20260709 13:01:56.337778  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.337953  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.338052  9003 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 4 LEARNER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:56.338707  9003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 415) took 1013 ms. Trace:
I20260709 13:01:56.339015  9003 rpcz_store.cc:276] 0709 13:01:55.325282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.327160 (+  1878us) service_pool.cc:224] Handling call
0709 13:01:55.374161 (+ 47001us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:56.338437 (+964276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.338631 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:56.337173  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.340036  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:56.176826  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:56.150707  9823 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:01:56.343159  9823 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:56.345583  8248 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:56.147995  9819 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.347869  9819 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:56.184808  9807 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:56.348598  9807 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.350464  9807 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.191742  9766 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:56.195174  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:56.159216  9831 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:56.351902  9831 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.152680  9834 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:56.257951  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:56.268721  9799 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:56.358649  9831 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.345376  9823 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:56.363901  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:01:56.285176  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 223821us
I20260709 13:01:56.195822  9822 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:56.369156  9822 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:56.246788  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:56.373466  9822 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:56.374109  9822 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:56.374642  9822 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:01:56.174945  9828 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:56.290284  9762 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.298734  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:56.245780  9826 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:56.379603  9826 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:56.380856  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:56.381712  9004 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:01:56.303623  9806 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:56.292587  9792 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:56.308619  9815 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.309782  9818 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:56.311440  9789 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.309315  9812 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.312162  9835 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:56.387297  9777 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:56.387066  9822 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:01:56.393627  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 473084us
W20260709 13:01:56.395277  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 312856us
I20260709 13:01:56.326854  9808 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:56.328045  9802 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:56.401122  9828 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:56.400960  9802 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.182251  9751 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.354768  9834 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.244706  9788 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:56.403034  9774 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:56.398798  8220 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:56.411487  8152 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45456 (request call id 277) took 3048 ms. Trace:
I20260709 13:01:56.414206  9834 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:56.413146  8220 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:56.421443  9802 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:56.422700  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.424012  9777 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:56.463982  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.466672  8972 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:01:56.427016  9789 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:56.463316  9806 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:56.472252  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:01:56.472016  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 45604.7us
W20260709 13:01:56.470124  8222 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.002s	sys 0.013s
I20260709 13:01:56.499714  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.477648  8220 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:01:56.549569  8222 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.012s	sys 0.013s
W20260709 13:01:56.553088  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 774016us
I20260709 13:01:56.554843  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:01:56.553567  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 76013us
I20260709 13:01:56.477000  9789 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:56.482805  9806 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:56.412072  8152 rpcz_store.cc:276] 0709 13:01:53.358053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.358240 (+   187us) service_pool.cc:224] Handling call
0709 13:01:56.406269 (+3048029us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:54.321107 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:54.321809 (+   702us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:54.321853 (+    44us) write_op.cc:435] Acquired schema lock
0709 13:01:54.321875 (+    22us) tablet.cc:662] Decoding operations
0709 13:01:54.322298 (+   423us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:54.322386 (+    88us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:54.322427 (+    41us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:01:54.322803 (+   376us) tablet.cc:701] Row locks acquired
0709 13:01:54.322822 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:01:54.322954 (+   132us) write_op.cc:270] Start()
0709 13:01:54.323078 (+   124us) write_op.cc:276] Timestamp: P: 1783602114322910 usec, L: 0
0709 13:01:54.323107 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:01:54.323754 (+   647us) log.cc:844] Serialized 591 byte log entry
0709 13:01:55.694385 (+1370631us) op_driver.cc:464] REPLICATION: finished
0709 13:01:56.062445 (+368060us) write_op.cc:301] APPLY: starting
0709 13:01:56.086158 (+ 23713us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:56.097013 (+ 10855us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:56.151136 (+ 54123us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:56.166439 (+ 15303us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:56.183387 (+ 16948us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:01:56.193287 (+  9900us) write_op.cc:312] APPLY: finished
0709 13:01:56.251108 (+ 57821us) log.cc:844] Serialized 23 byte log entry
0709 13:01:56.260481 (+  9373us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:56.266771 (+  6290us) write_op.cc:454] Released schema lock
0709 13:01:56.275533 (+  8762us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:56.297293 (+ 21760us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:56.297341 (+    48us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":330038,"apply.run_cpu_time_us":2592,"apply.run_wall_time_us":234764,"num_ops":1,"prepare.queue_time_us":891782,"prepare.run_cpu_time_us":66934,"prepare.run_wall_time_us":586272,"raft.queue_time_us":295194,"raft.run_cpu_time_us":31314,"raft.run_wall_time_us":859463,"replication_time_us":1324088,"thread_start_us":1727397,"threads_started":4,"wal-append.queue_time_us":531324,"wal-append.run_cpu_time_us":3150,"wal-append.run_wall_time_us":1030177}]]}
I20260709 13:01:56.560467  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:01:56.561292  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 194083us
I20260709 13:01:56.563162  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:56.472484  9828 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:56.555361  9751 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:56.558436  9789 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:56.568362  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.569448  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.570506  8332 raft_consensus.cc:2393] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 2: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:01:56.568969  8323 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 1.
I20260709 13:01:56.560848  9808 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:56.569207  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.562554  9806 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:56.580360  9834 raft_consensus.cc:1064] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
W20260709 13:01:56.580859  9834 raft_consensus.cc:1068] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: Unable to promote non-voter f865c4d761f5446eb12b3a029234772e: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:01:56.567550  9751 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:56.587260  9751 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:56.588204  9800 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:01:56.589476  8369 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.767s	user 0.000s	sys 0.057s
W20260709 13:01:56.589746  8369 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.000s	sys 0.057s
W20260709 13:01:56.590003  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 845738us
I20260709 13:01:56.571208  9823 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:01:56.566191  9828 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:56.578096  9788 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:56.499881  9844 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.099s
I20260709 13:01:56.600438  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.600849  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.603818  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.595438  9807 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:56.589233  9813 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.591277  9777 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:56.604576  9807 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.597769  9828 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:56.597764  9823 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:56.606482  9826 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:56.590863  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:01:56.606984  9826 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.607924  9823 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:56.608067  8295 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:56.595332  9774 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:01:56.608825  9815 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:56.608958  8295 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:56.609326  9815 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:01:56.609787  9815 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:56.610395  9842 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:56.610821  9842 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:56.611171  9842 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:56.611873  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.613004  8295 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:01:56.613016  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.614984  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.615899  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.616369  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.616895  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.608879  9774 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:56.618818  8398 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:56.619429  9774 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:56.620386  8974 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:56.620889  9779 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.621630  9837 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:56.604979  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.601398  8399 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:01:56.625463  8404 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:56.625670  9812 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:56.626832  8295 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:56.627527  9812 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:56.627907  9812 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:56.628258  9812 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:56.629065  9811 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:56.630925  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:56.631661  8222 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:56.635219  9780 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:56.635617  9780 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:01:56.636008  9780 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:56.637542  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:01:56.638536  9807 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.638569  9826 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.639647  9810 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:56.640368  9795 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:56.640828  8331 raft_consensus.cc:1240] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:56.641453  9798 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:01:56.644892  9829 log.cc:927] Time spent T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.818s	user 0.004s	sys 0.009s
I20260709 13:01:56.647737  9781 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.632349  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.617327  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.633315  8398 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:56.655982  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.656893  9812 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.658499  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.658998  9812 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.660058  8331 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:56.660486  8295 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:56.661381  8295 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:56.661759  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.664054  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.664156  9813 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:56.664582  9813 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:01:56.664932  9813 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:56.666474  8972 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:56.667444  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.667714  8297 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:56.672215  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.674073  8297 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:01:56.675508  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.676081  8325 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:01:56.676898  9006 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 2 LEARNER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:56.677721  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.678261  8325 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:56.678309  8297 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:56.679814  8295 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:56.680997  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:56.682139  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:56.686465  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:01:56.686985  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:56.687621  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:56.688113  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:56.688194  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:56.676484  9784 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:56.669020  9781 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:56.689502  9781 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:56.668540  9822 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.636896  8222 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:56.665906  9812 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:56.691120  8295 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:01:56.692437  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.694578  9781 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:56.694676  9808 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:01:56.700165  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 63346.8us
I20260709 13:01:56.702387  8298 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:56.702957  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.703850  8298 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:01:56.704228  8328 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:56.704712  9808 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:56.704847  9803 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:56.705137  9808 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:56.705279  9803 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:01:56.704154  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.705657  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.706167  8324 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:56.705647  9806 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.708103  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.710343  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.744802  8295 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:01:56.750013  8325 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:56.752471  9831 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:56.752789  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.752965  9831 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:56.696635  9761 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:56.705792  9803 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:56.757642  9821 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:56.657492  9789 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.758090  9784 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:56.750618  8222 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:56.766081  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.767057  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.767619  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.767946  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.786121  8251 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:56.786345  8248 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:56.769699  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.700408  9812 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:56.700068  9775 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:56.742461  9780 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:56.787125  8255 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:01:56.787693  8325 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:01:56.629125  9751 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.788939  8222 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:56.763010  9841 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:56.790336  9811 consensus_queue.cc:1059] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:56.762681  9844 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.770149  9831 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:56.796119  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.793788  9812 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:56.798333  8369 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:56.799079  8222 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:56.799589  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:01:56.772152  9847 log.cc:927] Time spent T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:01:56.795620  9842 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.828444  9842 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.829252  8329 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:56.793761  9814 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:56.794971  9788 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:56.830444  9788 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:01:56.830855  9788 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:56.832538  9808 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.801376  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.804035  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.805073  8151 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45434 (request call id 279) took 2489 ms. Trace:
I20260709 13:01:56.802457  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.806543  9811 raft_consensus.cc:3055] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:56.807078  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.807286  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.797627  9779 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 13:01:56.803081  9841 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:56.844744  9841 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:56.808112  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.809794  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.846336  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.847465  8332 raft_consensus.cc:1240] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:56.847426  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.797791  9823 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:01:56.811452  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.805233  8152 catalog_manager.cc:5697] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index 3 to 4, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:56.816162  8369 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:56.816742  8257 raft_consensus.cc:1240] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:56.852082  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.836740  9811 raft_consensus.cc:740] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:56.817416  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.817677  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:01:56.818034  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 114259us
I20260709 13:01:56.818187  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.810627  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.816835  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.852659  8253 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:56.818714  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.819131  9777 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:56.819432  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.821659  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:56.858007  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.822431  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.711759  9822 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.771709  9821 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:56.826184  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.796958  9775 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.816669  9845 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:56.819449  9780 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:56.803180  9774 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:01:56.842603  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 164366us
I20260709 13:01:56.843237  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.834450  9844 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:56.848924  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.809965  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.850178  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.817065  9836 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:56.856673  8369 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:56.857684  9809 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:56.848865  9815 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:56.866257  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.866966  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.835309  8151 rpcz_store.cc:276] 0709 13:01:54.315357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.315962 (+   605us) service_pool.cc:224] Handling call
0709 13:01:56.804996 (+2489034us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:55.803736 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:56.049848 (+246112us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:56.049884 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:01:56.049900 (+    16us) tablet.cc:662] Decoding operations
0709 13:01:56.050255 (+   355us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:56.050303 (+    48us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:56.050331 (+    28us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:01:56.059746 (+  9415us) tablet.cc:701] Row locks acquired
0709 13:01:56.063602 (+  3856us) write_op.cc:260] PREPARE: finished
0709 13:01:56.126904 (+ 63302us) write_op.cc:270] Start()
0709 13:01:56.139666 (+ 12762us) write_op.cc:276] Timestamp: P: 1783602116094517 usec, L: 0
0709 13:01:56.141567 (+  1901us) op_driver.cc:348] REPLICATION: starting
0709 13:01:56.218077 (+ 76510us) log.cc:844] Serialized 591 byte log entry
0709 13:01:56.666520 (+448443us) op_driver.cc:464] REPLICATION: finished
0709 13:01:56.666775 (+   255us) write_op.cc:301] APPLY: starting
0709 13:01:56.666857 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:56.666963 (+   106us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:56.667052 (+    89us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:56.667267 (+   215us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:56.668768 (+  1501us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:01:56.669993 (+  1225us) write_op.cc:312] APPLY: finished
0709 13:01:56.676790 (+  6797us) log.cc:844] Serialized 23 byte log entry
0709 13:01:56.690415 (+ 13625us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:56.699770 (+  9355us) write_op.cc:454] Released schema lock
0709 13:01:56.710292 (+ 10522us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:56.794344 (+ 84052us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:56.797469 (+  3125us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":86,"apply.run_cpu_time_us":2009,"apply.run_wall_time_us":91939,"num_ops":1,"prepare.queue_time_us":137632,"prepare.run_cpu_time_us":47986,"prepare.run_wall_time_us":801744,"raft.queue_time_us":83833,"raft.run_cpu_time_us":481,"raft.run_wall_time_us":481,"replication_time_us":524802,"thread_start_us":446155,"threads_started":3,"wal-append.queue_time_us":317742}]]}
I20260709 13:01:56.868784  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.879186  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.869971  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:56.870414  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.816368  9828 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:56.871655  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.846757  8327 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:56.869549  9836 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:56.873034  9845 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:56.909479  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.873206  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.874935  8326 raft_consensus.cc:2393] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 2: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:01:56.910157  8253 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:56.874857  9751 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.875082  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.862552  9821 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:56.875927  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:56.876281  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.885829  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:56.878310  8330 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:01:56.891310  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.915112  8369 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:56.915299  8325 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:01:56.893484  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.894636  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:56.864526  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.919507  9809 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:56.928040  9809 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:56.928467  9809 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:56.911221  9836 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:56.895840  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.932253  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.933615  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:56.898205  9828 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.898784  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:56.916074  9812 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.939160  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.892946  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:56.905028  8256 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:56.908097  9789 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.945711  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.863474  9806 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.896703  9803 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:56.912084  9781 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:56.916841  8369 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:56.895080  9815 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:56.921545  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.921079  9766 consensus_queue.cc:1059] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:01:56.919946  9827 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:56.924338  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:56.927114  9822 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.929222  8974 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:01:56.956377  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 90050.6us
I20260709 13:01:56.957326  8297 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:01:56.958666  9766 raft_consensus.cc:3055] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:56.959041  9766 raft_consensus.cc:740] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:56.959077  9781 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:56.959947  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:01:56.959901  9766 consensus_queue.cc:260] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.954556  9845 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:56.961094  9833 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:56.868037  9774 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:56.950060  8369 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:56.966907  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.961125  9766 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:56.918179  9761 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:56.954993  9008 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:56.930656  9775 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.967249  9810 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:56.979837  9761 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:56.979957  9795 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:56.983783  9007 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:56.966279  9774 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:56.986549  8369 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:56.961090  9811 consensus_queue.cc:260] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:56.989710  9834 raft_consensus.cc:1064] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:01:56.992172  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.990705  9802 consensus_queue.cc:1059] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:01:56.992776  8256 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:01:56.992718  9788 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:56.993302  9802 raft_consensus.cc:3055] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:56.993655  9802 raft_consensus.cc:740] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:56.994189  9811 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:56.994549  8332 raft_consensus.cc:1240] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:56.991416  8369 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:56.998003  8369 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:56.996743  9800 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:57.000216  8369 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:57.000240  8297 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:01:57.001479  9825 consensus_queue.cc:1059] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:01:56.962500  9813 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:57.004539  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:56.992172  9812 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:56.996045  9808 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:57.005513  8256 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:01:56.983809  9761 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:01:56.999188  9834 raft_consensus.cc:1068] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: Unable to promote non-voter f865c4d761f5446eb12b3a029234772e: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:01:57.007787  9851 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:57.001559  9795 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:57.017625  9795 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:57.018208  9795 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:56.998872  9800 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:57.001590  9840 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:57.002310  9822 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:57.022974  9822 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.023360  9822 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:57.003878  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:57.006708  9825 raft_consensus.cc:3055] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:57.003624  9845 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:56.994398  9802 consensus_queue.cc:260] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:57.019703  8323 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:57.020427  8297 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:01:57.025656  8256 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:57.022096  9800 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:57.021736  9840 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:57.030864  9825 raft_consensus.cc:740] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:01:57.041064  8297 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:01:57.039047  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:57.030792  9845 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:57.046855  9827 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:57.046929  9845 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:57.046367  8369 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:57.048445  8297 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:57.047327  9827 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.049868  8256 raft_consensus.cc:1240] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:57.055801  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:57.061580  9808 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:57.063755  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:57.064806  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:57.070464  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:57.072069  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:57.072582  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:57.045995  9851 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:57.073228  9810 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:57.062878  9827 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:57.073952  9810 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:57.075667  8332 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:57.045835  9840 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:57.074717  9781 consensus_queue.cc:1059] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:01:57.099411  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:57.101781  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:57.105259  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:57.103603  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:57.100871  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:57.107578  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:57.116782  8329 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:01:57.117610  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:57.053694  9854 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:57.116473  8252 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:57.156634  8253 raft_consensus.cc:1240] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:57.160537  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:57.100337  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:01:57.097559  9821 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:57.088476  9828 consensus_queue.cc:1059] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:01:57.104151  8251 raft_consensus.cc:1240] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:57.168763  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:57.158713  9853 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:57.084952  9810 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:57.170142  9853 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:57.069615  9802 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:57.098935  9831 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:57.102517  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:57.085884  9825 consensus_queue.cc:260] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:57.100562  9844 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:57.090152  9788 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:57.148892  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:57.180351  8332 raft_consensus.cc:2393] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 2: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:01:57.150218  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:57.151155  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:57.151307  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:57.152396  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:57.152933  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:57.162106  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:57.163329  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:57.166689  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:57.171482  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:57.088361  9803 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:01:57.230990  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 173192us
I20260709 13:01:57.177755  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:57.233376  8255 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:01:57.187083  8324 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:57.164196  9851 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:57.154078  9834 consensus_queue.cc:1059] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 13:01:57.179059  9854 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:57.179641  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:57.233505  9781 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:57.254113  9825 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:57.276533  9813 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:57.272857  8257 raft_consensus.cc:2393] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 2: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:01:57.282316  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:57.232308  9836 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:57.301481  8331 raft_consensus.cc:1240] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:57.280023  9843 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:57.282342  9837 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:57.269649  9851 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:57.298616  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:01:57.272425  9854 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:57.280304  9781 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:57.288544  9809 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:57.305644  9831 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:57.285708  9800 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:57.372855  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.315939  9843 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:57.298010  9857 consensus_queue.cc:1059] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.134s
I20260709 13:01:57.434079  9836 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:57.434976  9800 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:57.350432  9837 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:57.438397  9832 raft_consensus.cc:3055] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:57.438974  9832 raft_consensus.cc:740] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:01:57.440129  9832 consensus_queue.cc:260] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:57.436874  9800 raft_consensus.cc:3055] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:57.442039  9856 raft_consensus.cc:3055] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:57.442382  9800 raft_consensus.cc:740] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:57.442669  9856 raft_consensus.cc:740] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:57.443544  9800 consensus_queue.cc:260] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:57.445955  9832 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:57.370438  9809 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:01:57.455195  9862 log.cc:927] Time spent T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 13:01:57.535055  8249 raft_consensus.cc:2393] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 2: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:01:57.549160  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:57.551828  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:57.551851  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:57.369127  8406 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:57.563005  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:57.564162  8255 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:57.564905  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:57.567142  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:57.568957  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:57.568850  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:57.568984  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:57.576175  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:57.577672  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:57.582532  8326 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:57.584290  8256 raft_consensus.cc:1240] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:57.589644  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:57.591190  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:57.579547  8324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:01:57.585402  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:57.591907  8254 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:57.592058  8324 raft_consensus.cc:606] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:01:57.586264  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:57.587378  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:57.571421  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:57.609740  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:57.614300  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:57.616061  8253 raft_consensus.cc:2393] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 2: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:01:57.657352  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.733986  9853 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:57.782877  8254 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:01:57.795171  9843 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:57.795513  9762 raft_consensus.cc:3055] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:57.796216  9762 raft_consensus.cc:740] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:01:57.813213  8255 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:57.830106  9786 consensus_queue.cc:1059] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.095s
W20260709 13:01:57.832167  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.838212  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:57.839968  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:57.840549  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:57.844537  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:57.846437  9781 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:57.850227  9762 consensus_queue.cc:260] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:57.851779  9762 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:57.853137  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:57.859792  9800 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:57.860124  9833 consensus_meta.cc:287] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.898s	user 0.010s	sys 0.010s
I20260709 13:01:57.860867  9833 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:57.838315  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:57.830767  9837 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:01:57.861747  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 121930us
W20260709 13:01:57.868110  9774 consensus_meta.cc:287] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.882s	user 0.013s	sys 0.001s
I20260709 13:01:57.868388  9786 raft_consensus.cc:3055] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:01:57.866283  8332 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.083s	sys 0.136s
I20260709 13:01:57.866556  8400 raft_consensus.cc:1240] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 1. Current term is 2. Ops: []
W20260709 13:01:57.877239  8332 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.084s	sys 0.136s
I20260709 13:01:57.881307  8220 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:57.881852  9821 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:57.882795  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:57.883368  9803 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:57.885375  9803 consensus_queue.cc:1059] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:57.886574  9803 raft_consensus.cc:3055] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:57.886984  9803 raft_consensus.cc:740] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:57.888361  9809 consensus_queue.cc:1059] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:57.890973  9809 consensus_queue.cc:1059] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:57.892055  9834 raft_consensus.cc:3055] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:57.892525  9834 raft_consensus.cc:740] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:57.892069  9869 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:01:57.893939  8220 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:57.895049  9869 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:57.896960  8398 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:57.897604  8223 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:57.898212  9766 raft_consensus.cc:3055] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:57.898528  9856 consensus_queue.cc:260] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:57.869201  9786 raft_consensus.cc:740] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:01:57.901829  9795 consensus_meta.cc:287] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.883s	user 0.015s	sys 0.002s
W20260709 13:01:57.904510  9840 consensus_meta.cc:287] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.743s	user 0.002s	sys 0.030s
I20260709 13:01:57.904544  9834 consensus_queue.cc:260] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:57.869124  9774 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:57.906745  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:57.910552  8220 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:01:57.929100  9774 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:57.929962  9774 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:57.911892  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:01:57.930493  9774 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:57.930949  9774 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:57.910257  8399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "ff7a6fe533e94c1bae555b4cae427728"
 from {username='slave'} at 127.0.0.1:60896
I20260709 13:01:57.901749  9803 consensus_queue.cc:260] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:57.918089  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:57.938372  9003 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
W20260709 13:01:57.918107  9802 consensus_meta.cc:287] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.738s	user 0.008s	sys 0.010s
I20260709 13:01:57.940503  9803 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:57.917418  9827 consensus_meta.cc:287] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.843s	user 0.017s	sys 0.033s
I20260709 13:01:57.944010  9827 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:57.945188  8397 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:01:57.902689  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:57.913717  9766 raft_consensus.cc:740] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:57.908111  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:57.915124  9856 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:57.933502  8220 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:57.941027  8298 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:57.915288  9869 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:57.920500  9867 raft_consensus.cc:1064] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
W20260709 13:01:57.925410  9811 consensus_meta.cc:287] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.931s	user 0.018s	sys 0.040s
I20260709 13:01:57.919770  9840 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:57.949242  8399 raft_consensus.cc:606] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Received request to transfer leadership to TS ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:57.949753  8399 raft_consensus.cc:614] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:57.950246  8403 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 2. Current term is 3. Ops: [2.4-2.4]
I20260709 13:01:57.950095  9766 consensus_queue.cc:260] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:01:57.951334  9841 consensus_meta.cc:287] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.106s	user 0.031s	sys 0.047s
I20260709 13:01:57.952054  9766 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:57.925532  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.954813  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:57.955605  9851 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:57.956486  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:57.957981  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:57.958148  9869 consensus_queue.cc:1059] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:57.959043  9869 raft_consensus.cc:3055] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:57.959420  9869 raft_consensus.cc:740] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:57.959689  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:57.960680  8329 raft_consensus.cc:2393] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 3: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:01:57.960430  9869 consensus_queue.cc:260] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
W20260709 13:01:57.924027  9822 consensus_meta.cc:287] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.900s	user 0.014s	sys 0.015s
I20260709 13:01:57.962260  8325 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:57.961861  9822 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:57.963420  8330 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:01:57.964294  9822 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:01:57.964982  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.927994  8331 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:57.915868  8223 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:01:57.969501  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 53804.5us
I20260709 13:01:57.969625  9844 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:57.969972  9869 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:57.970419  9844 consensus_queue.cc:1059] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:57.971263  8223 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:01:57.971616  9844 raft_consensus.cc:3055] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:57.972015  9844 raft_consensus.cc:740] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:01:57.909168  9825 consensus_meta.cc:287] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.623s	user 0.016s	sys 0.000s
I20260709 13:01:57.973745  9775 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:57.973187  8223 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:01:57.975288  9845 consensus_meta.cc:287] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.913s	user 0.015s	sys 0.061s
I20260709 13:01:57.975643  9775 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:57.975744  9845 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:01:57.951757  9841 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:57.977067  8402 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:57.977633  8327 raft_consensus.cc:2393] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 2: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:01:57.978253  9811 consensus_queue.cc:1059] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:57.978642  9827 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:57.978857  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 105) took 1014 ms. Trace:
I20260709 13:01:57.919636  9834 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:57.979367  9811 raft_consensus.cc:3055] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:57.979846  9811 raft_consensus.cc:740] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:57.918469  9786 consensus_queue.cc:260] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:57.973022  9844 consensus_queue.cc:260] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:57.979967  8323 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:57.923509  8332 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:57.996599  8223 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:01:57.983659  9854 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:57.997587  9836 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:57.980785  9775 consensus_queue.cc:1059] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:01:58.004917  9841 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:58.005241  9786 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:57.979327  8327 rpcz_store.cc:276] 0709 13:01:56.964792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.966141 (+  1349us) service_pool.cc:224] Handling call
0709 13:01:57.978605 (+1012464us) spinlock_profiling.cc:243] Waited 952 ms on lock 0x7b54002c9f54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:01:57.978787 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2189027072}
I20260709 13:01:57.926726  8250 raft_consensus.cc:2393] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 2: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:01:58.006122  9836 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:58.007278  9836 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:58.007905  8257 raft_consensus.cc:1240] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:58.009378  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:58.011176  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:58.012231  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:01:58.013217  9003 raft_consensus.cc:2393] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 2: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:01:58.014786  8329 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 5: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:01:58.002086  9822 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:58.019433  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1375) took 1088 ms. Trace:
I20260709 13:01:58.014114  9840 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:58.025084  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:57.923170  9795 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:58.035141  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:01:58.038348  9810 consensus_meta.cc:287] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.867s	user 0.025s	sys 0.044s
I20260709 13:01:58.038923  9810 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:58.042454  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:58.043857  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:58.044649  8402 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 5: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:01:58.044687  9811 consensus_queue.cc:260] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:58.028091  8250 rpcz_store.cc:276] 0709 13:01:56.931400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.938910 (+  7510us) service_pool.cc:224] Handling call
0709 13:01:58.007534 (+1068624us) spinlock_profiling.cc:243] Waited 987 ms on lock 0x7b5400407c54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:01:58.019341 (+ 11807us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2269004544}
I20260709 13:01:58.045861  8406 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:58.046268  9811 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:58.048194  8223 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:01:58.050753  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.053658  9844 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:58.054497  8223 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:01:58.055177  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:58.057665  8399 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:01:58.025708  9822 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:58.059773  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:58.017885  9774 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:58.063176  8974 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.064204  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:58.064352  9810 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:58.065671  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:01:58.078250  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 59265.7us
I20260709 13:01:58.079627  9879 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:58.061268  9845 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:58.058640  9876 consensus_queue.cc:1059] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:58.072844  9822 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:58.088681  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 155417us
I20260709 13:01:58.083438  9762 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:58.095134  8405 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:58.079877  9775 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:58.098176  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.111040  8220 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.111984  9795 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:58.116995  8397 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:01:58.121398  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:58.143760  8297 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:58.145298  9841 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:58.145936  9841 raft_consensus.cc:697] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:58.146962  9841 consensus_queue.cc:237] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:01:58.153123  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.154935  9879 raft_consensus.cc:3055] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:58.155529  9879 raft_consensus.cc:740] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:01:58.157344  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 67278.4us
I20260709 13:01:58.158805  8297 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:58.133011  9878 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:58.161888  9761 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:58.162431  9761 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:58.162906  9761 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:58.165056  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:58.166369  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.134140  9836 raft_consensus.cc:3055] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:01:58.169346  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 67890.9us
I20260709 13:01:58.167363  8297 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:58.176504  9774 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:58.168129  8400 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:58.168783  9836 raft_consensus.cc:740] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:58.182643  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:58.186017  8297 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:58.187282  9788 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:58.187073  9836 consensus_queue.cc:260] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:01:58.187786  9788 raft_consensus.cc:697] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:58.188578  9836 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:58.188787  9788 consensus_queue.cc:237] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:58.190528  9762 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:58.191085  9762 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:58.196972  9825 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:01:58.202687  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 91768.8us
I20260709 13:01:58.197767  9825 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:58.197595  9854 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:58.205682  9854 raft_consensus.cc:697] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:58.177889  9879 consensus_queue.cc:260] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:58.207655  9825 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:58.198479  9822 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:58.210980  9882 consensus_queue.cc:1059] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:58.212153  9822 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:58.213454  9762 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:58.213642  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:58.214398  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:01:58.222783  9002 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:58.215901  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:58.217620  9879 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:58.217170  8220 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.206806  9854 consensus_queue.cc:237] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:58.225126  9882 raft_consensus.cc:3055] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:58.204496  8402 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
W20260709 13:01:58.241317  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 80808.2us
I20260709 13:01:58.229358  9882 raft_consensus.cc:740] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:58.245851  8220 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:01:58.261638  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:58.246984  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:58.264130  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.250487  9883 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:58.255309  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:01:58.246953  9784 raft_consensus.cc:993] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728: : Instructing follower e64761f43e4240319c41117cf886d22f to start an election
I20260709 13:01:58.252556  9867 consensus_queue.cc:237] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:58.266531  8220 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.268100  9836 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:58.268534  9822 raft_consensus.cc:1081] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Signalling peer e64761f43e4240319c41117cf886d22f to start an election
I20260709 13:01:58.263747  8297 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:58.244982  9761 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:58.269658  9784 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:58.270184  9784 raft_consensus.cc:697] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:01:58.271430  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.271245  9784 consensus_queue.cc:237] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:58.272078  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.273521  9856 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:58.273603  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.273964  9856 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:58.274469  9856 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:58.274709  9855 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:58.275187  9855 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:58.275270  8369 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:01:58.275712  9855 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:58.276161  9811 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:58.277382  9811 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:01:58.277909  9811 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:58.276698  8403 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:39306
I20260709 13:01:58.281837  8403 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:58.282310  8403 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:58.284642  9856 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:58.286592  9002 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:01:58.262609  9882 consensus_queue.cc:260] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:58.288015  9882 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:58.288080  8298 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:58.273813  9761 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:01:58.290771  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.290884  8220 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.291610  9803 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:58.292783  9803 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:58.277141  8369 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:01:58.289047  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:58.295014  8402 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:58.296952  8403 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:58.293833  9811 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:58.313640  8220 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.317930  8402 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:01:58.326210  8403 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:58.330175  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:01:58.330380  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 53370.2us
I20260709 13:01:58.329309  8297 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:58.332845  8369 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.334430  9841 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:58.335062  9841 raft_consensus.cc:697] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:58.336195  9841 consensus_queue.cc:237] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:58.337584  9845 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:58.338145  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:01:58.304409  9822 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:58.338785  9002 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:58.347158  9845 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:58.339257  9879 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:58.344885  9822 raft_consensus.cc:697] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:58.341252  9855 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:58.362847  9855 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:58.364739  8295 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:01:58.364771  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 51226.6us
I20260709 13:01:58.364228  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.366767  8220 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.364318  9856 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:58.367662  9784 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:58.373485  9822 consensus_queue.cc:237] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:58.374195  8369 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:01:58.372486  9762 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:58.376294  8220 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.378257  8220 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.378635  8295 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:58.372792  9784 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:58.377125  9883 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:58.380014  9882 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:58.381562  8369 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:01:58.380937  9784 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:58.381995  9879 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:58.383173  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.384100  9845 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:58.384869  9845 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:58.385239  9845 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:58.385679  9845 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:58.386127  8295 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:58.386956  9854 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:58.390206  8220 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.392135  9002 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:01:58.393070  8220 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.393358  8372 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:01:58.394098  9879 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:58.394280  8403 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:58.394676  8972 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.394904  9879 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:58.394294  9766 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:58.395828  9766 raft_consensus.cc:697] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:58.396032  9833 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:58.397457  8972 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:58.398268  9833 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:58.398180  8295 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:58.398774  9788 consensus_queue.cc:1059] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:58.399600  9854 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:58.396836  9766 consensus_queue.cc:237] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:58.400243  8295 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:58.400025  9761 raft_consensus.cc:3055] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:58.400895  9761 raft_consensus.cc:740] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:58.401080  9854 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:58.401981  9761 consensus_queue.cc:260] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:58.402534  8295 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:01:58.403306  9761 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:58.403352  9854 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:58.406572  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:01:58.408324  9811 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:58.380050  9856 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:58.410214  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.411787  9856 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:58.418184  9833 consensus_queue.cc:1059] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:58.419703  9825 raft_consensus.cc:3055] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:58.420100  9825 raft_consensus.cc:740] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:01:58.421077  9825 consensus_queue.cc:260] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:01:58.423815  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.425688  9825 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:58.433486  8220 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.434446  9882 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:58.412177  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.436107  8369 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:01:58.437609  9855 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:58.426298  8295 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:58.438566  9856 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:58.440462  8295 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:58.439690  9833 consensus_queue.cc:1059] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:01:58.442440  8295 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:01:58.439947  9854 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:58.442061  8151 catalog_manager.cc:5697] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 1 to 2, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 2 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:01:58.443997  9854 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:58.442659  9788 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:58.444729  9882 consensus_queue.cc:1059] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:58.445989  9879 raft_consensus.cc:3055] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:58.417044  9845 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:58.446394  9879 raft_consensus.cc:740] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:58.451594  8369 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:01:58.453238  9762 raft_consensus.cc:3055] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:58.454567  9762 raft_consensus.cc:740] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:01:58.455461  9788 consensus_queue.cc:1059] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:58.455698  9762 consensus_queue.cc:260] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:01:58.456552  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.456640  9761 raft_consensus.cc:3055] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:58.454098  8972 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:01:58.457095  9762 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:58.457713  9879 consensus_queue.cc:260] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:58.458124  8295 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:58.458303  8972 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:58.458849  9833 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:58.463313  9833 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:58.463719  9845 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:58.380820  9841 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:58.457110  9761 raft_consensus.cc:740] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:58.465056  8295 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:01:58.466586  8220 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.463773  9833 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:58.486477  9856 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:58.485311  9788 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:01:58.518641  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 67191.1us
I20260709 13:01:58.487699  9844 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:01:58.525671  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 60754.3us
I20260709 13:01:58.510211  9883 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:01:58.528540  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.516786  9851 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:58.533628  8257 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:55248
I20260709 13:01:58.546272  8257 raft_consensus.cc:606] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Received request to transfer leadership
I20260709 13:01:58.546523  8257 raft_consensus.cc:614] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:58.546695  8255 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:58.548813  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:58.550503  8295 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:01:58.527048  9879 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:58.530022  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:58.556509  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:58.534500  8250 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:50420
I20260709 13:01:58.557403  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:58.560642  8249 raft_consensus.cc:1240] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:58.562779  8251 raft_consensus.cc:2410] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:01:58.560307  9822 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:58.552342  9761 consensus_queue.cc:260] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:58.566718  9856 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.567198  9825 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:58.567277  9856 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:58.567967  9784 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:58.569509  8331 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:58.558441  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:58.570247  9784 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:58.570809  8256 raft_consensus.cc:2410] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:01:58.572531  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.574309  8295 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:01:58.575060  9002 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:58.576015  9822 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:58.575966  9786 consensus_queue.cc:1059] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:58.576668  9822 raft_consensus.cc:697] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:58.576902  9786 consensus_queue.cc:1243] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer ff7a6fe533e94c1bae555b4cae427728 log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:58.577521  9786 raft_consensus.cc:3055] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:58.577839  9786 raft_consensus.cc:740] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:58.578584  9786 consensus_queue.cc:260] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:58.578899  9854 consensus_queue.cc:1059] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:58.579756  9786 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:58.580263  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:01:58.580809  9821 raft_consensus.cc:3055] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:58.584229  8397 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:58.588804  8254 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:01:58.595934  8253 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:01:58.601661  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:58.602918  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:58.602984  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:58.603694  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:58.605217  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:58.605772  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:58.607089  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:58.607551  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:58.609923  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:58.610059  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:58.618032  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:58.621459  9784 consensus_queue.cc:1059] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
W20260709 13:01:58.622545  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 50077.3us
W20260709 13:01:58.632473  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.634578  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:58.639967  8399 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:58.641973  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:58.642735  8256 raft_consensus.cc:2410] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:01:58.599895  8250 raft_consensus.cc:606] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:01:58.644867  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:58.622582  9808 raft_consensus.cc:3055] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:01:58.648732  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.555320  9844 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:58.622628  9766 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:58.597086  9821 raft_consensus.cc:740] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:58.653661  8255 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:58.638013  9856 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:01:58.656288  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.644896  8250 raft_consensus.cc:614] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:58.660307  8251 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:01:58.648144  9808 raft_consensus.cc:740] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:58.689190  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:58.689411  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:01:58.689463  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 231249us
I20260709 13:01:58.690326  8251 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:58.694258  8253 raft_consensus.cc:2393] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 2: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:01:58.653714  9766 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:01:58.720642  8254 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:01:58.652307  9845 consensus_queue.cc:1059] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:01:58.657620  9836 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:01:58.726178  9005 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 2 LEARNER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:58.693922  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:58.743184  8251 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:01:58.747225  8403 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:58.748378  9833 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:58.749642  9761 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:58.749766  9808 consensus_queue.cc:260] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:58.772689  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:58.772909  9808 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:58.773517  8250 raft_consensus.cc:2393] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 2: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:01:58.751847  9844 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:58.752055  8327 raft_consensus.cc:1240] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:58.754400  9821 consensus_queue.cc:260] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:58.755491  9002 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:01:58.756362  9856 raft_consensus.cc:3055] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:58.756799  9836 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:58.661772  9855 consensus_queue.cc:1059] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.083s
I20260709 13:01:58.766697  8404 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:58.724921  9766 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:58.762960  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:58.771695  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:58.778388  8400 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:58.784406  8369 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.691264  9822 consensus_queue.cc:237] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:58.781541  9821 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:58.793296  8257 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:01:58.784317  9856 raft_consensus.cc:740] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:01:58.796774  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 74883us
I20260709 13:01:58.788803  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:58.806429  8249 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:58.801020  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:58.801945  9836 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:58.810894  8251 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:58.811146  9825 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.811638  9825 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:58.812152  8972 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:58.810612  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.814529  8253 raft_consensus.cc:1240] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:58.804278  9833 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:58.815282  8369 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.815856  9762 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:58.816356  9867 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:58.817140  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.818161  9833 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:58.818467  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:01:58.817369  9867 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:58.819581  8972 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:58.826197  9856 consensus_queue.cc:260] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:58.824025  8400 raft_consensus.cc:1240] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:58.827432  9856 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:58.828627  8251 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:01:58.828979  9762 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:58.798033  9897 consensus_queue.cc:1059] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:01:58.830399  9762 raft_consensus.cc:3055] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:58.830761  9762 raft_consensus.cc:740] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:01:58.830907  8399 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:58.831627  9762 consensus_queue.cc:260] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:58.832856  9762 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:58.792929  9899 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.820904  9811 raft_consensus.cc:3055] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:58.824002  8249 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:58.796588  9845 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:58.820336  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:58.858577  8223 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:01:58.858112  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 60360.8us
I20260709 13:01:58.858584  8369 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.862907  8295 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:01:58.868268  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.872439  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.850189  9811 raft_consensus.cc:740] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:58.850719  9844 consensus_queue.cc:1059] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:01:58.879190  9844 raft_consensus.cc:3055] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:58.879535  9844 raft_consensus.cc:740] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:58.880380  9844 consensus_queue.cc:260] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:58.833607  9855 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:01:58.858394  9836 consensus_queue.cc:1059] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:01:58.884819  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.885914  8295 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:58.873996  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:01:58.887977  8203 auto_rebalancer.cc:893] Removing replace marker failed: Replica 8c42259d7c4b463a9f485f42c320c81b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
W20260709 13:01:58.888414  8203 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:01:58.888675  8203 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:01:58.883049  9907 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:58.890002  9811 consensus_queue.cc:260] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:58.891227  9811 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:58.893227  8400 raft_consensus.cc:2393] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 3: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:01:58.894764  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:58.898063  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:58.869117  9841 consensus_queue.cc:1059] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:01:58.858949  9899 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:58.901047  9899 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:58.901916  9836 raft_consensus.cc:3055] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:58.883697  9803 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:58.902427  8248 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
I20260709 13:01:58.904466  9001 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:01:58.908332  9845 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:58.910033  8223 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:01:58.884477  9808 consensus_queue.cc:1059] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:01:58.913691  9008 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:58.885893  9825 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:58.916081  8372 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:01:58.916853  9001 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:58.902391  9836 raft_consensus.cc:740] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:01:58.918220  9836 consensus_queue.cc:260] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:58.919314  9836 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:58.921548  9803 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:58.883409  9822 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:58.922297  8223 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:01:58.926327  9774 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:58.926672  9774 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:01:58.927073  9774 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:58.929420  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:58.931452  8327 raft_consensus.cc:2393] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 3: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:01:58.932950  9845 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:58.933370  9845 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:58.933745  9845 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:58.937523  9855 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:58.938052  9855 raft_consensus.cc:697] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:58.938935  9855 consensus_queue.cc:237] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:58.939994  9906 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:58.887342  9844 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:58.924695  9766 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:58.943361  9766 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:58.929605  9867 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:58.925009  9761 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:01:58.948037  8220 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.950237  9882 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:58.950393  8220 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.951854  9869 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:58.954041  8400 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:58.957084  8403 raft_consensus.cc:1275] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:58.963289  9841 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:58.963704  9808 raft_consensus.cc:3055] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:58.964243  9808 raft_consensus.cc:740] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:58.963809  9854 raft_consensus.cc:3055] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:58.965577  9854 raft_consensus.cc:740] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:01:58.965983  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 80146.3us
I20260709 13:01:58.966445  9854 consensus_queue.cc:260] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:58.967257  8295 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:01:58.967797  9854 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:58.968228  9898 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:58.968875  9898 raft_consensus.cc:697] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:58.965155  9808 consensus_queue.cc:260] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:58.969900  9898 consensus_queue.cc:237] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:58.970551  9808 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:58.974162  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:58.975248  8249 raft_consensus.cc:2393] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 3: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:01:58.977267  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:58.977754  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:58.980335  8397 raft_consensus.cc:1275] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:58.980002  8295 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:58.989087  8403 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:58.991632  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:58.995306  8251 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:58.996272  8255 raft_consensus.cc:1275] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:59.000330  8249 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
W20260709 13:01:59.011530  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 126783us
I20260709 13:01:59.012390  8972 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:59.014544  8369 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:59.000389  9907 raft_consensus.cc:697] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:59.016527  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:01:59.017365  8295 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:01:59.018318  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.002110  9841 consensus_queue.cc:1059] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 13:01:59.020121  9902 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:59.011232  9822 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 13:01:59.022406  9903 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:59.026329  8369 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:59.027573  9912 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:59.005600  9904 consensus_queue.cc:1048] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:01:59.036228  8295 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:01:59.039039  9762 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:59.039665  9762 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:59.040112  9762 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:59.011351  9899 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:59.013021  9821 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:59.008610  9900 consensus_queue.cc:1048] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 13:01:59.052069  8369 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:59.045862  9899 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:01:59.054761  9912 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:59.057736  9905 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:59.081848  9912 raft_consensus.cc:697] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:59.082913  8327 raft_consensus.cc:1240] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:59.063310  9907 consensus_queue.cc:237] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:59.082867  9912 consensus_queue.cc:237] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:01:59.077050  9845 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:59.070051  8295 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:01:59.072244  8141 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ae82ced6ff014453bbcfd5f9f14b1f8d with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica e64761f43e4240319c41117cf886d22f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:01:59.130082  8141 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 58088.2us
I20260709 13:01:59.061717  9899 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:59.130791  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:59.131201  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:59.131934  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:59.132414  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:59.074501  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:59.133876  8253 raft_consensus.cc:1240] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:59.073762  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:59.134162  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:59.134405  8254 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:59.134610  8250 raft_consensus.cc:2393] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 2: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:01:59.135211  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:59.135501  8256 raft_consensus.cc:1240] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:59.135475  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:59.060297  9911 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:59.122479  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:59.136615  9911 raft_consensus.cc:697] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:59.136875  8249 raft_consensus.cc:2410] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:01:59.129992  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:59.137688  8251 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:59.137598  9911 consensus_queue.cc:237] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:59.130509  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:59.073760  9005 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:01:59.136171  8252 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:59.140895  9841 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:59.144412  9851 raft_consensus.cc:3055] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:59.144827  9851 raft_consensus.cc:740] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:59.145699  9851 consensus_queue.cc:260] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:59.146857  9851 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:59.134776  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:01:59.150267  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 80365.6us
I20260709 13:01:59.080502  9774 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:59.146209  9903 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:59.163470  9811 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.162235  9854 consensus_queue.cc:1048] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:01:59.167945  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:59.179777  8326 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:01:59.180365  8323 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:01:59.158478  9841 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:01:59.167919  8328 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:01:59.160072  8251 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:01:59.158455  9853 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 13:01:59.182627  8327 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:01:59.159025  9761 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:59.179327  9811 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:59.173486  8332 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:01:59.186890  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:59.167995  9836 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:59.169157  8252 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:01:59.187569  8331 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
I20260709 13:01:59.187906  9811 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:59.187659  9836 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:59.188565  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:59.189419  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:01:59.190079  9001 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:01:59.190366  9836 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:59.191298  8372 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:01:59.193030  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:59.198558  8249 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:01:59.199671  8330 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:59.200053  8251 raft_consensus.cc:1275] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:59.208941  8402 raft_consensus.cc:1275] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:59.212185  8248 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
I20260709 13:01:59.216164  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:01:59.216954  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:59.217075  8404 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:01:59.219568  8257 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:59.229162  8252 raft_consensus.cc:1275] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:59.199493  9761 raft_consensus.cc:697] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:01:59.231529  8222 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:01:59.232100  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:59.226877  9784 consensus_queue.cc:1048] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:01:59.238229  8328 raft_consensus.cc:2393] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 4: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:01:59.214316  9910 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:59.241474  8972 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:01:59.244551  9825 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:59.244724  9898 consensus_queue.cc:1048] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:01:59.245344  8153 catalog_manager.cc:5697] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: config changed from index 4 to 3, term changed from 2 to 3, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2), f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from VOTER to NON_VOTER, now has a pending config: VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1), VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2), VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3), VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 3 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:01:59.251039  9774 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:59.236001  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:59.253248  8326 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
I20260709 13:01:59.253496  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:01:59.255198  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:59.245015  9825 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:59.248968  8153 catalog_manager.cc:5697] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 3 to 4, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 4 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:59.278605  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.227941  9912 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:59.211004  9841 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:01:59.211335  9854 consensus_queue.cc:1048] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:01:59.225271  9766 consensus_queue.cc:1059] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 13:01:59.243273  9910 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:59.295424  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:01:59.295810  9886 log.cc:927] Time spent T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:01:59.296037  9008 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:59.296929  9915 log.cc:927] Time spent T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:01:59.293812  9845 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:59.297833  8326 raft_consensus.cc:2393] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 3: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:01:59.298174  9766 raft_consensus.cc:3055] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:59.298569  9766 raft_consensus.cc:740] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:59.298811  9821 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:59.288100  9825 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:59.299443  9766 consensus_queue.cc:260] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:59.301038  9766 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:59.301381  9910 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:59.293383  8153 catalog_manager.cc:5697] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 1 to 2, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:59.306005  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.292716  9912 raft_consensus.cc:697] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:59.295966  9003 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:59.254890  9761 consensus_queue.cc:237] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:01:59.387313  9912 consensus_queue.cc:237] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:59.376961  9762 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:01:59.379478  9916 log.cc:927] Time spent T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.195s	user 0.000s	sys 0.002s
I20260709 13:01:59.287700  9879 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:59.393568  9879 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:59.393283  9899 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:59.391749  8153 catalog_manager.cc:5697] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 1 to 2, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:59.395016  8153 catalog_manager.cc:5697] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 3 to 4, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 4 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:59.399597  9918 log.cc:927] Time spent T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 13:01:59.393990  9879 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:59.406236  8153 catalog_manager.cc:5697] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 1 to 2, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:59.391371  9008 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
W20260709 13:01:59.410989  9921 log.cc:927] Time spent T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:01:59.418529  9762 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:59.422997  9910 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:01:59.427529  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 120396us
I20260709 13:01:59.429888  8324 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:01:59.428787  8372 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:01:59.448989  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:59.449779  8248 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:01:59.452814  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:59.453465  8326 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:59.426033  8153 catalog_manager.cc:5697] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 3 to 4, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 4 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:59.460594  8326 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:01:59.461936  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:59.462837  9833 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:59.466300  8249 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:59.446156  9902 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.443955  9845 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:59.497617  9825 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:59.499764  9825 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:59.500165  9879 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:59.483765  9910 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:01:59.499787  9899 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:59.489830  9902 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:59.506207  9001 raft_consensus.cc:2368] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4.
I20260709 13:01:59.508160  9902 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:01:59.466609  9808 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:01:59.533161  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:01:59.533933  9008 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:59.496270  9845 raft_consensus.cc:697] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:01:59.543864  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 327022us
I20260709 13:01:59.545107  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:59.546118  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:59.566431  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:59.597721  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:59.598510  8328 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:59.601719  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:59.606410  8330 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:59.615520  8249 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
I20260709 13:01:59.616730  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:01:59.632073  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:59.594296  9833 raft_consensus.cc:697] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:01:59.519932  9821 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.086s
I20260709 13:01:59.665570  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:01:59.595108  9910 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:59.584270  9855 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:59.667634  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:01:59.697495  9833 consensus_queue.cc:237] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:59.700942  8249 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:59.703794  8254 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:59.713076  9008 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:01:59.714277  8328 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:01:59.629281  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:59.596429  9811 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.096s
I20260709 13:01:59.631544  9825 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:59.732431  8326 raft_consensus.cc:2393] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 3: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:01:59.708817  9921 log.cc:927] Time spent T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:01:59.762025  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:59.766741  8253 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:01:59.772301  8249 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:01:59.784406  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:01:59.757023  8327 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
I20260709 13:01:59.783267  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:01:59.775441  9879 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:01:59.825073  9931 log.cc:927] Time spent T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:01:59.800453  9902 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:59.908653  9902 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:59.909124  9902 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:59.915377  8256 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:59.798780  9882 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:59.889953  9929 log.cc:927] Time spent T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.089s	user 0.009s	sys 0.000s
I20260709 13:01:59.773901  9841 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:01:59.879722  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:59.922078  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.922721  8372 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:01:59.882688  8248 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:01:59.884156  8369 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:01:59.782912  9936 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:59.763721  9845 consensus_queue.cc:237] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:59.888484  8254 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:01:59.918372  8330 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
W20260709 13:01:59.927778  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 43759.1us
I20260709 13:01:59.800210  9899 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:59.933327  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 265449us
I20260709 13:01:59.934382  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:59.935362  9008 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:59.936959  9944 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:59.937814  9822 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:01:59.938331  9822 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:59.930029  8369 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:59.940413  9822 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:01:59.921443  9841 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:01:59.987497  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:01:59.988368  8324 raft_consensus.cc:2393] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 4: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:01:59.940876  9867 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:59.988124  9903 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.993577  9869 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:59.999370  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 69407.9us
I20260709 13:02:00.001057  8369 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:01:59.927284  9936 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:01:59.925112  9882 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:00.035429  9936 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:00.039207  9867 raft_consensus.cc:697] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:00.004045  9932 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:00.004534  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:00.056501  8254 raft_consensus.cc:2393] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 4: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:00.046430  8972 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:01:59.970275  9899 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:00.057682  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 53190.2us
W20260709 13:01:59.997370  8250 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.011s	sys 0.001s
I20260709 13:01:59.994560  9855 raft_consensus.cc:697] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:01:59.997654  9910 raft_consensus.cc:697] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:00.047415  9911 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.064994  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:00.072669  8249 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:02:00.067018  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:00.049167  9808 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:02:00.071472  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 70494.2us
I20260709 13:02:00.076833  9867 consensus_queue.cc:237] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:00.062649  8250 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.011s	sys 0.002s
I20260709 13:02:00.087503  8369 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:00.062440  9936 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:00.074389  9906 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:00.015043  9903 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:00.031801  9869 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:00.098274  9936 raft_consensus.cc:697] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:00.113607  9936 consensus_queue.cc:237] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:00.115824  9903 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:00.107398  9845 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:00.090221  9911 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:00.123991  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:00.124964  8257 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:00.085368  9808 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:00.135190  9808 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:00.104525  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:00.136543  8330 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 1.
W20260709 13:02:00.141809  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 54361.9us
I20260709 13:02:00.093129  9902 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:00.097496  9944 consensus_queue.cc:1059] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:02:00.159204  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:00.166306  8251 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:00.168157  8252 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:00.169728  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:00.161706  8141 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ae82ced6ff014453bbcfd5f9f14b1f8d with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica e64761f43e4240319c41117cf886d22f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:00.151916  9855 consensus_queue.cc:237] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:00.173406  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:00.174237  8397 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:00.157171  9808 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:00.157361  9911 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:00.185640  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:00.186403  8324 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
W20260709 13:02:00.186550  9959 log.cc:927] Time spent T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:02:00.184706  9821 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.188176  9821 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:00.199714  9901 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.200433  9901 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:00.200909  9945 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:00.201913  9007 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:00.183090  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:00.235949  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:00.206880  9882 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:00.236580  8254 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:02:00.246543  9958 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.247234  9958 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:00.227991  9762 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.184264  9808 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:00.181447  9910 consensus_queue.cc:237] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:00.256878  8223 leader_election.cc:400] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4.
I20260709 13:02:00.257269  8223 leader_election.cc:403] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 13:02:00.258105  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 98984.4us
I20260709 13:02:00.248976  9902 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:00.267727  9811 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.268299  9944 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:00.268479  9811 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:00.268862  9944 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:00.269469  8253 raft_consensus.cc:1275] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:00.260176  9945 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:00.292127  9908 log.cc:927] Time spent T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc: Append to log took a long time: real 0.121s	user 0.001s	sys 0.002s
I20260709 13:02:00.260842  9882 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4.
W20260709 13:02:00.298298  9919 log.cc:927] Time spent T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.870s	user 0.000s	sys 0.013s
W20260709 13:02:00.298506  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 106898us
I20260709 13:02:00.265949  9845 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:00.286770  9007 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 4. (term mismatch)
I20260709 13:02:00.255805  9762 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:00.303079  9007 pending_rounds.cc:85] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e: Aborting all ops after (but not including) 3
I20260709 13:02:00.284785  8254 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:00.301157  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1387) took 1260 ms. Trace:
I20260709 13:02:00.305214  8253 raft_consensus.cc:1275] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:00.301119  9952 raft_consensus.cc:3055] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:00.294559  9899 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:00.318851  9958 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:00.333552  9912 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:00.335587  9912 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:00.337579  9951 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.338120  9951 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:00.340302  9951 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:00.311631  9845 raft_consensus.cc:697] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:00.311383  9952 raft_consensus.cc:740] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:00.304080  8399 rpcz_store.cc:276] 0709 13:01:59.040244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.040388 (+   144us) service_pool.cc:224] Handling call
0709 13:01:59.040695 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:59.040990 (+   295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:59.041017 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:59.041506 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 13:01:59.220932 (+179426us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:59.220970 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.221033 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:00.228585 (+1007552us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b240020c498. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255579c4 00007f8825555acb 00007f8825710dc9 00007f8825710a55 00007f882571009d 00007f882834210e 00007f882835988c 00007f8828358bb6 00007f88283587d8 00007f8828362221
0709 13:02:00.299202 (+ 70617us) raft_consensus.cc:1685] finished
0709 13:02:00.299227 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:00.301041 (+  1814us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1292,"raft.queue_time_us":976,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":8152,"spinlock_wait_cycles":4347776,"thread_start_us":179108,"threads_started":1,"wal-append.queue_time_us":246807}
I20260709 13:02:00.303465  9007 pending_rounds.cc:107] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 2.4
I20260709 13:02:00.352650  9007 raft_consensus.cc:2917] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Aborting config change with OpId 2.4: Aborted: Op aborted by new leader
I20260709 13:02:00.354171  9007 raft_consensus.cc:2889] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 3 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:00.361860  9901 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:00.366225  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:00.367093  8253 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
I20260709 13:02:00.368183  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:00.369058  8252 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
I20260709 13:02:00.370724  8254 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:00.383787  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:00.384543  8330 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:00.397820  9821 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:00.415792  9841 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:00.423549  9940 log.cc:927] Time spent T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.396s	user 0.001s	sys 0.003s
I20260709 13:02:00.425339  9811 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:00.308979  9945 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:00.442065  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:00.333828  9906 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:00.449460  8972 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:00.466262  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:00.469899  8399 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:00.472685  9898 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:00.472692  9836 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.474742  9836 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:00.473587  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:00.484949  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:00.499003  9941 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.499437  9836 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:00.499636  9941 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:00.505613  9007 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 2 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:02:00.475988  9938 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.456765  9869 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:00.509938  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:00.456410  9803 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:00.478645  9902 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:00.429634  9945 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:00.524566  9803 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:00.540565  8330 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:00.521412  9902 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:00.551185  9879 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.506906  9912 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:00.632560  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 427613us
I20260709 13:02:00.531778  9762 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:00.528146  9906 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:00.684211  8397 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:00.505554  9836 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:00.531873  8401 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:00.651126  9808 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:00.721259  8296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.026s	sys 0.060s
W20260709 13:02:00.779143  8296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.027s	sys 0.060s
I20260709 13:02:00.521677  9845 consensus_queue.cc:237] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:00.534001  9938 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:02:00.734548  9944 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:00.758216  8223 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:00.759013  9882 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.762501  9950 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:00.498085  8295 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:00.764006  9008 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.021s	sys 0.090s
I20260709 13:02:00.754832  9907 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:00.796321  9907 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:00.798508  9907 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:00.780992  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:00.801944  8330 raft_consensus.cc:1275] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:00.802022  9007 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:00.781957  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:00.758230  9899 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.785516  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:00.685824  9803 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:00.791000  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:00.811693  8397 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:02:00.774057  9902 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:00.771608  9951 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:00.794718  9008 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.021s	sys 0.090s
I20260709 13:02:00.792362  9882 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:00.795471  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:00.802932  8399 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:02:00.804283  9956 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:00.661731  9952 consensus_queue.cc:260] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:00.806713  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 308747us
I20260709 13:02:00.807507  8405 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
I20260709 13:02:00.807658  9948 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:00.806164  8372 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:00.790369  9808 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:00.815387  8324 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:00.813103  8974 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:00.844430  8974 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:00.845206  8974 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:00.777091  9879 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:00.782007  9941 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:00.811733  9945 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:00.800205  9950 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:00.856261  9879 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:00.859331  9882 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:00.850390  9907 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:00.860361  9941 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:00.801021  9836 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:00.863437  9938 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:00.852020  8295 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:00.778784  9906 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:00.872972  8328 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:00.864289  9005 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:00.842114  9951 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:00.906541  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 54583.1us
I20260709 13:02:00.906790  8324 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:00.822185  9803 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.866580  9945 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:00.937956  9821 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:00.875020  9907 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:00.813163  9899 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:00.880422  9941 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:00.904961  9912 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.905006  9762 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:00.911268  9761 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:00.871219  9808 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:00.955636  9761 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:00.961656  9943 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.966881  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1352) took 1517 ms. Trace:
I20260709 13:02:00.938657  9821 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:00.970538  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:00.975324  8297 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.783s	user 0.019s	sys 0.014s
W20260709 13:02:00.975922  8297 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.019s	sys 0.014s
W20260709 13:02:00.976413  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 784511us
I20260709 13:02:00.951275  9907 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:00.936118  9945 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:00.952499  9941 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:00.981389  8324 raft_consensus.cc:1275] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:00.936203  9803 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:00.969208  9761 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:00.905380  9956 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:00.970109  9935 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:00.978490  9856 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:00.867208  9948 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:00.994946  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:00.975793  9867 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:00.974299  9762 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:01.001850  8324 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:02:00.965452  9912 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:02:00.997762  9935 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:00.978322  9936 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.016s	sys 0.049s
I20260709 13:02:00.983322  9943 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:00.990547  9944 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:02:00.977236  8297 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:01.015075  8329 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:01.016587  8298 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.773s	user 0.006s	sys 0.019s
W20260709 13:02:01.026999  8298 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.007s	sys 0.019s
W20260709 13:02:01.010918  9936 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.016s	sys 0.050s
W20260709 13:02:01.011081  9908 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.008s	sys 0.016s
W20260709 13:02:01.009526  9910 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.006s	sys 0.019s
I20260709 13:02:01.009895  9935 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:01.052248  9821 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:01.052392  9967 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:01.052799  9967 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.053162  9967 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:01.053751  9803 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:01.057608  9899 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:01.058199  9899 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:01.058540  9899 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.058889  9899 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:01.005103  9867 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:01.062589  9950 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:01.063704  9948 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:01.063823  9867 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:01.062062  9906 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:01.065058  8329 raft_consensus.cc:1275] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:01.001216  9856 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:01.013801  9957 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.765s	user 0.000s	sys 0.024s
I20260709 13:02:01.052064  9951 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:01.065359  9910 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.006s	sys 0.019s
I20260709 13:02:01.068742  9902 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:01.095625  9762 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:01.095748  9943 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:01.050163  9903 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.096475  9902 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:01.096737  9903 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:01.050379  9908 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.008s	sys 0.017s
I20260709 13:02:01.097721  9906 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:01.026419  9961 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.003s	sys 0.023s
W20260709 13:02:01.034335  9784 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.765s	user 0.007s	sys 0.019s
I20260709 13:02:01.049886  9898 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.051360  9963 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:01.108004  8328 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:01.062114  9808 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:01.090309  9957 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.822s	user 0.000s	sys 0.025s
I20260709 13:02:01.070173  9901 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:01.072600  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:01.111227  9961 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.003s	sys 0.023s
W20260709 13:02:01.200942  8222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.000s	sys 0.042s
I20260709 13:02:01.193765  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:01.117339  9784 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.008s	sys 0.019s
W20260709 13:02:01.207072  8222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.001s	sys 0.042s
I20260709 13:02:01.071076  9949 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.131s
W20260709 13:02:01.208691  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 672631us
I20260709 13:02:01.154464  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:01.226722  8324 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:02:01.220845  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:01.151686  9825 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.359s	user 0.017s	sys 0.020s
I20260709 13:02:01.112268  9906 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:01.193923  8329 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:02:01.112123  9964 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:01.154125  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:01.121558  9766 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.772s	user 0.004s	sys 0.032s
W20260709 13:02:01.230885  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 385530us
I20260709 13:02:01.244386  9964 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:01.118389  9905 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.003s	sys 0.017s
I20260709 13:02:01.230624  8222 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:01.233279  9825 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.478s	user 0.017s	sys 0.033s
W20260709 13:02:01.293744  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 63207.4us
W20260709 13:02:01.293893  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 48048.9us
I20260709 13:02:01.237968  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:01.239315  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:01.296119  8332 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:02:01.230216  9879 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.292294  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:01.197122  9901 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:01.297366  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:01.297550  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:01.298385  9906 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:01.304586  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:01.305734  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:01.306435  8331 raft_consensus.cc:2393] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 3: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:01.307127  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:01.307719  9951 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:01.308238  9951 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:01.309607  8331 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:01.310113  9001 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:01.309507  9879 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:01.299527  9825 raft_consensus.cc:697] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 3 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:00.974920  8250 rpcz_store.cc:276] 0709 13:01:59.449792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.450997 (+  1205us) service_pool.cc:224] Handling call
0709 13:01:59.451365 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:59.701951 (+250586us) spinlock_profiling.cc:243] Waited 1.9 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f88256b40ac 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256ce455 00007f88287b6060
0709 13:02:00.088619 (+386668us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:00.093097 (+  4478us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:00.135821 (+ 42724us) log.cc:844] Serialized 26 byte log entry
0709 13:02:00.549809 (+413988us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:00.624336 (+ 74527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.640985 (+ 16649us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:00.964228 (+323243us) raft_consensus.cc:1685] finished
0709 13:02:00.964258 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:00.966783 (+  2525us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19237,"raft.queue_time_us":51623,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"spinlock_wait_cycles":4363520,"thread_start_us":998712,"threads_started":2,"wal-append.queue_time_us":826366}
I20260709 13:02:01.157002  9963 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:02:01.281401  9766 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.892s	user 0.010s	sys 0.037s
I20260709 13:02:01.302663  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:01.240718  9836 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:01.317389  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:01.311914  9825 consensus_queue.cc:237] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:01.320242  9952 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:01.321587  9000 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:02:01.321959  8972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.830s	user 0.024s	sys 0.071s
W20260709 13:02:01.285553  9905 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.036s	user 0.005s	sys 0.025s
W20260709 13:02:01.318311  8975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.003s	sys 0.017s
W20260709 13:02:01.346282  8972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.024s	sys 0.071s
W20260709 13:02:01.346480  8975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.845s	user 0.003s	sys 0.017s
I20260709 13:02:01.336447  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:01.347328  8332 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:02:01.243723  8327 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:02:01.153175  9898 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:01.337371  9008 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.013s	sys 0.008s
I20260709 13:02:01.308614  9941 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:01.293187  9956 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:01.346692  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 855453us
I20260709 13:02:01.348940  8220 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:01.351244  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:01.303659  9803 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:01.352896  9936 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:01.353498  9932 consensus_queue.cc:1059] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:01.353675  9936 raft_consensus.cc:697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:01.354583  9932 raft_consensus.cc:3055] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:01.354933  9932 raft_consensus.cc:740] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:02:01.349587  9008 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.013s	sys 0.009s
I20260709 13:02:01.355790  9932 consensus_queue.cc:260] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:01.357314  9932 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:01.357384  9980 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:01.359092  9910 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:01.364044  8220 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:01.322386  9869 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:01.365734  9967 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:01.335824  9882 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.333346  9963 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:01.370582  8369 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:01.377471  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 619350us
W20260709 13:02:01.349047  9855 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.918s	user 0.005s	sys 0.039s
I20260709 13:02:01.375727  8329 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:02:01.382117  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:01.380522  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:01.385169  8257 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:01.387284  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:01.388175  8257 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 2: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:01.374521  9882 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
W20260709 13:02:01.383984  8326 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.000s	sys 0.041s
I20260709 13:02:01.393688  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:01.432246  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:01.391891  9855 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.015s	user 0.006s	sys 0.039s
W20260709 13:02:01.437561  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.436067  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:01.355068  9936 consensus_queue.cc:237] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:01.390517  9938 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:01.384478  9943 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:01.392380  9853 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.880s	user 0.014s	sys 0.018s
W20260709 13:02:01.431277  8326 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.000s	sys 0.042s
I20260709 13:02:01.505259  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:01.376379  9869 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:01.506584  8254 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:02:01.376812  9910 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.086s
I20260709 13:02:01.512883  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:01.581076  8332 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:02:01.418356  9856 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:01.420584  9845 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:01.432139  9906 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:01.586761  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:01.578840  9853 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.067s	user 0.016s	sys 0.020s
I20260709 13:02:01.462853  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:01.602957  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:01.389490  9867 consensus_queue.cc:1048] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.120s
I20260709 13:02:01.615748  8332 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:01.623227  8250 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:01.389675  9950 consensus_queue.cc:1048] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.120s
I20260709 13:02:01.624200  9867 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:01.624616  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:01.624702  9867 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:01.625869  8250 raft_consensus.cc:2393] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 2: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:01.626883  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:01.627779  8250 raft_consensus.cc:2393] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 2: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:01.624939  9950 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:01.629849  9950 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:01.503221  9983 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:01.503647  9938 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.512742  9855 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:01.679808  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 360893us
I20260709 13:02:01.604914  9987 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:01.691887  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 324930us
I20260709 13:02:01.693023  8220 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:01.701918  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:01.709038  9008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 424) took 1590 ms. Trace:
I20260709 13:02:01.666373  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:01.709582  9008 rpcz_store.cc:276] 0709 13:02:00.096026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.096160 (+   134us) service_pool.cc:224] Handling call
0709 13:02:00.096433 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:00.096616 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:00.823026 (+726410us) log.cc:844] Serialized 14 byte log entry
0709 13:02:01.685850 (+862824us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:01.685901 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:01.685941 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.686040 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.686180 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75014,"prepare.queue_time_us":959319,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":37290,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":45383,"thread_start_us":1577141,"threads_started":3}
I20260709 13:02:01.641314  9984 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:01.728979  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:01.729775  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:01.730229  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:01.644389  9967 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:01.650072  9983 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.648526  9951 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:01.643947  9836 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:01.619369  9845 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:01.638870  9906 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:01.677748  9855 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:01.833796  9943 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:01.937294  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 232103us
I20260709 13:02:01.793823  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:01.893956  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 201004us
I20260709 13:02:01.687674  9938 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:01.940723  8402 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
W20260709 13:02:01.802496  9975 log.cc:927] Time spent T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.503s	user 0.001s	sys 0.001s
W20260709 13:02:01.720505  9969 log.cc:927] Time spent T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.436s	user 0.000s	sys 0.002s
I20260709 13:02:01.793823  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:01.638381  9964 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:01.927243  9899 consensus_meta.cc:287] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.868s	user 0.008s	sys 0.003s
W20260709 13:02:01.917985  9995 log.cc:927] Time spent T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
W20260709 13:02:01.896597  9945 consensus_meta.cc:287] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.906s	user 0.012s	sys 0.000s
I20260709 13:02:01.876250  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:01.707988  9987 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:01.925379  9935 consensus_meta.cc:287] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.855s	user 0.011s	sys 0.000s
I20260709 13:02:01.694373  9989 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:02:01.931386  9963 consensus_meta.cc:287] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.512s	user 0.009s	sys 0.000s
I20260709 13:02:01.975725  8369 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:02.007382  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:01.946084  8220 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:01.931336  9952 consensus_meta.cc:287] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.587s	user 0.008s	sys 0.000s
I20260709 13:02:01.925637  9869 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:01.859990  9982 log.cc:927] Time spent T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 13:02:02.035059  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:01.887809  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:02.004529  9761 consensus_meta.cc:287] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.012s	user 0.011s	sys 0.000s
I20260709 13:02:01.730064  9978 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:01.730264  9991 consensus_queue.cc:1048] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.337s
W20260709 13:02:01.990818  9996 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:02:01.886804  9882 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:02.056130  9882 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:02.057111  9882 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:01.849272  9898 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:02.035003  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 59409.9us
I20260709 13:02:02.061188  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:01.949661  9932 consensus_meta.cc:287] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.592s	user 0.006s	sys 0.006s
I20260709 13:02:02.062772  8369 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:01.947750  9967 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:02.043907  8400 raft_consensus.cc:2393] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 3: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:02.039680  9935 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:02.044893  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:01.953073  9943 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:01.991676  9811 consensus_queue.cc:1048] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 13:02:02.042335  9869 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.773307  9984 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:02.035949  9899 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:02.040791  9989 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:02.046557  9952 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:02.040565  9963 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:02.047343  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:02.041359  8402 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:02.102468  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:02.103104  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:02.077312  9951 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:02.051852  9978 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:01.944188  9950 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:02.035982  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 90019.5us
I20260709 13:02:01.943792  9983 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:02.085954  9941 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:02.055461  9979 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:01.720412  9990 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.296s
I20260709 13:02:01.980001  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1491) took 1174 ms. Trace:
I20260709 13:02:02.063750  8369 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:02.004976  9907 consensus_meta.cc:287] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 1.020s	user 0.009s	sys 0.002s
I20260709 13:02:01.977517  9910 consensus_queue.cc:1048] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.068s
I20260709 13:02:02.070483  8406 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:02.072547  9906 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:02.073915  9762 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:02.075610  9935 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:02.062258  9991 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:02:02.035724  9945 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:02.061411  9882 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:01.947373  9836 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:02.098647  8398 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:01.943852  9961 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.102465  8404 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:02.185949 10000 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:02.117359  8220 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:02.118139  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:02.060032  9761 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:02.088708  9932 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:01.828877 10004 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.299s
W20260709 13:02:02.143751  8223 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.013s	sys 0.021s
I20260709 13:02:02.070626  9967 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:02.211422  8330 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:02.102063  9952 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:02.179126  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 115507us
I20260709 13:02:02.176275  8328 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:02.118399  9951 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.095463  9869 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:02.192842  8405 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:02:02.215955  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:02.100257  9007 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.030s	sys 0.031s
I20260709 13:02:02.201659  8402 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:02.186699  9950 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:02.202632  9932 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:02:02.209013  9803 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.006s	sys 0.033s
I20260709 13:02:02.181505  9907 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:02.205754  9899 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:02.206570  9989 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:02.206863  9903 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:02.207674  9943 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:02.208138  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 90856.6us
I20260709 13:02:02.223963  8400 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 4. (term mismatch)
I20260709 13:02:02.224404  8400 pending_rounds.cc:85] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 3
I20260709 13:02:02.224650  8400 pending_rounds.cc:107] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 2.4
I20260709 13:02:02.224964  8400 raft_consensus.cc:2917] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Aborting config change with OpId 2.4: Aborted: Op aborted by new leader
I20260709 13:02:02.192971  9762 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:02.226548  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:02.227118  9762 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:02.227221  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:02.228894  9907 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:02.230221  9882 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:02.230620  9882 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.230939  9882 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:02.232692  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:02.187111  9979 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:02.233709  9964 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.233733  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:02.234269  9964 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:02.234756  8400 raft_consensus.cc:2889] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:02.211053  9898 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:02.214939  8369 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:02.219553  8402 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:02.205035  9836 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:02.220007  9007 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.031s	sys 0.031s
I20260709 13:02:02.219159  9932 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:02.233160  8295 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:02.282878  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 49783.3us
I20260709 13:02:02.236167  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:02.285640  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:02.286387  8331 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:02:02.286542  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:02.209658  8223 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.013s	sys 0.021s
I20260709 13:02:02.235121  9979 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:02.284823  8295 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:02.215612  9951 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:02.263651  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 48843.8us
I20260709 13:02:02.293391  8369 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:02.295886  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:02.295187  8402 raft_consensus.cc:2393] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 2: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:02.215217  9952 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:02.298026  9979 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:02.255350  9803 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.836s	user 0.007s	sys 0.034s
I20260709 13:02:02.255605  8327 raft_consensus.cc:2393] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 3: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:02.305763  9803 raft_consensus.cc:697] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 4 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:02.299607  9964 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:02.313651  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:02.299814  9882 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:02.318398  9762 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:02.313651  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:02.318816  9869 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:02.260195  9821 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:02.256976  9853 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.282243  9910 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:02.293112  9971 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:02.342998  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:02.234800  9879 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:02.336196  9803 consensus_queue.cc:237] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:02.299866  9967 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:02.308781  9991 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:02.308974  9943 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:02.322933  9836 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.306087  9941 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:02:02.361930  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 122969us
I20260709 13:02:02.321023  9979 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:02.331367  9855 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:02.283973  9898 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:02.200294  9961 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:02.331614  9987 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:02.336894  9970 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.392500  8331 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:02.386842  9869 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:02.259634  9900 raft_consensus.cc:2955] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:02.345611  9907 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:02.345968  9971 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.413697  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:02.211159 10000 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:02.420161  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:02.394641 10022 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:02:02.423883  8295 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.922s (SENT) [suppressed 12 similar messages]
I20260709 13:02:02.363453  9943 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.429358  9943 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:02.363320  9991 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.389719  9910 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:02.409868  9869 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.352773  9762 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:02.348918  9853 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:02.452716  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:02.419255  9971 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:02.431226 10017 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:02.447894  9910 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:02.465157  9882 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:02.451900  9869 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:02.451956  9984 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:02.387079  9821 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:02.425402  9906 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:02.176833  8326 rpcz_store.cc:276] 0709 13:02:00.800234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.808056 (+  7822us) service_pool.cc:224] Handling call
0709 13:02:00.808417 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:00.808813 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:00.808847 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:00.809304 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 13:02:01.766239 (+956935us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:01.785069 (+ 18830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.827229 (+ 42160us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:01.937013 (+109784us) raft_consensus.cc:1685] finished
0709 13:02:01.938342 (+  1329us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.950646 (+ 12304us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":85,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"thread_start_us":768950,"threads_started":1,"wal-append.queue_time_us":1102736}
I20260709 13:02:02.466728  9882 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:02.477975  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:02.395951  9941 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:02.463416 10017 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:02.442287  9991 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:02.476405  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:02.415503  9979 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:02.393446  9836 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:02.501175  9882 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:02.503876  8332 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:02.425606  9907 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:02.416042  9970 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:02.537830 10014 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.522320  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1371) took 1089 ms. Trace:
I20260709 13:02:02.525450  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:02.539994  8331 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:02.546689  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 56537.8us
I20260709 13:02:02.535192  9821 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:02.539247  8255 rpcz_store.cc:276] 0709 13:02:01.432700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.432860 (+   160us) service_pool.cc:224] Handling call
0709 13:02:01.507266 (+ 74406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:01.639863 (+132597us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:01.827907 (+188044us) log.cc:844] Serialized 14 byte log entry
0709 13:02:02.197681 (+369774us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:02.197736 (+    55us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:02.197776 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.522013 (+324237us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54003471d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:02.522073 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:02.522245 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":329240,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":7034,"raft.queue_time_us":166454,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":40213,"spinlock_wait_cycles":304352512,"thread_start_us":508172,"threads_started":2,"wal-append.queue_time_us":599018}
I20260709 13:02:02.499732  9906 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:02.556396  8297 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:02.536666  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:02.560611  8297 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:02.567732  8297 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:02.569483  8297 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:02.422500 10022 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:02.595749  9956 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:02.557838  9762 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:02.594107  9879 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.596213  9956 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:02.570919  9808 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:02.611901  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 188043us
I20260709 13:02:02.616832  9879 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:02.594717  9910 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:02.614854  9956 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:02.598471  9898 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:02.566810  9978 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:02.573957  8400 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:02.623981  9970 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:02.623358  9907 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:02.687382  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 118023us
I20260709 13:02:02.554188 10014 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:02.623745  9951 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:02:02.696017  8297 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:02.709833  8331 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:02:02.682472 10013 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:02.711602  9964 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:02.695566  9898 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:02:02.704612  9808 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:02:02.718043  9853 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:02.743508  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:02.711983 10000 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:02.734512 10023 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:02:02.740857  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:02.746419  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:02.748639  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:02.752149  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:02.755846  8251 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:02.756938  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:02.757560  8248 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:02.746459  9803 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.745501  9970 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:02.761500  8400 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:02:02.745065  9979 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:02.752485  9961 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:02.753294  9879 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:02.749243 10031 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:02.748175 10022 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:02.749384  9882 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:02.769554  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 73630.5us
I20260709 13:02:02.788991  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:02.792881  8253 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:02:02.792176  8251 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:02.794193  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:02.794775  8251 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:02:02.795532  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:02.796123  8251 raft_consensus.cc:2368] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4.
I20260709 13:02:02.796603  8248 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
W20260709 13:02:02.797474  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 325543us
I20260709 13:02:02.769975 10028 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.785199  9970 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:02.800741 10016 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.800890  9970 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:02.801383  8248 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:02.778597  9941 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:02.792912 10020 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.186s	user 0.001s	sys 0.001s
I20260709 13:02:02.780084  9836 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:02.788748  9906 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:02.753082 10013 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:02.799737  8250 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:02.774820  9803 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:02.809237  8248 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:02.798892 10031 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:02.811383 10009 log.cc:927] Time spent T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 13:02:02.811529  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1365) took 1431 ms. Trace:
I20260709 13:02:02.799463  9853 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:02.801518 10017 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:02.816896  8254 rpcz_store.cc:276] 0709 13:02:01.380456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.381357 (+   901us) service_pool.cc:224] Handling call
0709 13:02:01.436607 (+ 55250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:01.578903 (+142296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.811299 (+1232396us) spinlock_profiling.cc:243] Waited 67.1 ms on lock 0x7b5400076cd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:02.811471 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":154221952}
W20260709 13:02:02.817514 10011 log.cc:927] Time spent T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.523s	user 0.005s	sys 0.000s
W20260709 13:02:02.832504  9952 consensus_meta.cc:287] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.503s	user 0.009s	sys 0.002s
I20260709 13:02:02.832921  9952 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:02.833063  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:02.833961  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 440525us
I20260709 13:02:02.825523  9853 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:02.835366  8329 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 1709) took 1674 ms (client timeout 1641 ms). Trace:
I20260709 13:02:02.814527  9912 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:02.800104  9991 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:02.802520 10016 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:02.817051 10031 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:02.818023  9901 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:02.832142  9932 consensus_meta.cc:287] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.531s	user 0.007s	sys 0.003s
W20260709 13:02:02.836261  8329 rpcz_store.cc:269] 0709 13:02:01.157663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.241787 (+ 84124us) service_pool.cc:224] Handling call
0709 13:02:02.831619 (+1589832us) spinlock_profiling.cc:243] Waited 75.4 ms on lock 0x7b54003c9454. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:02.831740 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":173486464}
I20260709 13:02:02.826356  9951 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:02.854810  8404 raft_consensus.cc:2368] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 2. Current term is 3.
I20260709 13:02:02.854079  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:02.856937  9971 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:02.929677  8250 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:02:02.972043  8253 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:02.973623  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:02.946300  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.002187  8254 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 6: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:02.813766 10028 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:02.854424  9932 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:03.002272 10013 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:03.115015 10013 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:03.115523 10013 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:03.117569 10017 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:03.117791  9867 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:03.118127 10017 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:03.108392  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.121812  8254 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:03.122884  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.123479  8254 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:02:03.124398  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:03.111272 10029 log.cc:927] Time spent T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:02:03.111591  9951 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:03.112452  9938 consensus_meta.cc:287] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.074s	user 0.012s	sys 0.001s
I20260709 13:02:03.125001  8254 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:03.072809  9991 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:03.150888  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.151698  8248 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:02:03.151129 10028 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:03.181919  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:03.146898  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.184775  8254 raft_consensus.cc:2368] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4.
I20260709 13:02:03.159265  9971 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:03.242583  9003 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56954 (request call id 331) took 1998 ms (client timeout 1922 ms). Trace:
I20260709 13:02:03.178719 10024 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:03.146652  9951 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:03.152599  9938 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:03.236047  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.266654  8248 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:03.227109  9979 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:02:03.261035 10016 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:03.272085  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.273883  8256 raft_consensus.cc:2368] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 2. Current term is 3.
I20260709 13:02:03.258796  9952 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:03.274703  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 325611us
I20260709 13:02:03.254554  8250 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:03.268210  9853 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:03.286510  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:03.275211  9983 consensus_meta.cc:287] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.098s	user 0.010s	sys 0.003s
I20260709 13:02:03.283257  9786 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:02:03.288594  9900 consensus_meta.cc:287] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.743s	user 0.008s	sys 0.004s
I20260709 13:02:03.289740  9932 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:03.290845  9786 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:03.296458  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.297060  8248 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:03.294891  9008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 338) took 1346 ms. Trace:
I20260709 13:02:03.297699  9008 rpcz_store.cc:276] 0709 13:02:01.948000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.949528 (+  1528us) service_pool.cc:224] Handling call
0709 13:02:01.949796 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:02.127884 (+178088us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:02.127917 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:02.127963 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:02.127991 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.294591 (+1166600us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b5400642b54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:03.294655 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.294816 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":409146240}
W20260709 13:02:03.263336  9003 rpcz_store.cc:269] 0709 13:02:01.259950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.297251 (+ 37301us) service_pool.cc:224] Handling call
0709 13:02:03.242217 (+1944966us) spinlock_profiling.cc:243] Waited 915 ms on lock 0x7b54004132d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:03.242449 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2103966592}
I20260709 13:02:03.291613  9983 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:03.304603  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.305286 10014 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:03.306667  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:03.306885  8405 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 198) took 1608 ms (client timeout 2018 ms). Trace:
I20260709 13:02:03.307433  9938 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:03.308151  8248 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
W20260709 13:02:03.309125  8398 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1679) took 1577 ms (client timeout 1874 ms). Trace:
I20260709 13:02:03.309471  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.310407  8248 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:03.311223 10031 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:03.311501  9761 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:03.315711  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1366) took 1934 ms. Trace:
I20260709 13:02:03.316215  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.316114  8252 rpcz_store.cc:276] 0709 13:02:01.381694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.381813 (+   119us) service_pool.cc:224] Handling call
0709 13:02:03.313339 (+1931526us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b5400346cd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:03.315647 (+  2308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3136375680}
I20260709 13:02:03.316964  8253 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:02:03.319518  9983 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:03.326833  9951 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:03.275257 10024 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:03.329128  8254 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:02:03.330461  8295 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:03.322371  9869 consensus_meta.cc:287] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.853s	user 0.011s	sys 0.000s
I20260709 13:02:03.324534  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 1418) took 1130 ms. Trace:
W20260709 13:02:03.307519  8405 rpcz_store.cc:269] 0709 13:02:01.698003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.701613 (+  3610us) service_pool.cc:224] Handling call
0709 13:02:03.306623 (+1605010us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b5400276854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:03.306775 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":847492480}
I20260709 13:02:03.332135  8295 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:03.335881  8404 rpcz_store.cc:276] 0709 13:02:02.194651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.209058 (+ 14407us) service_pool.cc:224] Handling call
0709 13:02:03.324234 (+1115176us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b54003a1bd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:03.324449 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1455133952}
I20260709 13:02:03.339314  8295 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:03.333770 10024 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:03.309700  8398 rpcz_store.cc:269] 0709 13:02:01.731319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.731465 (+   146us) service_pool.cc:224] Handling call
0709 13:02:03.308815 (+1577350us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b5400277c54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:03.309034 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":351197568}
I20260709 13:02:03.358119  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:03.358816  8398 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:03.359854  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:03.360987  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:03.340662  9979 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:03.361675  9979 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:03.362169  9979 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:03.343554  9845 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:03.365741  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1521) took 1029 ms. Trace:
I20260709 13:02:03.366164  8329 rpcz_store.cc:276] 0709 13:02:02.336280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.854394 (+518114us) service_pool.cc:224] Handling call
0709 13:02:02.911467 (+ 57073us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:02.911942 (+   475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:02.911981 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:03.017346 (+105365us) log.cc:844] Serialized 26 byte log entry
0709 13:02:03.261938 (+244592us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:03.261995 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.262093 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:03.365508 (+103415us) raft_consensus.cc:1685] finished
0709 13:02:03.365534 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.365682 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":548,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":24863,"thread_start_us":150777,"threads_started":1,"wal-append.queue_time_us":177827}
I20260709 13:02:03.338356  9964 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:03.373308  9964 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.373755  9964 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:03.375294  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:03.376020  8405 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:03.345142  9956 consensus_meta.cc:287] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.671s	user 0.010s	sys 0.001s
I20260709 13:02:03.346916  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:03.351639  8400 consensus_meta.cc:287] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.560s	user 0.009s	sys 0.000s
I20260709 13:02:03.338403  9869 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:03.353718  9786 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:03.354111  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1522) took 1017 ms. Trace:
W20260709 13:02:03.355444  9977 log.cc:927] Time spent T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 1.919s	user 0.002s	sys 0.013s
I20260709 13:02:03.375874  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.386444  8255 raft_consensus.cc:2376] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4. Re-sending same reply.
I20260709 13:02:03.386755  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:03.343839  9943 consensus_meta.cc:287] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.913s	user 0.011s	sys 0.000s
I20260709 13:02:03.313271 10031 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:03.382270  8403 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1684) took 1452 ms (client timeout 1928 ms). Trace:
I20260709 13:02:03.387706  8398 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:03.385560  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:03.385596  8332 rpcz_store.cc:276] 0709 13:02:02.336495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.115575 (+779080us) service_pool.cc:224] Handling call
0709 13:02:03.115887 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:03.116086 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:03.116111 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:03.116549 (+   438us) log.cc:844] Serialized 26 byte log entry
0709 13:02:03.248355 (+131806us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:03.248417 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.248513 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:03.353901 (+105388us) raft_consensus.cc:1685] finished
0709 13:02:03.353920 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.354051 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":131414,"threads_started":1,"wal-append.queue_time_us":171430}
I20260709 13:02:03.392549  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:03.393457  8398 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:03.391026  8403 rpcz_store.cc:269] 0709 13:02:01.929756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.980056 (+ 50300us) service_pool.cc:224] Handling call
0709 13:02:03.346713 (+1366657us) spinlock_profiling.cc:243] Waited 1.64 ms on lock 0x7b5800170798. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:02:03.382189 (+ 35476us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3775872}
I20260709 13:02:03.400641  8252 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:02:03.400840  8398 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:03.395426  8405 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:03.400931  8255 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 13:02:03.391101  9970 consensus_meta.cc:287] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.585s	user 0.010s	sys 0.002s
I20260709 13:02:03.385623  9903 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:03.405552  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:03.403848  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:03.406553  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:03.404412  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:03.416090  8255 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:03.417444  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:03.417449  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:03.386010  8400 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:03.419122  8403 raft_consensus.cc:2410] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 4 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:02:03.420658  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:03.418181  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 1423) took 1066 ms. Trace:
I20260709 13:02:03.425515  8404 rpcz_store.cc:276] 0709 13:02:02.351558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.403250 (+1051692us) service_pool.cc:224] Handling call
0709 13:02:03.418115 (+ 14865us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59008}
I20260709 13:02:03.385800  9956 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:03.439018  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:03.439692  8404 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:02:03.446909  8255 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:03.449250  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1422) took 1122 ms. Trace:
I20260709 13:02:03.449972  8398 rpcz_store.cc:276] 0709 13:02:02.326575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.448071 (+1121496us) service_pool.cc:224] Handling call
0709 13:02:03.448566 (+   495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.448826 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:03.448864 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:03.448927 (+    63us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:03.448973 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.449050 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.449180 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:03.433876  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1420) took 1216 ms. Trace:
I20260709 13:02:03.452448  8405 rpcz_store.cc:276] 0709 13:02:02.217673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.432802 (+1215129us) service_pool.cc:224] Handling call
0709 13:02:03.433129 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.433365 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:03.433406 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:03.433501 (+    95us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:03.433548 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.433652 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.433807 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:03.412334  9903 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:03.456333  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1424) took 1061 ms. Trace:
I20260709 13:02:03.456693  8404 rpcz_store.cc:276] 0709 13:02:02.395144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.440300 (+1045156us) service_pool.cc:224] Handling call
0709 13:02:03.455835 (+ 15535us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.456029 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:03.456056 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:03.456097 (+    41us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:03.456125 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.456180 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.456283 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:03.459172  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:03.397395  9943 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:02:03.460811  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:39294 (request call id 81) took 1138 ms. Trace:
I20260709 13:02:03.464150  8403 rpcz_store.cc:276] 0709 13:02:02.320429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.440400 (+1119971us) service_pool.cc:224] Handling call
0709 13:02:03.457961 (+ 17561us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:03.414052  9970 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:03.456782  9903 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:03.468334  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:03.470333  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:03.456190  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:03.472460  9009 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
W20260709 13:02:03.491611 10048 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:02:03.494374  9943 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:03.498777  9956 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:03.500305  9869 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:03.507663  8298 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:03.509712  9963 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:03.510736 10013 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:03.512475  9963 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:03.512888  9963 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:03.513265  9963 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:03.514134  9762 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:03.514649  9762 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:03.516782  9762 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:03.517231  9762 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:03.532405  9963 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:03.535862  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1519) took 1200 ms. Trace:
I20260709 13:02:03.538282  8331 rpcz_store.cc:276] 0709 13:02:02.335809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.720462 (+384653us) service_pool.cc:224] Handling call
0709 13:02:03.369250 (+648788us) spinlock_profiling.cc:243] Waited 1.04 ms on lock 0x7b58000e3498. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:02:03.369501 (+   251us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:03.369923 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:03.369959 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:03.370494 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 13:02:03.429979 (+ 59485us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:03.430041 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.430132 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:03.535428 (+105296us) raft_consensus.cc:1685] finished
0709 13:02:03.535456 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.535768 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":122,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":138,"spinlock_wait_cycles":2387328,"thread_start_us":59048,"threads_started":1,"wal-append.queue_time_us":162897}
I20260709 13:02:03.506356 10017 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:03.541307  9762 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:03.548375  9762 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:03.550747  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:03.566164  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:03.553165  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1523) took 1190 ms. Trace:
I20260709 13:02:03.536619  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:03.550861 10008 log.cc:927] Time spent T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 1.348s	user 0.000s	sys 0.014s
I20260709 13:02:03.556157  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:03.591964  9009 raft_consensus.cc:2368] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4.
W20260709 13:02:03.616909  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 62433us
I20260709 13:02:03.631342  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:03.541815  9964 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:03.631057  9786 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:03.614553  9005 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:02:03.638622  9786 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:03.644688  9786 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:03.645680  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:03.646790  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:03.647867  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:03.648430  9008 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 2 LEARNER]: Advancing to term 4
I20260709 13:02:03.648315  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:03.649565  8331 raft_consensus.cc:2393] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 3: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:03.650270  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:03.637799  8401 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1682) took 1749 ms (client timeout 1518 ms). Trace:
W20260709 13:02:03.652184  8401 rpcz_store.cc:269] 0709 13:02:01.888085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.942620 (+ 54535us) service_pool.cc:224] Handling call
0709 13:02:03.536343 (+1593723us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b5800170798. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:02:03.636184 (+ 99841us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6670720}
I20260709 13:02:03.653120  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1494) took 2827 ms. Trace:
I20260709 13:02:03.653133  9964 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:03.653573  8330 rpcz_store.cc:276] 0709 13:02:00.824229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.824419 (+   190us) service_pool.cc:224] Handling call
0709 13:02:00.824695 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:02.213631 (+1388936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.651482 (+1437851us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b540037cb54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:03.651615 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3189051776}
I20260709 13:02:03.654114  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.655717  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.656267  8248 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
I20260709 13:02:03.658305  8972 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:03.659193  9008 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 4 LEARNER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:03.659363  8972 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:03.660090  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:03.660593  8972 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:03.662115  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:03.664104  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:03.664225  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:03.665164  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.640947  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:03.666728  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:03.667547  8330 raft_consensus.cc:1240] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:03.668663  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.669255  9980 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:03.662510  9762 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:03.670593  9833 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:03.671350  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:03.694816  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:03.673893  9903 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:03.699666  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.675535  9963 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:03.720707  8332 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:03.684975  9991 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.677728  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.687572  9952 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.729019  8297 leader_election.cc:400] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 2. Current term is 3.
I20260709 13:02:03.683948  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:03.686754  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:03.729349  8297 leader_election.cc:403] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:03.729766  9009 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:03.729955  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 52186.5us
W20260709 13:02:03.732926  8141 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ae82ced6ff014453bbcfd5f9f14b1f8d with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica e64761f43e4240319c41117cf886d22f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:03.677707  9906 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:03.687938  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.688560  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:03.689502  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:03.734067  9008 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:03.691598  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:03.695632  8404 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:02:03.734328  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:03.734678  8403 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:03.735206  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:03.735275  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:03.736157  8404 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 6: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:03.736629  9833 raft_consensus.cc:697] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:03.736892  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:03.698098  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 188949us
I20260709 13:02:03.666179  9902 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:03.708525  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:03.737625  9833 consensus_queue.cc:237] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:03.738211  9902 raft_consensus.cc:697] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:03.739058  9902 consensus_queue.cc:237] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:03.708755 10060 log.cc:927] Time spent T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:02:03.709075 10004 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:03.710369  9903 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:03.693714  9001 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:02:03.742524  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:03.592190  8326 rpcz_store.cc:276] 0709 13:02:02.363272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.254892 (+891620us) service_pool.cc:224] Handling call
0709 13:02:03.266479 (+ 11587us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:03.293026 (+ 26547us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:03.295682 (+  2656us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:03.383439 (+ 87757us) log.cc:844] Serialized 26 byte log entry
0709 13:02:03.486489 (+103050us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:03.493180 (+  6691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.493258 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:03.552890 (+ 59632us) raft_consensus.cc:1685] finished
0709 13:02:03.552920 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.553090 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8364,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":3631,"thread_start_us":80366,"threads_started":1,"wal-append.queue_time_us":167373}
I20260709 13:02:03.711781  8249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1372) took 2273 ms. Trace:
I20260709 13:02:03.743628  8249 rpcz_store.cc:276] 0709 13:02:01.438317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.438475 (+   158us) service_pool.cc:224] Handling call
0709 13:02:03.274201 (+1835726us) spinlock_profiling.cc:243] Waited 24.1 ms on lock 0x7b5800002a98. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:02:03.288361 (+ 14160us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.288558 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:03.394550 (+105992us) log.cc:844] Serialized 14 byte log entry
0709 13:02:03.484449 (+ 89899us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:03.484500 (+    51us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:03.484535 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.485491 (+   956us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.711605 (+226114us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b54002ee350. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:03.711705 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":174759,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":1187,"raft.queue_time_us":2002,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"spinlock_wait_cycles":87515264,"thread_start_us":191912,"threads_started":2,"wal-append.queue_time_us":149947}
I20260709 13:02:03.745667  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.746153  8249 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:03.746841  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:03.747275  8249 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:03.711781  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:03.748247  8331 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 1.
W20260709 13:02:03.713173  8327 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 1509) took 2216 ms (client timeout 2539 ms). Trace:
W20260709 13:02:03.715663  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 415119us
I20260709 13:02:03.717680  9970 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:03.750276  8403 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:03.751185  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:03.751698  8403 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:02:03.752481  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:03.677798  8328 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 1711) took 2519 ms (client timeout 1874 ms). Trace:
I20260709 13:02:03.753618  9985 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:02:03.753571  8328 rpcz_store.cc:269] 0709 13:02:01.158297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.302433 (+144136us) service_pool.cc:224] Handling call
0709 13:02:03.677615 (+2375182us) spinlock_profiling.cc:243] Waited 844 ms on lock 0x7b5400090bd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:03.677711 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1940139776}
I20260709 13:02:03.754235  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:03.707772  9979 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:03.748981  8327 rpcz_store.cc:269] 0709 13:02:01.497281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.504463 (+  7182us) service_pool.cc:224] Handling call
0709 13:02:03.712983 (+2208520us) spinlock_profiling.cc:243] Waited 495 ms on lock 0x7b540020ec54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:03.713114 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1137497472}
I20260709 13:02:03.755406  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:03.756384  9979 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:03.756637  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:03.756534  9991 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:03.756680  8222 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:03.759099  8222 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:03.759187  9991 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:03.725040  8257 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:03.759853  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:03.761050  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:03.707471  9762 raft_consensus.cc:697] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:03.761551  9786 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:03.762022  9786 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:03.762445  9786 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:03.762148  9762 consensus_queue.cc:237] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:03.693790  9007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 436) took 2340 ms. Trace:
I20260709 13:02:03.728682  8329 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:03.733752  9906 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:03.702270  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:03.743158  9001 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:03.759935  8222 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:03.765535  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 44852.7us
I20260709 13:02:03.765743  8406 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:02:03.771202  8222 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:03.771368  8297 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:03.735445  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.766288  9903 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:02:03.785612 10004 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:03.785688  9989 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:03.786545 10004 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:03.760063  9900 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:03.789968  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:03.790057  9900 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:03.792546  8369 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:03.792642  9900 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:03.793593 10005 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:03.767999  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.794508 10005 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:03.794847 10005 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:03.795110  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:03.795267  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.795785  8257 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:03.796798  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:03.743144  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:03.797578  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:03.798308  9943 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:03.763963  9007 rpcz_store.cc:276] 0709 13:02:01.344870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.348021 (+  3151us) service_pool.cc:224] Handling call
0709 13:02:01.372762 (+ 24741us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:01.380938 (+  8176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:01.380969 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:01.418513 (+ 37544us) log.cc:844] Serialized 26 byte log entry
0709 13:02:02.292071 (+873558us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:02.292119 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.292189 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:03.267503 (+975314us) spinlock_profiling.cc:243] Waited 4.72 ms on lock 0x7b180007f4e0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d1522 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:02:03.550344 (+282841us) spinlock_profiling.cc:243] Waited 6.65 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:03.685311 (+134967us) raft_consensus.cc:1685] finished
0709 13:02:03.685333 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.685470 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":128608,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":70096,"spinlock_wait_cycles":26162816,"thread_start_us":708701,"threads_started":1,"wal-append.queue_time_us":726951}
I20260709 13:02:03.798738  9943 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:03.799152  9943 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:03.799218  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:03.801016  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:03.801542  9869 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:03.802806  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.803865  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:03.804352  8248 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:03.795194 10005 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:03.769819  9867 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.772109  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:03.806836  8369 leader_election.cc:400] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4.
I20260709 13:02:03.807185  8369 leader_election.cc:403] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:02:03.753291  8403 raft_consensus.cc:2393] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 3: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:03.808755  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:03.810195  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:03.767635  8249 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:03.772719  8222 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:03.756418  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:03.774212 10017 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:03.769649 10024 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:03.769783  9952 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:02:03.778010  9009 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:03.779177  9906 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:03.779714  9980 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:03.780222  8405 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
I20260709 13:02:03.793524  9912 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:03.798211  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:03.755398  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1419) took 1543 ms. Trace:
I20260709 13:02:03.754289  9985 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:03.770622  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 68430.9us
I20260709 13:02:03.771136  8372 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:03.814287  8257 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:03.815135  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:03.831980  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.832392  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:03.821501  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.820470  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:03.815100  8222 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:03.823273  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:03.824822  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:03.833417  8972 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:03.825372  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 54117.9us
I20260709 13:02:03.826924  8298 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:03.834666  8297 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:03.834822  8369 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:03.836397  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:03.837116  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:03.837937  8403 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:02:03.838940  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:03.829217  8248 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:03.840773  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:03.841579  8248 raft_consensus.cc:2393] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 4: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:03.842480  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:03.846084  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 75029us
I20260709 13:02:03.848469  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:03.850399  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:03.850512  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:03.850973  8255 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:03.851240  8372 leader_election.cc:400] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4.
I20260709 13:02:03.849587  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.854142  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.820422 10016 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:03.806063  9867 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:03.818073 10004 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:03.818161  9902 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:03.856594  8972 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:03.820830  9991 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:03.865203  9952 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:03.999886  9910 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:03.936012  8251 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:03.984437  9985 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:04.001926  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:04.002189  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1420) took 1247 ms. Trace:
I20260709 13:02:04.002808  8251 rpcz_store.cc:276] 0709 13:02:02.754884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.799727 (+ 44843us) service_pool.cc:224] Handling call
0709 13:02:03.935776 (+1136049us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.000711 (+ 64935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.002027 (+  1316us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400406850. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:04.002138 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2612114688}
I20260709 13:02:04.003310  9766 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:04.004069  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:04.003908  9766 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:03.861938  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:04.005611  9950 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.005847 10005 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:04.006116  9950 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:04.006491  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:04.007591  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:04.008224  9950 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:04.008589  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:03.947309  8256 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1377) took 2430 ms (client timeout 1574 ms). Trace:
I20260709 13:02:03.897411  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:04.010250  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 153730us
I20260709 13:02:04.010816  8331 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
W20260709 13:02:03.893916  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 66291us
I20260709 13:02:03.945211  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:03.991262  8974 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:03.898708  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:03.934303  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.933892  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:03.916415  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:03.824640  9989 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:03.898039  8329 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:03.900702  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:04.015992  8330 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:04.016446  8972 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:03.868920  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1431) took 1034 ms. Trace:
I20260709 13:02:04.019032  8252 rpcz_store.cc:276] 0709 13:02:02.834128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.317190 (+483062us) service_pool.cc:224] Handling call
0709 13:02:03.334868 (+ 17678us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:03.401237 (+ 66369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.868603 (+467366us) spinlock_profiling.cc:243] Waited 65.5 ms on lock 0x7b540026c854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:03.868847 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":150643584}
W20260709 13:02:03.998654  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 143451us
W20260709 13:02:03.894594  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 59998.2us
I20260709 13:02:04.018211  8257 raft_consensus.cc:2393] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 2: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:03.947315  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:04.015964  8974 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:04.013332  8249 raft_consensus.cc:2376] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3. Re-sending same reply.
W20260709 13:02:04.009685  8256 rpcz_store.cc:269] 0709 13:02:01.505599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.602682 (+ 97083us) service_pool.cc:224] Handling call
0709 13:02:03.935609 (+2332927us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b5400406854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:03.935817 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3796084480}
I20260709 13:02:04.030316  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:04.031448  8256 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
W20260709 13:02:03.829576  9000 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.626s	user 0.014s	sys 0.014s
W20260709 13:02:03.899238  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 72414us
I20260709 13:02:03.848156  9833 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:03.830896  9898 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:04.041049  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:03.926187 10031 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:03.898414  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:03.981846  9903 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 2. Current term is 3.
I20260709 13:02:03.852365  9906 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:03.896947  9808 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.008203  9786 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:03.856179  9845 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:03.844635 10024 raft_consensus.cc:697] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 4 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:03.849629  9932 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:04.050966  8252 raft_consensus.cc:2368] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 2. Current term is 3.
I20260709 13:02:04.040565  8295 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:03.867341  9943 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:03.896030  9803 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:03.932814  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:04.105353  8326 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:04.106293  9943 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:04.034636  9007 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:03.950717  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:04.037163  8369 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:03.843160  8403 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
W20260709 13:02:04.110646  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 73574.1us
I20260709 13:02:04.111063  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:03.848347  8250 raft_consensus.cc:2393] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 3: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:04.111524  8369 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:04.112241  9811 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:03.860625  9944 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:03.968564  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:04.048331  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:03.867486  9945 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:04.011062  9991 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:04.038760  9901 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:03.857750  9907 consensus_queue.cc:1059] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.201s
I20260709 13:02:04.081660  8404 raft_consensus.cc:2376] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2. Re-sending same reply.
I20260709 13:02:04.021173  8255 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:04.050664  9766 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:04.154129  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 97737.4us
I20260709 13:02:04.005178  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:03.865504  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:04.158704  8251 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 6: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:02:04.099785  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 284741us
I20260709 13:02:04.100405  8405 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:04.159511  8222 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:04.159649  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:04.100405  8330 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:04.032030 10016 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.858141  9936 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.223s
I20260709 13:02:04.029951  9980 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:04.049860  9000 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.846s	user 0.015s	sys 0.015s
I20260709 13:02:03.849413  9912 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:04.049964  9833 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.148942  8252 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:04.053670  9898 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:04.112715  9811 raft_consensus.cc:697] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:04.151835  9762 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:04.152290 10035 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:04.115900  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 75430us
I20260709 13:02:04.177608  8295 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:04.103680 10031 raft_consensus.cc:697] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:04.160553  8222 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:04.038826 10063 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:02:04.165063  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:04.152505  9786 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:04.168388  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:04.168653  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:04.170528  8330 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:04.182874  8405 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:04.184379  8331 raft_consensus.cc:2393] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 3: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:04.184754  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:04.179647 10035 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.182094  9980 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:04.179257 10005 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:04.180011  9766 raft_consensus.cc:3055] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:04.188692  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:04.189122  9766 raft_consensus.cc:740] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:04.189944  9766 consensus_queue.cc:260] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:04.190814  9867 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:04.190989  9766 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:04.192050  9811 consensus_queue.cc:237] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:04.181866  9761 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:04.194581  8323 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:04.194178 10024 consensus_queue.cc:237] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:04.158310  8328 consensus_queue.cc:237] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:04.102286  9845 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.102839  9906 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.178721  8295 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:04.152889  9855 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:03.868266 10017 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:04.154666  9991 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:04.145747  9944 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:04.114259  9856 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:04.187844 10035 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:04.185235  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:04.225016  9989 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4.
I20260709 13:02:04.165472 10026 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:02:04.211654  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 195285us
I20260709 13:02:04.211966  8295 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:04.208626  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 48161.2us
I20260709 13:02:04.241328  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:04.241987  8331 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:04.178056  9833 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:04.174280  9901 raft_consensus.cc:697] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:04.218725  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:04.215446  9970 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:04.114566  9808 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:02:04.219434  9856 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:04.220213  9944 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:04.221645  9855 raft_consensus.cc:697] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:04.222615  8326 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:04.217541 10017 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:04.223338  9000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 335) took 2539 ms. Trace:
I20260709 13:02:04.161942 10004 consensus_queue.cc:1059] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:02:04.180809  9898 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:04.265192  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:04.170804  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 154930us
I20260709 13:02:04.225327  8405 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:04.267062  8974 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:04.161913 10016 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:04.032521  9902 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:04.242195  8222 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:04.242880  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:04.269291  8972 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:04.269795  8222 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:04.242107  8295 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:04.244585  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1408) took 2402 ms. Trace:
I20260709 13:02:04.270485  8974 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:04.270709  8399 rpcz_store.cc:276] 0709 13:02:01.835529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.125363 (+289834us) service_pool.cc:224] Handling call
0709 13:02:02.181156 (+ 55793us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.373611 (+1192455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:03.373643 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:03.373688 (+    45us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:03.373718 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.237526 (+863808us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400267fd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:04.237574 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.237695 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2742086400}
I20260709 13:02:04.214562  9906 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:04.214658  9845 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:04.252903  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:04.272794  9970 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:04.198836  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:04.247200  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1513) took 1912 ms. Trace:
I20260709 13:02:04.282665  8325 rpcz_store.cc:276] 0709 13:02:02.334412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.532802 (+198390us) service_pool.cc:224] Handling call
0709 13:02:02.832021 (+299219us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:02.848452 (+ 16431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:02.848499 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:03.004242 (+155743us) log.cc:844] Serialized 26 byte log entry
0709 13:02:03.260686 (+256444us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:03.260728 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.260811 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:03.305973 (+ 45162us) raft_consensus.cc:1685] finished
0709 13:02:03.305992 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.237913 (+931921us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b54003c9450. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:04.247134 (+  9221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6832,"raft.queue_time_us":7008,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":7669,"spinlock_wait_cycles":686996864,"thread_start_us":187289,"threads_started":1,"wal-append.queue_time_us":233287}
I20260709 13:02:04.217072  9786 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:04.284587  9786 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:04.285041  9786 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:04.286918  9808 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:04.289312  8372 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:04.290974  8223 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:04.293640 10072 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:04.296185  9901 consensus_queue.cc:237] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:04.259218  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:04.297605  8331 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:04.300427  9004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 336) took 2569 ms. Trace:
I20260709 13:02:04.255502 10026 raft_consensus.cc:697] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:04.300897  9004 rpcz_store.cc:276] 0709 13:02:01.692930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.056135 (+363205us) service_pool.cc:224] Handling call
0709 13:02:02.056905 (+   770us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:02.225029 (+168124us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:02.394285 (+169256us) log.cc:844] Serialized 14 byte log entry
0709 13:02:02.690994 (+296709us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:02.695403 (+  4409us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:02.707413 (+ 12010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.356962 (+649549us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b18003beb40. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882569f2a6 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1
0709 13:02:03.358338 (+  1376us) spinlock_profiling.cc:243] Waited 5.38 ms on lock 0x7b18003beb40. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:02:03.645141 (+286803us) spinlock_profiling.cc:243] Waited 28.3 ms on lock 0x7b18003beb40. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d1522 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:02:04.244185 (+599044us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b540057d5d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:04.247325 (+  3140us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.261978 (+ 14653us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":462292,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":4254,"raft.queue_time_us":4060,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":4380,"spinlock_wait_cycles":470429440,"thread_start_us":409287,"threads_started":2,"wal-append.queue_time_us":330502}
I20260709 13:02:04.302245 10026 consensus_queue.cc:237] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:04.264285  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 150889us
I20260709 13:02:04.305783  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1444) took 1152 ms. Trace:
I20260709 13:02:04.306142  8254 rpcz_store.cc:276] 0709 13:02:03.153143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.265001 (+111858us) service_pool.cc:224] Handling call
0709 13:02:04.305596 (+1040595us) spinlock_profiling.cc:243] Waited 56.3 ms on lock 0x7b54003fd754. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:04.305715 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":129463552}
I20260709 13:02:04.227684  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:04.250135  9900 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:04.309015  9900 raft_consensus.cc:697] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:04.309506  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:04.310024  9900 consensus_queue.cc:237] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:04.311416  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:04.312206 10077 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:04.312994 10078 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:04.272176  8295 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:04.313496 10078 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:04.313920 10078 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:04.314393  8295 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:04.315330  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:04.316182  8295 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:03.827847  8402 rpcz_store.cc:276] 0709 13:02:02.211577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.334462 (+122885us) service_pool.cc:224] Handling call
0709 13:02:02.334761 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:02.334953 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:02.678130 (+343177us) log.cc:844] Serialized 14 byte log entry
0709 13:02:03.755085 (+1076955us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:03.755135 (+    50us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:03.755178 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.755237 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.755338 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1002230,"prepare.queue_time_us":266163,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":1039895,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":81,"thread_start_us":369178,"threads_started":2,"wal-append.queue_time_us":108526,"wal-append.run_cpu_time_us":1387,"wal-append.run_wall_time_us":1032466}
I20260709 13:02:04.293036  8223 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:04.263625  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:04.246656  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:04.278456  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:04.296988  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.302543  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1432) took 1505 ms. Trace:
I20260709 13:02:04.310086  9855 consensus_queue.cc:237] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:04.267163  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:04.359030  8405 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:04.341995  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:04.322202  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 51759.6us
I20260709 13:02:04.262408 10017 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:04.350952  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 58020.8us
I20260709 13:02:04.356923  8398 rpcz_store.cc:276] 0709 13:02:02.796817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.451625 (+654808us) service_pool.cc:224] Handling call
0709 13:02:03.452035 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:03.697075 (+245040us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:03.697117 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:03.697585 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 13:02:03.954386 (+256801us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:04.107411 (+153025us) log.cc:844] Serialized 14 byte log entry
0709 13:02:04.107629 (+   218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.302077 (+194448us) spinlock_profiling.cc:243] Waited 5.82 ms on lock 0x7b54000fc3d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:04.302141 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:04.302203 (+    62us) raft_consensus.cc:1685] finished
0709 13:02:04.302298 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.302474 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":287969,"prepare.queue_time_us":224610,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":240057,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":5531,"spinlock_wait_cycles":13375872,"thread_start_us":369672,"threads_started":2,"wal-append.queue_time_us":344904}
I20260709 13:02:04.351084  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:04.275040  9985 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:02:04.319216  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.352177  8328 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:04.351536  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:04.317382  8295 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:04.254649  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1432) took 1404 ms. Trace:
W20260709 13:02:04.373378  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 56072.1us
I20260709 13:02:04.279912  8222 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:04.364019  8223 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:04.373700  8253 rpcz_store.cc:276] 0709 13:02:02.850628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.317633 (+467005us) service_pool.cc:224] Handling call
0709 13:02:03.985664 (+668031us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b5800002a98. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:02:03.985885 (+   221us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:03.986273 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:03.986305 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:03.986808 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 13:02:04.230999 (+244191us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:04.231038 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.231105 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:04.231145 (+    40us) raft_consensus.cc:1685] finished
0709 13:02:04.231157 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.254565 (+ 23408us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5366,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":115,"spinlock_wait_cycles":38965632,"thread_start_us":243824,"threads_started":1,"wal-append.queue_time_us":239808}
I20260709 13:02:04.378993  8972 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:04.379249  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 73428.6us
I20260709 13:02:04.226557 10031 consensus_queue.cc:237] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:04.369815  9803 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:04.379308  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 90083.3us
I20260709 13:02:04.380517  8398 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 2. Current term is 3. Ops: []
W20260709 13:02:04.380890  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.381409  8295 leader_election.cc:400] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 2. Current term is 3.
I20260709 13:02:04.381748  8295 leader_election.cc:403] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:02:04.382460  8295 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:04.383677  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:04.315361  9898 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:04.385411  9945 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:04.386008  9945 raft_consensus.cc:697] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:04.337464  8397 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:04.387156  9903 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.368032  8331 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:04.387797  9903 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:04.391716  8223 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:04.382400  8372 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:04.393926  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:04.383365  8398 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:04.399117  8398 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:04.278321 10013 raft_consensus.cc:2764] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:02:04.274859 10063 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:04.278530  9989 raft_consensus.cc:3055] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:04.322392  9912 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:04.314437  9979 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:04.407629  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:04.403699  8399 raft_consensus.cc:2410] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:04.406459  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:04.408972  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:04.338966  9856 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:04.377892  9964 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:04.335543  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:04.566828  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:04.390880  9803 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.322773  9906 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:04.460444  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:04.318102  9971 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:04.408273  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:04.317615  9970 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:04.523396  8325 raft_consensus.cc:2393] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 5: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:04.508453  9963 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:04.514137  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:04.370373  9985 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:04.322407  9936 consensus_queue.cc:1059] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.187s
I20260709 13:02:04.413234  9910 consensus_queue.cc:1059] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.167s
I20260709 13:02:04.610824  8404 raft_consensus.cc:2368] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5.
I20260709 13:02:04.383637  9784 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:04.572955  9803 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:04.508128 10073 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:04.521313  8405 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
W20260709 13:02:04.574162  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 63538.3us
I20260709 13:02:04.615670  8402 raft_consensus.cc:1240] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:04.577952  8323 raft_consensus.cc:2410] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 5 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
W20260709 13:02:04.583091  8397 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 201) took 2856 ms (client timeout 2143 ms). Trace:
I20260709 13:02:04.578831  9971 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.510890  9989 raft_consensus.cc:740] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:02:04.412770  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 53125.2us
I20260709 13:02:04.514981  9957 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:04.412067 10072 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:04.412369  9980 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:04.384047  9935 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.189s
I20260709 13:02:04.402832 10083 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:04.595187  9902 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:04.596462  9903 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:04.570840 10063 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:04.605958  9985 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:04.411269  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:04.686322  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:04.390172  9808 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 2. Current term is 3.
I20260709 13:02:04.568068  9766 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:04.521910  9990 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:02:04.697516  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 417618us
W20260709 13:02:04.616772  8397 rpcz_store.cc:269] 0709 13:02:01.720524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.720646 (+   122us) service_pool.cc:224] Handling call
0709 13:02:03.947148 (+2226502us) spinlock_profiling.cc:243] Waited 1.68 ms on lock 0x7b5800170798. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:02:04.576276 (+629128us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b5400067f54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:04.576397 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":516670592}
I20260709 13:02:04.700714  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:04.701809  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:04.702761  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:04.622349 10073 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:04.637537  9936 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:04.638664  9906 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:04.640429  9898 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:04.642302  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:04.743083  8323 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:04.643029  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:04.687618 10063 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:04.644451  9991 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:04.647034  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:04.591579 10080 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:02:04.756280  9991 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:04.757233  8405 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:04.611110  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:04.720209 10073 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:04.634214  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:04.386940  9945 consensus_queue.cc:237] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:04.669672  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:04.616355 10013 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:04.629678  9971 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:04.690930  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 299569us
W20260709 13:02:04.699263  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 307610us
I20260709 13:02:04.767614  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:04.697139  8297 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:04.768276  8323 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:04.768512  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:04.768648  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 71602.3us
I20260709 13:02:04.667514 10083 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:04.770084  8297 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:02:04.771219  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.731647  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:04.650800  9856 consensus_queue.cc:1059] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:02:04.744446  8332 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:04.643711  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:04.777561  8328 raft_consensus.cc:2393] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 3: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:04.625581  9784 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:04.753314  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:04.663867  9957 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:04.759357  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 296222us
I20260709 13:02:04.697724  9990 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:04.648684  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:04.759689  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:04.737623  9980 raft_consensus.cc:3055] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:04.764556  8325 raft_consensus.cc:2393] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 2: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:04.261209  9000 rpcz_store.cc:276] 0709 13:02:01.683830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.684917 (+  1087us) service_pool.cc:224] Handling call
0709 13:02:02.054234 (+369317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:02.088543 (+ 34309us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:02.090542 (+  1999us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:02.192574 (+102032us) log.cc:844] Serialized 26 byte log entry
0709 13:02:03.718786 (+1526212us) spinlock_profiling.cc:243] Waited 7.95 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f88256b40ac 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b
0709 13:02:04.182859 (+464073us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:04.202597 (+ 19738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.216529 (+ 13932us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:04.223038 (+  6509us) raft_consensus.cc:1685] finished
0709 13:02:04.223076 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.223283 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":115471,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"spinlock_wait_cycles":18283776,"thread_start_us":1971559,"threads_started":1,"wal-append.queue_time_us":1783360}
I20260709 13:02:04.745594 10072 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.775079  8220 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:04.753796 10063 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:04.715145  9766 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:04.971167  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 214) took 1249 ms. Trace:
W20260709 13:02:05.143584  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 392792us
W20260709 13:02:04.946676  8972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.027s	sys 0.048s
I20260709 13:02:05.069748  8151 catalog_manager.cc:5697] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 1 to 2, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 2 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:05.238528  8972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.836s	user 0.034s	sys 0.090s
W20260709 13:02:04.814420  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 44467.2us
I20260709 13:02:04.966164  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 1443) took 1068 ms. Trace:
I20260709 13:02:04.787983  8330 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:05.245396  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 843356us
I20260709 13:02:05.067250  8153 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45448 (request call id 281) took 6103 ms. Trace:
I20260709 13:02:05.249570  8402 rpcz_store.cc:276] 0709 13:02:03.721540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.390479 (+668939us) service_pool.cc:224] Handling call
0709 13:02:04.615420 (+224941us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.616273 (+   853us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.970897 (+354624us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b54003e6550. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:04.971088 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":516688640}
I20260709 13:02:05.245504  8151 catalog_manager.cc:5697] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 1 to 2, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 2 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:05.069239  8152 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55754 (request call id 196) took 6197 ms. Trace:
W20260709 13:02:05.251026  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 476063us
W20260709 13:02:05.250257  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:05.246464  8404 rpcz_store.cc:276] 0709 13:02:03.898241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.309233 (+410992us) service_pool.cc:224] Handling call
0709 13:02:04.965910 (+656677us) spinlock_profiling.cc:243] Waited 300 ms on lock 0x7b5400277c54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:04.966087 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":691004160}
I20260709 13:02:04.722142  9989 consensus_queue.cc:260] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
W20260709 13:02:05.264070  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.785290  8397 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:05.055691  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1441) took 1226 ms. Trace:
I20260709 13:02:05.256135  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:05.025389 10086 log.cc:927] Time spent T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.375s	user 0.001s	sys 0.001s
W20260709 13:02:05.033020 10035 consensus_meta.cc:287] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.789s	user 0.007s	sys 0.000s
W20260709 13:02:05.013115  9845 consensus_meta.cc:287] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.676s	user 0.004s	sys 0.004s
I20260709 13:02:05.075898  9906 raft_consensus.cc:3055] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:04.810406  9786 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:05.276624  8152 rpcz_store.cc:276] 0709 13:01:58.871946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.873136 (+  1190us) service_pool.cc:224] Handling call
0709 13:01:58.896393 (+ 23257us) spinlock_profiling.cc:243] Waited 4.51 ms on lock 0x7b54003bf454. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828728090 00007f88287c6f3a 00007f882e57d1cc 00007f882e67b37e 00007f882cb3c2a4 00007f882cb3c232 00007f882cb3c161 00007f882cb3c0dc 00007f882cb3b392 00007f8826af948c 00007f8826af88c6
0709 13:02:05.069028 (+6172635us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1513874,"reported_tablets":1,"spinlock_wait_cycles":10364672}
W20260709 13:02:05.174317  9855 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.000s	sys 0.010s
W20260709 13:02:04.940981 10026 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.015s	sys 0.052s
I20260709 13:02:05.251816  8222 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:05.299494  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 47792.5us
I20260709 13:02:05.300383  8222 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:05.221582  8332 raft_consensus.cc:2410] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:05.282132  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:04.766337 10013 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:02:05.088500  9944 consensus_meta.cc:287] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.823s	user 0.001s	sys 0.006s
I20260709 13:02:04.876330  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:05.177253 10078 consensus_meta.cc:287] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.863s	user 0.004s	sys 0.008s
W20260709 13:02:05.073244 10102 log.cc:927] Time spent T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.264s	user 0.001s	sys 0.000s
W20260709 13:02:05.038131 10016 consensus_meta.cc:287] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.762s	user 0.007s	sys 0.002s
W20260709 13:02:05.042128 10017 consensus_meta.cc:287] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.641s	user 0.000s	sys 0.011s
I20260709 13:02:05.244395  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1440) took 1419 ms. Trace:
I20260709 13:02:04.927237  9856 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:05.288549  8329 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:05.201726 10038 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.769142  8327 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
W20260709 13:02:05.161073 10079 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.000s	sys 0.010s
I20260709 13:02:05.238683  9990 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:04.787173  9808 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:05.311493 10077 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:05.283209  8151 catalog_manager.cc:5697] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 1 to 2, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 2 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:04.905262  9761 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:05.277257  9900 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:05.285069  8220 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:02:05.277158  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:05.369588  8329 raft_consensus.cc:2410] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:05.297676  8297 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:05.275044  8405 rpcz_store.cc:276] 0709 13:02:03.829491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.615657 (+786166us) service_pool.cc:224] Handling call
0709 13:02:04.633768 (+ 18111us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.757737 (+123969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.055331 (+297594us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54000febd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:05.055602 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":283387776}
I20260709 13:02:05.296733  9903 consensus_queue.cc:1059] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:02:04.765915  9833 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:05.309922  8399 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 6: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:05.215708  9980 raft_consensus.cc:740] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:02:05.302506 10026 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.932s	user 0.034s	sys 0.081s
I20260709 13:02:04.787232 10083 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:05.339445 10016 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:05.291636  9906 raft_consensus.cc:740] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:02:05.298806  9855 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.000s	sys 0.015s
I20260709 13:02:05.341315  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1524) took 2922 ms. Trace:
I20260709 13:02:05.310340 10013 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:05.303045 10094 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:05.385707  8324 rpcz_store.cc:276] 0709 13:02:02.419095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.321623 (+902528us) service_pool.cc:224] Handling call
0709 13:02:03.322070 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:03.322385 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:03.322430 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:04.238203 (+915773us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b2c000d4468. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:02:04.245047 (+  6844us) log.cc:844] Serialized 26 byte log entry
0709 13:02:04.411387 (+166340us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:04.411426 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.411496 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:05.341028 (+929532us) raft_consensus.cc:1685] finished
0709 13:02:05.341046 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.341247 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23294208,"thread_start_us":142025,"threads_started":1,"wal-append.queue_time_us":392009}
I20260709 13:02:05.336308 10078 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:05.309146  9786 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:05.366760  9808 raft_consensus.cc:697] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:05.291406 10035 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:05.289943  8328 raft_consensus.cc:1240] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:05.249054  8153 rpcz_store.cc:276] 0709 13:01:58.963587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.964359 (+   772us) service_pool.cc:224] Handling call
0709 13:02:05.067092 (+6102733us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:00.116363 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:00.117010 (+   647us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:00.117053 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:02:00.117075 (+    22us) tablet.cc:662] Decoding operations
0709 13:02:00.117943 (+   868us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:00.118003 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:00.118029 (+    26us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:02:00.118580 (+   551us) tablet.cc:701] Row locks acquired
0709 13:02:00.118595 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:02:00.118741 (+   146us) write_op.cc:270] Start()
0709 13:02:00.118850 (+   109us) write_op.cc:276] Timestamp: P: 1783602120118699 usec, L: 0
0709 13:02:00.118874 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:02:00.119546 (+   672us) log.cc:844] Serialized 3122 byte log entry
0709 13:02:01.320830 (+1201284us) op_driver.cc:464] REPLICATION: finished
0709 13:02:02.216778 (+895948us) write_op.cc:301] APPLY: starting
0709 13:02:02.218275 (+  1497us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:02.327152 (+108877us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:02.347060 (+ 19908us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:02.603752 (+256692us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:02.675737 (+ 71985us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 13:02:02.680639 (+  4902us) write_op.cc:312] APPLY: finished
0709 13:02:02.719507 (+ 38868us) log.cc:844] Serialized 59 byte log entry
0709 13:02:02.774891 (+ 55384us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:02.787422 (+ 12531us) write_op.cc:454] Released schema lock
0709 13:02:02.789167 (+  1745us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:02.809041 (+ 19874us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:02.810853 (+  1812us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":875208,"apply.run_cpu_time_us":6856,"apply.run_wall_time_us":591873,"num_ops":7,"prepare.queue_time_us":621151,"prepare.run_cpu_time_us":3644,"prepare.run_wall_time_us":39461,"replication_time_us":1201771,"thread_start_us":837942,"threads_started":3,"wal-append.queue_time_us":88317,"wal-append.run_cpu_time_us":1315,"wal-append.run_wall_time_us":1007420}]]}
I20260709 13:02:05.293702  9845 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:05.348419 10101 log.cc:927] Time spent T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
W20260709 13:02:05.354009  8296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.839s	user 0.044s	sys 0.191s
W20260709 13:02:05.397802  8296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.883s	user 0.045s	sys 0.191s
I20260709 13:02:05.398780  8327 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:05.398658  9980 consensus_queue.cc:260] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:05.399673  9980 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:05.399879 10035 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:05.359586  8151 catalog_manager.cc:5697] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 1 to 2, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 2 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:05.402806  8151 catalog_manager.cc:5697] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 1 to 2, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 2 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:05.403707  9898 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:05.403972  9002 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:05.404249  9898 raft_consensus.cc:697] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:05.404899  9808 consensus_queue.cc:237] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:05.405164  9898 consensus_queue.cc:237] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:05.355528  8223 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.024s	sys 0.006s
I20260709 13:02:05.403975  8331 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 3. Current term is 4. Ops: []
W20260709 13:02:05.406051  8223 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.024s	sys 0.007s
W20260709 13:02:05.406349  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 638191us
W20260709 13:02:05.376667  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 79068.6us
I20260709 13:02:05.407922  8297 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:05.340060  8401 rpcz_store.cc:276] 0709 13:02:03.818913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.093001 (+1274088us) service_pool.cc:224] Handling call
0709 13:02:05.126364 (+ 33363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:05.130320 (+  3956us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:05.130650 (+   330us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:05.176093 (+ 45443us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:05.176151 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.176255 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.237937 (+ 61682us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:05.408334  9784 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:05.349617 10024 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.055s	sys 0.123s
I20260709 13:02:05.408986  8297 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:05.409827  9945 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:05.354167 10079 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.000s	sys 0.013s
I20260709 13:02:05.410425  9786 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:05.411104  9786 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:05.411450  9786 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:05.411841  9786 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:05.411974  9833 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:05.412869 10004 raft_consensus.cc:3055] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:02:05.413048  9803 consensus_meta.cc:287] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.762s	user 0.000s	sys 0.008s
I20260709 13:02:05.413278 10004 raft_consensus.cc:740] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:05.413560 10079 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.414008 10079 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:05.414211 10004 consensus_queue.cc:260] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:05.414696 10078 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:05.415125 10004 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:05.416074 10079 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:05.391690 10026 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:05.418514  8403 raft_consensus.cc:2393] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 3: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:05.419715  9845 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:05.408875  9906 consensus_queue.cc:260] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:05.420939  8403 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 1438) took 1697 ms (client timeout 2021 ms). Trace:
I20260709 13:02:05.355753 10038 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:05.374004 10077 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:05.244818 10072 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:05.381212  8972 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:05.339840  9902 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:02:04.960039 10073 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:05.343881 10017 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:05.387681  9009 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:02:04.811525 10092 consensus_queue.cc:1059] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.526s
I20260709 13:02:05.349313  9856 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:05.365845  9950 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.006s	sys 0.013s
I20260709 13:02:05.374140  9761 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:05.382316  9900 raft_consensus.cc:697] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 5 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:05.405560  9766 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:05.407197  8223 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:05.408874  9784 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:02:05.409300 10024 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.867s	user 0.055s	sys 0.123s
I20260709 13:02:05.312405  9944 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:05.421166  9906 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:05.435209  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:05.410420 10090 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:05.421311  8403 rpcz_store.cc:269] 0709 13:02:03.723434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.110824 (+387390us) service_pool.cc:224] Handling call
0709 13:02:05.420764 (+1309940us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54002675d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:05.420888 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3005200128}
I20260709 13:02:05.447399  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:05.449293  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 68148us
W20260709 13:02:05.449820  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:05.467248  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:05.380316  8406 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.812s	user 0.019s	sys 0.006s
W20260709 13:02:05.468779  8406 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.900s	user 0.019s	sys 0.006s
W20260709 13:02:05.452896  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 45757.6us
I20260709 13:02:05.470153  8223 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:05.429277  9971 consensus_meta.cc:287] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.663s	user 0.005s	sys 0.001s
I20260709 13:02:05.425211  9989 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:05.410475  8297 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:02:05.483160  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 72751.6us
I20260709 13:02:05.425846  9910 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.423308  9970 raft_consensus.cc:3055] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:05.427537 10016 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:05.430420  9957 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:05.479033  9007 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 4 LEARNER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:05.450883  9833 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:05.451184 10035 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:05.480324 10024 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:05.485729  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:05.392503 10094 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:05.452544  9803 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:05.485388 10092 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:05.429013  9990 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:05.495433  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:05.489033  9970 raft_consensus.cc:740] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:05.479977 10080 consensus_queue.cc:1059] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:02:05.483446 10093 consensus_queue.cc:1059] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:02:05.497004  9903 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:05.480382  9901 consensus_queue.cc:1059] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
W20260709 13:02:05.536986  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 236712us
W20260709 13:02:05.510921  9985 consensus_meta.cc:287] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.812s	user 0.010s	sys 0.000s
I20260709 13:02:05.497843  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:02:05.539649  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 254652us
I20260709 13:02:05.504894  9833 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:05.485237  9971 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:05.557382  8220 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:05.547804  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 177838us
I20260709 13:02:05.512058  9002 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:02:05.507069  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1403) took 1728 ms. Trace:
I20260709 13:02:05.490764  9910 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:05.533313 10024 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.558936  8220 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:02:05.540315  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 55990.4us
I20260709 13:02:05.603978  8297 leader_election.cc:400] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5.
I20260709 13:02:05.604339  8297 leader_election.cc:403] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:02:05.508848 10035 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:05.578518  8397 raft_consensus.cc:1240] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 2. Current term is 3. Ops: []
W20260709 13:02:05.570683  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:05.556106  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:05.579073  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:05.582763  8369 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:05.594051  8325 raft_consensus.cc:1240] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:05.588603  8399 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:05.605904  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:05.663143  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 80484us
W20260709 13:02:05.611941 10105 log.cc:927] Time spent T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:02:05.611675 10031 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:05.612504  9963 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:05.612984 10072 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:05.614526 10038 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:05.615278  8974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.150s	user 0.103s	sys 0.149s
W20260709 13:02:05.675259  8974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.210s	user 0.103s	sys 0.149s
W20260709 13:02:05.675923  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 1.40552e+06us
I20260709 13:02:05.588092  9970 consensus_queue.cc:260] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:05.617040 10106 log.cc:927] Time spent T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:02:05.645227  9009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 445) took 1718 ms. Trace:
I20260709 13:02:05.579898  9761 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:05.645965  8372 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:05.665195 10100 log.cc:927] Time spent T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 13:02:05.662315  8403 raft_consensus.cc:2393] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 5: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:05.601374  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:05.662350  8402 raft_consensus.cc:2393] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 3: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:05.661823  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:05.705617  8405 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:05.556720  9985 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:05.596525  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:05.563575  9833 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:05.551340  9900 consensus_queue.cc:237] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:05.593060  9950 consensus_queue.cc:1059] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.077s
W20260709 13:02:05.657649  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 51360.9us
I20260709 13:02:05.689541  9009 rpcz_store.cc:276] 0709 13:02:03.927362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.927499 (+   137us) service_pool.cc:224] Handling call
0709 13:02:03.928263 (+   764us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:05.444835 (+1516572us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.640104 (+195269us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b5400533254. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:05.645159 (+  5055us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3355946752}
I20260709 13:02:05.645246 10016 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:05.703383  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:05.819851  8248 raft_consensus.cc:2393] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 4: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:05.707237  8249 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:05.708315  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:05.823277  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:05.666428  9856 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:05.659857 10104 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:05.720206  8255 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:05.846482  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:05.847456  8255 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 3: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:05.847596  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:05.848589  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:05.669019  9845 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:05.775524  8257 raft_consensus.cc:1240] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:05.775170  9761 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.660243 10024 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:05.829057  8405 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:05.690557 10073 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:05.822655  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:05.689436  9970 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:05.852955  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 258679us
W20260709 13:02:05.857375  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 298507us
I20260709 13:02:05.867694  8220 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:05.663956  9808 consensus_queue.cc:1059] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.112s
I20260709 13:02:05.716166 10017 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:05.852988  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:05.824348  8251 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:02:05.850782  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:05.781479  9944 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:05.866868  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 196276us
I20260709 13:02:05.888667  8220 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:05.821034  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:05.870677  8255 raft_consensus.cc:2393] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 3: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:05.817215 10107 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:05.890394 10107 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:05.817185 10077 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:05.893803  8249 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:05.871421  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:05.898067  8254 raft_consensus.cc:2393] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 2: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:05.899116  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:05.899919  8254 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:05.900924  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:05.870112  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:05.872252  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:05.872788  8402 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:05.902642  8405 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:05.893215 10107 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:05.903528  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:05.904438  8402 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:05.904939  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:05.889146  8369 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:05.905768  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:05.907142  8369 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:05.914474  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:05.915675  8254 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 6: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:05.925717  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:05.916945  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:05.934320  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 60439.9us
W20260709 13:02:05.935398  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 46812.7us
W20260709 13:02:05.936031  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:05.936165  8220 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:05.936968  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:05.933945  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:05.939294  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:05.863365  9855 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.941924  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:05.943164  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:05.952999  8251 raft_consensus.cc:2393] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 3: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:05.941290  8248 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:02:05.947609  8222 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:02:05.956123  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 270031us
I20260709 13:02:05.957672  8249 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:05.862917  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:05.854375  9845 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:05.871317  9856 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:05.948401  8255 raft_consensus.cc:2368] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4.
I20260709 13:02:05.964473  8251 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:02:05.871008 10078 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:02:06.020439  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:06.023751  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 57200.1us
I20260709 13:02:05.830973 10079 consensus_queue.cc:1059] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.119s
I20260709 13:02:05.933547  9950 raft_consensus.cc:3055] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:02:06.064292  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 418458us
I20260709 13:02:05.938395 10073 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:05.978930  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:05.865942  9761 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:06.067941  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:06.087934  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:06.088092  9009 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:02:06.088622  8399 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:05.940025  9803 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:06.097324  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:06.038349  9845 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:05.855517 10104 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:06.022994  9856 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:05.564939  8250 rpcz_store.cc:276] 0709 13:02:03.774793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.112083 (+337290us) service_pool.cc:224] Handling call
0709 13:02:04.112518 (+   435us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:04.145950 (+ 33432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.150497 (+  4547us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:04.151221 (+   724us) log.cc:844] Serialized 52 byte log entry
0709 13:02:04.273259 (+122038us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:04.522173 (+248914us) log.cc:844] Serialized 14 byte log entry
0709 13:02:05.084189 (+562016us) spinlock_profiling.cc:243] Waited 7.09 ms on lock 0x7b18000c00a0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256ce455 00007f88287b6060 00007f88287c8716 00007f88287d9287 00007f88287d9208
0709 13:02:05.456094 (+371905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.472449 (+ 16355us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:05.480157 (+  7708us) raft_consensus.cc:1685] finished
0709 13:02:05.491015 (+ 10858us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.503445 (+ 12430us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":153459,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":52,"raft.queue_time_us":188474,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":14478,"spinlock_wait_cycles":16301824,"thread_start_us":479702,"threads_started":2,"wal-append.queue_time_us":182036,"wal-append.run_cpu_time_us":2888,"wal-append.run_wall_time_us":1335226}
W20260709 13:02:06.102645 10115 log.cc:927] Time spent T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.373s	user 0.001s	sys 0.000s
W20260709 13:02:06.103101 10109 log.cc:927] Time spent T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.463s	user 0.000s	sys 0.001s
I20260709 13:02:06.106621  8399 raft_consensus.cc:3055] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:06.066591  9950 raft_consensus.cc:740] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:06.066817  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:06.102501 10110 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.414s	user 0.000s	sys 0.001s
I20260709 13:02:06.107121  8399 raft_consensus.cc:740] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:06.087911  9971 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:06.096293 10103 log.cc:927] Time spent T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.505s	user 0.000s	sys 0.001s
W20260709 13:02:06.147019  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:05.971678  9990 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:06.089788 10122 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:06.022718 10017 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.078s
I20260709 13:02:06.026059 10117 consensus_queue.cc:1059] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.134s
I20260709 13:02:06.091952  9005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 452) took 1311 ms. Trace:
I20260709 13:02:05.970940  9855 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:06.112614 10113 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:06.062985 10080 consensus_queue.cc:1059] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:02:06.096886 10079 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:06.101315  9856 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:05.946475 10094 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:06.133774  8399 consensus_queue.cc:260] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:06.184098  8399 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:06.136904  9912 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:06.137696  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:06.138028  9950 consensus_queue.cc:260] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:06.187606  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1433) took 2465 ms. Trace:
W20260709 13:02:06.191499  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 123120us
I20260709 13:02:06.140111  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:06.067981 10078 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:06.143193  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:06.144325  9784 raft_consensus.cc:3055] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:06.144865  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:06.202456  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:06.203284  9008 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 4 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
W20260709 13:02:06.161571  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 293061us
W20260709 13:02:06.162637  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 226560us
I20260709 13:02:06.114602  9991 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:06.092458 10111 consensus_queue.cc:1059] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 13:02:06.095372  9910 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:06.147342 10035 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:06.172619 10013 consensus_meta.cc:287] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.784s	user 0.006s	sys 0.004s
I20260709 13:02:06.225327 10111 raft_consensus.cc:3055] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:06.224676 10013 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:06.225739 10111 raft_consensus.cc:740] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:06.227869 10013 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:06.228549  9906 consensus_meta.cc:287] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.756s	user 0.009s	sys 0.000s
I20260709 13:02:06.195761  8372 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:06.230815 10078 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:06.206020  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:06.206707  8223 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:06.232916  9991 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:06.233039  8223 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:06.233342  9991 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:06.233777  9991 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:06.234228  9980 consensus_meta.cc:287] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.834s	user 0.010s	sys 0.000s
I20260709 13:02:06.234388 10026 raft_consensus.cc:3055] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:06.234850 10026 raft_consensus.cc:740] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:06.236034  9855 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:06.235759 10026 consensus_queue.cc:260] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:06.206013  8220 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:06.181208  9833 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:06.189566 10079 raft_consensus.cc:697] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:06.182705 10132 consensus_queue.cc:1059] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:02:06.237285 10026 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:06.245908  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:06.246914  9009 raft_consensus.cc:2393] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 4: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:06.237927 10111 consensus_queue.cc:260] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:06.249202 10111 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:06.250327  8222 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:06.250761  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.251868  8222 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:06.189489 10113 raft_consensus.cc:697] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:06.191362  9985 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:06.214897  9786 consensus_meta.cc:287] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.803s	user 0.012s	sys 0.000s
I20260709 13:02:06.201334  9784 raft_consensus.cc:740] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:02:06.219595  9989 consensus_meta.cc:287] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.732s	user 0.010s	sys 0.000s
W20260709 13:02:06.221652 10063 consensus_meta.cc:287] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.936s	user 0.005s	sys 0.006s
I20260709 13:02:06.221903  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:06.222720  8250 raft_consensus.cc:1240] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:06.238289  8220 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:02:06.199658 10004 consensus_meta.cc:287] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.784s	user 0.006s	sys 0.006s
I20260709 13:02:06.255184  9009 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:06.253531  8222 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:06.238226 10093 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:06.238059 10122 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:02:06.260640  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 335292us
I20260709 13:02:06.265317 10107 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:06.241580  9963 consensus_meta.cc:287] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.554s	user 0.006s	sys 0.003s
I20260709 13:02:06.282997  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:06.305095  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:06.306614  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:06.239812  9943 raft_consensus.cc:3055] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:06.238683 10013 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:06.259056  8399 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:02:06.307798  8399 pending_rounds.cc:85] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 2
I20260709 13:02:06.308151  8399 pending_rounds.cc:107] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:02:06.308492  8399 raft_consensus.cc:2889] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:06.239090  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:06.265503 10073 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:06.187973  8406 rpcz_store.cc:276] 0709 13:02:03.722156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.392691 (+670535us) service_pool.cc:224] Handling call
0709 13:02:04.393053 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:04.564267 (+171214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.564298 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:04.568328 (+  4030us) log.cc:844] Serialized 26 byte log entry
0709 13:02:05.469262 (+900934us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:05.469318 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.469408 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:06.187413 (+718005us) raft_consensus.cc:1685] finished
0709 13:02:06.187444 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:06.187553 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":161311,"raft.queue_time_us":161944,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":17056,"thread_start_us":900615,"threads_started":1,"wal-append.queue_time_us":970689}
I20260709 13:02:06.298844  8369 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:06.262305  9950 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:06.296808  8402 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
W20260709 13:02:06.302546  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 64348.6us
I20260709 13:02:06.313351  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:06.313730  8220 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:06.313975  8406 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:06.314916  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:06.315268  8220 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:06.317103  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:06.280303 10024 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:06.320572  8369 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:06.323246  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:06.280117 10078 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:02:06.299510  8401 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:02:06.330020 10135 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:06.301754  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:06.277031  9833 consensus_queue.cc:1059] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:02:06.285982  9980 consensus_queue.cc:1059] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:02:06.285300 10080 consensus_queue.cc:1059] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:02:06.352425  8974 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:06.239166 10079 consensus_queue.cc:237] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:06.278746  9786 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:06.320717 10073 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.321157  9963 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5.
I20260709 13:02:06.316182  9943 raft_consensus.cc:740] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:06.331087 10013 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:06.330029  9784 consensus_queue.cc:260] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:06.390146  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:06.391032  8404 raft_consensus.cc:2393] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 5: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:06.338091  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:06.340029 10098 raft_consensus.cc:3055] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:06.341851 10138 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:06.342748  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:06.343066 10093 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:06.322570 10004 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:06.393359  8255 raft_consensus.cc:2393] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 5: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:06.343591 10142 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:06.343506  9957 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:06.312436 10107 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:06.335407 10097 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:06.343614  9985 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:06.405202  9991 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:06.396512 10004 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.406158 10004 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:06.344875  8297 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:06.341903 10077 raft_consensus.cc:3055] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:06.354271  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:06.406940  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 62146.6us
I20260709 13:02:06.322628  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:06.356637  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:06.335403  8369 connection.cc:256] RPC call timeout handler was delayed by -1.9189805830000068s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1786s for a response
I20260709 13:02:06.357977  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:06.330830  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:06.359297  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:02:06.351078 10126 log.cc:927] Time spent T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 13:02:06.377928 10127 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:06.378391  8327 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:02:06.363440  9902 raft_consensus.cc:3055] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:06.392683  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:06.285665 10063 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:06.383275  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:06.429607  8248 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:06.430025  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:06.430503  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:06.435451  8252 raft_consensus.cc:2393] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 3: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:06.436452  8252 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:50420
I20260709 13:02:06.437163  8252 raft_consensus.cc:606] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:02:06.355293 10135 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:06.447177  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:06.376818  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:06.448426  8256 raft_consensus.cc:2393] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 5: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:02:06.386807  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 133375us
W20260709 13:02:06.391458  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 76270.1us
I20260709 13:02:06.397140  8257 raft_consensus.cc:2393] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 3: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:06.384828 10073 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:06.384266 10072 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:06.390480 10083 consensus_meta.cc:287] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 1.002s	user 0.011s	sys 0.000s
I20260709 13:02:06.396121 10098 raft_consensus.cc:740] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:06.449303  9906 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:06.462136 10098 consensus_queue.cc:260] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:06.462448  9906 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:06.463182 10098 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:06.464481  9906 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:06.464937 10063 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:06.465636  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:06.465951  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:06.466792  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:06.467908  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:06.442440  9985 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:06.401502  8401 raft_consensus.cc:2393] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 3: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:06.402267  9856 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:06.404496 10107 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:06.394770  9901 raft_consensus.cc:3055] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:06.404939 10097 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:06.486585  8403 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:06.487216  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:06.486979  8400 raft_consensus.cc:1240] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:06.475760  9001 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:06.407815  8399 raft_consensus.cc:2393] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 2: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:06.408262  9963 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:06.488606  8251 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:06.488554  9963 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:06.481112  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:06.409130  9991 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:06.409912  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:06.413203  8406 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
I20260709 13:02:06.489779  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:06.490454  9001 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:06.416047  9943 consensus_queue.cc:260] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:06.491113  8404 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:06.491300  9943 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:06.491983  8406 raft_consensus.cc:1275] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:06.409616 10141 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:06.464998  9910 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:06.496855  9910 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:06.496905  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:06.497503  9910 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:06.498557  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:06.499168  8255 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:06.500126  8255 raft_consensus.cc:1240] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:06.486052  8974 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:06.493304  8406 raft_consensus.cc:1275] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:06.381094 10113 consensus_queue.cc:237] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:06.426009  9902 raft_consensus.cc:740] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:06.396793 10138 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.513006 10138 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:06.396324  9833 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.451202  8220 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:06.517771  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 66687.5us
I20260709 13:02:06.519341  8220 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:06.513515  9902 consensus_queue.cc:260] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:06.429646 10125 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.450399  8369 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:06.434129 10090 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:06.418066 10127 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:06.459582  8257 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:06.428225  9786 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:06.521075 10092 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:06.519177  9833 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:06.539016 10127 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:06.442565 10142 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:06.487414  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:06.514822 10141 consensus_queue.cc:1059] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:02:06.489259  9963 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:06.353893  8329 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:06.544658  8253 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:02:06.174690  9005 rpcz_store.cc:276] 0709 13:02:04.776956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.786887 (+  9931us) service_pool.cc:224] Handling call
0709 13:02:04.934201 (+147314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.977247 (+ 43046us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:05.481649 (+504402us) log.cc:844] Serialized 14 byte log entry
0709 13:02:05.955943 (+474294us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:05.978975 (+ 23032us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:06.023357 (+ 44382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:06.063550 (+ 40193us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:06.088345 (+ 24795us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4251,"prepare.queue_time_us":552967,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":70936,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":38892,"thread_start_us":748492,"threads_started":3,"wal-append.queue_time_us":538022}
I20260709 13:02:06.410952 10077 raft_consensus.cc:740] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:06.503684  8252 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:06.507104  8974 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:06.547060  8252 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:06.524029 10104 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:06.478632  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:06.524402  8220 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:06.454762  8222 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:06.461141 10083 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:06.552287  8253 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:06.502386  9901 raft_consensus.cc:740] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:06.527302  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:06.506002  9856 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:06.529598  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 79278.2us
I20260709 13:02:06.520969 10078 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:06.484405  9845 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:06.542958 10119 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.432s	user 0.005s	sys 0.000s
I20260709 13:02:06.542035  8403 raft_consensus.cc:2410] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:02:06.488029  8401 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 5: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:06.524189  9803 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:06.527792 10107 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:06.510880  8256 raft_consensus.cc:1275] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:06.482455  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:06.553905  9902 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:06.553663 10142 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:06.556373 10123 raft_consensus.cc:3055] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:06.538012  9943 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:06.559912  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 52893us
I20260709 13:02:06.559990  8220 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:06.560662  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 106032us
I20260709 13:02:06.603106  8222 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:06.561992  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:06.562227  8406 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 2. Current term is 3. Ops: []
W20260709 13:02:06.567507  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.604578  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:06.605691  8369 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:06.606491  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:06.606992  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:06.607115  8329 raft_consensus.cc:2376] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6. Re-sending same reply.
W20260709 13:02:06.567998  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.568851  8253 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:06.608197  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:06.608853  8324 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:02:06.608995  8220 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:06.609808  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:06.572154  8398 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:06.610132  8324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:37920
W20260709 13:02:06.563485  9970 consensus_meta.cc:287] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.680s	user 0.008s	sys 0.003s
I20260709 13:02:06.528657 10094 consensus_queue.cc:1059] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:02:06.569833 10092 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:06.562816 10080 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:06.606766 10122 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:06.578192 10104 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:06.536659 10125 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:06.569676  9903 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:06.601809  8397 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:06.543447 10135 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:06.610368  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.610673  8324 raft_consensus.cc:606] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:02:06.612586  8222 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:06.613080  9856 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:06.607076 10123 raft_consensus.cc:740] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:06.621948 10111 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:06.609263  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:06.577023  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 220866us
I20260709 13:02:06.611214  9985 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:06.618144  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:06.617026  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:06.616891  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:06.621985  8406 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:06.638605  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:06.622963 10094 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:06.623714  9945 raft_consensus.cc:3055] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:06.607250 10141 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.613867 10079 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:06.667084 10135 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:06.667112 10141 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:06.615181  9808 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.629902 10152 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:06.630532 10122 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:06.669159  9808 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:06.670303  9945 raft_consensus.cc:740] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:06.670668  8403 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:06.632583  8295 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:06.672246 10072 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:06.617712  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:06.615370  9952 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:06.634125  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 153) took 1164 ms. Trace:
I20260709 13:02:06.673395  8323 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:06.568172 10078 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:06.669615 10122 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.630769  9833 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:06.632287  8329 raft_consensus.cc:2393] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 5: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:06.621222  8401 raft_consensus.cc:1240] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:06.673522  8331 rpcz_store.cc:276] 0709 13:02:05.469865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.470154 (+   289us) service_pool.cc:224] Handling call
0709 13:02:05.470552 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:05.470976 (+   424us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:05.471020 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:05.471572 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 13:02:05.592604 (+121032us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:05.592640 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.592706 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:06.541097 (+948391us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:06.628373 (+ 87276us) raft_consensus.cc:1685] finished
0709 13:02:06.628403 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:06.634037 (+  5634us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2169,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":4760,"spinlock_wait_cycles":30759296,"thread_start_us":94701,"threads_started":1,"wal-append.queue_time_us":381310}
I20260709 13:02:06.675400  8220 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:06.624188 10117 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.041s
W20260709 13:02:06.675799  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 57737us
I20260709 13:02:06.672889 10072 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:06.638891 10111 raft_consensus.cc:697] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:02:06.697865  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 77690.8us
I20260709 13:02:06.698447  8249 raft_consensus.cc:1240] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:06.690521  8369 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:06.636941  9903 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:06.701476  8369 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:06.678188  9952 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:06.702209  9943 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:06.676594 10122 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:06.643958  9985 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:06.674005  9898 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:06.705861  8328 raft_consensus.cc:2410] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:06.679369 10097 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:06.686178 10092 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:06.689692 10152 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:06.660018  9963 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:06.715221 10112 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:06.716153 10078 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:06.717890  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:06.716518  9985 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:06.711889  8369 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:06.704630  9952 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:06.720192 10155 log.cc:927] Time spent T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:02:06.720285  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:06.721205  9008 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 3: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:06.671239  9945 consensus_queue.cc:260] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:06.722087  9945 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:06.724598 10158 log.cc:927] Time spent T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 13:02:06.725086  9903 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:06.683595 10077 consensus_queue.cc:260] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:06.705989  8326 raft_consensus.cc:1275] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:06.719027 10092 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:02:06.732203  9845 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:06.732652 10142 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:06.727468  8369 leader_election.cc:400] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4.
I20260709 13:02:06.735411  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:06.726485 10077 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:06.726457  9808 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:06.723210 10083 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:06.744725  8369 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:06.729604  9856 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:06.714406 10073 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:06.748181  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:06.753501  8248 raft_consensus.cc:2393] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 5: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:06.737985  9971 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.733884  9002 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:06.754159  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:06.758595 10125 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:06.760638  9855 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:06.761214  9855 raft_consensus.cc:697] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:06.761994 10083 consensus_queue.cc:1059] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:06.762140  9855 consensus_queue.cc:237] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:06.765368  9906 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:02:06.766559  9971 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:06.770041 10092 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:06.752677 10135 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:02:06.780236  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.759155 10112 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:06.766031 10083 raft_consensus.cc:3055] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:06.713078 10123 consensus_queue.cc:260] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:06.794209 10160 log.cc:927] Time spent T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:02:06.794432  9963 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:06.783133 10078 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.796211  8400 raft_consensus.cc:1240] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:06.796526 10078 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:06.800206 10078 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:06.800603  9003 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:06.801277  8401 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:02:06.786576 10112 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.787032 10083 raft_consensus.cc:740] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:06.748661 10111 consensus_queue.cc:237] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
W20260709 13:02:06.807657  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 108803us
W20260709 13:02:06.810742  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 137855us
I20260709 13:02:06.806305  8401 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:06.818467  8298 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:06.819571  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:06.807008 10112 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:06.820492  9007 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:06.808465 10123 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:06.817718  8295 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:06.827520  8372 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:06.829493 10157 log.cc:927] Time spent T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:02:06.829627  9898 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:06.826066  8298 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:06.831545  9963 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:06.831792  8298 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:06.832971  8298 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:06.833732 10135 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:06.834128 10135 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.834472 10135 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:06.838133  9900 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:06.840404  8298 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:06.842352  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:06.810145 10073 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:06.826294 10159 log.cc:927] Time spent T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.128s	user 0.002s	sys 0.000s
I20260709 13:02:06.840027  9903 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:06.848517  9898 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:06.865862  8330 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:02:06.866794  8330 raft_consensus.cc:606] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:02:06.867280  8330 raft_consensus.cc:614] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:06.875129  8295 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:06.879012  9903 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:06.879516  9903 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.879887  9903 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:06.883010  9910 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:06.900076  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:06.850667 10083 consensus_queue.cc:260] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:06.907166  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:06.917861  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.919077  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:06.920068  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:06.920794  8399 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:06.923530  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:06.923754  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:06.860205 10164 consensus_queue.cc:1059] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 13:02:06.928140  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:06.929179  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:06.930465  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:06.933168  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:06.934356 10117 consensus_queue.cc:1059] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:06.934453 10092 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:06.864099  9952 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:06.937085  8328 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
W20260709 13:02:06.939715  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 337724us
I20260709 13:02:06.942997  8328 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:06.944680  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:06.945289  8331 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:06.877280 10004 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:06.957131  9009 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:06.957254  9001 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:02:06.958317  9006 raft_consensus.cc:1240] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:06.958954  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:06.960651  8372 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:06.961380 10135 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:06.963168  9989 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:06.963127 10138 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:06.981768  9989 raft_consensus.cc:697] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:06.982745  8250 raft_consensus.cc:2393] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 5: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:06.983023  9989 consensus_queue.cc:237] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:06.992602  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:06.993814  8405 raft_consensus.cc:2393] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 5: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:06.991575 10138 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:07.007174 10135 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:07.024399  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 149331us
I20260709 13:02:06.927317  8326 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:02:07.026782 10031 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:07.029670  8255 raft_consensus.cc:1240] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:07.030426  8404 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:06.941596  9945 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:07.041706  8330 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:06.954460  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:07.013180 10112 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:07.018864 10063 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:07.051874 10164 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 13:02:07.048945  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:07.062618  8331 raft_consensus.cc:2393] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 5: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:07.063688  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:07.064489  8331 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
W20260709 13:02:07.065925 10168 log.cc:927] Time spent T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:02:07.065914  8331 raft_consensus.cc:1240] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3. Ops: []
W20260709 13:02:07.054949  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:07.053784  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.953877  9902 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.101s
I20260709 13:02:06.953862 10080 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 13:02:07.045598  8326 raft_consensus.cc:606] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:02:07.053035  8401 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:07.080538 10098 consensus_meta.cc:287] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.617s	user 0.005s	sys 0.006s
I20260709 13:02:07.048332  9945 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:07.073729 10164 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:07.050261 10031 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 13:02:07.080168  8326 raft_consensus.cc:614] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:07.077730  9971 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:07.089664 10035 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:07.108395  9761 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:07.111680  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:07.111781 10072 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:07.106909 10035 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:07.115206 10035 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:07.116994  9945 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:07.120018  8369 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:07.120256 10122 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:07.121321  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:07.122457  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:07.122906 10152 raft_consensus.cc:3055] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:07.123189 10135 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:07.123456 10152 raft_consensus.cc:740] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:07.123903 10135 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:07.124563 10135 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:07.127171  9952 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:07.127400  8369 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:07.131058  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:07.134907 10004 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:07.140524  9903 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:07.152812  9784 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:07.166802 10104 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.167273 10104 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:07.156946 10079 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:07.174139  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:07.174934  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:07.175200  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:07.148517 10107 consensus_meta.cc:287] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.575s	user 0.009s	sys 0.000s
I20260709 13:02:07.129390  9910 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:07.099726 10164 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:07.152096 10127 consensus_meta.cc:287] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.612s	user 0.013s	sys 0.000s
I20260709 13:02:07.175804  8249 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:07.184005  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:07.188948  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:07.181234  8223 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:07.190119  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:07.190932  8249 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 3: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:07.191741  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:07.183051 10031 consensus_queue.cc:1059] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:02:07.184563 10170 consensus_queue.cc:1059] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:07.185376  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:07.203220  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:07.185837 10004 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:07.187179  9006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 366) took 1086 ms. Trace:
I20260709 13:02:07.186491  8326 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:07.206419  9006 rpcz_store.cc:276] 0709 13:02:06.100685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.100893 (+   208us) service_pool.cc:224] Handling call
0709 13:02:06.910866 (+809973us) spinlock_profiling.cc:243] Waited 3.82 ms on lock 0x7b5800175898. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:02:06.958155 (+ 47289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:06.958633 (+   478us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.186951 (+228318us) spinlock_profiling.cc:243] Waited 36.9 ms on lock 0x7b5400527bd0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:07.187110 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93758208}
I20260709 13:02:07.153172 10152 consensus_queue.cc:260] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:07.218693  8248 raft_consensus.cc:2393] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 5: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:07.184238  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:07.198592  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:07.221915  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:07.222827  8369 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:07.214823  8223 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:07.225076  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.225157  8254 raft_consensus.cc:2393] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 3: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:07.229054  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:07.193080 10107 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:07.223346  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:07.197667 10092 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:07.236670 10112 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:07.236814  8401 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:07.245270  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:07.249080  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:07.250355  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:07.251405  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:07.251971  8323 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:07.252516  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:07.203673 10127 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:07.254518  8397 raft_consensus.cc:1275] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:07.255199 10127 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:07.255664  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1410) took 1542 ms. Trace:
I20260709 13:02:07.256038  8251 rpcz_store.cc:276] 0709 13:02:05.713409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.023686 (+310277us) service_pool.cc:224] Handling call
0709 13:02:06.488453 (+464767us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:06.489153 (+   700us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.255466 (+766313us) spinlock_profiling.cc:243] Waited 464 ms on lock 0x7b540016f150. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:07.255602 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1068135296}
I20260709 13:02:07.257184  9903 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:07.193827 10090 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:07.257673  8251 raft_consensus.cc:1240] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:02:07.257961 10090 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:07.261615  8369 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:07.261633  9989 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:07.262467 10112 consensus_queue.cc:1059] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:07.262831  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:07.263382  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:07.263933 10104 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:07.265090  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.269385 10122 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:07.290908 10104 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:07.291026  9901 consensus_queue.cc:260] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:07.292439  9901 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:07.284225  9989 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.293879  9989 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:07.299435  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:07.300207  8329 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:07.220990 10031 consensus_queue.cc:1243] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer 8c42259d7c4b463a9f485f42c320c81b log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:02:07.272866  8252 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:07.226078 10004 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:07.268539 10092 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:07.282557  8251 raft_consensus.cc:1240] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:07.283645  9006 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:07.258761 10072 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:07.282027  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:07.287518 10164 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:07.288164 10112 raft_consensus.cc:3055] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:07.219630  9985 consensus_queue.cc:1059] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
W20260709 13:02:07.269848  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 55164.9us
I20260709 13:02:07.310607  8332 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:07.311443  8323 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
W20260709 13:02:07.312824  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.277208  9952 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:07.316589  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:07.317055  8329 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:07.318595  8369 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:07.321907  9985 raft_consensus.cc:3055] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:07.322309  9985 raft_consensus.cc:740] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:07.326568  8328 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:02:07.328584  8372 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:07.331866  8369 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:07.333811  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:07.333947 10135 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:07.334635  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:07.334932  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:07.335172  8323 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:07.336309  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:07.340026  8326 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:02:07.313941 10112 raft_consensus.cc:740] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:07.340456  8369 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:07.344518  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:07.342123 10112 consensus_queue.cc:260] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:07.345728 10142 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:07.346782 10097 raft_consensus.cc:3055] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:07.332960 10092 consensus_queue.cc:1059] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:02:07.347215 10079 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:07.347271 10097 raft_consensus.cc:740] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:07.347747 10079 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:07.348009 10092 raft_consensus.cc:3055] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:07.348438 10092 raft_consensus.cc:740] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:07.350716  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1479) took 1019 ms. Trace:
I20260709 13:02:07.351058  8252 rpcz_store.cc:276] 0709 13:02:06.331395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.549394 (+217999us) service_pool.cc:224] Handling call
0709 13:02:06.549653 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.305967 (+756314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.350501 (+ 44534us) spinlock_profiling.cc:243] Waited 723 ms on lock 0x7b540034c6d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:07.350663 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1662873472}
I20260709 13:02:07.351872  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1470) took 1024 ms. Trace:
I20260709 13:02:07.352093  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:07.352269  8250 rpcz_store.cc:276] 0709 13:02:06.327649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.351657 (+ 24008us) service_pool.cc:224] Handling call
0709 13:02:07.351677 (+1000020us) spinlock_profiling.cc:243] Waited 548 ms on lock 0x7b5400178754. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:07.351817 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1259339008}
I20260709 13:02:07.352996  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:07.327378  9855 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:07.353523  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:07.354056  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:07.351076  8295 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:07.357870 10170 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:07.358256 10170 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.358572 10170 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:07.329972  9004 raft_consensus.cc:1240] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:07.353615  9945 raft_consensus.cc:3055] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:07.360512  8220 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:07.354723  8324 raft_consensus.cc:2393] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 3: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:07.348212 10079 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:07.361555  9006 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:07.346365  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:07.362468 10122 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:07.317937 10004 raft_consensus.cc:697] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:07.357097  8295 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:07.352682  8252 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:07.364897 10095 raft_consensus.cc:3055] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:07.364120  8222 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:07.341409  9952 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:07.364826 10107 consensus_queue.cc:1048] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:07.368369  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:07.368673  8220 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:07.380543  8220 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:07.374517 10123 raft_consensus.cc:2764] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:02:07.381584  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:07.383594  8324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:02:07.384243  8324 raft_consensus.cc:606] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:02:07.384650  8324 raft_consensus.cc:614] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:07.385640  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.387313 10148 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:07.374423 10031 consensus_queue.cc:1059] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:02:07.389060  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 220) took 1240 ms. Trace:
I20260709 13:02:07.381588 10073 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:07.389571  8404 rpcz_store.cc:276] 0709 13:02:06.148902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.491806 (+342904us) service_pool.cc:224] Handling call
0709 13:02:06.492062 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.030825 (+538763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.388757 (+357932us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b5400276854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:07.388986 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1235807104}
I20260709 13:02:07.379086 10122 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:07.393872  8220 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:07.326504  9985 consensus_queue.cc:260] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:07.396906  8220 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:07.368968  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:07.370060  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:07.397888  8406 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:07.371037  8323 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:07.362268  8397 raft_consensus.cc:1275] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:07.372956  8372 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:07.373739  9008 raft_consensus.cc:1240] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:07.354703  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:07.399283  8252 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:07.367578  9945 raft_consensus.cc:740] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:07.404956  8220 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:07.377647 10164 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:07.377007 10095 raft_consensus.cc:740] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:07.407678  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:07.381847  9803 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:07.445219  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:07.348330 10097 consensus_queue.cc:260] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:07.400417 10073 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:07.406915  8323 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:07.375164 10135 consensus_queue.cc:1059] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:02:07.401567  9943 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:07.459535  9952 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:07.459519 10004 consensus_queue.cc:237] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:07.460393  9952 raft_consensus.cc:697] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:07.461326  9952 consensus_queue.cc:237] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:07.455255  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 50377.6us
I20260709 13:02:07.404533  9855 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:07.465713  9855 raft_consensus.cc:697] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:07.349284 10092 consensus_queue.cc:260] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:07.468214 10118 raft_consensus.cc:1064] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:02:07.466699  9855 consensus_queue.cc:237] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:07.457996  9945 consensus_queue.cc:260] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:07.469189  9945 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:07.467800  8220 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:07.469779  8325 raft_consensus.cc:1240] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:07.470333 10072 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:07.470811 10072 raft_consensus.cc:697] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:07.471567 10122 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:07.471786 10072 consensus_queue.cc:237] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:07.396358  9957 consensus_queue.cc:1059] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.059s
I20260709 13:02:07.472426  9985 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:07.473093  9957 raft_consensus.cc:3055] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:07.473464  9957 raft_consensus.cc:740] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:07.406574  8250 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:07.473722 10095 consensus_queue.cc:260] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:07.474819 10095 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:07.475430  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.476708 10142 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:07.477397  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:07.452044 10138 consensus_queue.cc:1048] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:02:07.478047  9002 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
W20260709 13:02:07.443154 10183 log.cc:927] Time spent T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:02:07.479420  8325 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:07.453781 10117 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:07.472172  8974 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:07.458438 10184 log.cc:927] Time spent T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:02:07.468123 10092 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:07.480717  8223 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:07.481012  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 231) took 1212 ms. Trace:
W20260709 13:02:07.493502  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.493628  8255 rpcz_store.cc:276] 0709 13:02:06.268308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.500762 (+232454us) service_pool.cc:224] Handling call
0709 13:02:07.029510 (+528748us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.030110 (+   600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.455883 (+425773us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b5400308750. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:07.480954 (+ 25071us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1215558912}
I20260709 13:02:07.485146  8974 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:07.399727 10107 consensus_queue.cc:1048] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.058s
I20260709 13:02:07.500591  8325 raft_consensus.cc:1275] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:02:07.500650  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:07.501215  8325 pending_rounds.cc:85] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 1
I20260709 13:02:07.501549  8325 pending_rounds.cc:107] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:02:07.501827  8325 raft_consensus.cc:2889] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:07.498127  8397 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:07.499859  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:07.502269  8406 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:07.505564  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:07.505801  8222 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:07.507809  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:07.508641  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 92049.8us
I20260709 13:02:07.510890  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:07.512192  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:07.516433  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:07.483815  9943 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.519356  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:07.520674  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:07.522100  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:07.523063  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.512197  9803 raft_consensus.cc:697] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:07.525267  9803 consensus_queue.cc:237] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:07.492863  9980 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:07.489414 10125 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:07.542558  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.506704 10127 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:07.518594 10073 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:07.528425 10138 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:07.548838  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:07.526649 10123 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:07.512269 10063 consensus_queue.cc:1059] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
W20260709 13:02:07.580889  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.584060  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:07.599352  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.553992  9963 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:07.550758 10073 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:07.605793  8402 raft_consensus.cc:1240] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:07.538213  9943 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:07.611899  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1709) took 3837 ms. Trace:
I20260709 13:02:07.613449  8398 rpcz_store.cc:276] 0709 13:02:03.774168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.518351 (+744183us) service_pool.cc:224] Handling call
0709 13:02:04.518685 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:06.610419 (+2091734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.611675 (+1001256us) spinlock_profiling.cc:243] Waited 2.05 s on lock 0x7b540020fb54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:07.611838 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4722598272}
I20260709 13:02:07.614914  8398 raft_consensus.cc:1240] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:07.617214  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:07.588729 10138 raft_consensus.cc:697] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:07.557500 10162 consensus_queue.cc:1059] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
W20260709 13:02:07.632805  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 96106.9us
I20260709 13:02:07.549324 10125 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:07.604763 10098 raft_consensus.cc:3055] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:07.636873  8256 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:02:07.639068  8256 pending_rounds.cc:85] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 3
I20260709 13:02:07.639338  8256 pending_rounds.cc:107] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:02:07.639609  8256 raft_consensus.cc:2889] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:07.639684  8405 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
I20260709 13:02:07.612754 10073 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:07.548230 10163 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:02:07.646216  8297 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:07.650415  9943 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:07.652884  9007 raft_consensus.cc:2410] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:07.655161  8397 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:07.608100 10090 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:07.611403 10165 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:02:07.664470  8297 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:07.638227 10098 raft_consensus.cc:740] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:07.632185 10129 consensus_queue.cc:1059] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:02:07.677482  8297 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:07.679591  8256 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 13:02:07.682322 10192 log.cc:927] Time spent T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:02:07.672062 10162 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:07.684441 10162 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:07.687821 10079 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:07.672224 10198 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:07.689953 10198 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:07.690836 10090 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:07.694756 10165 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:07.696659  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:07.654915 10164 consensus_queue.cc:1059] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:02:07.702589 10164 raft_consensus.cc:3055] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:07.703054 10164 raft_consensus.cc:740] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:07.704030 10164 consensus_queue.cc:260] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:07.707602  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:07.708422  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.707507 10118 consensus_queue.cc:237] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:07.709901  8252 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:07.712056  8252 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:07.713495  9784 raft_consensus.cc:2955] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 4 to 6, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:07.721978 10098 consensus_queue.cc:260] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:07.723310 10098 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:07.723588  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 1760) took 1118 ms. Trace:
I20260709 13:02:07.724095  8332 rpcz_store.cc:276] 0709 13:02:06.605167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.609581 (+  4414us) service_pool.cc:224] Handling call
0709 13:02:07.723374 (+1113793us) spinlock_profiling.cc:243] Waited 695 ms on lock 0x7b540037c654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:07.723516 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1599048576}
I20260709 13:02:07.642292  9970 consensus_queue.cc:1059] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:02:07.726922  9970 raft_consensus.cc:3055] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:07.727289  9970 raft_consensus.cc:740] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:07.728312  8257 raft_consensus.cc:1240] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:07.728233  9970 consensus_queue.cc:260] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:07.729545  8257 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:07.729774  9903 raft_consensus.cc:1027] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: attempt to promote peer f865c4d761f5446eb12b3a029234772e: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
W20260709 13:02:07.730381 10191 log.cc:927] Time spent T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.221s	user 0.001s	sys 0.000s
I20260709 13:02:07.731873 10125 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:07.733821  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:07.741700  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 64329.2us
I20260709 13:02:07.742193  8332 raft_consensus.cc:2393] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 4: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:02:07.721326  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.742936  8297 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:07.727308  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1471) took 1317 ms. Trace:
I20260709 13:02:07.747201  8403 rpcz_store.cc:276] 0709 13:02:06.410428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.566087 (+155659us) service_pool.cc:224] Handling call
0709 13:02:06.566421 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:06.675799 (+109378us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.727110 (+1051311us) spinlock_profiling.cc:243] Waited 94.4 ms on lock 0x7b54003e6554. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:07.727256 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":217248896}
I20260709 13:02:07.748411  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:07.751431  8256 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:02:07.713514 10199 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:02:07.713163  8252 raft_consensus.cc:1240] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:07.731971  8250 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:07.744334  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:07.744647  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:07.757477  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:07.747082 10162 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:07.759640  8297 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:07.763363  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:07.673630  9989 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:07.745712  9985 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.754861  9970 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:07.740943 10163 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:02:07.772921 10203 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.773590 10203 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:07.776087 10203 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:07.770843  8220 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:07.777510  9985 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:07.779726  9985 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:07.779769 10072 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:07.780368 10072 raft_consensus.cc:697] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:07.781247 10072 consensus_queue.cc:237] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:07.783246 10141 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:07.791173  8324 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:07.800992  8404 raft_consensus.cc:1240] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:02:07.766301  9945 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:07.769049 10118 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:07.809056  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.772068 10198 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:07.811211  9980 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:07.811518  9989 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:07.757513 10138 consensus_queue.cc:237] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:07.815075 10079 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:07.816098  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:07.771472  9971 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:02:07.827975  9900 raft_consensus.cc:3055] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:07.828527  9900 raft_consensus.cc:740] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:07.829363  9900 consensus_queue.cc:260] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:07.832221  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:07.833169  9005 raft_consensus.cc:2393] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 2: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:07.834327  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:07.835618  8254 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:07.836337  8330 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:07.837805 10170 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:07.840301 10170 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:07.840546 10188 log.cc:927] Time spent T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.566s	user 0.002s	sys 0.001s
I20260709 13:02:07.841859  8323 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:07.843123 10073 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:07.843778 10148 raft_consensus.cc:993] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b: : Instructing follower e64761f43e4240319c41117cf886d22f to start an election
I20260709 13:02:07.846879  8220 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:07.847486  8324 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:02:07.859014  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:07.863782  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:07.875478  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:07.908870  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:08.020023  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:08.022428  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:08.058049  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.835636  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:08.072286  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.859412 10123 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:07.860868 10198 raft_consensus.cc:1081] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Signalling peer e64761f43e4240319c41117cf886d22f to start an election
I20260709 13:02:08.078594 10122 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.107887 10122 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.110795 10122 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:08.111481 10122 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:08.112089 10122 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.071723 10118 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:08.120221  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.078737 10098 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:08.079116 10111 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.072135 10117 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:08.126358  8331 raft_consensus.cc:2955] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:08.088912  9945 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.139335  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:08.134516  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 517353us
I20260709 13:02:08.146414  8399 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:08.146623  8972 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:08.147715  8297 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:08.088440 10204 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:08.086458  9985 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:08.147665  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:08.154183  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:08.154886  8399 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:08.102829 10207 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:07.867547 10092 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.244s
I20260709 13:02:08.165160  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:08.150205 10072 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:08.168140  8399 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:08.154778  9980 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:08.139312 10098 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.133790 10118 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:08.152076  9945 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:08.174223  8399 raft_consensus.cc:1240] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:08.146142 10111 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:08.149488 10117 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.176759  8402 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:08.157511  9985 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:08.170717 10141 raft_consensus.cc:2764] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:02:08.185038  8402 raft_consensus.cc:1240] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:08.185690  8399 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:08.187685  8399 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:02:08.155309 10204 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.176455 10118 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:08.178061 10098 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:08.164050 10207 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.197109  9945 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:08.198779  9985 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:08.191905 10073 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:08.174378 10072 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.202905 10207 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:08.202852  9002 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:08.208617  9784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:57170 (request call id 76) took 4154 ms. Trace:
I20260709 13:02:08.208834 10111 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:08.209337 10204 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:08.209059  9784 rpcz_store.cc:276] 0709 13:02:04.113696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.153126 (+ 39430us) service_pool.cc:224] Handling call
0709 13:02:04.200315 (+ 47189us) log.cc:844] Serialized 487 byte log entry
0709 13:02:04.314219 (+113904us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b18001a6a40. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d1522 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:02:08.208524 (+3894305us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14140,"raft.queue_time_us":14993,"raft.run_cpu_time_us":1466,"raft.run_wall_time_us":49300,"spinlock_wait_cycles":31171968}
I20260709 13:02:08.210202 10117 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:08.210228 10072 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:08.243281  8250 raft_consensus.cc:1240] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:08.250566  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:08.251991  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:08.252876  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:08.254525  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 106733us
I20260709 13:02:08.255336  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:08.259742  8403 raft_consensus.cc:1240] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:08.261142  8139 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ae82ced6ff014453bbcfd5f9f14b1f8d with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 5)
I20260709 13:02:08.261598  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:08.261946  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:08.263681  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.255353  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:08.267647  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:08.268297  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.268590  9001 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:08.263164  8250 raft_consensus.cc:2393] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 4: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:08.274085  8401 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 3: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:02:08.275023  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.275663  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:08.282817  8372 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:08.287101  8222 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:08.288368  9007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56954 (request call id 374) took 1331 ms. Trace:
I20260709 13:02:08.288561 10219 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:08.288566 10125 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:08.289290 10219 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.289338 10125 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:08.289780 10204 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.289956 10219 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:08.289996 10125 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:08.290815 10204 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.284603  8372 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:08.296514  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:08.298081 10079 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:02:08.299304  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.302086  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:08.303658  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:08.305155 10072 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.306084 10072 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.307183 10125 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.276073 10207 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.303750  8154 catalog_manager.cc:5697] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: config changed from index 3 to 4, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1), VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2), VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3), VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 3 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:08.313581  8972 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:08.313874  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:08.314813  9002 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:08.315109  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:08.316212  8972 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:08.319021 10098 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:08.321187  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:08.322311  8331 raft_consensus.cc:2393] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 4: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:08.322721  8151 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45456 (request call id 281) took 7586 ms. Trace:
I20260709 13:02:08.297242 10204 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:08.307214  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:08.325186  8250 raft_consensus.cc:1240] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:02:08.326982  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:08.332098  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:08.317353  8327 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:08.317727 10127 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:08.321625  8972 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:08.340955  8372 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:08.342248  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 48311.7us
I20260709 13:02:08.309192 10072 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:08.346202 10079 raft_consensus.cc:697] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:08.345283  8404 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:08.288936  9007 rpcz_store.cc:276] 0709 13:02:06.956291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.958646 (+  2355us) service_pool.cc:224] Handling call
0709 13:02:08.287807 (+1329161us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b54005d7ad4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:08.288072 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1593525504}
I20260709 13:02:08.354748  9980 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:08.362529  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:08.323220  8151 rpcz_store.cc:276] 0709 13:02:00.736253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.736373 (+   120us) service_pool.cc:224] Handling call
0709 13:02:08.322382 (+7586009us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:05.853995 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:06.113255 (+259260us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:06.116371 (+  3116us) write_op.cc:435] Acquired schema lock
0709 13:02:06.116394 (+    23us) tablet.cc:662] Decoding operations
0709 13:02:06.301199 (+184805us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:06.302580 (+  1381us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:06.319756 (+ 17176us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:02:06.400409 (+ 80653us) tablet.cc:701] Row locks acquired
0709 13:02:06.400433 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:02:06.413986 (+ 13553us) write_op.cc:270] Start()
0709 13:02:06.414092 (+   106us) write_op.cc:276] Timestamp: P: 1783602126413938 usec, L: 0
0709 13:02:06.414111 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:02:06.415684 (+  1573us) log.cc:844] Serialized 2055 byte log entry
0709 13:02:06.881578 (+465894us) op_driver.cc:464] REPLICATION: finished
0709 13:02:07.030886 (+149308us) write_op.cc:301] APPLY: starting
0709 13:02:07.043399 (+ 12513us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:07.073979 (+ 30580us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:07.080874 (+  6895us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:07.108858 (+ 27984us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:07.109081 (+   223us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:02:07.109140 (+    59us) write_op.cc:312] APPLY: finished
0709 13:02:07.109422 (+   282us) log.cc:844] Serialized 47 byte log entry
0709 13:02:07.109634 (+   212us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:07.109762 (+   128us) write_op.cc:454] Released schema lock
0709 13:02:07.109817 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:07.110228 (+   411us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:07.110266 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":163746,"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":63485,"apply.run_cpu_time_us":3189,"apply.run_wall_time_us":90858,"num_ops":5,"prepare.queue_time_us":298134,"prepare.run_cpu_time_us":21423,"prepare.run_wall_time_us":709684,"raft.queue_time_us":321747,"raft.run_cpu_time_us":11330,"raft.run_wall_time_us":59694,"replication_time_us":460417,"thread_start_us":580464,"threads_started":4,"wal-append.queue_time_us":116182,"wal-append.run_cpu_time_us":23776,"wal-append.run_wall_time_us":1601320}]]}
I20260709 13:02:08.363474  8329 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:08.363461 10125 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:08.361634 10204 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:08.374851  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:08.380017 10092 consensus_queue.cc:1059] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:08.380555 10207 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:08.381628  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 78047.9us
I20260709 13:02:08.382216 10111 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:02:08.396294  9761 raft_consensus.cc:3055] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:08.399045  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:08.407225  8398 raft_consensus.cc:1240] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:08.412230  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:08.425386  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 74584.9us
I20260709 13:02:08.355732  9980 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.417086 10204 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.432917  8250 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:08.433861 10123 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:08.434404  8297 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:08.435859 10090 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.438220 10092 raft_consensus.cc:3055] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:08.445497 10092 raft_consensus.cc:740] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:08.445123 10079 consensus_queue.cc:237] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.446862 10092 consensus_queue.cc:260] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:08.448575 10092 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:08.449151 10035 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:08.449620 10035 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.450069 10035 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:08.449651 10090 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.453351  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:08.454053  9007 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:08.455164 10221 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:08.458213  9001 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 3 LEARNER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:08.458746  9980 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:08.458755  9000 raft_consensus.cc:1240] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:08.459332  9006 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:08.459561 10123 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:08.456947 10221 consensus_queue.cc:1059] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:08.462250 10123 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:08.462353  9945 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:08.465411 10220 raft_consensus.cc:3055] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:08.465999 10220 raft_consensus.cc:740] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:08.423744  9761 raft_consensus.cc:740] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:08.440974 10204 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:08.466921 10213 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:08.491358 10118 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.493777  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:08.494158 10118 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:08.495376  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:08.496690 10118 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:08.497334  9903 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:02:08.497637  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.498090  9903 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:08.498557  9003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 361) took 2409 ms. Trace:
I20260709 13:02:08.499598  8220 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:08.499709  9980 consensus_queue.cc:1059] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:08.500998  9903 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:08.507112  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:08.507928  8256 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:08.508777  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:08.509994  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:08.509877 10220 consensus_queue.cc:260] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:08.510802  8257 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:08.510655  9761 consensus_queue.cc:260] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.511485 10220 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:08.511897  9761 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:08.514521  8297 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:08.455165  9957 consensus_queue.cc:260] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:08.516712  9957 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:08.516862  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1567) took 1397 ms. Trace:
I20260709 13:02:08.517350  8323 rpcz_store.cc:276] 0709 13:02:07.119397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.480306 (+360909us) service_pool.cc:224] Handling call
0709 13:02:07.497756 (+ 17450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.842318 (+344562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.516224 (+673906us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b54003cb254. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:08.516660 (+   436us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":790889728}
W20260709 13:02:08.519325  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.519834  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:08.502595  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 181203us
W20260709 13:02:08.522761  8972 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.839s (SENT) [suppressed 22 similar messages]
I20260709 13:02:08.515894  9903 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:08.516875  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.514585 10073 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:08.525166 10073 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.525700 10073 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:08.501513  9980 raft_consensus.cc:3055] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:08.528084  9980 raft_consensus.cc:740] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:08.501739 10138 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:08.532786 10083 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:08.529131  9980 consensus_queue.cc:260] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.535388  9980 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:08.535640 10165 raft_consensus.cc:3055] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:08.536163 10165 raft_consensus.cc:740] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:08.542965  9963 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:08.543535  9963 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:08.550940  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1492) took 1306 ms. Trace:
I20260709 13:02:08.551386  8251 rpcz_store.cc:276] 0709 13:02:07.180593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.508905 (+328312us) service_pool.cc:224] Handling call
0709 13:02:07.540173 (+ 31268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.540414 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:07.554921 (+ 14507us) log.cc:844] Serialized 14 byte log entry
0709 13:02:08.284680 (+729759us) spinlock_profiling.cc:243] Waited 1.36 ms on lock 0x7b18002520a0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e23a 00007f88287c8a2a
0709 13:02:08.367756 (+ 83076us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:08.374084 (+  6328us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:08.440114 (+ 66030us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.468290 (+ 28176us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:08.487027 (+ 18737us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34423,"prepare.queue_time_us":13303,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":1865,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":40362,"spinlock_wait_cycles":3118464,"thread_start_us":58246,"threads_started":1,"wal-append.queue_time_us":889337}
I20260709 13:02:08.553495  8399 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:39308
W20260709 13:02:08.563176  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.564427 10122 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:08.567294  9910 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.568123  9910 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:08.572968 10180 log.cc:927] Time spent T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 1.227s	user 0.001s	sys 0.005s
I20260709 13:02:08.566924  8369 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:02:08.576321  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.577737  8974 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:08.578671 10227 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:08.579552 10227 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.580181 10094 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:08.580693 10094 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:08.581104 10094 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:08.584328  8406 raft_consensus.cc:1240] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 13:02:08.591452  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:08.592849 10185 log.cc:927] Time spent T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 1.227s	user 0.007s	sys 0.005s
W20260709 13:02:08.592921  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 78741us
I20260709 13:02:08.593509  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 233) took 1097 ms. Trace:
I20260709 13:02:08.593920  8398 rpcz_store.cc:276] 0709 13:02:07.496173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.615632 (+119459us) service_pool.cc:224] Handling call
0709 13:02:07.615881 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.433554 (+817673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.593187 (+159633us) spinlock_profiling.cc:243] Waited 791 ms on lock 0x7b54002689d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:08.593413 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1818969856}
W20260709 13:02:08.594432 10187 log.cc:927] Time spent T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 1.249s	user 0.005s	sys 0.000s
W20260709 13:02:08.594558  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.597536  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1442) took 1420 ms. Trace:
I20260709 13:02:08.598016  8328 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:08.597889  8248 rpcz_store.cc:276] 0709 13:02:07.177230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.236534 (+ 59304us) service_pool.cc:224] Handling call
0709 13:02:07.236824 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:07.242474 (+  5650us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:07.242502 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:07.242970 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 13:02:07.318402 (+ 75432us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:07.318458 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.318532 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:08.593593 (+1275061us) spinlock_profiling.cc:243] Waited 30.3 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:08.595807 (+  2214us) raft_consensus.cc:1685] finished
0709 13:02:08.595825 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:08.597329 (+  1504us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5047,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":109,"spinlock_wait_cycles":69772672,"thread_start_us":75064,"threads_started":1,"wal-append.queue_time_us":101776}
I20260709 13:02:08.599143 10112 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:08.600359  9008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 380) took 1295 ms. Trace:
I20260709 13:02:08.600759  9008 rpcz_store.cc:276] 0709 13:02:07.305045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.308683 (+  3638us) service_pool.cc:224] Handling call
0709 13:02:07.373392 (+ 64709us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.404102 (+ 30710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.599989 (+1195887us) spinlock_profiling.cc:243] Waited 64.4 ms on lock 0x7b54004c2550. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:08.600192 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":148117760}
I20260709 13:02:08.602401  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:08.603318  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:08.603631  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:08.604200  8254 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1448) took 1262 ms (client timeout 1623 ms). Trace:
I20260709 13:02:08.604445  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:08.604717  8254 rpcz_store.cc:269] 0709 13:02:07.341781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.345351 (+  3570us) service_pool.cc:224] Handling call
0709 13:02:08.603869 (+1258518us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b54003f1bd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:08.604009 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1064769920}
I20260709 13:02:08.605269  8327 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:02:08.605643  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:08.605927  8324 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:08.606609  8332 raft_consensus.cc:2393] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 3: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:08.604873 10204 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.607777  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:08.608768  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:08.609220  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:08.609504  8254 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
I20260709 13:02:08.609987  8248 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:08.608781  8326 raft_consensus.cc:2393] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 4: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:08.611806  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1479) took 1114 ms. Trace:
I20260709 13:02:08.612238  8403 rpcz_store.cc:276] 0709 13:02:07.497117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.749383 (+252266us) service_pool.cc:224] Handling call
0709 13:02:08.259095 (+509712us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.260313 (+  1218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.611616 (+351303us) spinlock_profiling.cc:243] Waited 507 ms on lock 0x7b54000fcdd0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:08.611751 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1165016960}
I20260709 13:02:08.499092  9003 rpcz_store.cc:276] 0709 13:02:06.089288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.090987 (+  1699us) service_pool.cc:224] Handling call
0709 13:02:06.793369 (+702382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:06.801007 (+  7638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.498097 (+1697090us) spinlock_profiling.cc:243] Waited 638 ms on lock 0x7b54006dcc50. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:08.498357 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1466299264}
I20260709 13:02:08.615358 10169 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:02:08.615559  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 48945.1us
I20260709 13:02:08.615773  9898 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.616036 10169 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:08.616429  9898 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.618834 10169 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:08.618908  9898 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:08.620967  8298 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:08.621667 10170 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:08.622586 10227 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:08.626262 10207 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:08.626935 10097 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.617167  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:08.611819  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:08.630424 10204 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:08.632776  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 44618.8us
I20260709 13:02:08.637041  9007 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:08.644145  8399 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:08.577026 10104 raft_consensus.cc:2955] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 4 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:08.636121 10035 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:08.664307  8399 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:08.660979  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:08.666189  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:08.666530  8257 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
W20260709 13:02:08.668763  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 146035us
W20260709 13:02:08.672248  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 51872.9us
I20260709 13:02:08.647715 10072 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:02:08.673457  9008 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 4 LEARNER]: Advancing to term 6
I20260709 13:02:08.537308 10165 consensus_queue.cc:260] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.687933 10165 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:08.692355 10164 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:08.692559  8397 raft_consensus.cc:1275] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:08.693934  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:08.695528  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:08.696075 10232 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.267s	user 0.004s	sys 0.000s
I20260709 13:02:08.697685 10152 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.703241  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:08.671223 10207 consensus_queue.cc:1059] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:08.704516 10207 raft_consensus.cc:3055] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:08.704895 10207 raft_consensus.cc:740] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:08.705870 10207 consensus_queue.cc:260] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.707243 10207 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:08.676326  9989 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:08.676776  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1748) took 1313 ms. Trace:
I20260709 13:02:08.713569  8400 rpcz_store.cc:276] 0709 13:02:07.363724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.367015 (+  3291us) service_pool.cc:224] Handling call
0709 13:02:08.616928 (+1249913us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b5800170798. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699eee 0000000000375939 00007f882d70d1e0 00007f882d7d60c4 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:02:08.676702 (+ 59774us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28861824}
I20260709 13:02:08.714991  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:08.715760  8400 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:08.716776  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:08.713572 10204 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.718468 10204 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:02:08.720914 10204 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:08.678053  8404 raft_consensus.cc:2393] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 2: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:08.722605  8404 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:08.723359  9910 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:08.724164  8404 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:08.678114  8972 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:08.725911  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 47875.8us
I20260709 13:02:08.682325  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 1770) took 1577 ms. Trace:
I20260709 13:02:08.713025  9900 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:08.728300  8325 rpcz_store.cc:276] 0709 13:02:07.104917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.480078 (+375161us) service_pool.cc:224] Handling call
0709 13:02:07.484120 (+  4042us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.502328 (+ 18208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.681887 (+1179559us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b5400064d54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:08.682147 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37069184}
I20260709 13:02:08.729748 10098 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:08.682207  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:08.682966  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 1773) took 1485 ms. Trace:
W20260709 13:02:08.684222  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.665823 10170 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.703397 10090 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:08.708447  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.672570  8372 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:08.675419  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 54547.8us
W20260709 13:02:08.709146  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.704070  9007 raft_consensus.cc:2393] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 3: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:08.712965  9989 raft_consensus.cc:697] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:02:08.734536  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 62042.3us
I20260709 13:02:08.663165  9898 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:08.735107  8222 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:08.673882  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:08.740338 10127 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.741361  9989 consensus_queue.cc:237] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.743712  8401 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:08.732304  8330 rpcz_store.cc:276] 0709 13:02:07.187236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.725931 (+538695us) service_pool.cc:224] Handling call
0709 13:02:07.836147 (+110216us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.836836 (+   689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.663844 (+827008us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b540037ad50. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:08.672963 (+  9119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":233086208}
W20260709 13:02:08.763144  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:08.765372  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.770030  9008 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 6 LEARNER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:02:08.771067  8400 raft_consensus.cc:2393] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 4: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:08.737983 10098 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.746985 10079 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:08.763372  9945 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.796236  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:08.746768  9005 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:08.703262 10219 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:08.804587  8222 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:08.772204 10127 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.794209 10122 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:08.813670  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:08.817298  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 85912.2us
I20260709 13:02:08.794034 10213 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.770669  8404 raft_consensus.cc:2955] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:02:08.824661  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:08.825146  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:08.825637  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.824512  9945 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.768359 10094 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.845233 10094 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:08.842743  9008 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:08.846346  8325 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
I20260709 13:02:08.814337 10079 raft_consensus.cc:697] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:02:08.861915  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 62292.5us
W20260709 13:02:08.851339 10038 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.317s	user 0.004s	sys 0.009s
I20260709 13:02:08.854332  8250 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:02:08.848727 10111 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:08.845512 10217 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.117s
I20260709 13:02:08.865463  8372 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:08.867303  8222 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:02:08.865231 10038 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.518s	user 0.004s	sys 0.010s
I20260709 13:02:08.875973  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:08.876109  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:08.825532 10098 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.880788  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:08.885676  9008 raft_consensus.cc:2410] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:08.888307  9784 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 13:02:08.874634 10204 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:08.895900 10031 consensus_queue.cc:1059] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:08.897576  8253 raft_consensus.cc:1240] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:08.861620 10213 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.888653 10107 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:08.874208 10138 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:02:08.900123  9003 raft_consensus.cc:1275] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:08.886502 10035 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:08.887600  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:08.899217 10117 consensus_queue.cc:1048] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 13:02:08.903168 10111 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.917857  8400 raft_consensus.cc:2393] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 3: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:08.917917 10111 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:08.918999  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:08.920164 10111 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:08.900980 10163 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:02:08.920313  8399 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:08.921751 10202 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:02:08.913836 10238 log.cc:927] Time spent T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:02:08.922585  8405 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:08.890414 10090 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.925649  9008 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:02:08.925822 10090 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:08.928779 10090 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:08.928818  9005 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:08.930321 10127 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:08.929942 10073 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:08.884474 10094 consensus_queue.cc:1059] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:02:08.932547 10127 raft_consensus.cc:3055] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:08.932901 10127 raft_consensus.cc:740] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:08.933748 10127 consensus_queue.cc:260] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.934514 10098 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:08.898435 10204 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.936998  9803 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:08.939272  8399 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:08.941882  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:08.942255  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:08.942807 10079 consensus_queue.cc:237] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.943315  9970 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.943948  9970 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:08.941891  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:08.946977  9002 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
W20260709 13:02:08.948562  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 46567.2us
W20260709 13:02:08.951876  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.951851  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:08.952667  9008 raft_consensus.cc:2393] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 4: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:08.939697  9803 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:08.958582  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:08.959182  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:08.959607  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.916617 10219 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:08.959977  8972 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:08.957715  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:08.915897 10170 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:08.940191 10204 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:08.968155  8331 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
W20260709 13:02:08.968555 10235 log.cc:927] Time spent T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.230s	user 0.001s	sys 0.001s
I20260709 13:02:08.963197 10217 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:08.969053 10207 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:08.968926  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:08.970283 10127 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:08.973707  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:09.023672  9007 raft_consensus.cc:2393] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 4: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:08.915553  9963 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:08.974262 10073 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:08.977100  9009 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:08.948298  8400 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:08.958043  9803 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:09.037807  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:09.049073  8323 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:08.990644  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:08.990998  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:08.995103  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 165478us
I20260709 13:02:08.914184 10107 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.001830  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:09.002672  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:09.003641  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:09.004398  9971 consensus_queue.cc:1059] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:02:09.006719 10219 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.013389  8974 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:09.049669  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:09.061748  8257 raft_consensus.cc:2410] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:08.997275 10207 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.011924 10202 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:09.032397 10240 log.cc:927] Time spent T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 13:02:09.054993  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 81174us
I20260709 13:02:09.028059  9005 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:09.029896  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:08.978081  9898 consensus_queue.cc:1059] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
W20260709 13:02:09.068526  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 76552.7us
I20260709 13:02:09.037582  9903 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.077874  9008 raft_consensus.cc:1275] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:02:09.046474 10213 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:09.046591 10138 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:09.082319  8249 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:02:08.964745  9945 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:09.085704  9008 pending_rounds.cc:85] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e: Aborting all ops after (but not including) 5
I20260709 13:02:09.090135  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:09.090837  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.069377 10207 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:09.095232  8404 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
I20260709 13:02:09.099103 10112 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.062269  9970 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:09.032585 10217 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.093950  9008 pending_rounds.cc:107] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:02:09.125365  9008 raft_consensus.cc:2889] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:09.125416  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:09.093698 10165 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:09.069242 10163 consensus_queue.cc:1059] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:02:09.088727 10072 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.092829 10117 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.133160  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:09.099116  9761 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.134106  8398 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 2. Current term is 3. Ops: [2.5-2.5]
W20260709 13:02:09.135527  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 48850.9us
I20260709 13:02:09.135598  8403 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
I20260709 13:02:09.131323  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:09.110879  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:09.113036 10073 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.137794  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:09.110005  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:09.134414  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 47978.5us
I20260709 13:02:09.109697  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:09.136870  8252 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:09.141746  9007 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:09.135174  8332 raft_consensus.cc:2376] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2. Re-sending same reply.
I20260709 13:02:09.082373 10202 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.143739  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:09.112643 10111 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:09.093159 10219 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:09.093098  9898 consensus_queue.cc:1243] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:02:09.128026 10165 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:09.158376  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:09.161011  8298 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:09.164208  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:09.167936  8328 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:02:09.169552 10227 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:09.169293 10204 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:09.172536 10198 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:09.173249 10198 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.174757 10204 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:09.175807 10198 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:09.202521  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.131652  9903 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.207154  8250 raft_consensus.cc:2393] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 4: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:09.207958  8323 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:09.208307 10217 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:09.147121 10138 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:02:09.202420  9957 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:09.222592 10111 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.223261  9957 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.223591 10138 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:09.223686  9957 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:09.225039 10111 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:09.225989  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:09.226646  8399 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:09.226830 10202 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:09.228606 10038 raft_consensus.cc:3055] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:09.229027 10038 raft_consensus.cc:740] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:09.209561  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:09.204071  8327 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:09.142483  9761 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.231617 10207 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:09.204229  9004 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:09.234172 10207 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:09.141865 10112 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.236497 10107 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:09.236972 10112 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:09.167420 10165 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:09.238458  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:09.239620  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:09.240712  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:09.152899 10073 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.210031  8406 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:02:09.243258  8324 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:02:09.210500  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:09.157116  9003 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:02:09.218273 10219 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:09.220291 10090 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:09.160692 10117 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:09.222901 10233 log.cc:927] Time spent T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.420s	user 0.003s	sys 0.000s
I20260709 13:02:09.209949  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:09.139259 10072 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.230072 10038 consensus_queue.cc:260] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:09.263002  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:09.263375  8222 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:09.253316 10217 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:09.257189  8403 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:09.162606  9784 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 13:02:09.255614 10207 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.267781  8332 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:09.257921  8295 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:09.251174  8399 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:09.269335  8406 raft_consensus.cc:2393] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 5: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:09.273449  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:09.282202  8222 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:09.285138  8220 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:09.273864 10198 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:09.288872  8369 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:09.281602  8330 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:02:09.263751  8401 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
W20260709 13:02:09.295539  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 51517.5us
I20260709 13:02:09.297886 10259 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:09.298362 10259 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.298807 10259 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:09.301326  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.301789  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:09.302208  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:09.302800  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:09.303963  8972 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:09.274194 10111 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:09.265280 10090 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.304754 10127 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:09.283269 10219 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.283043 10207 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:09.308480  9761 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:09.283847  9900 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:09.290784  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:09.345237  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:09.332185  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 47105.8us
I20260709 13:02:09.272174 10112 consensus_queue.cc:1059] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:02:09.295792 10204 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:09.348644  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.302129 10073 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:09.349910  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 77241.4us
I20260709 13:02:09.350419 10111 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:09.350674 10219 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.292214 10152 consensus_queue.cc:1059] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:02:09.288524 10217 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:09.302129  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:09.354697  9903 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:09.355304 10260 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:09.358242 10198 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.361739  8327 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:09.437003 10198 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:09.439144 10198 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:09.362422 10204 raft_consensus.cc:3055] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:09.439810 10204 raft_consensus.cc:740] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:09.363956 10117 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:09.440819 10204 consensus_queue.cc:260] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.342557 10038 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:09.340337 10127 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:09.331745  9957 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:09.366217  8324 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:09.367678  8974 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:09.445958  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 78375.2us
I20260709 13:02:09.368052  8972 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:09.369056  8297 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:09.371507  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:09.447639  8974 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:08.693631 10078 consensus_queue.cc:1175] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e64761f43e4240319c41117cf886d22f" responder_term: 5 status { last_received { term: 2 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 4. Current term is 5. Ops: [4.6-4.6]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:09.376232  9803 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:09.377758  8249 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1487) took 2198 ms (client timeout 1974 ms). Trace:
I20260709 13:02:09.378674  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 1789) took 1048 ms. Trace:
I20260709 13:02:09.380218  8220 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:09.385351  8223 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:09.360888 10165 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:09.390552 10259 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.390884  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:09.683790  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 304299us
I20260709 13:02:09.684242  8404 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:09.394202 10090 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:09.421027  8403 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:09.282618 10138 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 13:02:09.666376 10227 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:09.669798 10263 log.cc:927] Time spent T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.248s	user 0.000s	sys 0.009s
W20260709 13:02:09.693794  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 422250us
I20260709 13:02:09.694411 10138 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:09.693843 10227 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:09.418133  9963 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:09.433250 10207 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:09.695523 10207 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:09.695933 10207 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.696296 10207 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:09.696796 10259 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:09.434901 10217 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:09.436196  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:09.669674  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 222165us
I20260709 13:02:09.623479 10092 raft_consensus.cc:3055] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:09.701594 10092 raft_consensus.cc:740] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:09.702521 10165 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:09.702461 10092 consensus_queue.cc:260] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.704120 10227 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:09.704533 10219 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:09.706048 10202 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:09.706751  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:09.706586 10202 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.708688 10202 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:09.617218  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:09.471999  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1494) took 1702 ms. Trace:
I20260709 13:02:09.463970  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:09.679142 10262 log.cc:927] Time spent T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.252s	user 0.000s	sys 0.009s
I20260709 13:02:09.619146 10097 raft_consensus.cc:3055] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:09.621264 10111 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:09.659855 10078 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:09.450135 10260 raft_consensus.cc:697] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 4 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:09.459818 10204 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:09.619099  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:09.668228  9970 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.623172 10220 raft_consensus.cc:2955] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 2 to 5, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:02:09.617241  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 248272us
I20260709 13:02:09.617455 10035 raft_consensus.cc:2764] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:02:09.613554 10170 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:09.620841 10031 consensus_queue.cc:1059] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:09.717913 10097 raft_consensus.cc:740] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:02:09.614539  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 262845us
I20260709 13:02:09.458354  9989 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 13:02:09.623678  9005 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:09.619849  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 236) took 1290 ms. Trace:
I20260709 13:02:09.710503  8402 rpcz_store.cc:276] 0709 13:02:07.769933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.187378 (+417445us) service_pool.cc:224] Handling call
0709 13:02:08.311734 (+124356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:08.343488 (+ 31754us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:08.343527 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:08.365123 (+ 21596us) log.cc:844] Serialized 26 byte log entry
0709 13:02:08.487852 (+122729us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:08.649512 (+161660us) log.cc:844] Serialized 14 byte log entry
0709 13:02:08.649945 (+   433us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.650030 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:08.699906 (+ 49876us) spinlock_profiling.cc:243] Waited 1.38 ms on lock 0x7b1800358c60. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d1522 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:02:08.796135 (+ 96229us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b1800358c60. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d1557 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:02:09.222150 (+426015us) spinlock_profiling.cc:243] Waited 4.05 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:09.351499 (+129349us) raft_consensus.cc:1685] finished
0709 13:02:09.351523 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.471653 (+120130us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54000fd7d0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:09.471788 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":153986,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":55,"raft.queue_time_us":13167,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":3226,"spinlock_wait_cycles":320535808,"thread_start_us":151777,"threads_started":2,"wal-append.queue_time_us":310535}
I20260709 13:02:09.449707 10127 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:09.450330 10213 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:09.445180 10079 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:02:09.624465  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:02:09.682942  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 244340us
W20260709 13:02:09.747189  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.719858 10198 consensus_queue.cc:1059] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
W20260709 13:02:09.452147  8249 rpcz_store.cc:269] 0709 13:02:07.179129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.191505 (+ 12376us) service_pool.cc:224] Handling call
0709 13:02:09.377395 (+2185890us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b540033ead4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:09.377592 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3948826496}
I20260709 13:02:09.453150  8331 rpcz_store.cc:276] 0709 13:02:08.330543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.341663 (+ 11120us) service_pool.cc:224] Handling call
0709 13:02:09.378490 (+1036827us) spinlock_profiling.cc:243] Waited 548 ms on lock 0x7b540037a354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:09.378590 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1260987648}
I20260709 13:02:09.694725  8223 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:09.697949  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:09.653743 10251 log.cc:927] Time spent T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.477s	user 0.002s	sys 0.007s
I20260709 13:02:09.710124  8406 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:09.716710  8220 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:02:09.718786  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 350819us
I20260709 13:02:09.709203  9000 raft_consensus.cc:2955] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:09.724541 10260 consensus_queue.cc:237] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.710402 10152 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:09.729852  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:09.735723  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 289589us
I20260709 13:02:09.614432  9900 raft_consensus.cc:697] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:09.742219  8398 rpcz_store.cc:276] 0709 13:02:08.333660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.596078 (+262418us) service_pool.cc:224] Handling call
0709 13:02:08.607818 (+ 11740us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:09.135687 (+527869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.484030 (+348343us) spinlock_profiling.cc:243] Waited 523 ms on lock 0x7b5400276854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:09.619775 (+135745us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1203742464}
I20260709 13:02:09.744803 10097 consensus_queue.cc:260] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:09.758080  9803 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:09.758435 10097 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:09.751359  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 56728.3us
I20260709 13:02:09.761866  9957 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:09.749315 10202 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.736544  9970 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.762923 10202 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.764372  9970 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:09.764866 10202 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:09.753566 10152 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.767143 10152 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:09.751691  8220 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:09.752188  8403 raft_consensus.cc:2410] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:09.754266  8295 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:09.755263  9900 consensus_queue.cc:237] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:09.753643 10111 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:09.753438 10219 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:09.750221  9004 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:09.746209 10213 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.775444  8223 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:09.776997  8220 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:09.750602  8372 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:09.775135  8295 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:02:09.777637  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.776413  9970 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:09.778637  9970 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.779023  9970 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:09.742467 10170 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.781347 10259 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:09.786619 10117 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:09.787153 10253 log.cc:927] Time spent T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.543s	user 0.007s	sys 0.007s
I20260709 13:02:09.787688  8220 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:09.788380 10138 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:09.789037  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:09.789733  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:09.801203 10170 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:09.801216  9005 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:09.803751 10261 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:09.804212 10261 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:09.804632 10261 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:09.805595 10202 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:02:09.806177 10267 log.cc:927] Time spent T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.179s	user 0.000s	sys 0.005s
I20260709 13:02:09.794129  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:09.809763  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:09.810407  8403 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:09.805630  8401 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:09.809728  8406 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:02:09.785280  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:09.818046  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.793977  8222 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:09.776494 10217 raft_consensus.cc:3055] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:09.797801  8295 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:09.781728 10213 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:09.777912 10272 consensus_queue.cc:1048] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:02:09.794644 10117 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.774901 10111 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:09.826695 10111 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:09.827118 10092 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:09.821730 10217 raft_consensus.cc:740] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:09.806131 10202 raft_consensus.cc:697] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:09.830580 10217 consensus_queue.cc:260] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:09.834198 10217 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:09.834102 10202 consensus_queue.cc:237] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:09.838820 10118 raft_consensus.cc:3055] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:09.795571  8220 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:09.839314 10118 raft_consensus.cc:740] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:02:09.839988  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 44509us
I20260709 13:02:09.840217 10118 consensus_queue.cc:260] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:09.840943 10259 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:09.841348 10259 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:09.841797 10259 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:09.847765  8154 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45434 (request call id 280) took 8125 ms. Trace:
I20260709 13:02:09.826627 10117 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:09.801682  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:09.783754 10207 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:09.855844 10207 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:09.857460  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 59694.9us
I20260709 13:02:09.859719  8222 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:09.792199 10072 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:09.791548 10163 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:09.869379  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 74236us
I20260709 13:02:09.870339  8223 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:09.871599  8223 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:09.873817  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.822093  9803 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:09.874985  9803 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.875360  9803 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:09.878054  8295 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:09.880363  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:09.880262 10127 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:09.881825  8972 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:09.882551 10127 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:09.882917 10078 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:09.883185 10127 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:09.883621 10127 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.883461 10213 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:09.883986 10127 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:09.885655 10038 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:02:09.887486  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 67539.2us
W20260709 13:02:09.796203  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 45687.8us
I20260709 13:02:09.863869 10122 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:09.862260 10097 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.864956 10118 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:09.883499 10078 raft_consensus.cc:697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:09.863739 10123 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.896522  9004 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:09.898828  8223 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:09.886359 10035 consensus_queue.cc:1059] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:02:09.886653 10038 raft_consensus.cc:697] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:09.897899 10170 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.891512 10090 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:09.891769 10219 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:09.900876  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:09.955394  8330 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:09.862350  8403 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:09.914085  9761 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.914538  9971 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:02:09.891105  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 58198us
I20260709 13:02:09.915019 10163 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.915750  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:09.963840  8331 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:09.920172  9970 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.920872  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:09.921126 10213 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:09.922593  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:09.885138 10261 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.927635  8324 raft_consensus.cc:1275] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:09.929337  8406 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:09.931175 10152 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:09.970588  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 72137.8us
I20260709 13:02:09.971993  8223 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:09.967936  8372 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:09.952736  9005 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:09.931490  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:09.936980  8151 catalog_manager.cc:5697] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:09.939904  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:09.982967  8398 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:09.942001  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:09.942538  8297 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:09.945062  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:09.946689 10117 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:09.943583 10031 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:09.966769 10163 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.943791  9957 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.998179  9957 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.948954  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:09.999742  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:10.000322  9003 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:02:09.949589  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:10.001811  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:10.002318  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:09.848186  8154 rpcz_store.cc:276] 0709 13:02:01.722172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.838721 (+116549us) service_pool.cc:224] Handling call
0709 13:02:09.847688 (+8008967us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:08.596463 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:08.614247 (+ 17784us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:08.614298 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:02:08.614315 (+    17us) tablet.cc:662] Decoding operations
0709 13:02:08.619237 (+  4922us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:08.619352 (+   115us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:08.619392 (+    40us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:02:08.619789 (+   397us) tablet.cc:701] Row locks acquired
0709 13:02:08.619852 (+    63us) write_op.cc:260] PREPARE: finished
0709 13:02:08.620052 (+   200us) write_op.cc:270] Start()
0709 13:02:08.620219 (+   167us) write_op.cc:276] Timestamp: P: 1783602128619984 usec, L: 0
0709 13:02:08.620267 (+    48us) op_driver.cc:348] REPLICATION: starting
0709 13:02:08.628248 (+  7981us) log.cc:844] Serialized 591 byte log entry
0709 13:02:09.150226 (+521978us) op_driver.cc:464] REPLICATION: finished
0709 13:02:09.618016 (+467790us) write_op.cc:301] APPLY: starting
0709 13:02:09.618124 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:09.618267 (+   143us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:09.628684 (+ 10417us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:09.695597 (+ 66913us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:09.695840 (+   243us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:02:09.695916 (+    76us) write_op.cc:312] APPLY: finished
0709 13:02:09.696147 (+   231us) log.cc:844] Serialized 23 byte log entry
0709 13:02:09.696353 (+   206us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:09.696471 (+   118us) write_op.cc:454] Released schema lock
0709 13:02:09.696543 (+    72us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:09.697052 (+   509us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:09.697093 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":946496,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":467486,"apply.run_cpu_time_us":10252,"apply.run_wall_time_us":78995,"num_ops":1,"prepare.queue_time_us":248476,"prepare.run_cpu_time_us":26211,"prepare.run_wall_time_us":328535,"raft.queue_time_us":133934,"raft.run_cpu_time_us":47142,"raft.run_wall_time_us":318733,"replication_time_us":511033,"spinlock_wait_cycles":14720,"thread_start_us":654763,"threads_started":4,"wal-append.queue_time_us":329285}]]}
I20260709 13:02:10.003229  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:10.003921  8252 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:10.004287  8372 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:09.953292  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:09.912875 10079 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:10.005314  8405 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:10.006418 10079 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:10.006886 10079 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.007160  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:10.006820 10038 consensus_queue.cc:237] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:10.007512 10079 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:10.007862  8249 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
I20260709 13:02:10.008860  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:10.009609  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.009857  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:10.010612  8253 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:10.011404 10035 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:10.011067 10078 consensus_queue.cc:237] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:10.012270  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:10.013586  9000 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:10.013711  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:10.013924  8405 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:09.958004 10122 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:10.019141  9957 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:10.019578 10122 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:09.948462 10271 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:09.958215  9784 raft_consensus.cc:3055] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:09.953385  8401 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:09.935988 10111 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.958382 10217 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:09.962428 10170 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:09.962728 10090 raft_consensus.cc:697] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 4 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:09.985801  8222 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:02:09.988461  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 45997.4us
I20260709 13:02:09.989183  8402 raft_consensus.cc:2368] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 3. Current term is 4.
I20260709 13:02:09.994901  8399 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:09.997092 10031 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:10.009943  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:09.975713  8329 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
I20260709 13:02:09.965063  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:10.055023  8325 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:10.005009  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:09.954064 10127 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:09.984135  8332 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:10.057554  8256 raft_consensus.cc:2368] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 3. Current term is 4.
I20260709 13:02:10.025758  8249 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:10.060582  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:10.032099  8331 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:09.964102  9761 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:09.967381  9971 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:10.035329  8974 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:09.989701 10072 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:02:09.957857 10123 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:10.034617  8298 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:10.073249  9007 raft_consensus.cc:1240] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:02:09.979285 10202 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:09.897907 10198 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:09.981478 10104 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:10.077064 10202 raft_consensus.cc:697] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:09.976926  8326 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:10.043901  8151 catalog_manager.cc:5697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:10.046178  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:02:10.055570  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 69852.8us
I20260709 13:02:10.052547  8297 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:10.086032  8297 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:10.037253  9784 raft_consensus.cc:740] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:10.056447  8324 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:10.062666  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:10.045914  9001 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:02:10.064323  8325 raft_consensus.cc:1275] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:10.065924  8328 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:02:10.067355  8974 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:10.110214  8974 leader_election.cc:400] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 3. Current term is 4.
I20260709 13:02:10.110528  8974 leader_election.cc:403] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:02:10.037494  8248 raft_consensus.cc:2393] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 4: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:10.111407 10213 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:10.111907 10213 raft_consensus.cc:697] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:10.112462 10207 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:10.114874  9761 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:10.116268 10170 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:10.117054  8257 raft_consensus.cc:2393] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 4: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:10.021358 10138 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:02:10.069983  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:10.023298 10259 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:10.076979  9963 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:10.075783 10198 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.064805 10092 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:10.086891 10220 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:10.086649 10202 consensus_queue.cc:237] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:10.087589 10270 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:02:10.087913 10266 log.cc:927] Time spent T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.242s	user 0.001s	sys 0.001s
I20260709 13:02:10.088406 10090 consensus_queue.cc:237] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:10.088801  9803 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:10.089258  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:10.095577  8253 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:10.096643  8297 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:10.097990  8331 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:10.099066  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:10.093005  8151 catalog_manager.cc:5697] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 2 to 3, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 3 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
I20260709 13:02:10.265350  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:10.123013  8972 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:10.279419  8249 raft_consensus.cc:2393] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 4: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:10.265604  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:10.247419  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 193880us
I20260709 13:02:10.241461  8295 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:10.280694  9009 raft_consensus.cc:2393] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 6: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
W20260709 13:02:10.280675  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 184126us
I20260709 13:02:10.282333  8255 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:10.280042  8222 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:10.292177  8151 catalog_manager.cc:5697] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 1 to 2, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 2 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:10.293870  8151 catalog_manager.cc:5697] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:10.296203  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 54875.3us
I20260709 13:02:10.296065  8151 catalog_manager.cc:5697] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:10.297480  8295 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:10.298637  9761 raft_consensus.cc:2764] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:02:10.297534  8151 catalog_manager.cc:5697] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 1 to 2, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:10.150403  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:10.194676  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:10.373672  8254 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:10.299187  8295 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:10.381150  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 99400.6us
I20260709 13:02:10.382382  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:10.383225  8253 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:10.147097 10276 log.cc:927] Time spent T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:02:10.387058  8151 catalog_manager.cc:5697] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:10.388937  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 89850.9us
I20260709 13:02:10.390048  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:10.409718  8405 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:10.394876  8331 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:02:10.118805  9900 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:10.396854  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:10.403602  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:10.159298 10269 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.195695 10259 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:10.120820 10261 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:10.155628 10272 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:10.269767  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:10.265707 10138 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:10.234702 10268 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.276736 10117 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:10.283783  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:10.271633 10031 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:10.247371  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:10.411464  8151 catalog_manager.cc:5697] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:10.514688  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:10.787972  8973 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.027s	sys 0.097s
I20260709 13:02:10.420899  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:10.746547  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 585402us
I20260709 13:02:10.542320  8253 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
W20260709 13:02:10.911834  8973 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.028s	sys 0.108s
I20260709 13:02:10.304127 10152 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:10.912492  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:10.839829 10227 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.914713 10227 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:10.378598 10205 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:10.919036 10205 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.592142  9900 raft_consensus.cc:697] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:10.698818  8250 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 4: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:10.147020  9971 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:10.938122  9001 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:10.938354  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:10.940850  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:10.941691  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:10.942027  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.942914  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:10.943866  9005 raft_consensus.cc:2393] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 5: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:10.949028 10038 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:10.949651 10038 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:10.951655 10038 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:10.284389 10198 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:10.593825  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:10.601918  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 187628us
I20260709 13:02:10.963387  9970 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:10.594091 10163 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:10.965585  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:10.966358 10268 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:10.968169  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1593) took 1001 ms. Trace:
I20260709 13:02:10.968487 10268 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:10.968652  8324 rpcz_store.cc:276] 0709 13:02:09.966696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.987596 (+ 20900us) service_pool.cc:224] Handling call
0709 13:02:10.056089 (+ 68493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:10.097409 (+ 41320us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:10.967987 (+870578us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b540011cbd0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:10.968113 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26644224}
I20260709 13:02:10.411931 10213 consensus_queue.cc:237] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:10.970469  9784 consensus_queue.cc:260] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:10.971676  9784 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:10.971926  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:10.751740  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:10.977160  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:10.981527  8250 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:10.965335 10272 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:10.982791  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:10.983232  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:10.983875  8248 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:10.983981 10272 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:10.984503  8250 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:10.987833  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:10.697683  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1778) took 1572 ms. Trace:
W20260709 13:02:10.928932  8972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.007s	sys 0.003s
I20260709 13:02:10.978514 10138 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:10.992014  8972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.007s	sys 0.003s
W20260709 13:02:10.992569  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 869627us
I20260709 13:02:10.993254  8972 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:10.991312  8397 rpcz_store.cc:276] 0709 13:02:09.125046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.303286 (+178240us) service_pool.cc:224] Handling call
0709 13:02:09.327153 (+ 23867us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:09.341940 (+ 14787us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:09.341974 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:09.362628 (+ 20654us) log.cc:844] Serialized 26 byte log entry
0709 13:02:09.684807 (+322179us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:09.684860 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.684937 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:10.697346 (+1012409us) raft_consensus.cc:1685] finished
0709 13:02:10.697375 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:10.697593 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47706,"raft.queue_time_us":16431,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":6604,"thread_start_us":321767,"threads_started":1,"wal-append.queue_time_us":353535}
I20260709 13:02:10.996238  8972 leader_election.cc:400] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 3. Current term is 4.
I20260709 13:02:10.996410 10138 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:10.986515  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:10.512053  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:10.533519  9963 raft_consensus.cc:697] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:10.997788  8254 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:10.998371  9963 consensus_queue.cc:237] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:10.999372  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:11.000762  8251 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:02:10.299677 10104 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:10.968843  8253 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:11.004287  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:11.002096  8250 raft_consensus.cc:2393] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 6: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:10.981202  8402 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:10.374194 10123 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:11.010455  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:11.025789  8253 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:10.301645 10170 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:02:11.017321  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:11.004873  9006 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.000s	sys 0.009s
W20260709 13:02:11.048538  8398 consensus_meta.cc:287] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 1.062s	user 0.010s	sys 0.000s
I20260709 13:02:11.049712  8398 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:11.050544  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 248) took 1114 ms. Trace:
I20260709 13:02:11.051071  8398 rpcz_store.cc:276] 0709 13:02:09.936228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.939624 (+  3396us) service_pool.cc:224] Handling call
0709 13:02:11.050468 (+1110844us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:10.969048 10268 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.415165 10220 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:11.003643  8151 catalog_manager.cc:5697] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 2 to 4, now has a pending config: VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1), VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2), VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3), VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 4 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } } pending_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:11.065037  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:11.065918  8330 raft_consensus.cc:2393] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 6: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:11.003603 10122 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.004316 10204 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.920326 10205 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:11.069038  8251 raft_consensus.cc:1240] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 6. Ops: []
I20260709 13:02:11.085448  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:11.087496  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:11.087970  8253 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:11.086216  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:11.089236  8329 raft_consensus.cc:2410] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:11.003935 10127 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:10.927845  9803 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:11.062306  9008 raft_consensus.cc:2955] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:11.091039  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:11.091944  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:11.094027  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1466) took 1095 ms. Trace:
I20260709 13:02:11.094525  8251 rpcz_store.cc:276] 0709 13:02:09.998168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.015140 (+1016972us) service_pool.cc:224] Handling call
0709 13:02:11.068867 (+ 53727us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.093742 (+ 24875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.093965 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:11.049567 10170 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.061290 10152 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.101809 10170 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:11.060750  9957 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:11.111119 10272 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:11.112638  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:11.061865 10261 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.118063  8223 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:11.072774 10035 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:11.121767  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:11.144811  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:11.111582 10097 consensus_meta.cc:287] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.161s	user 0.003s	sys 0.005s
I20260709 13:02:11.090309 10104 raft_consensus.cc:697] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:11.151883  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49220 (request call id 250) took 1164 ms. Trace:
I20260709 13:02:11.018224 10269 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:11.122310 10072 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.157378  8257 rpcz_store.cc:276] 0709 13:02:09.987814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.988817 (+  1003us) service_pool.cc:224] Handling call
0709 13:02:11.151628 (+1162811us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b5400408154. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:11.151801 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":235809664}
I20260709 13:02:11.153947  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:11.121889 10284 log.cc:927] Time spent T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:02:11.154939 10111 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:11.156126  8220 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:11.157610  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 141043us
W20260709 13:02:11.159945  8369 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.037s	sys 0.061s
I20260709 13:02:11.161132  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:11.162204  9007 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 1.
W20260709 13:02:11.161033  8369 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.038s	sys 0.061s
W20260709 13:02:11.161928  8332 consensus_meta.cc:287] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.102s	user 0.009s	sys 0.000s
I20260709 13:02:11.164708  8332 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:11.121394 10227 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:11.167706  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 49711.1us
I20260709 13:02:11.168583  8220 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:11.080842 10268 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:11.172465  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 116686us
I20260709 13:02:11.175835  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 1587) took 1220 ms. Trace:
I20260709 13:02:11.090528 10122 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:11.178938  8332 rpcz_store.cc:276] 0709 13:02:09.944744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.944858 (+   114us) service_pool.cc:224] Handling call
0709 13:02:11.165264 (+1220406us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:11.186801  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:11.121655  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:11.194818  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 97148us
I20260709 13:02:11.195217  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:11.196064  8397 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 4: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:11.156088  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:11.158504 10138 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.202975  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:11.158443 10127 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:11.200703  8399 raft_consensus.cc:1275] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:11.096123 10204 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:11.175531  9989 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:11.160579  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:11.161893 10097 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.199090  8295 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:02:11.206715  8370 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.007s	sys 0.043s
W20260709 13:02:11.226145  8370 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.008s	sys 0.044s
W20260709 13:02:11.176451 10207 consensus_meta.cc:287] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 1.064s	user 0.009s	sys 0.000s
I20260709 13:02:11.212713  8251 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:11.122171 10152 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:11.145936 10261 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:11.056392  9900 consensus_queue.cc:237] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:11.155838 10035 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:11.124055  9957 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:11.224205  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:11.191308  9803 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:11.228204  8248 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:11.230396  8295 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:11.144197 10272 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:11.259610 10170 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:11.202934  9903 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.234752  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:11.205497 10219 consensus_meta.cc:287] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.248s	user 0.004s	sys 0.006s
I20260709 13:02:11.238305  9000 raft_consensus.cc:1275] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:02:11.238710 10220 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:11.241179  9971 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:11.241315 10213 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.217833 10104 consensus_queue.cc:237] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:11.241549 10117 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:11.215894 10293 consensus_queue.cc:1059] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:02:11.242101  9803 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:11.242522 10291 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:11.214138  8250 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
I20260709 13:02:11.242864 10035 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:11.204586 10227 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:02:11.244989 10204 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:11.238402 10299 consensus_queue.cc:1059] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.398s
I20260709 13:02:11.216316  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:11.217265 10268 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:11.245364  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:11.216137 10294 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.246873 10298 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.252281 10302 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:11.251228 10295 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.240612  8295 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:11.245016  9957 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:11.257978 10289 log.cc:927] Time spent T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.186s	user 0.000s	sys 0.000s
I20260709 13:02:11.258582 10292 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:11.175101 10072 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
W20260709 13:02:11.226459  8369 connection.cc:256] RPC call timeout handler was delayed by -1.2304611280000017s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1567s for a response
I20260709 13:02:11.229251  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:11.209726 10127 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.262550  8330 raft_consensus.cc:2393] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 3: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:02:11.219166 10078 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.947s	user 0.015s	sys 0.007s
I20260709 13:02:11.288729  8329 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:02:11.289961  8222 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:11.229105 10090 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:11.240502 10207 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 3. Current term is 4.
I20260709 13:02:11.302713  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:11.287863  9000 pending_rounds.cc:85] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e: Aborting all ops after (but not including) 5
I20260709 13:02:11.241199 10304 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.246634 10290 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:02:11.308614  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.287801  9971 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.232782 10138 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:11.233196  9989 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:11.344055  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:12.159909  8369 connection.cc:256] RPC call timeout handler was delayed by -1.22326112799999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
W20260709 13:02:11.625859  8402 consensus_meta.cc:287] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.610s	user 0.008s	sys 0.000s
I20260709 13:02:12.161374  8402 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:11.563898  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:12.163091  8257 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
W20260709 13:02:11.618465  8256 consensus_meta.cc:287] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.334s	user 0.009s	sys 0.000s
I20260709 13:02:12.164136  8256 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
W20260709 13:02:12.164629  8256 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1470) took 2165 ms (client timeout 2090 ms). Trace:
W20260709 13:02:12.164953  8256 rpcz_store.cc:269] 0709 13:02:09.999257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.069639 (+ 70382us) service_pool.cc:224] Handling call
0709 13:02:12.164558 (+2094919us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:11.287515 10219 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:12.166437  8248 raft_consensus.cc:2393] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 4: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:12.158249  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:12.167886  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 291068us
W20260709 13:02:12.168741  8297 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.005s	sys 0.006s
W20260709 13:02:12.169099  8297 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.005s	sys 0.006s
I20260709 13:02:11.648186  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:12.172735  8253 raft_consensus.cc:2393] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 5: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:02:11.364559  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.560331  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:12.174841  8404 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:12.176080  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:12.177098  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:12.170542  8975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.005s	sys 0.005s
I20260709 13:02:12.162192  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1805) took 1571 ms. Trace:
W20260709 13:02:12.177822  8975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.005s	sys 0.005s
I20260709 13:02:12.178035  8402 rpcz_store.cc:276] 0709 13:02:10.590460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.599089 (+  8629us) service_pool.cc:224] Handling call
0709 13:02:12.162119 (+1563030us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:12.179358  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:12.180173  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1487) took 1136 ms. Trace:
I20260709 13:02:12.177851  8405 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:12.181659  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:12.180183  8402 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
W20260709 13:02:12.158087  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 989035us
I20260709 13:02:11.559149  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:12.185307  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:12.186234  8403 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 1.
I20260709 13:02:12.184255  8220 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:12.218705 10317 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:12.219534 10317 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:12.229514 10317 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:12.230333 10317 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:12.231025 10317 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:12.245158 10317 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:12.246027 10317 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.246735 10317 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:12.250146 10317 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:12.180807  8253 rpcz_store.cc:276] 0709 13:02:11.044184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.096386 (+ 52202us) service_pool.cc:224] Handling call
0709 13:02:11.647993 (+551607us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b5800002a98. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:02:12.180117 (+532124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9705856}
I20260709 13:02:12.261932  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:12.262092  8405 raft_consensus.cc:2393] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 3: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:02:11.563293 10079 consensus_meta.cc:287] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 1.555s	user 0.011s	sys 0.000s
W20260709 13:02:11.764050 10306 log.cc:927] Time spent T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 13:02:11.564791 10198 consensus_meta.cc:287] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.611s	user 0.006s	sys 0.000s
I20260709 13:02:12.166013  9000 pending_rounds.cc:107] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:02:11.852934 10038 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:12.292716  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:11.287544  9903 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:11.337579 10170 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:11.860157 10097 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:12.308188  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1467) took 2309 ms. Trace:
I20260709 13:02:12.308712  8248 rpcz_store.cc:276] 0709 13:02:09.998452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.262841 (+2264389us) service_pool.cc:224] Handling call
0709 13:02:12.298190 (+ 35349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.307759 (+  9569us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:12.307800 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:12.307862 (+    62us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:12.307911 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.307992 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.308118 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:12.293350  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:12.293820  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:12.294550  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:12.310604  8402 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
W20260709 13:02:11.829955  9900 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.017s	sys 0.008s
W20260709 13:02:12.180202 10078 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.904s	user 0.020s	sys 0.025s
I20260709 13:02:12.297566  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:12.177793 10326 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:11.562294 10122 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:11.961650 10293 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.000s	sys 0.017s
I20260709 13:02:11.560043 10269 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:11.562620 10261 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:12.193701 10127 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:12.314533  9900 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.069s	user 0.027s	sys 0.013s
W20260709 13:02:12.305466  8298 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.002s	sys 0.031s
I20260709 13:02:11.968883 10303 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:12.310629  8257 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:11.337976 10291 raft_consensus.cc:697] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 6 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:12.312584  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:12.166340  9971 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:11.338553 10279 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:12.298096 10079 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:11.301050 10213 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:12.328931  8397 raft_consensus.cc:2393] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 4: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:12.301934  9000 raft_consensus.cc:2889] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:12.314821  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1808) took 1136 ms. Trace:
I20260709 13:02:12.422792  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1598) took 2383 ms. Trace:
I20260709 13:02:12.380825  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:12.423460  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 239196us
W20260709 13:02:12.423846  8298 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.004s	sys 0.031s
I20260709 13:02:12.283892 10290 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:12.458482  8220 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:12.459964  8220 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:12.460143  8298 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:12.281234 10035 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.263345 10298 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:12.264938 10294 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:12.317149 10311 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:12.367221  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 1.11774e+06us
I20260709 13:02:12.302090 10317 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:12.448398  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 1530) took 1095 ms. Trace:
I20260709 13:02:11.560001  8400 raft_consensus.cc:2955] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:12.361825 10170 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:12.453071  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 1.16255e+06us
I20260709 13:02:12.282907 10304 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:12.179551 10292 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:12.312537 10219 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:12.473613 10261 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:12.264209 10302 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:12.468292  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:12.343231 10293 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.981s	user 0.002s	sys 0.024s
W20260709 13:02:12.374392 10205 consensus_meta.cc:287] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.284s	user 0.013s	sys 0.000s
I20260709 13:02:12.434930  8974 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:12.315821 10323 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.422883  9903 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:12.472813  8295 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:12.320668  8327 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 1822) took 1208 ms (client timeout 1562 ms). Trace:
I20260709 13:02:12.457898  8325 rpcz_store.cc:276] 0709 13:02:10.039137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.121524 (+ 82387us) service_pool.cc:224] Handling call
0709 13:02:10.121836 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:10.148757 (+ 26921us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:02:10.148790 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:10.159762 (+ 10972us) log.cc:844] Serialized 26 byte log entry
0709 13:02:10.371146 (+211384us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:10.371975 (+   829us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:10.372058 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.413863 (+2041805us) raft_consensus.cc:1685] finished
0709 13:02:12.413892 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.422703 (+  8811us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":475,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":79649,"thread_start_us":101950,"threads_started":1,"wal-append.queue_time_us":509950}
I20260709 13:02:12.476395  8404 raft_consensus.cc:2393] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 3: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:12.421810  9989 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:12.380753 10220 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.478804  8397 rpcz_store.cc:276] 0709 13:02:11.353116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.179301 (+826185us) service_pool.cc:224] Handling call
0709 13:02:12.448343 (+269042us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:12.307441 10310 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.264348 10295 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:12.343251 10320 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.360494  9803 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:12.307191 10198 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:12.483745  8222 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:12.487082  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:12.457324 10269 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:12.466518 10122 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:12.467104 10078 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:12.320897 10097 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:12.478691 10317 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:12.464149  9009 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
W20260709 13:02:12.518021  9008 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.845s	user 0.000s	sys 0.022s
W20260709 13:02:12.592871  9008 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.919s	user 0.000s	sys 0.023s
I20260709 13:02:12.320071 10038 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:02:12.467895 10322 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:12.469864 10325 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.484323 10268 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:02:12.540444  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 105585us
I20260709 13:02:12.600772  9008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 511) took 1583 ms. Trace:
I20260709 13:02:12.476028 10035 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:12.557183  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.457654  8401 rpcz_store.cc:276] 0709 13:02:11.170561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.175746 (+  5185us) service_pool.cc:224] Handling call
0709 13:02:12.306505 (+1130759us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:12.564550  9006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 509) took 2321 ms. Trace:
I20260709 13:02:12.330646 10326 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:12.590524  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 117759us
I20260709 13:02:12.448793 10303 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:12.614833 10292 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:12.616305 10297 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:12.557812 10335 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.483296 10170 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:12.596534  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:12.594556 10310 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:12.597752 10322 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:12.598331 10294 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:12.598390 10339 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:12.601657  9008 rpcz_store.cc:276] 0709 13:02:11.017880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.024033 (+  6153us) service_pool.cc:224] Handling call
0709 13:02:11.061904 (+ 37871us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.062169 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:11.210227 (+148058us) log.cc:844] Serialized 14 byte log entry
0709 13:02:12.593487 (+1383260us) log.cc:844] Serialized 14 byte log entry
0709 13:02:12.600457 (+  6970us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:12.600509 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:12.600554 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.600621 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.600722 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":99951,"prepare.queue_time_us":507732,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":651753,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":29370,"thread_start_us":1334060,"threads_started":2,"wal-append.queue_time_us":942354}
W20260709 13:02:12.606952  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:12.558277 10315 log.cc:927] Time spent T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:02:12.483615 10311 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:12.608337 10038 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:12.608807 10285 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:12.597286 10308 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:12.609058 10298 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:12.609400 10302 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:12.616834 10297 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:12.610293 10336 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:12.519102 10272 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:12.611747  8256 raft_consensus.cc:2368] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3.
I20260709 13:02:12.487541 10329 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:12.610389 10331 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:12.615197  8332 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:12.550292  9900 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.557595 10333 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:12.596143  8257 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:12.578506 10138 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:12.555382 10205 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:12.616881 10335 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:12.892283  8254 consensus_meta.cc:287] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.595s	user 0.011s	sys 0.000s
I20260709 13:02:12.916215  8254 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
W20260709 13:02:12.638916  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:12.917644  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:12.620675  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 136989us
I20260709 13:02:12.919438  8222 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:12.919981  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:12.727479  8251 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1485) took 1721 ms (client timeout 2076 ms). Trace:
I20260709 13:02:12.612946  9006 rpcz_store.cc:276] 0709 13:02:10.243206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.243439 (+   233us) service_pool.cc:224] Handling call
0709 13:02:10.269366 (+ 25927us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.074631 (+805265us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:11.569915 (+495284us) log.cc:844] Serialized 14 byte log entry
0709 13:02:11.970582 (+400667us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:12.030390 (+ 59808us) log.cc:844] Serialized 26 byte log entry
0709 13:02:12.030588 (+   198us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:12.030829 (+   241us) log.cc:844] Serialized 14 byte log entry
0709 13:02:12.168177 (+137348us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.168249 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.561265 (+393016us) raft_consensus.cc:1685] finished
0709 13:02:12.561286 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.561416 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50394,"prepare.queue_time_us":1217788,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":67,"raft.queue_time_us":1680073,"raft.run_cpu_time_us":470,"raft.run_wall_time_us":125857,"spinlock_wait_cycles":10782464,"thread_start_us":1112783,"threads_started":3,"wal-append.queue_time_us":780667}
W20260709 13:02:12.922086  8251 rpcz_store.cc:269] 0709 13:02:11.006276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.102361 (+ 96085us) service_pool.cc:224] Handling call
0709 13:02:12.727249 (+1624888us) spinlock_profiling.cc:243] Waited 53.7 ms on lock 0x7b5400407c54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:12.727400 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123578112}
W20260709 13:02:12.730577  8329 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 1819) took 1628 ms (client timeout 2010 ms). Trace:
I20260709 13:02:12.918771  8295 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:12.595594 10317 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:12.913014  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1471) took 2913 ms. Trace:
I20260709 13:02:12.562657 10323 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:12.603170  9989 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.925215  8325 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
W20260709 13:02:12.928041  8329 rpcz_store.cc:269] 0709 13:02:11.102305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.118663 (+ 16358us) service_pool.cc:224] Handling call
0709 13:02:12.730354 (+1611691us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b54003c9954. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:12.730499 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":378547072}
W20260709 13:02:12.997769  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 76298us
W20260709 13:02:12.998778  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 80081.6us
I20260709 13:02:13.000735  8248 rpcz_store.cc:276] 0709 13:02:09.999555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.309990 (+2310435us) service_pool.cc:224] Handling call
0709 13:02:12.332186 (+ 22196us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.484527 (+152341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:12.484557 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:12.484604 (+    47us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:12.484639 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.912658 (+428019us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b54003471d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:12.912735 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.912930 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326656768}
I20260709 13:02:12.999136  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:13.004664  8401 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:02:13.007357  9008 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:12.614121  9903 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:12.594831 10072 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:13.017138 10219 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.019577  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:13.004103  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:12.998848 10122 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:13.007763  8401 pending_rounds.cc:85] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 2
I20260709 13:02:12.907111 10290 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:12.602784 10320 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:12.594297 10269 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:12.997212  9963 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:13.014564  9903 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:13.008307 10308 raft_consensus.cc:697] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:13.000065  9900 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:12.996495 10329 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.015719  9989 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:13.017539 10335 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:12.638090 10292 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.021968  9006 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:12.903630 10097 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:12.906018 10198 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:12.590018  8327 rpcz_store.cc:269] 0709 13:02:11.108429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.216011 (+107582us) service_pool.cc:224] Handling call
0709 13:02:12.316789 (+1100778us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:13.031466  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:13.137069  8972 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.006s	sys 0.025s
W20260709 13:02:13.137492  8972 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.006s	sys 0.025s
I20260709 13:02:12.716611 10326 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:12.723604 10035 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:12.997063 10170 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:12.647360 10079 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:13.035605  8324 raft_consensus.cc:2393] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 6: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:12.729513  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:13.018784  8325 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 177) took 1806 ms (client timeout 1530 ms). Trace:
I20260709 13:02:12.644728  8397 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:13.039067  8257 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1484) took 2033 ms (client timeout 2148 ms). Trace:
W20260709 13:02:13.147724  8325 rpcz_store.cc:269] 0709 13:02:11.198722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.596232 (+1397510us) service_pool.cc:224] Handling call
0709 13:02:13.005378 (+409146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:13.005110 10285 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:12.696802  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:13.050139  9002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 512) took 2022 ms. Trace:
W20260709 13:02:13.053905  8369 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.003s	sys 0.002s
I20260709 13:02:13.056277 10346 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.595781 10078 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:12.915339 10336 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:12.717185 10303 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:12.899327 10337 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:12.594924 10291 consensus_queue.cc:237] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:12.610773 10325 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:13.004132 10334 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:13.057899  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 434626us
I20260709 13:02:12.647756 10304 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:13.033461  8401 pending_rounds.cc:107] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 3.3
W20260709 13:02:13.076903 10104 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.008s	sys 0.005s
I20260709 13:02:12.618494 10268 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:13.035075 10261 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.082644  8332 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:13.034942 10038 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.596026 10220 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:13.101095  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 1812) took 2038 ms. Trace:
I20260709 13:02:12.643875 10330 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:02:13.137820  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 527131us
W20260709 13:02:13.085770 10092 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.671s	user 0.017s	sys 0.028s
W20260709 13:02:13.070013 10299 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.030s	sys 0.029s
I20260709 13:02:13.036782 10219 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:13.093894 10295 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:13.148124  8257 rpcz_store.cc:269] 0709 13:02:11.005991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.179972 (+1173981us) service_pool.cc:224] Handling call
0709 13:02:13.039006 (+859034us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:13.038923  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:13.149227  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:13.073793  9963 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:12.998476 10331 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:12.999707 10333 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:12.998562 10339 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:13.099268 10329 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:13.516767  8397 raft_consensus.cc:1275] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:02:13.293169  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.725322  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1476) took 3724 ms. Trace:
I20260709 13:02:13.725402  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1532) took 2370 ms. Trace:
W20260709 13:02:13.725631  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.654903  8331 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:13.160323  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:13.649556  8256 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1521) took 2419 ms (client timeout 1974 ms). Trace:
I20260709 13:02:13.149292  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:13.627907  8223 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.4:33825 timed out after 2.144s (SENT) [suppressed 16 similar messages]
W20260709 13:02:13.590440  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.725878  8397 rpcz_store.cc:276] 0709 13:02:11.354790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.615882 (+1261092us) service_pool.cc:224] Handling call
0709 13:02:12.644552 (+ 28670us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.725164 (+1080612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.725352 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:13.759296  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 131371us
W20260709 13:02:13.726716  8256 rpcz_store.cc:269] 0709 13:02:11.230151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.261669 (+1031518us) service_pool.cc:224] Handling call
0709 13:02:13.649295 (+1387626us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b540033ead4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:13.649476 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":748132224}
I20260709 13:02:13.720541  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49220 (request call id 260) took 1111 ms. Trace:
I20260709 13:02:13.138203 10097 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:13.531071 10350 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
W20260709 13:02:13.718153 10104 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.197s	user 0.024s	sys 0.020s
W20260709 13:02:13.663925  9971 consensus_meta.cc:287] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.211s	user 0.005s	sys 0.005s
W20260709 13:02:13.720206 10092 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.315s	user 0.022s	sys 0.053s
I20260709 13:02:13.727000 10118 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.785024  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:13.786068  8256 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:13.722747  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:13.786799  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:13.238116  8400 consensus_meta.cc:287] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.611s	user 0.007s	sys 0.002s
I20260709 13:02:13.717928  9002 rpcz_store.cc:276] 0709 13:02:11.028137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.061234 (+ 33097us) service_pool.cc:224] Handling call
0709 13:02:11.120635 (+ 59401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.303583 (+182948us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.303619 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:12.294185 (+990566us) log.cc:844] Serialized 26 byte log entry
0709 13:02:12.562633 (+268448us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:12.568226 (+  5593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.591874 (+ 23648us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:13.049797 (+457923us) raft_consensus.cc:1685] finished
0709 13:02:13.049839 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:13.050051 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45737,"raft.queue_time_us":131748,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":127,"thread_start_us":222781,"threads_started":2,"wal-append.queue_time_us":726110}
I20260709 13:02:13.774784  8325 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:13.788581  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:13.789779 10335 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:13.789165  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:13.776383 10323 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:13.793870 10323 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.794340 10323 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:13.796440 10323 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:13.797281  8325 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
W20260709 13:02:13.798614  8325 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 1825) took 2689 ms (client timeout 2719 ms). Trace:
W20260709 13:02:13.799050  8325 rpcz_store.cc:269] 0709 13:02:11.109525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.160042 (+2050517us) service_pool.cc:224] Handling call
0709 13:02:13.798537 (+638495us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:13.800020 10170 consensus_meta.cc:287] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.640s	user 0.009s	sys 0.000s
I20260709 13:02:13.800382  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:13.800479 10170 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:13.801378  8325 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:13.703039 10313 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.760700  8327 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:13.813148 10331 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:13.814934 10347 log.cc:927] Time spent T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:02:13.650583 10138 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.815632 10138 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:13.769469 10337 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:13.714026 10205 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:13.664916 10269 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:13.647730 10322 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:13.725764  8255 rpcz_store.cc:276] 0709 13:02:10.001038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.567132 (+2566094us) service_pool.cc:224] Handling call
0709 13:02:12.604014 (+ 36882us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.604206 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:13.147058 (+542852us) log.cc:844] Serialized 14 byte log entry
0709 13:02:13.724973 (+577915us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:13.725024 (+    51us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:13.725096 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.725170 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:13.725278 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1113374,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":95,"thread_start_us":987393,"threads_started":2}
I20260709 13:02:13.783999 10092 raft_consensus.cc:697] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 4 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:13.724535  8401 raft_consensus.cc:2889] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:13.370503  9900 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:13.136449 10292 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:13.238474  9989 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:13.151199 10127 consensus_meta.cc:287] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.677s	user 0.010s	sys 0.000s
I20260709 13:02:13.724118 10336 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:13.372294 10311 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:13.248718 10310 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:13.761044 10334 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.246742 10320 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:13.762523  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 1820) took 2654 ms. Trace:
I20260709 13:02:13.763511  8248 rpcz_store.cc:276] 0709 13:02:12.609622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.013694 (+404072us) service_pool.cc:224] Handling call
0709 13:02:13.720484 (+706790us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:13.720877  8328 rpcz_store.cc:276] 0709 13:02:11.062764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.062965 (+   201us) service_pool.cc:224] Handling call
0709 13:02:11.090720 (+ 27755us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.246006 (+155286us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.246049 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:11.246087 (+    38us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:11.246118 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.100791 (+1854673us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b5400364554. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:13.100860 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:13.101021 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":356414336}
I20260709 13:02:13.810137 10118 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:13.790328 10335 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:13.790613  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:13.810269  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:13.760896  9001 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:13.819175  8257 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
W20260709 13:02:13.821406  8295 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.015s	sys 0.024s
W20260709 13:02:13.372772 10314 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.019s	sys 0.032s
W20260709 13:02:13.848225 10314 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.286s	user 0.029s	sys 0.056s
I20260709 13:02:13.841640 10334 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:13.736438 10261 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:13.754608 10038 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:13.802129  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 1834) took 1247 ms. Trace:
I20260709 13:02:13.855700  8325 rpcz_store.cc:276] 0709 13:02:12.555092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.800083 (+1244991us) service_pool.cc:224] Handling call
0709 13:02:13.802059 (+  1976us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:13.778417 10097 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:13.824502  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:13.831039  8369 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.303s	user 0.003s	sys 0.023s
I20260709 13:02:13.832005  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:13.724372 10346 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:13.247107 10308 consensus_queue.cc:237] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:13.784171  9971 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:13.821962 10295 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:13.826169 10303 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.841922  8331 rpcz_store.cc:276] 0709 13:02:11.107805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.120762 (+ 12957us) service_pool.cc:224] Handling call
0709 13:02:13.654458 (+2533696us) spinlock_profiling.cc:243] Waited 1.42 ms on lock 0x7b58000e3498. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:02:13.654688 (+   230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.758693 (+104005us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.761694 (+  3001us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3260288}
I20260709 13:02:13.790102 10152 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:13.845736  8256 raft_consensus.cc:2393] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 3: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:02:13.846527  8295 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.016s	sys 0.024s
I20260709 13:02:13.847206  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:13.822263 10313 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:13.852454  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:13.882339  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 1831) took 1373 ms. Trace:
I20260709 13:02:13.882763  8328 rpcz_store.cc:276] 0709 13:02:12.508994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.842875 (+1333881us) service_pool.cc:224] Handling call
0709 13:02:13.882280 (+ 39405us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:13.883780  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1526) took 2532 ms. Trace:
I20260709 13:02:13.884089  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:13.868534 10299 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.565s	user 0.037s	sys 0.048s
I20260709 13:02:13.857065  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:13.850315  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:13.868526  8369 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:13.873916  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:13.836984 10127 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:13.880142  8248 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
W20260709 13:02:13.880155  8295 connection.cc:256] RPC call timeout handler was delayed by -1.4237306030000241s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1638s for a response
W20260709 13:02:13.892297  8295 connection.cc:256] RPC call timeout handler was delayed by -1.2586515440000312s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.209s for a response
I20260709 13:02:13.887828  8325 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:13.885179  8328 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
W20260709 13:02:13.892771  8295 connection.cc:256] RPC call timeout handler was delayed by -1.1578515439999819s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2202s for a response
I20260709 13:02:13.890081  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:13.893118  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 1828) took 1385 ms. Trace:
W20260709 13:02:13.893307  8295 connection.cc:256] RPC call timeout handler was delayed by -1.029530603000012s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2076s for a response
I20260709 13:02:13.893707  8251 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
I20260709 13:02:13.893575  8325 rpcz_store.cc:276] 0709 13:02:12.508030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.856732 (+1348702us) service_pool.cc:224] Handling call
0709 13:02:13.893062 (+ 36330us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:13.872288 10083 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.891312  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:13.898692  8331 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:13.899363  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 1830) took 1390 ms. Trace:
W20260709 13:02:13.853470  9001 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56954 (request call id 415) took 1251 ms (client timeout 1638 ms). Trace:
I20260709 13:02:13.894827  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:13.918817  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 1833) took 1407 ms. Trace:
I20260709 13:02:13.919242  8325 rpcz_store.cc:276] 0709 13:02:12.511535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.894591 (+1383056us) service_pool.cc:224] Handling call
0709 13:02:13.918752 (+ 24161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:13.916883  9963 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:13.875166 10311 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:13.920254  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:13.920867  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1477) took 3896 ms. Trace:
I20260709 13:02:13.893160  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 1832) took 1383 ms. Trace:
I20260709 13:02:13.884225  8400 rpcz_store.cc:276] 0709 13:02:11.351035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.559329 (+208294us) service_pool.cc:224] Handling call
0709 13:02:11.559667 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.559885 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:13.787947 (+2228062us) log.cc:844] Serialized 14 byte log entry
0709 13:02:13.883371 (+ 95424us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:13.883446 (+    75us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:13.883512 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.883583 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:13.883703 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":81,"thread_start_us":94800,"threads_started":1}
I20260709 13:02:13.899817  8331 rpcz_store.cc:276] 0709 13:02:12.508689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.878820 (+1370131us) service_pool.cc:224] Handling call
0709 13:02:13.899285 (+ 20465us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:13.875412 10320 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:13.870635  9989 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:13.921557  8328 rpcz_store.cc:276] 0709 13:02:12.509276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.883700 (+1374424us) service_pool.cc:224] Handling call
0709 13:02:13.893110 (+  9410us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:13.921411  8255 rpcz_store.cc:276] 0709 13:02:10.001324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.886233 (+3884909us) service_pool.cc:224] Handling call
0709 13:02:13.897165 (+ 10932us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.897338 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:13.897372 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:13.897413 (+    41us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:13.897475 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.897527 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:13.897650 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:13.922616  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1480) took 3820 ms. Trace:
I20260709 13:02:13.878103 10322 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:13.916414  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:13.925654  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 57187.9us
I20260709 13:02:13.926312  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:13.955054  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:13.955549  8256 rpcz_store.cc:276] 0709 13:02:10.079206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.887252 (+3808046us) service_pool.cc:224] Handling call
0709 13:02:13.898944 (+ 11692us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.899143 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:13.899180 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:13.899235 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:13.899282 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.899361 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:13.899478 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:13.974476  8325 raft_consensus.cc:2410] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:13.898065 10220 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:13.975335  8397 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:13.922700 10299 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:13.981112  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:13.981830  8331 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:13.982700  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:13.982985  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 1821) took 2874 ms. Trace:
I20260709 13:02:13.983228  8331 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
I20260709 13:02:13.983362  8328 rpcz_store.cc:276] 0709 13:02:11.108157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.974735 (+2866578us) service_pool.cc:224] Handling call
0709 13:02:13.982458 (+  7723us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.982621 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:13.982644 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:13.982681 (+    37us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:13.982727 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.982778 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:13.982869 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:13.984068  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:13.984583  8331 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
I20260709 13:02:13.984939  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 1827) took 2874 ms. Trace:
I20260709 13:02:13.985793  8255 raft_consensus.cc:1240] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:13.986446  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1512) took 3260 ms. Trace:
I20260709 13:02:13.986747  8255 rpcz_store.cc:276] 0709 13:02:10.726641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.974824 (+3248183us) service_pool.cc:224] Handling call
0709 13:02:13.985646 (+ 10822us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.986214 (+   568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.986395 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:13.901073  8250 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 1.
I20260709 13:02:13.888974 10303 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:13.978487  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:13.980253  8248 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: []
W20260709 13:02:13.954281 10270 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.317s	user 0.047s	sys 0.054s
I20260709 13:02:13.985278  8328 rpcz_store.cc:276] 0709 13:02:11.110127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.984217 (+2874090us) service_pool.cc:224] Handling call
0709 13:02:13.984480 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.984634 (+   154us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:13.984660 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:13.984698 (+    38us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:13.984742 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.984790 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:13.984880 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:13.987438 10280 log.cc:927] Time spent T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 3.577s	user 0.021s	sys 0.015s
I20260709 13:02:13.886089 10295 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:13.988016  8255 raft_consensus.cc:1275] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:14.029923  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1513) took 3302 ms. Trace:
I20260709 13:02:14.033006  8369 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:14.038738  8248 rpcz_store.cc:276] 0709 13:02:10.727039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.974751 (+3247712us) service_pool.cc:224] Handling call
0709 13:02:13.980072 (+  5321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.029681 (+ 49609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.029870 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.040086  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1490) took 2870 ms. Trace:
I20260709 13:02:14.031168  8972 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:14.040046  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:14.040757  8255 rpcz_store.cc:276] 0709 13:02:11.169741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.987477 (+2817736us) service_pool.cc:224] Handling call
0709 13:02:13.987741 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.039622 (+ 51881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.039986 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:13.954272 10170 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:14.045203 10362 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:13.975637 10219 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:14.049613  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:14.042021  8325 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:13.923573 10083 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:13.989943  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1601) took 2785 ms. Trace:
W20260709 13:02:14.030489 10270 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.438s	user 0.047s	sys 0.054s
I20260709 13:02:13.981002 10261 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:13.892752 10092 consensus_queue.cc:237] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:14.044364  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:14.051673  8331 rpcz_store.cc:276] 0709 13:02:11.204334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.985102 (+2780768us) service_pool.cc:224] Handling call
0709 13:02:13.985361 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.989521 (+  4160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:13.989562 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:13.989619 (+    57us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:13.989682 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.989753 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:13.989878 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:13.918370  9001 rpcz_store.cc:269] 0709 13:02:12.601397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.603716 (+  2319us) service_pool.cc:224] Handling call
0709 13:02:13.852271 (+1248555us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:13.955687 10311 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:14.054684  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:14.053653  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:14.062860  8248 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:14.058892  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:14.053682  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:14.037844 10333 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:14.065232  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:14.062825  8255 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:14.050787 10359 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.075521  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:14.075773 10346 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:14.076421  9005 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56940 (request call id 514) took 3006 ms (client timeout 1738 ms). Trace:
I20260709 13:02:14.078635  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1475) took 4077 ms. Trace:
I20260709 13:02:14.079016  8253 rpcz_store.cc:276] 0709 13:02:10.000747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.457471 (+2456724us) service_pool.cc:224] Handling call
0709 13:02:12.480244 (+ 22773us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:12.480593 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:12.480625 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:13.655656 (+1175031us) spinlock_profiling.cc:243] Waited 1.08 ms on lock 0x7b2c000014c8. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:02:13.701711 (+ 46055us) log.cc:844] Serialized 548 byte log entry
0709 13:02:13.895153 (+193442us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:13.895198 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.895273 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:14.078320 (+183047us) raft_consensus.cc:1685] finished
0709 13:02:14.078340 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.078567 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2486656,"thread_start_us":176877,"threads_started":1,"wal-append.queue_time_us":341714}
I20260709 13:02:14.070288 10331 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:14.081135 10331 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:14.065232  8972 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:14.076476 10346 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:14.092607  9006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 422) took 1484 ms. Trace:
I20260709 13:02:14.082515 10038 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:14.061960 10138 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:14.084676 10313 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:14.053228 10270 raft_consensus.cc:697] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:14.056180 10299 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:14.071543  8328 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
W20260709 13:02:14.098632  8332 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 1826) took 2977 ms (client timeout 1596 ms). Trace:
I20260709 13:02:13.977972  9989 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:13.989734 10322 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:14.085613 10118 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:14.071532 10362 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:14.058073  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:14.089200  9000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 503) took 3932 ms. Trace:
I20260709 13:02:14.127489  8400 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:14.127408  9000 rpcz_store.cc:276] 0709 13:02:10.156879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.158765 (+  1886us) service_pool.cc:224] Handling call
0709 13:02:10.934432 (+775667us) spinlock_profiling.cc:243] Waited 1.93 ms on lock 0x7b5800175898. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:02:10.934711 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.486844 (+1552133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.088949 (+1602105us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b54004c2f54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:14.089139 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":601482880}
W20260709 13:02:14.076995  9005 rpcz_store.cc:269] 0709 13:02:11.070517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.078862 (+  8345us) service_pool.cc:224] Handling call
0709 13:02:14.075346 (+2996484us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b5800175898. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:02:14.076313 (+   967us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25051008}
I20260709 13:02:14.128098 10270 consensus_queue.cc:237] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:14.109338 10308 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:14.088781  9971 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:14.129089 10308 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:14.088526 10083 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:14.129551 10308 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:14.090807  8327 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
W20260709 13:02:14.130215  8327 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 176) took 2932 ms (client timeout 2512 ms). Trace:
W20260709 13:02:14.130601  8327 rpcz_store.cc:269] 0709 13:02:11.197840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.138814 (+1940974us) service_pool.cc:224] Handling call
0709 13:02:14.130140 (+991326us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.124943  8328 pending_rounds.cc:85] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 5
I20260709 13:02:14.086385  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:14.131739  8328 pending_rounds.cc:107] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 3.6
W20260709 13:02:14.089752 10317 consensus_meta.cc:287] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.057s	user 0.011s	sys 0.000s
I20260709 13:02:14.131982  8250 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:14.132277 10317 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:14.132500 10334 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:14.094249 10078 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:14.133622  9989 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:14.134290  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:14.134452 10317 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:14.132037  8328 raft_consensus.cc:2889] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:14.083101 10285 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:14.095757  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:14.139147  8248 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:14.081816 10097 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:14.083099 10292 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:14.090948  8397 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:13.978216 10320 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:14.125296  8332 rpcz_store.cc:269] 0709 13:02:11.109827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.245082 (+135255us) service_pool.cc:224] Handling call
0709 13:02:14.086600 (+2841518us) spinlock_profiling.cc:243] Waited 444 ms on lock 0x7b54003ca854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:14.086705 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1020804864}
I20260709 13:02:14.096958  9006 rpcz_store.cc:276] 0709 13:02:12.608276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.149770 (+541494us) service_pool.cc:224] Handling call
0709 13:02:13.150056 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.775573 (+625517us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:13.775603 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:13.778658 (+  3055us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:13.785771 (+  7113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.092367 (+306596us) spinlock_profiling.cc:243] Waited 625 ms on lock 0x7b54005a8cd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:14.092421 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.092548 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2404,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":2443,"spinlock_wait_cycles":1437165824}
I20260709 13:02:14.093919  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1589) took 4148 ms. Trace:
I20260709 13:02:14.133052  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:14.095036 10295 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:14.087284  8972 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:14.135416  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1602) took 2930 ms. Trace:
I20260709 13:02:14.152755  8248 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:02:14.118268  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 71584.1us
I20260709 13:02:14.128212  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1816) took 1112 ms. Trace:
I20260709 13:02:14.196421  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1821) took 1179 ms. Trace:
W20260709 13:02:14.197263  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 109995us
I20260709 13:02:14.089818 10331 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:14.197109  8328 rpcz_store.cc:276] 0709 13:02:11.204665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.056520 (+2851855us) service_pool.cc:224] Handling call
0709 13:02:14.071237 (+ 14717us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.135158 (+ 63921us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.135341 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:14.197269  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 1.62787e+06us
I20260709 13:02:14.203341  8398 raft_consensus.cc:2410] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:14.158684  8257 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6. Ops: [5.5-5.5]
W20260709 13:02:14.198717  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.159569  8256 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:02:14.156307  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:14.200009  8397 rpcz_store.cc:276] 0709 13:02:13.016648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.894697 (+878049us) service_pool.cc:224] Handling call
0709 13:02:14.196353 (+301656us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.197750  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1491) took 3027 ms. Trace:
I20260709 13:02:14.195865  8323 rpcz_store.cc:276] 0709 13:02:09.945982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.946091 (+   109us) service_pool.cc:224] Handling call
0709 13:02:09.949385 (+  3294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:09.949796 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:09.949825 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:09.950312 (+   487us) log.cc:844] Serialized 26 byte log entry
0709 13:02:10.069306 (+118994us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:10.069378 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:10.069476 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:13.986854 (+3917378us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:14.093649 (+106795us) raft_consensus.cc:1685] finished
0709 13:02:14.093672 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.093859 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":370,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":76,"spinlock_wait_cycles":1452059392,"thread_start_us":87696,"threads_started":1,"wal-append.queue_time_us":301811}
I20260709 13:02:14.193728  8326 raft_consensus.cc:2368] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5.
I20260709 13:02:14.088907 10311 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:14.097666 10333 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:14.136458 10362 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.189162 10337 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:14.195809 10335 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:14.203792 10285 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:14.216310 10285 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.216724 10285 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:14.217916  8254 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1503) took 3826 ms (client timeout 2581 ms). Trace:
W20260709 13:02:14.218417  8254 rpcz_store.cc:269] 0709 13:02:10.391586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.999045 (+607459us) service_pool.cc:224] Handling call
0709 13:02:14.217717 (+3218672us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400407254. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:14.217835 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2976384256}
W20260709 13:02:14.219198  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 153958us
I20260709 13:02:14.219635  8254 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:14.219827  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:14.222241  8972 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:14.223575  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:14.204684  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1514) took 2999 ms. Trace:
I20260709 13:02:14.224965  8257 rpcz_store.cc:276] 0709 13:02:11.205445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.157981 (+2952536us) service_pool.cc:224] Handling call
0709 13:02:14.158399 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:14.204414 (+ 46015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.204628 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.205379  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:14.226357  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1814) took 1211 ms. Trace:
I20260709 13:02:14.205410  8248 rpcz_store.cc:276] 0709 13:02:11.170135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.151803 (+2981668us) service_pool.cc:224] Handling call
0709 13:02:14.152298 (+   495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.197506 (+ 45208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.197697 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.209822  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:14.228765  8405 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
I20260709 13:02:14.228715  8403 rpcz_store.cc:276] 0709 13:02:13.014881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.155949 (+1141068us) service_pool.cc:224] Handling call
0709 13:02:14.226294 (+ 70345us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.099567 10346 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:14.213483  8330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 1817) took 3148 ms (client timeout 2737 ms). Trace:
I20260709 13:02:14.221108  8369 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:14.227977  8397 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
W20260709 13:02:14.235997  8330 rpcz_store.cc:269] 0709 13:02:11.065265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.087277 (+ 22012us) service_pool.cc:224] Handling call
0709 13:02:14.213262 (+3125985us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b5400064854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:14.213385 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":400604416}
I20260709 13:02:14.229636  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 256) took 1172 ms. Trace:
I20260709 13:02:14.237156  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1813) took 1222 ms. Trace:
I20260709 13:02:14.213078  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 1818) took 3147 ms. Trace:
I20260709 13:02:14.237658  8397 rpcz_store.cc:276] 0709 13:02:13.014584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.204944 (+1190360us) service_pool.cc:224] Handling call
0709 13:02:14.237077 (+ 32133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.238159  8324 rpcz_store.cc:276] 0709 13:02:11.065650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.111795 (+ 46145us) service_pool.cc:224] Handling call
0709 13:02:14.212878 (+3101083us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b5400090bd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:14.213018 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4317455744}
I20260709 13:02:14.233727  8972 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:14.234992  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1515) took 3014 ms. Trace:
I20260709 13:02:14.237597  8405 rpcz_store.cc:276] 0709 13:02:13.057468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.209547 (+1152079us) service_pool.cc:224] Handling call
0709 13:02:14.229558 (+ 20011us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.240233  8254 rpcz_store.cc:276] 0709 13:02:11.205736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.219185 (+3013449us) service_pool.cc:224] Handling call
0709 13:02:14.219493 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.219986 (+   493us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.220146 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.241526  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:14.242278  8405 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:14.205188  8297 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:14.242883  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1815) took 1227 ms. Trace:
I20260709 13:02:14.243371  8405 rpcz_store.cc:276] 0709 13:02:13.015160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.241206 (+1226046us) service_pool.cc:224] Handling call
0709 13:02:14.242831 (+  1625us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.244067  8297 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:14.244246  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:14.245214  8405 raft_consensus.cc:2410] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:14.244259  8369 leader_election.cc:400] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3.
I20260709 13:02:14.245635  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1818) took 1229 ms. Trace:
I20260709 13:02:14.245826  8369 leader_election.cc:403] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:02:14.246042  8405 rpcz_store.cc:276] 0709 13:02:13.015900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.244019 (+1228119us) service_pool.cc:224] Handling call
0709 13:02:14.245583 (+  1564us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.246515  9989 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3.
I20260709 13:02:14.246922  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:14.247823  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1820) took 1231 ms. Trace:
I20260709 13:02:14.248229  8405 rpcz_store.cc:276] 0709 13:02:13.016404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.246696 (+1230292us) service_pool.cc:224] Handling call
0709 13:02:14.247766 (+  1070us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.207083 10362 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:14.249679  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1534) took 2894 ms. Trace:
I20260709 13:02:14.249701 10299 consensus_queue.cc:1059] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:14.250121  8405 rpcz_store.cc:276] 0709 13:02:11.355317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.248879 (+2893562us) service_pool.cc:224] Handling call
0709 13:02:14.249141 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.249323 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:14.249351 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:14.249394 (+    43us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:14.249465 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.249530 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.249631 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.251444  9989 raft_consensus.cc:3055] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:14.089010 10359 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:14.248013 10308 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:14.199435  8400 rpcz_store.cc:276] 0709 13:02:13.015399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.978292 (+962893us) service_pool.cc:224] Handling call
0709 13:02:14.128150 (+149858us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.275580  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1481) took 4167 ms. Trace:
I20260709 13:02:14.252805  8248 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:14.254129  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:14.266775  8257 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:02:14.312772  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 47951.8us
I20260709 13:02:14.276839  8297 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:14.314005  8251 rpcz_store.cc:276] 0709 13:02:10.093604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.916164 (+3822560us) service_pool.cc:224] Handling call
0709 13:02:14.081825 (+165661us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b5800002a98. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:02:14.260382 (+178557us) spinlock_profiling.cc:243] Waited 4.47 ms on lock 0x7b5800002a98. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699e9c 00007f8825699a5b 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:02:14.260619 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.260854 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:14.260893 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:14.261013 (+   120us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:14.261112 (+    99us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.261195 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.261355 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13100800}
I20260709 13:02:14.319120  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1516) took 3113 ms. Trace:
I20260709 13:02:14.252288  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:14.319917  8257 rpcz_store.cc:276] 0709 13:02:11.205987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.225877 (+3019890us) service_pool.cc:224] Handling call
0709 13:02:14.240230 (+ 14353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.318625 (+ 78395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.319010 (+   385us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.237632 10271 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:14.321336  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1817) took 1305 ms. Trace:
I20260709 13:02:14.322708  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1517) took 3116 ms. Trace:
W20260709 13:02:14.255517 10282 log.cc:927] Time spent T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 3.502s	user 0.012s	sys 0.018s
I20260709 13:02:14.325886  8403 rpcz_store.cc:276] 0709 13:02:13.015645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.231506 (+1215861us) service_pool.cc:224] Handling call
0709 13:02:14.321276 (+ 89770us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.326072  8248 rpcz_store.cc:276] 0709 13:02:11.206257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.228523 (+3022266us) service_pool.cc:224] Handling call
0709 13:02:14.252450 (+ 23927us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.315176 (+ 62726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.322375 (+  7199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.239038 10346 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:14.325481  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 258) took 1235 ms. Trace:
I20260709 13:02:14.251986  9989 raft_consensus.cc:740] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:14.253679 10213 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:14.230209  8250 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:14.328599  8325 raft_consensus.cc:2410] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:02:14.234692 10295 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:14.329674  8397 rpcz_store.cc:276] 0709 13:02:13.090533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.238941 (+1148408us) service_pool.cc:224] Handling call
0709 13:02:14.325288 (+ 86347us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.313514 10292 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:14.258405  9903 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:14.237805 10304 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:14.301520 10311 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:14.252508 10335 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:14.314445 10118 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:14.204018 10336 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:14.314546 10083 consensus_queue.cc:1059] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:02:14.336197 10308 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:14.206888 10333 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:14.344288  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:14.345153  8297 connection.cc:256] RPC call timeout handler was delayed by -1.8087765450000006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17420000000000002s for a response
I20260709 13:02:14.336798 10292 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:14.354679  8330 raft_consensus.cc:1275] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:14.355670  8397 raft_consensus.cc:2410] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:14.356878  9989 consensus_queue.cc:260] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:14.358258  9989 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:14.356859  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:14.346122 10311 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.370149  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:14.343504 10304 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.364706  8397 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:14.370400  8329 raft_consensus.cc:2393] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 4: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:14.348142 10118 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:14.337740  9900 consensus_queue.cc:1048] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:02:14.372334  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:14.376048  8328 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:14.373292  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:14.373816  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:14.364349 10292 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:14.368649  8400 raft_consensus.cc:3055] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:14.370494 10311 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:14.368929  8255 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:14.379798  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 257) took 2116 ms. Trace:
I20260709 13:02:14.363677 10334 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:14.373868 10304 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:14.379256  8257 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:14.385241  8254 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:14.385084  8251 rpcz_store.cc:276] 0709 13:02:12.263186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.327547 (+2064361us) service_pool.cc:224] Handling call
0709 13:02:14.379019 (+ 51472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.379369 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:14.379408 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:14.379465 (+    57us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:14.379513 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.379598 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.379732 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1677}
I20260709 13:02:14.391717  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:14.392428  8253 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:14.389721 10303 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:14.390231 10362 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:14.375363 10118 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:14.421399 10362 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:14.421597  8252 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1473) took 4421 ms (client timeout 2202 ms). Trace:
W20260709 13:02:14.422031  8252 rpcz_store.cc:269] 0709 13:02:10.000133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.075059 (+ 74926us) service_pool.cc:224] Handling call
0709 13:02:14.421390 (+4346331us) spinlock_profiling.cc:243] Waited 3.43 s on lock 0x7b5400308c54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:14.421525 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7884939648}
I20260709 13:02:14.424149  8248 raft_consensus.cc:1275] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:14.424743  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1497) took 1278 ms. Trace:
I20260709 13:02:14.383216  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:14.382371  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:14.394596  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 258) took 1827 ms. Trace:
I20260709 13:02:14.397028  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1496) took 1288 ms. Trace:
I20260709 13:02:14.426060  8327 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 6: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:14.397891 10333 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:14.426832 10339 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:14.431348  9001 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:14.432446  8297 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:14.432514  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:14.434043  9006 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:14.435256  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:14.435688  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.437520 10296 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:14.408438 10304 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:14.438259 10296 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.438628 10296 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:14.409906  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:14.439832  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:14.411651  8404 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 1529) took 3058 ms (client timeout 1742 ms). Trace:
I20260709 13:02:14.414768 10336 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:14.383157 10329 consensus_meta.cc:287] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.620s	user 0.011s	sys 0.000s
I20260709 13:02:14.415241  9008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 418) took 1812 ms. Trace:
I20260709 13:02:14.416188 10286 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.418854 10227 consensus_queue.cc:1059] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:14.382861  8400 raft_consensus.cc:740] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:14.425055  8248 rpcz_store.cc:276] 0709 13:02:13.145880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.423509 (+1277629us) service_pool.cc:224] Handling call
0709 13:02:14.423829 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.424525 (+   696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.424694 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.433964  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:14.437564  9000 raft_consensus.cc:1240] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:14.440668  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:14.426219  8255 rpcz_store.cc:276] 0709 13:02:13.107522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.391447 (+1283925us) service_pool.cc:224] Handling call
0709 13:02:14.395229 (+  3782us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.395388 (+   159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:14.395411 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:14.395449 (+    38us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:14.395491 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.395537 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.395621 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:14.442260  8329 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 1604) took 3215 ms (client timeout 1905 ms). Trace:
W20260709 13:02:14.490641  8329 rpcz_store.cc:269] 0709 13:02:11.226813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.019245 (+1792432us) service_pool.cc:224] Handling call
0709 13:02:14.442022 (+1422777us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b540011e9d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:14.442176 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3102887168}
I20260709 13:02:14.425750  8257 rpcz_store.cc:276] 0709 13:02:12.566883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.334615 (+1767732us) service_pool.cc:224] Handling call
0709 13:02:14.379049 (+ 44434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.391672 (+ 12623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.394532 (+  2860us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":60672}
I20260709 13:02:14.408672  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:14.445573  8328 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:14.448141  8222 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:14.492241  8406 raft_consensus.cc:2410] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:02:14.448860  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.406114 10268 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:14.434087 10127 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:14.440562  8404 rpcz_store.cc:269] 0709 13:02:11.352847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.175693 (+822846us) service_pool.cc:224] Handling call
0709 13:02:14.411381 (+2235688us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b5400267fd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:14.411540 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":688145152}
I20260709 13:02:14.457638  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:14.457700  9008 rpcz_store.cc:276] 0709 13:02:12.602314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.014384 (+412070us) service_pool.cc:224] Handling call
0709 13:02:13.014705 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.718105 (+703400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:13.718137 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:13.718183 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:13.718213 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.414952 (+696739us) spinlock_profiling.cc:243] Waited 703 ms on lock 0x7b54006ce3d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:14.415022 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.415171 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1616412544}
I20260709 13:02:14.459663  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:14.472402  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.368542 10365 consensus_queue.cc:1059] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:02:14.482668  8325 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:02:14.486008  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:14.459477 10369 log.cc:927] Time spent T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:02:14.511664  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:14.487951 10320 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:14.496096  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 48010.4us
I20260709 13:02:14.491084  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:14.496762  8253 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:14.491201  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1541) took 2017 ms. Trace:
I20260709 13:02:14.501230  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:14.502614  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:14.508899  8332 raft_consensus.cc:2410] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:02:14.506421  8220 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:14.391230 10285 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:14.486243  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:14.460299 10092 consensus_queue.cc:1048] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:02:14.528398  8330 raft_consensus.cc:2368] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4.
I20260709 13:02:14.518471  8222 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:14.528513  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:14.436496  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:14.529331  9006 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:14.529064  8404 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:14.519452  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 1829) took 2011 ms. Trace:
I20260709 13:02:14.520956  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:14.522284  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:14.465952 10329 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:14.527688  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:14.529500  8222 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:14.535936  8253 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 1.
I20260709 13:02:14.526952  8403 rpcz_store.cc:276] 0709 13:02:12.473337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.326880 (+1853543us) service_pool.cc:224] Handling call
0709 13:02:14.333819 (+  6939us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.490825 (+157006us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:14.490863 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:14.490906 (+    43us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:14.490939 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.491003 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.491136 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5855,"raft.queue_time_us":160907,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":11194,"thread_start_us":156648,"threads_started":1}
I20260709 13:02:14.502012 10073 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:14.509382  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:14.540884  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:14.533123  8331 rpcz_store.cc:276] 0709 13:02:12.508381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.072036 (+1563655us) service_pool.cc:224] Handling call
0709 13:02:14.105340 (+ 33304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.105634 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:14.331985 (+226351us) log.cc:844] Serialized 14 byte log entry
0709 13:02:14.519030 (+187045us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:14.519100 (+    70us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:14.519172 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.519257 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.519389 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":389872,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":2758,"raft.run_cpu_time_us":64,"raft.run_wall_time_us":65,"thread_start_us":387723,"threads_started":2}
I20260709 13:02:14.542841  8222 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:14.492213 10118 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:14.501753 10260 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:14.482811 10286 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:14.516470 10271 raft_consensus.cc:697] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:14.495738 10217 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
W20260709 13:02:14.551909  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 229766us
I20260709 13:02:14.544417  8222 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:14.544714 10290 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:14.500618 10331 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:14.558274  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:14.559679  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:14.560397  8253 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:14.534828 10072 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:14.524801  9971 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:14.551180  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 44840.3us
I20260709 13:02:14.568293  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:14.569407  8220 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:14.554860  8223 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:14.529028 10320 raft_consensus.cc:3055] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:14.556370  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:14.562511  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:14.539623  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:14.571009  9008 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:14.566730  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:14.534684  9002 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:14.571024  8404 raft_consensus.cc:2393] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 3: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:14.559008  8324 raft_consensus.cc:2376] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3. Re-sending same reply.
I20260709 13:02:14.549733 10090 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:14.549844 10138 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:14.550017 10304 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:14.561962 10290 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:14.677349  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:14.668419  8405 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:14.696697  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:14.739893  8324 raft_consensus.cc:2376] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6. Re-sending same reply.
I20260709 13:02:14.698191 10313 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:14.568395 10317 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:14.569418 10260 raft_consensus.cc:697] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:14.571856 10320 raft_consensus.cc:740] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:14.663689  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:14.691015  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:14.743876  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:14.553279 10073 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:14.746955  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:14.772164  8329 raft_consensus.cc:2393] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 3: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:14.664666 10090 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:14.691675 10138 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.753602  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:14.730533 10290 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:14.696508 10271 consensus_queue.cc:237] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:14.664144 10337 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:14.706935 10083 consensus_queue.cc:1059] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:02:14.743916 10317 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.766650  9000 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:14.767433 10331 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:14.768002  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:14.745033 10313 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:14.550836  8400 consensus_queue.cc:260] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:14.706158 10359 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:14.741056  8253 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:14.827383  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:14.821264  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:14.821413  9006 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 1.
I20260709 13:02:14.823524  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:14.826771  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 267750us
W20260709 13:02:14.762349  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 254451us
I20260709 13:02:14.833740  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:14.833876 10329 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:14.831678 10291 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:14.834962 10286 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:14.728493 10371 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.137s
I20260709 13:02:14.822167 10090 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:14.839501  9009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56940 (request call id 510) took 3926 ms (client timeout 2089 ms). Trace:
I20260709 13:02:14.826794 10138 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:14.846287  8253 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:14.846123  8249 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
W20260709 13:02:14.846724  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 272189us
I20260709 13:02:14.839395 10317 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:14.854629  8326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 1603) took 3640 ms (client timeout 1721 ms). Trace:
I20260709 13:02:14.852973 10073 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:14.857221  8326 rpcz_store.cc:269] 0709 13:02:11.214574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.729111 (+1514537us) service_pool.cc:224] Handling call
0709 13:02:14.854364 (+2125253us) spinlock_profiling.cc:243] Waited 985 ms on lock 0x7b540020ec54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:14.854532 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2265664896}
I20260709 13:02:14.859560 10292 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:14.840548 10213 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.843925 10083 consensus_queue.cc:1243] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:02:14.861701 10213 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:14.862951  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:14.863554  9005 raft_consensus.cc:2376] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4. Re-sending same reply.
I20260709 13:02:14.863822 10213 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:14.864475  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:14.865468 10375 raft_consensus.cc:3055] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:14.851238  8400 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:14.865865 10375 raft_consensus.cc:740] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:14.866564  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:14.866772 10375 consensus_queue.cc:260] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:14.855367  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:14.868165  9006 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:14.841064 10313 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:14.847406 10320 consensus_queue.cc:260] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:14.856141  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:14.850874  8220 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:14.893922  8223 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:14.895407  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 44605.1us
I20260709 13:02:14.850538 10291 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:14.871098  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:14.886041  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:14.849355  9009 rpcz_store.cc:269] 0709 13:02:10.913272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.913436 (+   164us) service_pool.cc:224] Handling call
0709 13:02:14.839315 (+3925879us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400598dd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:14.839430 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2923802880}
I20260709 13:02:14.896557  8220 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:14.911275  8220 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:14.912549  8220 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:14.837145 10260 consensus_queue.cc:237] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:14.936874 10117 consensus_queue.cc:1048] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.003s
W20260709 13:02:14.937629  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 43789.5us
I20260709 13:02:14.938922  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:14.940227  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:14.848469 10337 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:14.942121  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 71060.2us
I20260709 13:02:14.946163  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:14.947620  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:14.951311  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.954450  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:14.955067  8369 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:14.861075 10296 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:14.960120  8972 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:15.012866  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 52805.2us
I20260709 13:02:15.005918 10291 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:14.846915 10377 consensus_queue.cc:1175] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ff7a6fe533e94c1bae555b4cae427728" responder_term: 4 status { last_received { term: 2 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:02:15.007743 10335 consensus_meta.cc:287] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.657s	user 0.009s	sys 0.000s
W20260709 13:02:15.009810  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 67535.7us
I20260709 13:02:15.087828  8220 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:15.015092  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:14.934685 10333 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:15.089418  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:14.941401 10285 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:15.090277  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:14.941152 10259 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:15.089375  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 74342.7us
I20260709 13:02:15.119578  8972 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:14.946163 10092 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:14.944813 10217 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:14.955753 10331 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:15.107667  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 169818us
I20260709 13:02:15.010316 10320 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:14.940430 10083 consensus_queue.cc:1048] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.164s
I20260709 13:02:15.055783 10375 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:15.129937  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:15.113772 10291 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:15.085265 10359 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.085048 10339 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:15.072777  9005 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:02:15.049933 10382 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:15.135632  8255 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:15.106690 10335 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:15.089486 10118 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:15.073342 10292 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:15.145968  9006 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
W20260709 13:02:15.148571  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 195725us
I20260709 13:02:15.109184 10378 raft_consensus.cc:2764] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:02:15.132615  9001 raft_consensus.cc:2393] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 4: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:15.049671 10304 consensus_queue.cc:1059] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 13:02:15.132972 10213 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:15.126529 10371 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:15.049930 10337 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:15.176647 10331 consensus_queue.cc:1048] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:02:15.206832  9001 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:15.208117 10331 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:15.202540 10292 raft_consensus.cc:3055] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:15.210566 10292 raft_consensus.cc:740] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:15.183149 10213 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.211642 10337 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:15.212076 10213 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:15.213373  9989 raft_consensus.cc:3055] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:15.202634 10379 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:02:15.213930  9989 raft_consensus.cc:740] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:15.214802  9989 consensus_queue.cc:260] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:15.216096  9989 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:15.219803  8398 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 253) took 2163 ms (client timeout 1779 ms). Trace:
W20260709 13:02:15.220345  8398 rpcz_store.cc:269] 0709 13:02:13.056563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.134033 (+1077470us) service_pool.cc:224] Handling call
0709 13:02:15.219543 (+1085510us) spinlock_profiling.cc:243] Waited 68.1 ms on lock 0x7b54003e6054. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:15.219718 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156700544}
I20260709 13:02:15.223024  8369 leader_election.cc:400] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5.
I20260709 13:02:15.223461  8369 leader_election.cc:403] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:15.226150  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:15.211673 10292 consensus_queue.cc:260] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:15.226811  8325 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:15.227386 10292 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:15.139364 10380 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:15.145947 10271 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:15.132290 10268 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:02:15.133232 10073 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:02:15.157799  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:15.158063 10339 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:15.181027  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:15.162029 10291 raft_consensus.cc:697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 6 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:15.170044 10329 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.228066 10379 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:02:15.227794 10290 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:15.228267  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:15.228847  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:15.160001  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:15.181969  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:15.234000  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:15.236027 10325 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:15.236819  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:15.238056  8331 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:02:15.238759  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:15.239523  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:15.283552 10271 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:15.240039  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:15.240262  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:15.241057  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:15.243109  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:15.243506  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:15.244674  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:15.247905  9007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 417) took 2645 ms. Trace:
I20260709 13:02:15.248239 10297 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:15.249508 10296 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:15.250547  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1518) took 4036 ms. Trace:
W20260709 13:02:15.258117  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.162506 10359 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:15.260476  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:15.261344  9003 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56940 (request call id 505) took 5103 ms (client timeout 1747 ms). Trace:
W20260709 13:02:15.262776  8974 connection.cc:256] RPC call timeout handler was delayed by -1.1674555800000235s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15100000000000002s for a response
I20260709 13:02:15.267449  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:15.273041  8222 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:15.272197  8297 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:15.182263 10346 consensus_meta.cc:287] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.844s	user 0.010s	sys 0.001s
W20260709 13:02:15.285946  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 198109us
I20260709 13:02:15.237941 10213 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:15.288168  8372 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:15.331176  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:15.332038  8397 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:15.333057  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:15.333354  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 60382.3us
W20260709 13:02:15.335157  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.303045  8398 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:02:15.337369  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:15.338625  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:15.339236  8398 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:15.303478  8406 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:15.304107  8251 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:15.342247 10219 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.342767 10219 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:15.344048 10123 raft_consensus.cc:1064] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b: attempting to promote NON_VOTER f865c4d761f5446eb12b3a029234772e to VOTER
I20260709 13:02:15.344750 10219 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:15.306870  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:15.348091  8329 raft_consensus.cc:2368] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5.
I20260709 13:02:15.307863  8323 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 5: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:15.308735  8328 raft_consensus.cc:2376] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3. Re-sending same reply.
I20260709 13:02:15.349541  8222 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:15.309561  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:15.311563  8254 rpcz_store.cc:276] 0709 13:02:11.214574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.241636 (+3027062us) service_pool.cc:224] Handling call
0709 13:02:14.385083 (+143447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.392020 (+  6937us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.250317 (+858297us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b5400178750. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:15.250477 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":327486848}
I20260709 13:02:15.310170  9007 rpcz_store.cc:276] 0709 13:02:12.602092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.613246 (+ 11154us) service_pool.cc:224] Handling call
0709 13:02:12.613633 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.776273 (+1162640us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:13.776300 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:13.776340 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:13.776366 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.247253 (+1470887us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b540057d5d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:15.247340 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.247581 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2672147328}
I20260709 13:02:15.352730  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:02:15.314028  8974 connection.cc:256] RPC call timeout handler was delayed by -1.0348848599999769s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1779s for a response
W20260709 13:02:15.358177  8298 leader_election.cc:336] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Timed out: RequestConsensusVote RPC to 127.6.255.4:33825 timed out after 1.638s (SENT)
I20260709 13:02:15.325484  8325 raft_consensus.cc:1275] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:15.322824 10213 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:15.345564 10090 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:15.353402  8254 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:15.296164 10339 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:15.326021  8372 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:15.315285 10118 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:15.376005  8974 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:02:15.358695  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.368314  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 1548) took 1027 ms. Trace:
I20260709 13:02:15.297811  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:15.323875  8220 leader_election.cc:400] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4.
I20260709 13:02:15.306932  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:15.305801  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:15.475142  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:15.324427 10138 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:15.474427 10379 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5.
I20260709 13:02:15.484365 10138 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:15.318506 10359 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:15.307224 10271 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:15.396950 10313 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:15.475376 10386 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:15.487027  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1488) took 4318 ms. Trace:
I20260709 13:02:15.487553 10313 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:15.486824 10386 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:15.486159 10322 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:15.489296 10386 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:15.489444  8400 raft_consensus.cc:1275] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:02:15.489907  8400 pending_rounds.cc:85] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 5
I20260709 13:02:15.489944  8220 leader_election.cc:403] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:15.490289  8400 pending_rounds.cc:107] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:02:15.490607  8400 raft_consensus.cc:2889] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:15.490797  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 166976us
I20260709 13:02:15.490882  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:15.491225  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1540) took 3018 ms. Trace:
I20260709 13:02:15.491505  9007 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:15.491902  8253 raft_consensus.cc:2393] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 5: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:15.491549  8400 rpcz_store.cc:276] 0709 13:02:12.473034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.328033 (+1854999us) service_pool.cc:224] Handling call
0709 13:02:14.352995 (+ 24962us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:15.489176 (+1136181us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b54000dbc20. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f8828735dc0 00007f88287c0d87 00007f88287c1470 00007f88287bdd8f 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c
0709 13:02:15.491004 (+  1828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.491178 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":403819904}
I20260709 13:02:15.492983  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:15.493105  9971 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:15.493625  9971 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:15.493997  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:15.494674  8253 raft_consensus.cc:2368] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 5. Current term is 6.
I20260709 13:02:15.494510  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:15.495352  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:15.327119 10375 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.496342 10375 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:15.498253 10333 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:02:15.498410 10375 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:15.481667 10118 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.501088 10118 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:15.501843  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:15.502770  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1539) took 3316 ms. Trace:
I20260709 13:02:15.502971  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:15.503132  8405 rpcz_store.cc:276] 0709 13:02:12.186729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.251629 (+2064900us) service_pool.cc:224] Handling call
0709 13:02:14.251880 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.743456 (+491576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.502579 (+759123us) spinlock_profiling.cc:243] Waited 416 ms on lock 0x7b54002684d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:15.502725 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":956934528}
I20260709 13:02:15.504269  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:15.504914  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1835) took 1101 ms. Trace:
I20260709 13:02:15.505070 10090 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:15.293673 10382 consensus_queue.cc:1059] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.212s
I20260709 13:02:15.505885  8223 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:15.506875 10072 raft_consensus.cc:3055] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:15.507314 10072 raft_consensus.cc:740] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:15.507444 10382 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:15.507869 10382 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.508188 10382 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:15.508288 10072 consensus_queue.cc:260] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:15.510188  8397 rpcz_store.cc:276] 0709 13:02:14.340630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.332795 (+992165us) service_pool.cc:224] Handling call
0709 13:02:15.368241 (+ 35446us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:15.512252  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:15.512367  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:15.303475 10290 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:15.513325  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:15.366726  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1533) took 3999 ms. Trace:
I20260709 13:02:15.514449 10123 consensus_queue.cc:237] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:15.514179  8401 rpcz_store.cc:276] 0709 13:02:11.355058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.638388 (+1283330us) service_pool.cc:224] Handling call
0709 13:02:12.683677 (+ 45289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.838172 (+1154495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.327604 (+1489432us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b5400277754. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:15.354764 (+ 27160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":736990592}
I20260709 13:02:15.513165  8397 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:15.487398  8256 rpcz_store.cc:276] 0709 13:02:11.169135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.976997 (+2807862us) service_pool.cc:224] Handling call
0709 13:02:14.159283 (+182286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:14.204231 (+ 44948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.486853 (+1282622us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b5400346cd0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:15.486974 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":302136576}
I20260709 13:02:15.519024  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 1556) took 1100 ms. Trace:
I20260709 13:02:15.486174  9009 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:15.519503  8397 rpcz_store.cc:276] 0709 13:02:14.418175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.512090 (+1093915us) service_pool.cc:224] Handling call
0709 13:02:15.518944 (+  6854us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:15.503535  8329 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:15.326952 10227 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:15.522220  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:15.321241  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:15.514482  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:15.301951  8249 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:15.526623  8331 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:15.597126  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:15.527673  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 169011us
I20260709 13:02:15.289999 10378 consensus_queue.cc:1059] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.144s
I20260709 13:02:15.598191  8974 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:15.432893  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:15.524724  8254 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
W20260709 13:02:15.528980  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 202908us
I20260709 13:02:15.299110 10329 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:15.395442 10204 raft_consensus.cc:3055] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:02:15.310992 10311 consensus_meta.cc:287] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.862s	user 0.002s	sys 0.008s
I20260709 13:02:15.307302  8324 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 5: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:15.590361  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:15.591128  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:15.610654  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:15.593892  8398 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:15.611941  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 1555) took 1194 ms. Trace:
I20260709 13:02:15.612362  8398 rpcz_store.cc:276] 0709 13:02:14.417757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.338379 (+920622us) service_pool.cc:224] Handling call
0709 13:02:15.611876 (+273497us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:15.354175 10380 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:15.505244  8405 rpcz_store.cc:276] 0709 13:02:14.403262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.504044 (+1100782us) service_pool.cc:224] Handling call
0709 13:02:15.504861 (+   817us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:15.431175 10381 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:15.622802  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:15.480806  8972 leader_election.cc:336] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.530s (SENT)
I20260709 13:02:15.623309  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:15.354094  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:15.491559  8220 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:15.308445  8327 raft_consensus.cc:2393] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 3: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:15.493112  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:15.624076 10311 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:15.641762  8399 raft_consensus.cc:2393] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 4: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:15.643249 10311 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:15.600272  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:15.303792  9000 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:15.646378  8254 raft_consensus.cc:2393] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 5: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:15.604453  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:15.604799  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:15.433336 10219 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.606006  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:15.656875  8332 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:15.607400  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1826) took 1279 ms. Trace:
I20260709 13:02:15.657739  8397 rpcz_store.cc:276] 0709 13:02:14.330243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.521932 (+1191689us) service_pool.cc:224] Handling call
0709 13:02:15.607339 (+ 85407us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:15.658910  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:15.314680 10346 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:15.659682  8397 raft_consensus.cc:2376] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6. Re-sending same reply.
I20260709 13:02:15.660353  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1834) took 1257 ms. Trace:
I20260709 13:02:15.660727  8397 rpcz_store.cc:276] 0709 13:02:14.402947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.658581 (+1255634us) service_pool.cc:224] Handling call
0709 13:02:15.660287 (+  1706us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:15.661762  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:15.326824 10317 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:15.356215  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:15.433842 10198 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:15.475229 10083 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.478554  9005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56940 (request call id 519) took 1024 ms. Trace:
W20260709 13:02:15.591585 10338 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 13:02:15.581236 10213 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:15.591976 10299 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:15.624226  8252 raft_consensus.cc:2393] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 6: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:15.624840  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:15.702512  8248 raft_consensus.cc:2368] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 5. Current term is 6.
W20260709 13:02:15.632750  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 151890us
I20260709 13:02:15.631680  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:15.634550  8403 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:15.703243  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:15.703693  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 1543) took 1365 ms. Trace:
I20260709 13:02:15.704166  8403 rpcz_store.cc:276] 0709 13:02:14.338534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.329975 (+991441us) service_pool.cc:224] Handling call
0709 13:02:15.703635 (+373660us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:15.614507  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:15.637732  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:15.706578  9009 raft_consensus.cc:2393] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 4: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:15.581089 10227 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:15.590190 10313 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:15.650431  8400 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:15.711884  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 1558) took 1275 ms. Trace:
I20260709 13:02:15.712360  8400 rpcz_store.cc:276] 0709 13:02:14.436416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.492878 (+1056462us) service_pool.cc:224] Handling call
0709 13:02:15.711802 (+218924us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:15.431316 10331 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.183s
I20260709 13:02:15.651847  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:15.716635  8328 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:15.652264  8330 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
W20260709 13:02:15.652657  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 161102us
I20260709 13:02:15.720265  8220 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:15.654574  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:15.475006  9989 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.147s
I20260709 13:02:15.298794 10291 consensus_queue.cc:237] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:15.606138 10204 raft_consensus.cc:740] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:15.613276  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:15.637288  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:15.732574  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:15.693918  9005 rpcz_store.cc:276] 0709 13:02:14.453956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.864233 (+410277us) service_pool.cc:224] Handling call
0709 13:02:15.478359 (+614126us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54006c07d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:15.478488 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":329433856}
I20260709 13:02:15.731348  8398 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:15.639777 10381 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:02:15.313894  9003 rpcz_store.cc:269] 0709 13:02:10.157577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.159429 (+  1852us) service_pool.cc:224] Handling call
0709 13:02:15.261087 (+5101658us) spinlock_profiling.cc:243] Waited 3.41 s on lock 0x7b540067acd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:15.261240 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7835439488}
I20260709 13:02:15.695708  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:15.747931  8251 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:15.656437 10290 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:15.698105  8332 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:15.698638  8402 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 4: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:15.659253  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:15.700412  8298 leader_election.cc:336] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Timed out: RequestConsensusVote RPC to 127.6.255.4:33825 timed out after 1.791s (SENT)
I20260709 13:02:15.697157 10083 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:15.635262  8222 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:15.763399  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 128187us
W20260709 13:02:15.764336  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.764631  8398 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:15.709118  8324 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 5: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:15.766767  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:15.767947  8251 raft_consensus.cc:2368] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5.
I20260709 13:02:15.734047 10138 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:15.769531  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:15.770426  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:15.724169  8220 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:15.725193  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:15.727821  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:15.651539 10387 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.093s
I20260709 13:02:15.725705 10260 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.664499 10219 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:15.708745 10299 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:15.713821 10227 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:15.744485  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:15.747124  8401 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:15.751133 10381 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:15.741496  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:15.804131  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:15.754243  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:15.755889  8223 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:15.805744  8297 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:15.655915 10320 consensus_meta.cc:287] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.516s	user 0.006s	sys 0.004s
I20260709 13:02:15.755432  8298 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:15.726255 10384 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.007s	sys 0.026s
W20260709 13:02:16.098235 10384 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.892s	user 0.007s	sys 0.027s
I20260709 13:02:15.726507 10325 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:02:15.776973  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:16.101330  8398 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:16.116369 10339 consensus_meta.cc:287] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.535s	user 0.010s	sys 0.000s
I20260709 13:02:15.771711  8222 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:02:16.157825  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 385976us
I20260709 13:02:15.782720  8329 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 6
W20260709 13:02:16.160475 10271 consensus_meta.cc:287] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.675s	user 0.001s	sys 0.013s
I20260709 13:02:16.160877 10271 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:16.161850  8397 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:15.782091  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 58011.6us
I20260709 13:02:16.166802  8220 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:15.732234  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:16.168119 10380 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:16.168418  8327 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:16.168784 10380 raft_consensus.cc:697] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:16.169515 10271 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:16.169662 10380 consensus_queue.cc:237] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:15.788010  8325 raft_consensus.cc:2393] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 3: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:16.172343  8325 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:16.168141  8220 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:16.173161 10227 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:15.726699 10313 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:15.798382  8253 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:16.174855 10138 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:16.175390 10138 raft_consensus.cc:697] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:16.176622 10138 consensus_queue.cc:237] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:15.800786  8403 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:16.178063  8220 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:16.179823 10227 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:15.738601  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:16.181274 10380 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:15.805821  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 50036.5us
I20260709 13:02:16.185011 10268 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:16.185196 10380 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:16.186663 10380 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.694073 10198 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:16.187101 10380 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:16.188656 10198 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:16.189033 10380 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:15.807086  8332 raft_consensus.cc:2393] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 5: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:02:16.190301  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 148619us
I20260709 13:02:16.190522  8332 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:02:15.822391  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:15.801568 10299 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:15.825392  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 342442us
I20260709 13:02:15.786518 10290 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:15.831987 10335 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:16.197619 10335 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.198158 10335 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:15.837257  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:15.900279  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.914288  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:16.202036  9005 raft_consensus.cc:2410] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:02:16.202376  9003 raft_consensus.cc:2393] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 4: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:16.202977  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:16.203603  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:16.203717  9005 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:15.941184  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:16.205456  9007 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:16.206404  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:16.207414  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:16.207568  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.207991  9007 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:16.208783  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:15.943835  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:16.205202  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:16.210207  9005 raft_consensus.cc:2393] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 4: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:16.210310  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:16.211133  8401 raft_consensus.cc:2393] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 3: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:16.211264  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:16.211938  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:16.212208  9005 raft_consensus.cc:2393] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 4: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:16.212677  8401 raft_consensus.cc:2368] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4.
I20260709 13:02:15.833788 10204 consensus_queue.cc:260] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:16.033627 10118 consensus_meta.cc:287] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.532s	user 0.015s	sys 0.011s
I20260709 13:02:16.213801 10118 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:16.035017  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 201021us
I20260709 13:02:16.215849 10118 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:16.035542  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:16.216745  8251 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:16.217136 10338 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.217677 10338 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:16.218103 10384 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.034313 10346 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:16.036487  8402 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 1547) took 1696 ms (client timeout 1715 ms). Trace:
I20260709 13:02:16.218606 10384 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:16.219084  8402 rpcz_store.cc:269] 0709 13:02:14.340324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.440387 (+100063us) service_pool.cc:224] Handling call
0709 13:02:15.827905 (+1387518us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400276354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:16.036415 (+208510us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2776809344}
I20260709 13:02:16.219735 10338 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:16.037441  8250 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:15.788705 10365 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.220623  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 259) took 3625 ms. Trace:
I20260709 13:02:16.220994 10365 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:16.221145  8250 rpcz_store.cc:276] 0709 13:02:12.594981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.353093 (+1758112us) service_pool.cc:224] Handling call
0709 13:02:15.776249 (+1423156us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b5800002a98. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699e9c 00007f8825699a5b 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:02:15.800200 (+ 23951us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:16.220365 (+420165us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.220565 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5257600}
I20260709 13:02:16.222407  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:16.223443 10365 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:16.039156  9000 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:16.224601  9000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56948 (request call id 504) took 1792 ms. Trace:
I20260709 13:02:16.224982  9000 rpcz_store.cc:276] 0709 13:02:14.432154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.156469 (+724315us) service_pool.cc:224] Handling call
0709 13:02:16.224545 (+1068076us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.225970  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:16.046646 10317 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:15.766968 10259 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:16.226686  9000 raft_consensus.cc:2368] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4.
I20260709 13:02:16.047371 10329 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:16.047809 10279 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:16.227424 10279 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:16.227473  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:16.227926 10279 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:16.228669  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:16.050787  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:16.040313  9971 consensus_meta.cc:287] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.546s	user 0.005s	sys 0.018s
I20260709 13:02:16.036036 10219 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:16.052786  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:16.229318  8405 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:16.230779  8249 raft_consensus.cc:2376] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6. Re-sending same reply.
I20260709 13:02:16.231931  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:16.232698  8372 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:16.036110 10390 log.cc:927] Time spent T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.280s	user 0.009s	sys 0.000s
I20260709 13:02:16.054800  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49220 (request call id 271) took 1613 ms. Trace:
I20260709 13:02:16.035815  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:16.233731  8372 leader_election.cc:400] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4.
I20260709 13:02:16.234299  8372 leader_election.cc:403] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:02:16.234325 10320 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:16.223639  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:16.234932 10320 raft_consensus.cc:697] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:16.235018  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:16.235644  8402 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:16.235929  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:16.235872 10320 consensus_queue.cc:237] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:16.236457  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:16.062542 10388 raft_consensus.cc:3055] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:16.237144  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:16.235047  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:16.066282  8974 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:16.064239  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:16.238153  8405 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 5
W20260709 13:02:16.238322  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 172009us
W20260709 13:02:16.069689  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.070421 10292 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:16.073899 10374 raft_consensus.cc:3055] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:16.239583 10291 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:16.239784 10374 raft_consensus.cc:740] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:16.240032 10291 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:16.073746  8252 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:16.240427 10291 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:16.241091  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1524) took 1010 ms. Trace:
I20260709 13:02:16.241478  8252 rpcz_store.cc:276] 0709 13:02:15.230976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.711650 (+480674us) service_pool.cc:224] Handling call
0709 13:02:16.240954 (+529304us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b540034c6d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:16.241041 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":782229760}
I20260709 13:02:16.220551 10384 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:16.243600  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1526) took 1011 ms. Trace:
I20260709 13:02:16.243957  8252 rpcz_store.cc:276] 0709 13:02:15.231952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.242498 (+1010546us) service_pool.cc:224] Handling call
0709 13:02:16.242915 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:16.243125 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:16.243161 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:16.243238 (+    77us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:16.243289 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.243400 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:16.243532 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.077855  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:16.237167  8402 raft_consensus.cc:2393] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 6: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:16.246369  8331 raft_consensus.cc:1240] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:16.246886  8405 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:16.247725  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:16.248359  8405 raft_consensus.cc:2393] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 3: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:16.248689  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 1554) took 1871 ms. Trace:
I20260709 13:02:16.080363 10072 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:16.126684  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:16.249780  8251 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:16.238195  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 276) took 1292 ms. Trace:
I20260709 13:02:16.250896  8251 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:16.246603  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:16.250715  8257 rpcz_store.cc:276] 0709 13:02:14.945692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.017968 (+ 72276us) service_pool.cc:224] Handling call
0709 13:02:15.091627 (+ 73659us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:15.091913 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:15.091953 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:15.092529 (+   576us) log.cc:844] Serialized 26 byte log entry
0709 13:02:15.583011 (+490482us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:15.583065 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.583167 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:16.238005 (+654838us) raft_consensus.cc:1685] finished
0709 13:02:16.238025 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:16.238143 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":418959,"threads_started":1,"wal-append.queue_time_us":570057}
I20260709 13:02:16.252410  8251 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:16.130323 10083 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:16.086864  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:16.213306  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:16.130898  8404 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:16.254004  9009 raft_consensus.cc:2393] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 5: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
W20260709 13:02:16.254637  8404 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 1545) took 1915 ms (client timeout 1835 ms). Trace:
I20260709 13:02:16.254818  9005 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6. Ops: []
W20260709 13:02:16.132616  8406 consensus_meta.cc:287] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.786s	user 0.008s	sys 0.003s
I20260709 13:02:16.255388  8406 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:16.255609  8250 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:16.255911  8406 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1831) took 1854 ms (client timeout 1828 ms). Trace:
W20260709 13:02:16.256264  8406 rpcz_store.cc:269] 0709 13:02:14.401867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.159721 (+757854us) service_pool.cc:224] Handling call
0709 13:02:16.255833 (+1096112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.239516 10279 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:16.257385  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:16.257951  8406 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:02:16.134298  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:16.261637  8255 raft_consensus.cc:2410] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:16.255060  9009 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:02:16.261716 10335 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:16.233080  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:16.263582 10083 consensus_queue.cc:1059] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:16.233708  8248 rpcz_store.cc:276] 0709 13:02:14.441444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.667123 (+225679us) service_pool.cc:224] Handling call
0709 13:02:16.054623 (+1387500us) spinlock_profiling.cc:243] Waited 947 ms on lock 0x7b5400408654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:16.054742 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2176962944}
I20260709 13:02:16.145687 10359 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:16.148142  9903 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:16.148361 10396 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.106s	user 0.011s	sys 0.010s
I20260709 13:02:16.252707  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:16.149587 10404 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.091s	user 0.012s	sys 0.010s
W20260709 13:02:16.149513 10400 log.cc:927] Time spent T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.101s	user 0.015s	sys 0.007s
I20260709 13:02:16.267388  9004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 513) took 5213 ms. Trace:
I20260709 13:02:16.129848  8154 catalog_manager.cc:5697] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: config changed from index 2 to 5, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:16.153581  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 345805us
W20260709 13:02:16.153770 10398 log.cc:927] Time spent T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.115s	user 0.002s	sys 0.000s
I20260709 13:02:16.155097  9008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 533) took 1191 ms. Trace:
W20260709 13:02:16.153627 10395 log.cc:927] Time spent T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.116s	user 0.001s	sys 0.001s
W20260709 13:02:16.156462 10397 log.cc:927] Time spent T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.116s	user 0.001s	sys 0.001s
I20260709 13:02:16.159102 10381 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:15.779497  8328 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:16.167914  8397 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:16.335414  8323 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:02:16.334821  8972 leader_election.cc:400] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 5. Current term is 6.
I20260709 13:02:16.336302  8972 leader_election.cc:403] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:16.173590 10122 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:15.750309  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:16.175421 10313 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:15.794557 10260 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:16.207261 10271 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:16.214099 10204 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:16.346448  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 272) took 1027 ms. Trace:
I20260709 13:02:16.350982  8397 rpcz_store.cc:276] 0709 13:02:15.318988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.661493 (+342505us) service_pool.cc:224] Handling call
0709 13:02:16.346240 (+684747us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b5400277254. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:16.346378 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1147590912}
I20260709 13:02:16.353250  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:16.354269  8397 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:16.345149 10268 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:02:16.355451  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 268) took 1039 ms. Trace:
I20260709 13:02:16.229743  9971 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:16.357770  8254 raft_consensus.cc:2393] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 3: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:16.336256 10338 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4.
I20260709 13:02:16.359388  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1527) took 1126 ms. Trace:
I20260709 13:02:16.359755  8254 rpcz_store.cc:276] 0709 13:02:15.232542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.738281 (+505739us) service_pool.cc:224] Handling call
0709 13:02:16.359207 (+620926us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b54003471d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:16.359334 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":404320000}
W20260709 13:02:16.051856  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.360657  9971 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:16.053846  8399 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 1559) took 1617 ms (client timeout 1642 ms). Trace:
W20260709 13:02:16.364038  8399 rpcz_store.cc:269] 0709 13:02:14.436695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.459398 (+ 22703us) service_pool.cc:224] Handling call
0709 13:02:16.053581 (+1594183us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540019bed4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:16.053768 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2614688640}
I20260709 13:02:16.365228  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:16.366015  8399 raft_consensus.cc:2393] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 4: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:16.366853  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:16.367127 10381 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:16.367476  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 260) took 1053 ms. Trace:
I20260709 13:02:16.367843  8399 rpcz_store.cc:276] 0709 13:02:15.314484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.366650 (+1052166us) service_pool.cc:224] Handling call
0709 13:02:16.367430 (+   780us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.368378  8327 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
I20260709 13:02:16.368656  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:16.369421  9006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 502) took 1937 ms. Trace:
I20260709 13:02:16.370039  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:16.370775  8328 raft_consensus.cc:2393] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 4: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:16.237593 10388 raft_consensus.cc:740] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:16.334707  9903 consensus_queue.cc:1059] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:02:16.240680 10374 consensus_queue.cc:260] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:16.248955  8974 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:02:16.375947  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 126999us
I20260709 13:02:16.378449  8974 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:16.209025  8369 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:16.263017  8330 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:57170
I20260709 13:02:16.263877 10335 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:16.335794 10359 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:02:16.265026 10384 raft_consensus.cc:3055] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:16.264969 10279 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:16.269282 10291 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:16.268741  9000 raft_consensus.cc:1240] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:16.270176 10292 consensus_queue.cc:1059] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:16.267793  9004 rpcz_store.cc:276] 0709 13:02:11.053692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.060179 (+  6487us) service_pool.cc:224] Handling call
0709 13:02:14.661703 (+3601524us) spinlock_profiling.cc:243] Waited 6.04 ms on lock 0x7b5800175898. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:02:15.327576 (+665873us) spinlock_profiling.cc:243] Waited 3.89 ms on lock 0x7b5800175898. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699e9c 00007f8825699a5b 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:02:15.327774 (+   198us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:15.328217 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:15.328270 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:15.328923 (+   653us) log.cc:844] Serialized 26 byte log entry
0709 13:02:15.735985 (+407062us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:16.181611 (+445626us) log.cc:844] Serialized 14 byte log entry
0709 13:02:16.181818 (+   207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.181902 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:16.267146 (+ 85244us) raft_consensus.cc:1685] finished
0709 13:02:16.267179 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:16.267300 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1124,"prepare.queue_time_us":287914,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":8107,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":12003,"spinlock_wait_cycles":22826752,"thread_start_us":791290,"threads_started":2,"wal-append.queue_time_us":713354}
I20260709 13:02:16.276507  8406 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:16.421260  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:16.277350  8154 catalog_manager.cc:5697] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: config changed from index 4 to 6, term changed from 3 to 4, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to <none>, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New cstate: current_term: 4 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:16.422079  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:16.422818  8323 raft_consensus.cc:2393] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 2: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:16.248970  8405 rpcz_store.cc:276] 0709 13:02:14.377802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.247526 (+1869724us) service_pool.cc:224] Handling call
0709 13:02:16.248647 (+  1121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.382381 10338 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:16.043505 10382 consensus_meta.cc:287] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.535s	user 0.012s	sys 0.000s
I20260709 13:02:16.292498  9002 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 6 LEARNER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260709 13:02:16.149616  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 449024us
I20260709 13:02:16.274066  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:16.293347  8403 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:16.445909  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1842) took 1096 ms. Trace:
I20260709 13:02:16.446369  8403 rpcz_store.cc:276] 0709 13:02:15.349074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.728451 (+379377us) service_pool.cc:224] Handling call
0709 13:02:16.445834 (+717383us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.253257  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:16.447518  8402 raft_consensus.cc:2376] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4. Re-sending same reply.
I20260709 13:02:16.448016  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 259) took 1133 ms. Trace:
I20260709 13:02:16.443429  8154 catalog_manager.cc:5697] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 2 to 3, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 3 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:02:16.307466  8329 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:16.209218  9007 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:16.297158  8223 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:16.449446  8154 catalog_manager.cc:5697] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 1 to 2, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 2 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
W20260709 13:02:16.450415  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 153267us
I20260709 13:02:16.451328  8154 catalog_manager.cc:5697] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 4 to 5, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 5 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:16.303092  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1528) took 1012 ms. Trace:
I20260709 13:02:16.330736  8297 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:16.453908  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 123174us
I20260709 13:02:16.454802  8297 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:16.455675  8297 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:16.456729  8297 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:16.458248  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:16.317286  8253 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:16.331382  9008 rpcz_store.cc:276] 0709 13:02:14.963424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.236895 (+273471us) service_pool.cc:224] Handling call
0709 13:02:15.237185 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:15.237584 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:15.237632 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:15.238099 (+   467us) log.cc:844] Serialized 26 byte log entry
0709 13:02:15.700728 (+462629us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:16.024640 (+323912us) log.cc:844] Serialized 14 byte log entry
0709 13:02:16.052639 (+ 27999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.052753 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:16.154833 (+102080us) raft_consensus.cc:1685] finished
0709 13:02:16.154860 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:16.155024 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19359,"prepare.queue_time_us":336714,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":21066,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":138,"thread_start_us":757542,"threads_started":2,"wal-append.queue_time_us":573399}
I20260709 13:02:16.226096  8222 leader_election.cc:400] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4.
I20260709 13:02:16.461019  8222 leader_election.cc:403] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:02:16.224740  8325 raft_consensus.cc:1240] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:02:16.462029  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1620) took 1125 ms. Trace:
I20260709 13:02:16.462447  8325 rpcz_store.cc:276] 0709 13:02:15.336792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.173146 (+836354us) service_pool.cc:224] Handling call
0709 13:02:16.224501 (+ 51355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:16.461679 (+237178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.461959 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.051565  9001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 532) took 1089 ms. Trace:
I20260709 13:02:16.379735  8330 raft_consensus.cc:606] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Received request to transfer leadership
W20260709 13:02:16.255043  8404 rpcz_store.cc:269] 0709 13:02:14.339679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.752785 (+413106us) service_pool.cc:224] Handling call
0709 13:02:16.254501 (+1501716us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54003e7954. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:16.254584 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2957031808}
W20260709 13:02:16.464597  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 340330us
I20260709 13:02:16.369285  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 267) took 1053 ms. Trace:
I20260709 13:02:16.465216  8399 rpcz_store.cc:276] 0709 13:02:15.316277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.368435 (+1052158us) service_pool.cc:224] Handling call
0709 13:02:16.369239 (+   804us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.356950  8397 rpcz_store.cc:276] 0709 13:02:15.316515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.352913 (+1036398us) service_pool.cc:224] Handling call
0709 13:02:16.355381 (+  2468us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.465030  8154 catalog_manager.cc:5697] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:16.140192  8398 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:16.466082  8295 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:16.399588  8332 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:16.452919  8252 rpcz_store.cc:276] 0709 13:02:15.233063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.244761 (+1011698us) service_pool.cc:224] Handling call
0709 13:02:16.245010 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:16.245156 (+   146us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:16.245181 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:16.245216 (+    35us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:16.245250 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.245307 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:16.245407 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.468152  8154 catalog_manager.cc:5697] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 2 to 4, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 4 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:16.471837  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1513) took 1080 ms. Trace:
I20260709 13:02:16.472219  8253 rpcz_store.cc:276] 0709 13:02:15.391589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.495138 (+103549us) service_pool.cc:224] Handling call
0709 13:02:16.471684 (+976546us) spinlock_profiling.cc:243] Waited 302 ms on lock 0x7b540026c854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:16.471788 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":695248128}
I20260709 13:02:16.472589  8154 catalog_manager.cc:5697] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 2 to 3, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 3 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:16.288648 10299 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:16.139631 10333 consensus_meta.cc:287] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.641s	user 0.008s	sys 0.005s
I20260709 13:02:16.411535  8328 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:16.357961  8326 raft_consensus.cc:2393] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 4: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:16.398681  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:16.383445 10374 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:16.412037  9971 raft_consensus.cc:3055] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:16.373827 10388 consensus_queue.cc:260] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:16.382906 10297 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:16.378310 10213 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:16.501760 10072 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:16.504207  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:16.488189 10329 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:16.278697  8250 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:16.423633  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:16.543383  8405 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:16.091893  8151 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55754 (request call id 197) took 7404 ms. Trace:
I20260709 13:02:16.544514  8151 rpcz_store.cc:276] 0709 13:02:08.687136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.687317 (+   181us) service_pool.cc:224] Handling call
0709 13:02:08.978629 (+291312us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b54003bf454. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828728090 00007f88287c6f3a 00007f882e57d1cc 00007f882e67b37e 00007f882cb3c2a4 00007f882cb3c232 00007f882cb3c161 00007f882cb3c0dc 00007f882cb3b392 00007f8826af948c 00007f8826af88c6
0709 13:02:16.091794 (+7113165us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:12.254071 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:12.254754 (+   683us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:12.254790 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:02:12.254809 (+    19us) tablet.cc:662] Decoding operations
0709 13:02:12.256128 (+  1319us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:12.256226 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:12.256275 (+    49us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:02:12.261558 (+  5283us) tablet.cc:701] Row locks acquired
0709 13:02:12.285584 (+ 24026us) write_op.cc:260] PREPARE: finished
0709 13:02:12.316874 (+ 31290us) write_op.cc:270] Start()
0709 13:02:12.416467 (+ 99593us) write_op.cc:276] Timestamp: P: 1783602132312297 usec, L: 0
0709 13:02:12.438378 (+ 21911us) op_driver.cc:348] REPLICATION: starting
0709 13:02:12.562000 (+123622us) log.cc:844] Serialized 4289 byte log entry
0709 13:02:14.064177 (+1502177us) op_driver.cc:464] REPLICATION: finished
0709 13:02:14.236525 (+172348us) write_op.cc:301] APPLY: starting
0709 13:02:14.236611 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:14.328690 (+ 92079us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:14.336562 (+  7872us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:14.395132 (+ 58570us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:14.395340 (+   208us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 13:02:14.395380 (+    40us) write_op.cc:312] APPLY: finished
0709 13:02:14.395627 (+   247us) log.cc:844] Serialized 77 byte log entry
0709 13:02:14.395804 (+   177us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:14.395921 (+   117us) write_op.cc:454] Released schema lock
0709 13:02:14.395975 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:14.396344 (+   369us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:14.396371 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":14708,"reported_tablets":15,"spinlock_wait_cycles":492939648,"child_traces":[["op",{"apply.queue_time_us":169583,"apply.run_cpu_time_us":4185,"apply.run_wall_time_us":159787,"num_ops":10,"prepare.queue_time_us":1087054,"prepare.run_cpu_time_us":91310,"prepare.run_wall_time_us":1594897,"raft.queue_time_us":172279,"raft.run_cpu_time_us":18197,"raft.run_wall_time_us":290091,"replication_time_us":1610124,"thread_start_us":1996942,"threads_started":4,"wal-append.queue_time_us":1155394,"wal-append.run_cpu_time_us":24939,"wal-append.run_wall_time_us":1712379}]]}
I20260709 13:02:16.424749 10271 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.422122  8406 raft_consensus.cc:2393] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 5: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:16.293728  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:16.444159  8298 leader_election.cc:336] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Timed out: RequestConsensusVote RPC to 127.6.255.4:33825 timed out after 1.616s (SENT)
I20260709 13:02:16.554215  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 270) took 1237 ms. Trace:
I20260709 13:02:16.554263  8298 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:16.383826 10384 raft_consensus.cc:740] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:16.448313  8402 rpcz_store.cc:276] 0709 13:02:15.314211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.253009 (+938798us) service_pool.cc:224] Handling call
0709 13:02:16.447968 (+194959us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.369774  9006 rpcz_store.cc:276] 0709 13:02:14.431647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.189821 (+758174us) service_pool.cc:224] Handling call
0709 13:02:15.204676 (+ 14855us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:15.205409 (+   733us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:15.205473 (+    64us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:15.288285 (+ 82812us) log.cc:844] Serialized 513 byte log entry
0709 13:02:15.767681 (+479396us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:15.767725 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.767796 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:16.369207 (+601411us) raft_consensus.cc:1685] finished
0709 13:02:16.369235 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:16.369367 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12249,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"thread_start_us":421927,"threads_started":1,"wal-append.queue_time_us":694333}
W20260709 13:02:16.380002  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 170980us
I20260709 13:02:16.639587  8369 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:16.639748  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 50286.7us
I20260709 13:02:16.555649 10388 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:16.475755  8154 catalog_manager.cc:5697] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 1 to 3, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 3 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:02:16.477391  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:16.641774  8254 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:16.478052  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:16.643576  8323 raft_consensus.cc:2368] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4.
I20260709 13:02:16.642113  8154 catalog_manager.cc:5697] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: config changed from index 3 to 4, term changed from 4 to 5, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to <none>, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1), VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2), VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3), VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 5 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:02:16.478788  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 252722us
I20260709 13:02:16.478065  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:02:16.481307  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 102889us
I20260709 13:02:16.479532  8297 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:16.487458  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:16.488179  9007 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:16.646145  8974 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:16.488416  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:16.426306 10338 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:16.490620  8252 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 13:02:16.491135  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:16.648973  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1841) took 1322 ms. Trace:
I20260709 13:02:16.493551  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:16.419994 10339 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:16.470556 10325 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:16.474967 10279 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:16.498035  8400 raft_consensus.cc:2393] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 2: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:16.498162  8295 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:16.464421 10260 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:16.383785 10365 consensus_queue.cc:1175] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f865c4d761f5446eb12b3a029234772e" responder_term: 4 status { last_received { term: 2 index: 5 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: [3.5-3.5]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:16.635975 10387 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.415277  9009 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:16.481530 10122 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:16.487916 10359 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:16.492625 10333 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4.
I20260709 13:02:16.492147  9903 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:16.445031 10382 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:16.490756 10335 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:16.500239  8327 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:16.717947 10268 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 5. Current term is 6.
I20260709 13:02:16.719374 10268 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:16.720343 10407 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:16.721024 10407 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:16.721487 10407 raft_consensus.cc:697] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:16.722329 10407 consensus_queue.cc:237] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:16.554621  8406 rpcz_store.cc:276] 0709 13:02:15.317138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.420939 (+1103801us) service_pool.cc:224] Handling call
0709 13:02:16.554156 (+133217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.725001  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:16.725752  8406 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:16.555173  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 111019us
I20260709 13:02:16.496009  9971 raft_consensus.cc:740] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:16.539881  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:16.635624  8405 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:16.636672  8402 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:16.766920  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1812) took 3755 ms. Trace:
I20260709 13:02:16.503417 10408 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:16.476517  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:16.644701  8154 catalog_manager.cc:5697] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 1 to 2, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 2 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:02:16.767489  8405 rpcz_store.cc:276] 0709 13:02:13.011217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.557881 (+3546664us) service_pool.cc:224] Handling call
0709 13:02:16.635460 (+ 77579us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:16.766609 (+131149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.766869 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.768935  8154 catalog_manager.cc:5697] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 1 to 2, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 2 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
W20260709 13:02:16.647392  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.552639 10329 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:16.649324  8403 rpcz_store.cc:276] 0709 13:02:15.326732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.447203 (+1120471us) service_pool.cc:224] Handling call
0709 13:02:16.648915 (+201712us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.587841  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:16.588006  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49220 (request call id 282) took 1060 ms. Trace:
I20260709 13:02:16.652963  8249 raft_consensus.cc:2393] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 4: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:16.507237 10297 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:16.504928  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 262) took 1190 ms. Trace:
I20260709 13:02:16.639819 10292 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.649271 10291 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:16.648442 10227 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:16.587744 10271 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:16.652042 10338 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:16.740206  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.656742 10279 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:16.639417 10083 consensus_queue.cc:1059] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 13:02:16.762255  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:16.656883 10420 raft_consensus.cc:3055] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:16.938823 10420 raft_consensus.cc:740] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:16.940081 10420 consensus_queue.cc:260] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:16.941346 10418 log.cc:927] Time spent T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 13:02:16.945495 10420 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:16.934621 10325 raft_consensus.cc:697] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:16.767011  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 254) took 3710 ms. Trace:
I20260709 13:02:16.946660 10083 raft_consensus.cc:3055] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:16.946410  8402 rpcz_store.cc:276] 0709 13:02:13.056897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.588505 (+3531608us) service_pool.cc:224] Handling call
0709 13:02:16.636500 (+ 47995us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:16.766716 (+130216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.766949 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.947257 10083 raft_consensus.cc:740] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:16.640758  8369 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:16.948158 10083 consensus_queue.cc:260] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:16.948875  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 308030us
I20260709 13:02:16.946805 10325 consensus_queue.cc:237] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:16.950801  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:16.770723  8154 catalog_manager.cc:5697] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 3 to 5, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 5 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
W20260709 13:02:16.772950  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 127037us
W20260709 13:02:16.952030  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:16.952610  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.764729  8255 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:16.723963 10335 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:16.759866  9005 raft_consensus.cc:1240] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:16.782188  8405 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:16.885480  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 405835us
I20260709 13:02:16.910419  8404 rpcz_store.cc:276] 0709 13:02:15.314973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.490080 (+1175107us) service_pool.cc:224] Handling call
0709 13:02:16.504876 (+ 14796us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.962724  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 255) took 3905 ms. Trace:
I20260709 13:02:16.963238  8297 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:16.963228  8405 rpcz_store.cc:276] 0709 13:02:13.057187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.781704 (+3724517us) service_pool.cc:224] Handling call
0709 13:02:16.782020 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:16.962466 (+180446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.962658 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:16.915616  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 417300us
W20260709 13:02:16.915982  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 269784us
W20260709 13:02:16.919426  8972 leader_election.cc:336] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.512s (SENT)
I20260709 13:02:16.921998  8256 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
W20260709 13:02:16.781400 10416 log.cc:927] Time spent T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.066s	user 0.002s	sys 0.000s
I20260709 13:02:16.714696 10387 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:16.995981  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.930769 10313 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.949272 10083 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:16.975917  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57170 (request call id 77) took 1374 ms. Trace:
I20260709 13:02:17.002434  8330 rpcz_store.cc:276] 0709 13:02:15.595749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.262884 (+667135us) service_pool.cc:224] Handling call
0709 13:02:16.955397 (+692513us) spinlock_profiling.cc:243] Waited 25.1 ms on lock 0x7b4c0033eb40. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8826a8aa20 00007f8826a9ba29 00007f8826a61644 00007f8826a88f2d 00007f8826a88b93 00007f88287b8c92 00007f88287b884a 00007f882d78660a 00007f8826d0c7d4 00007f8826d0c762 00007f8826d0c691
0709 13:02:16.955530 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57656704}
I20260709 13:02:16.884428  8250 rpcz_store.cc:276] 0709 13:02:15.496243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.236895 (+740652us) service_pool.cc:224] Handling call
0709 13:02:16.556330 (+319435us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:17.014739  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1824) took 3302 ms. Trace:
I20260709 13:02:17.020349  8404 rpcz_store.cc:276] 0709 13:02:13.712057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.013846 (+3301789us) service_pool.cc:224] Handling call
0709 13:02:17.014218 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.014406 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.014430 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:17.014469 (+    39us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:17.014515 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.014572 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.014683 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.952706 10279 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:16.928397  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 199145us
I20260709 13:02:16.937441  8399 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:16.681960 10384 consensus_queue.cc:260] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:16.970965 10335 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:16.952584  8154 catalog_manager.cc:5697] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:02:16.777072  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 261) took 1462 ms. Trace:
I20260709 13:02:16.956086  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 283) took 1459 ms. Trace:
I20260709 13:02:16.781639 10329 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:16.932770  9004 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:16.966693  8974 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:16.967638  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:16.773682 10408 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:16.973417  8222 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:16.973986  8403 raft_consensus.cc:1240] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:17.060407  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1822) took 3532 ms. Trace:
I20260709 13:02:17.060869  8403 rpcz_store.cc:276] 0709 13:02:13.527890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.932211 (+3404321us) service_pool.cc:224] Handling call
0709 13:02:16.973753 (+ 41542us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.060133 (+ 86380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.060356 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.974494  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:16.964790  8972 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:16.928126  8406 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:17.030570  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1825) took 2709 ms. Trace:
I20260709 13:02:17.031987  8254 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:02:16.945988 10413 log.cc:927] Time spent T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.294s	user 0.000s	sys 0.000s
I20260709 13:02:16.929355 10292 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:17.048125  8154 catalog_manager.cc:5697] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 1 to 3, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 3 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:02:17.070830  8154 catalog_manager.cc:5697] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } } }
W20260709 13:02:17.005084 10410 log.cc:927] Time spent T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
W20260709 13:02:17.072307  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 121529us
I20260709 13:02:17.073136  8403 raft_consensus.cc:1240] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:17.073622  8298 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:17.075071  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.075228  8298 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:17.075587  8369 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:17.021268 10204 consensus_meta.cc:287] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.665s	user 0.010s	sys 0.000s
I20260709 13:02:16.954162 10299 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:17.000433 10317 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.062088  8397 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:17.069070 10421 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:02:17.067201  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 147753us
I20260709 13:02:17.062511  8404 rpcz_store.cc:276] 0709 13:02:14.324869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.029726 (+2704857us) service_pool.cc:224] Handling call
0709 13:02:17.030044 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.030228 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.030254 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:17.030297 (+    43us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:17.030327 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.030381 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.030502 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:17.062041 10428 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:02:17.062492  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 95870.4us
I20260709 13:02:17.047662  8400 rpcz_store.cc:276] 0709 13:02:15.314725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.822026 (+507301us) service_pool.cc:224] Handling call
0709 13:02:16.776816 (+954790us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b54000fd2d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:16.776997 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":698814208}
I20260709 13:02:17.073887  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1560) took 2636 ms. Trace:
I20260709 13:02:17.052610  8257 rpcz_store.cc:276] 0709 13:02:15.496498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.252074 (+755576us) service_pool.cc:224] Handling call
0709 13:02:16.252321 (+   247us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:16.252614 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:16.480418 (+227804us) log.cc:844] Serialized 14 byte log entry
0709 13:02:16.650638 (+170220us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:16.697841 (+ 47203us) log.cc:844] Serialized 26 byte log entry
0709 13:02:16.698038 (+   197us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:16.698089 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.698211 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:16.941496 (+243285us) raft_consensus.cc:1685] finished
0709 13:02:16.941517 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:16.956029 (+ 14512us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5381,"prepare.queue_time_us":400036,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":8836,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":5467,"thread_start_us":376565,"threads_started":2,"wal-append.queue_time_us":439138}
I20260709 13:02:16.953104  9971 consensus_queue.cc:260] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:17.036871  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1832) took 2634 ms. Trace:
W20260709 13:02:17.077791 10419 log.cc:927] Time spent T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.477s	user 0.000s	sys 0.001s
I20260709 13:02:17.026430 10313 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:17.068289 10426 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:17.072307 10198 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:17.137615 10198 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:17.138085 10198 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:17.140467 10271 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:17.142073 10297 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:17.141350 10271 raft_consensus.cc:2955] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Committing config change with OpId 3.6: config changed from index 4 to 6, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:17.142562 10436 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:17.144271 10382 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:17.144621 10292 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:17.110344  8327 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:17.015897 10138 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:17.064800  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 91404.2us
I20260709 13:02:17.110296  8972 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:17.135113  8400 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:17.147897  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 271) took 1830 ms. Trace:
I20260709 13:02:17.148567  8222 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:17.148200  8400 rpcz_store.cc:276] 0709 13:02:15.318602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.120836 (+1802234us) service_pool.cc:224] Handling call
0709 13:02:17.134794 (+ 13958us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.147668 (+ 12874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.147844 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27264}
I20260709 13:02:17.079169 10431 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:17.064244 10408 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:16.466271  9001 rpcz_store.cc:276] 0709 13:02:14.961978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.208085 (+246107us) service_pool.cc:224] Handling call
0709 13:02:15.208438 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:15.208711 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:15.699516 (+490805us) log.cc:844] Serialized 14 byte log entry
0709 13:02:16.051091 (+351575us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:16.051160 (+    69us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:16.051231 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.051321 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:16.051492 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":120226,"prepare.queue_time_us":644076,"prepare.run_cpu_time_us":16047,"prepare.run_wall_time_us":158196,"raft.queue_time_us":121445,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":97,"thread_start_us":687789,"threads_started":2,"wal-append.queue_time_us":334966}
I20260709 13:02:17.120007  8403 rpcz_store.cc:276] 0709 13:02:14.437012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.061798 (+2624786us) service_pool.cc:224] Handling call
0709 13:02:17.072976 (+ 11178us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.073584 (+   608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.073821 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:17.122236  8399 rpcz_store.cc:276] 0709 13:02:14.402208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.035796 (+2633588us) service_pool.cc:224] Handling call
0709 13:02:17.036187 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.036432 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.036472 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:17.036532 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:17.036591 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.036667 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.036801 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:17.121790  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 46057.1us
W20260709 13:02:17.128459  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 162677us
W20260709 13:02:17.108714 10414 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.427s	user 0.000s	sys 0.001s
W20260709 13:02:17.131253  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 168013us
I20260709 13:02:17.135113  8404 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:17.174718  8297 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:17.174328  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 269) took 1857 ms. Trace:
I20260709 13:02:17.175693  8400 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:17.176522  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1565) took 1856 ms. Trace:
I20260709 13:02:17.157842 10387 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:17.152305  8397 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
I20260709 13:02:17.177740  8400 rpcz_store.cc:276] 0709 13:02:15.320597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.149723 (+1829126us) service_pool.cc:224] Handling call
0709 13:02:17.175523 (+ 25800us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.176227 (+   704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.176446 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:17.086035 10359 raft_consensus.cc:697] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:17.168319  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1823) took 3631 ms. Trace:
W20260709 13:02:17.169775  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:17.170019  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.115458 10204 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:16.997536  9903 raft_consensus.cc:697] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 5 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:17.124110 10339 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:17.172344  8369 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:17.176023  8404 rpcz_store.cc:276] 0709 13:02:15.316792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.121010 (+1804218us) service_pool.cc:224] Handling call
0709 13:02:17.134794 (+ 13784us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.174092 (+ 39298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.174275 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36736}
I20260709 13:02:17.164727  8222 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:02:17.221163  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 48918.3us
W20260709 13:02:17.221329  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 56686.3us
I20260709 13:02:17.222010  8222 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:17.222235  8369 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:02:17.174789  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 99617.6us
W20260709 13:02:17.213929  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 439477us
I20260709 13:02:17.223369  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:17.223986  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.166117 10299 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:17.166687 10382 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:17.116988 10317 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:17.166715 10292 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:17.228479  8298 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:17.217289  8402 rpcz_store.cc:276] 0709 13:02:13.537013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.947555 (+3410542us) service_pool.cc:224] Handling call
0709 13:02:16.947939 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.167439 (+219500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.167519 (+    80us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:17.167642 (+   123us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:17.167724 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.167905 (+   181us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.168185 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":186199,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"thread_start_us":211653,"threads_started":1}
I20260709 13:02:17.171289 10122 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4.
I20260709 13:02:17.167837 10431 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:17.237252  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1848) took 1034 ms. Trace:
I20260709 13:02:17.258520  8405 rpcz_store.cc:276] 0709 13:02:16.203218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.236364 (+1033146us) service_pool.cc:224] Handling call
0709 13:02:17.236695 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.236879 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.236908 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:17.236979 (+    71us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:17.237023 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.237084 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.237191 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:17.238870  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 66341us
W20260709 13:02:17.241956  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 67276.5us
I20260709 13:02:17.242769  8400 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:17.170665 10426 raft_consensus.cc:697] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 5 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:17.175299 10434 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:17.237514 10387 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:17.166841 10138 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:17.264467  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1838) took 1938 ms. Trace:
I20260709 13:02:17.237821 10382 raft_consensus.cc:697] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:17.308816  8400 rpcz_store.cc:276] 0709 13:02:15.326059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.178956 (+1852897us) service_pool.cc:224] Handling call
0709 13:02:17.241372 (+ 62416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.264199 (+ 22827us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.264406 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:17.230341 10260 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:17.230010 10436 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:17.249311  8222 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:17.268949  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:17.252978  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1840) took 1926 ms. Trace:
I20260709 13:02:17.308029  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1626) took 1840 ms. Trace:
I20260709 13:02:17.228686 10440 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:17.294203  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.307228  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:17.311000  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 50783.1us
I20260709 13:02:17.232929  9003 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:17.238652 10445 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:17.311966  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:17.276191 10213 consensus_meta.cc:287] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.774s	user 0.011s	sys 0.001s
I20260709 13:02:17.350811  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:17.403456  8329 raft_consensus.cc:2393] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 4: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:02:17.404184 10435 log.cc:927] Time spent T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 13:02:17.404808 10213 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:17.267555 10299 raft_consensus.cc:697] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:17.266184 10292 raft_consensus.cc:697] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 6 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:17.424986  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:17.434717  8329 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:17.435115  8330 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:02:17.348558  8331 rpcz_store.cc:276] 0709 13:02:15.467894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.247918 (+780024us) service_pool.cc:224] Handling call
0709 13:02:16.248205 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:16.248525 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:16.248551 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:16.248962 (+   411us) log.cc:844] Serialized 26 byte log entry
0709 13:02:16.480705 (+231743us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:16.480761 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.486659 (+  5898us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:17.307765 (+821106us) raft_consensus.cc:1685] finished
0709 13:02:17.307787 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.307948 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1728,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":82,"thread_start_us":231343,"threads_started":1,"wal-append.queue_time_us":307194}
I20260709 13:02:17.437798  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:17.352857  8402 raft_consensus.cc:1275] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:17.259219 10219 consensus_queue.cc:1059] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.098s
I20260709 13:02:17.441217  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1849) took 1214 ms. Trace:
I20260709 13:02:17.441704  8402 rpcz_store.cc:276] 0709 13:02:16.226782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.308085 (+1081303us) service_pool.cc:224] Handling call
0709 13:02:17.352452 (+ 44367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.440893 (+ 88441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.441149 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:17.443852  8329 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:17.383231  8253 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:17.413319  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:17.445791  8326 raft_consensus.cc:2368] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4.
I20260709 13:02:17.446061  8332 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 4. Current term is 5. Ops: [3.6-4.7]
I20260709 13:02:17.266134 10431 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:17.314323 10436 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.311463 10446 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.312662 10433 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:17.311866 10381 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.348912  8404 rpcz_store.cc:276] 0709 13:02:15.326513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.220992 (+1894479us) service_pool.cc:224] Handling call
0709 13:02:17.252392 (+ 31400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.252623 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.252654 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:17.252698 (+    44us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:17.252750 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.252808 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.252919 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:17.456418  8295 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:17.470088 10313 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:17.470045 10292 consensus_queue.cc:237] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:17.475724 10381 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:17.439453 10428 consensus_queue.cc:1048] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:02:17.440086 10359 consensus_queue.cc:237] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:17.477604 10446 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:17.480294  8295 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:17.307276  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:17.413741  8327 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:17.484161  8295 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:17.277098 10443 raft_consensus.cc:2764] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:02:17.318140 10260 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.487249  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:17.483126 10446 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:17.488519  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:17.505980  9006 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:17.489033 10438 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:02:17.494546  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 266014us
I20260709 13:02:17.498799  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:17.515136  8328 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
W20260709 13:02:17.502532  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.472049 10436 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:17.481657 10440 raft_consensus.cc:2764] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:02:17.523830  8372 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:17.525944  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 301391us
I20260709 13:02:17.533994  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:17.429869  9903 consensus_queue.cc:237] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:17.503750 10260 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:17.492990 10198 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:17.514288 10313 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:17.518669 10438 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:17.569808  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:17.570096  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:17.555306  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1561) took 3028 ms. Trace:
I20260709 13:02:17.588271  8406 rpcz_store.cc:276] 0709 13:02:14.526703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.063661 (+2536958us) service_pool.cc:224] Handling call
0709 13:02:17.064036 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:17.173822 (+109786us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:02:17.173858 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:17.180220 (+  6362us) log.cc:844] Serialized 26 byte log entry
0709 13:02:17.450784 (+270564us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:17.450833 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.450916 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:17.555021 (+104105us) raft_consensus.cc:1685] finished
0709 13:02:17.555049 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.555210 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":77934,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":107,"thread_start_us":285417,"threads_started":2,"wal-append.queue_time_us":282411}
I20260709 13:02:17.504530 10433 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:17.569203  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:17.570068  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 320678us
I20260709 13:02:17.570641  8220 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:17.596526  8220 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:17.550629  8369 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:17.535461 10446 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:17.605123  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 54538.5us
I20260709 13:02:17.606621  8369 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:17.612283  8325 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:17.616053 10433 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.574715 10438 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:17.572291 10313 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.606038 10446 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.554781 10198 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:17.548290 10083 consensus_queue.cc:1059] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.079s
I20260709 13:02:17.550580 10439 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:17.603984 10442 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:17.651152  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:17.730664  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 134185us
I20260709 13:02:17.749055  8325 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:17.749527  8220 leader_election.cc:400] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4.
I20260709 13:02:17.495455 10426 consensus_queue.cc:237] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:17.665660  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 150459us
I20260709 13:02:17.665163  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:17.551226 10204 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.092s
W20260709 13:02:17.668689  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 166684us
I20260709 13:02:17.752606  8295 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:17.754007  8295 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:17.536758 10443 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.100s
I20260709 13:02:17.758528  8220 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:17.545696 10299 consensus_queue.cc:237] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:17.652936 10454 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:17.651132 10313 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:17.666997 10446 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:17.691728  9004 consensus_meta.cc:287] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.615s	user 0.007s	sys 0.010s
I20260709 13:02:17.665974 10317 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:17.660408 10433 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:17.699434  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1547) took 1228 ms. Trace:
I20260709 13:02:17.702797  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:17.703100 10431 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:17.704296 10448 log.cc:927] Time spent T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 13:02:17.704905 10338 consensus_meta.cc:287] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.742s	user 0.008s	sys 0.001s
I20260709 13:02:17.665462 10122 consensus_queue.cc:1059] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
W20260709 13:02:17.705267 10408 consensus_meta.cc:287] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.008s
I20260709 13:02:17.800159  8325 raft_consensus.cc:1275] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:17.785836 10458 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:17.693638  8327 raft_consensus.cc:1275] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:17.706696  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:17.805878  8329 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:17.708038  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:17.702791  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:17.704365  9008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 518) took 1488 ms. Trace:
I20260709 13:02:17.686959 10138 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:17.714617  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 112513us
I20260709 13:02:17.692811 10451 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.730607 10467 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:17.695830 10457 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:17.733003 10462 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:17.736106 10463 consensus_queue.cc:1059] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:17.649214  8330 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:17.613616 10381 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:17.799729 10317 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:17.550375 10382 consensus_queue.cc:237] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:17.759316  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 152721us
I20260709 13:02:17.737574  9002 raft_consensus.cc:1240] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:17.789274  8253 rpcz_store.cc:276] 0709 13:02:16.474959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.477885 (+  2926us) service_pool.cc:224] Handling call
0709 13:02:17.699204 (+1221319us) spinlock_profiling.cc:243] Waited 619 ms on lock 0x7b54003fd754. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:17.699358 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1424506624}
I20260709 13:02:17.766173 10118 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:02:17.796360  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.797353  9009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 438) took 1753 ms. Trace:
I20260709 13:02:17.798646  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 1857) took 2474 ms. Trace:
I20260709 13:02:17.850857  8369 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:17.706022 10449 log.cc:927] Time spent T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
W20260709 13:02:17.802965  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 100809us
I20260709 13:02:17.804762  8325 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:17.808847  9007 raft_consensus.cc:1240] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:17.807690  9008 rpcz_store.cc:276] 0709 13:02:16.215979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.490844 (+274865us) service_pool.cc:224] Handling call
0709 13:02:16.491117 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:16.934375 (+443258us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:16.975412 (+ 41037us) log.cc:844] Serialized 14 byte log entry
0709 13:02:17.094822 (+119410us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:17.094875 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:17.094931 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.704066 (+609135us) spinlock_profiling.cc:243] Waited 443 ms on lock 0x7b54004c2554. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:17.704129 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.704293 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":41352,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":11189,"raft.queue_time_us":14838,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":10048,"spinlock_wait_cycles":1018360704,"thread_start_us":119009,"threads_started":1,"wal-append.queue_time_us":169723}
I20260709 13:02:17.791319 10123 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:02:17.860440 10456 log.cc:927] Time spent T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 13:02:17.860317 10123 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:17.863714 10123 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:17.864246 10213 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:17.864981 10213 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:17.866194  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:17.866850  9001 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:17.867580 10460 consensus_queue.cc:1059] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:17.817809  8251 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:17.805950 10408 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:17.821092  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 54088.7us
I20260709 13:02:17.821705  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:17.828154  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.850260 10462 raft_consensus.cc:697] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:17.768400 10454 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.789263  9004 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:17.837457 10359 consensus_queue.cc:1059] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:02:17.845992  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:17.850672  9009 rpcz_store.cc:276] 0709 13:02:16.043823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.262884 (+219061us) service_pool.cc:224] Handling call
0709 13:02:16.263164 (+   280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:16.679569 (+416405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.797044 (+1117475us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b54005a8cd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:17.797283 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":348947200}
I20260709 13:02:17.954604  9006 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:17.859792 10453 log.cc:927] Time spent T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:02:17.796784  9903 consensus_queue.cc:1048] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:02:17.851244  8324 rpcz_store.cc:276] 0709 13:02:15.324492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.777691 (+453199us) service_pool.cc:224] Handling call
0709 13:02:16.133842 (+356151us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:16.495019 (+361177us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:02:16.495051 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:16.496071 (+  1020us) log.cc:844] Serialized 26 byte log entry
0709 13:02:16.706017 (+209946us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:16.706073 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.706183 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:16.945347 (+239164us) raft_consensus.cc:1685] finished
0709 13:02:16.945366 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.798350 (+852984us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b54003c9450. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:17.798579 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13516,"raft.queue_time_us":82199,"raft.run_cpu_time_us":69,"raft.run_wall_time_us":73,"spinlock_wait_cycles":817671040,"thread_start_us":459608,"threads_started":2,"wal-append.queue_time_us":423854}
I20260709 13:02:17.833067 10458 raft_consensus.cc:697] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:02:17.965494  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.965377  9006 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:17.918181  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:17.819975  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:17.967108  8252 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:17.923537  9002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 522) took 1693 ms. Trace:
I20260709 13:02:17.923796  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:17.968916  8404 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:02:17.923839  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:17.969534  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.831395 10407 consensus_queue.cc:1059] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.084s
I20260709 13:02:17.801620 10338 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:17.953893  9971 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:17.981505  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 112398us
I20260709 13:02:17.930549 10464 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:17.926095  8295 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:02:17.927138  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 168594us
I20260709 13:02:17.984392  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1837) took 2658 ms. Trace:
I20260709 13:02:17.984987  8220 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:17.809541  8222 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:17.857093 10118 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:02:17.985865  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 176328us
W20260709 13:02:17.932590  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 136473us
I20260709 13:02:17.946084  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 284) took 1469 ms. Trace:
I20260709 13:02:17.984920  8397 rpcz_store.cc:276] 0709 13:02:15.325806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.178319 (+1852513us) service_pool.cc:224] Handling call
0709 13:02:17.178708 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:17.179114 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.179155 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:17.179686 (+   531us) log.cc:844] Serialized 26 byte log entry
0709 13:02:17.430351 (+250665us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:17.430401 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.430518 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:17.984167 (+553649us) raft_consensus.cc:1685] finished
0709 13:02:17.984186 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.984330 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1147,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":18591,"thread_start_us":192456,"threads_started":1,"wal-append.queue_time_us":262418}
I20260709 13:02:17.986599  8254 rpcz_store.cc:276] 0709 13:02:16.475878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.642397 (+166519us) service_pool.cc:224] Handling call
0709 13:02:16.642655 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.062636 (+419981us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.945221 (+882585us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b5400178254. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:17.945378 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":893954432}
W20260709 13:02:17.930071  8256 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1509) took 2539 ms (client timeout 2005 ms). Trace:
W20260709 13:02:17.987914  8256 rpcz_store.cc:269] 0709 13:02:15.390538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.741160 (+350622us) service_pool.cc:224] Handling call
0709 13:02:17.929880 (+2188720us) spinlock_profiling.cc:243] Waited 957 ms on lock 0x7b540033e354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:17.929986 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2201518976}
I20260709 13:02:17.942222  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1545) took 1489 ms. Trace:
I20260709 13:02:17.947245  8402 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:17.955118  8250 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
I20260709 13:02:17.957753 10468 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:02:17.934803 10440 raft_consensus.cc:993] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728: : Instructing follower 8c42259d7c4b463a9f485f42c320c81b to start an election
W20260709 13:02:17.967643  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 201645us
I20260709 13:02:18.003449  8220 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:17.924703 10431 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:17.969151  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 118315us
I20260709 13:02:17.970978  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:18.005270  8406 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
I20260709 13:02:17.930037 10365 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.836002 10451 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:18.027678 10451 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:18.035166  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:18.036219  9007 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 4: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:02:17.914371  8972 leader_election.cc:336] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.545s (SENT)
I20260709 13:02:17.978313  9005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 515) took 2342 ms. Trace:
I20260709 13:02:17.954489 10454 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:18.053099  8252 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:02:17.987391  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:17.924587 10469 consensus_queue.cc:1059] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:17.988771  8223 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:17.986727  8974 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:17.988801  8255 rpcz_store.cc:276] 0709 13:02:16.452539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.459903 (+  7364us) service_pool.cc:224] Handling call
0709 13:02:16.482173 (+ 22270us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:16.954608 (+472435us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.941884 (+987276us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b5400407254. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:17.942124 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":648797952}
W20260709 13:02:17.979461 10420 consensus_meta.cc:287] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 1.033s	user 0.012s	sys 0.000s
I20260709 13:02:17.983708 10320 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.993796  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:17.993968  8403 raft_consensus.cc:1240] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:17.926013  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:18.074445  8400 raft_consensus.cc:2393] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 2: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:02:17.967110  9004 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56940 (request call id 545) took 1666 ms (client timeout 1849 ms). Trace:
I20260709 13:02:18.001139  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:18.001284  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:18.075500  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 88791.6us
I20260709 13:02:17.980955  9000 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:18.008605  8404 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:02:18.005229  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:02:18.012502 10335 consensus_meta.cc:287] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.945s	user 0.012s	sys 0.007s
I20260709 13:02:18.012631 10213 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:17.988344 10118 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:17.993664 10474 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:18.013645 10465 log.cc:927] Time spent T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:02:18.013993 10338 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:18.085233 10329 consensus_meta.cc:287] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.030s	user 0.004s	sys 0.008s
I20260709 13:02:18.014770 10458 consensus_queue.cc:237] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:18.019486 10313 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:18.021865  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:17.996753  9009 raft_consensus.cc:2955] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 4 to 6, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:18.025559 10365 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:18.038599  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:18.044497 10279 consensus_meta.cc:287] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.993s	user 0.010s	sys 0.000s
I20260709 13:02:18.044781 10198 raft_consensus.cc:3055] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:02:18.107591 10374 consensus_meta.cc:287] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 1.608s	user 0.011s	sys 0.002s
I20260709 13:02:18.047986 10260 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:18.048552 10443 raft_consensus.cc:3055] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:18.145767 10443 raft_consensus.cc:740] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:18.146834 10443 consensus_queue.cc:260] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:17.968696  9002 rpcz_store.cc:276] 0709 13:02:16.230093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.502545 (+272452us) service_pool.cc:224] Handling call
0709 13:02:17.713588 (+1211043us) spinlock_profiling.cc:243] Waited 2 ms on lock 0x7b5800175898. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699eee 0000000000375939 00007f882d70d1e0 00007f882d7d60c4 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:02:17.713757 (+   169us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.855262 (+141505us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.923474 (+ 68212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4596480}
I20260709 13:02:17.999775 10122 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:18.148595 10122 raft_consensus.cc:697] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:18.149547 10122 consensus_queue.cc:237] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:18.050377 10440 raft_consensus.cc:1081] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Signalling peer 8c42259d7c4b463a9f485f42c320c81b to start an election
W20260709 13:02:18.079649  9004 rpcz_store.cc:269] 0709 13:02:16.300814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.494029 (+193215us) service_pool.cc:224] Handling call
0709 13:02:17.962347 (+1468318us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:18.052160 10408 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:18.054316  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:18.176498  8406 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 4: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:18.055945  9001 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:18.059047  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:18.179109  8397 raft_consensus.cc:2376] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5. Re-sending same reply.
I20260709 13:02:18.060707  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1568) took 2735 ms. Trace:
I20260709 13:02:18.062465  8328 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "038cd0877a674c2996410c9f327df911"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:02:18.052979 10428 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:18.074658  8405 raft_consensus.cc:2393] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 4: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:18.088109  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:18.089453  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:18.028227 10451 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:02:18.091835  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1566) took 2771 ms. Trace:
I20260709 13:02:18.100106  8297 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:18.014045 10462 consensus_queue.cc:237] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:18.068969 10475 raft_consensus.cc:3055] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:18.103794  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:18.083559 10325 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:02:18.090004 10387 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.026s	sys 0.038s
I20260709 13:02:18.084808 10320 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:18.127539 10446 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:18.090495 10477 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:18.098145 10407 consensus_queue.cc:1059] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
W20260709 13:02:18.129810  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:18.130101  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 124888us
I20260709 13:02:18.110649  8974 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:18.059718 10138 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
W20260709 13:02:18.046458 10388 consensus_meta.cc:287] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 1.395s	user 0.011s	sys 0.000s
W20260709 13:02:18.107250 10436 consensus_meta.cc:287] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.581s	user 0.010s	sys 0.000s
W20260709 13:02:18.051755  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 125675us
I20260709 13:02:18.152524  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:18.129638 10463 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.129676 10213 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.098681 10476 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.096s
I20260709 13:02:18.132671 10329 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:18.099527 10335 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:02:18.180285  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:18.685050 10433 consensus_meta.cc:287] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.882s	user 0.001s	sys 0.007s
I20260709 13:02:18.691588 10443 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:18.796241  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 881434us
W20260709 13:02:18.799057  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:18.799813  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.870820  8326 raft_consensus.cc:1240] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4. Ops: []
W20260709 13:02:18.681320 10333 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.002s	sys 0.013s
I20260709 13:02:18.143733 10279 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:19.325242 10333 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.183s	user 0.006s	sys 0.019s
I20260709 13:02:18.370472 10381 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.327356 10381 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.328049 10279 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:19.329675  8403 rpcz_store.cc:276] 0709 13:02:15.320893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.171226 (+1850333us) service_pool.cc:224] Handling call
0709 13:02:17.218937 (+ 47711us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:18.074179 (+855242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.091628 (+ 17449us) spinlock_profiling.cc:243] Waited 774 ms on lock 0x7b54003e6f54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:18.091782 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1781203712}
I20260709 13:02:18.143337 10292 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:18.692834  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:18.040267  9005 rpcz_store.cc:276] 0709 13:02:15.636372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.256000 (+619628us) service_pool.cc:224] Handling call
0709 13:02:16.256275 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.004675 (+748400us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.978030 (+973355us) spinlock_profiling.cc:243] Waited 425 ms on lock 0x7b5400476b54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:17.978244 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":977151744}
I20260709 13:02:18.177513 10204 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.333671  8397 raft_consensus.cc:2410] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:02:19.115640  8220 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.049s	user 0.000s	sys 0.018s
I20260709 13:02:19.332963  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:19.175786  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:18.138813 10198 raft_consensus.cc:740] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:18.377907 10384 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:18.316241  9000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 445) took 1350 ms. Trace:
I20260709 13:02:19.333104  8399 rpcz_store.cc:276] 0709 13:02:15.325062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.173698 (+1848636us) service_pool.cc:224] Handling call
0709 13:02:17.175036 (+  1338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:17.177498 (+  2462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.177540 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:17.214958 (+ 37418us) log.cc:844] Serialized 26 byte log entry
0709 13:02:17.573934 (+358976us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:17.573971 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.574041 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:18.060453 (+486412us) raft_consensus.cc:1685] finished
0709 13:02:18.060481 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.060634 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":265237,"threads_started":1,"wal-append.queue_time_us":439777,"wal-append.run_cpu_time_us":2903,"wal-append.run_wall_time_us":1386051}
W20260709 13:02:18.316876  9003 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56940 (request call id 541) took 2124 ms (client timeout 1578 ms). Trace:
W20260709 13:02:19.347818  8220 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.282s	user 0.002s	sys 0.036s
I20260709 13:02:18.800470  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:19.144338  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:18.361171  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.223390  9009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 527) took 1227 ms. Trace:
I20260709 13:02:18.378085 10452 raft_consensus.cc:3055] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:18.403085 10219 raft_consensus.cc:3055] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:19.185365  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:19.135576  8222 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:02:18.289974 10438 consensus_meta.cc:287] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.604s	user 0.010s	sys 0.000s
W20260709 13:02:18.913226 10118 consensus_meta.cc:287] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.805s	user 0.007s	sys 0.000s
I20260709 13:02:18.930042 10329 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:19.348693  8295 leader_election.cc:400] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 5. Current term is 6.
I20260709 13:02:19.366729  8295 leader_election.cc:403] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:19.355613 10204 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:18.779501 10477 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:02:18.799569 10387 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.442s	user 0.035s	sys 0.046s
W20260709 13:02:18.927289 10454 consensus_meta.cc:287] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.846s	user 0.007s	sys 0.003s
I20260709 13:02:19.349232  8401 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
W20260709 13:02:18.316152 10485 log.cc:927] Time spent T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 13:02:19.222391 10072 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.403573 10299 raft_consensus.cc:3055] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:19.375813 10083 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.838768 10325 raft_consensus.cc:697] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 5 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:18.409358 10260 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:19.352823 10335 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:19.354096  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:18.183081  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:18.139350 10469 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:19.329234 10320 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:19.224042  9971 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:19.365917 10213 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:18.779130 10475 raft_consensus.cc:740] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:19.368458  8405 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:18.309190 10439 raft_consensus.cc:3055] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:18.371891 10365 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:19.372603 10198 consensus_queue.cc:260] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.373914 10387 raft_consensus.cc:697] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:19.393968 10388 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:19.394157 10118 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.366848  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1860) took 1363 ms. Trace:
I20260709 13:02:18.152792 10474 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:19.373169 10463 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:18.801726 10467 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.396623 10475 consensus_queue.cc:260] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.398519 10436 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.399371 10475 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:19.379671 10426 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:19.400094 10426 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.401744 10454 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.395145 10387 consensus_queue.cc:237] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.403272 10219 raft_consensus.cc:740] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:19.379761 10083 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:18.797420  8972 leader_election.cc:336] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.583s (SENT)
W20260709 13:02:18.355587  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 366710us
I20260709 13:02:18.801674 10313 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:19.380911 10299 raft_consensus.cc:740] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:19.386931 10498 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:19.387710  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 55750.8us
I20260709 13:02:19.387480  9971 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.392328  8331 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:19.392457  9008 raft_consensus.cc:2393] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 5: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:19.390785  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 278) took 1472 ms. Trace:
I20260709 13:02:19.380460 10072 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.395144 10388 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.398733 10467 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:18.379130 10442 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:19.402577 10426 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:19.379982 10431 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:02:19.357582  9003 rpcz_store.cc:269] 0709 13:02:16.192780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.204912 (+ 12132us) service_pool.cc:224] Handling call
0709 13:02:16.273877 (+ 68965us) spinlock_profiling.cc:243] Waited 67.5 ms on lock 0x7b5800175898. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699e9c 00007f8825699a5b 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:02:18.316669 (+2042792us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b5400718ed4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:18.316779 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":789742592}
I20260709 13:02:19.416865 10504 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.603729  8324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:57170
I20260709 13:02:19.404775  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:19.401155 10436 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:19.407720 10292 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:19.429276 10476 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:19.429791 10476 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.430171 10476 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:19.430718 10118 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:19.431411 10072 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:19.432130 10072 consensus_queue.cc:1059] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:19.433285  8223 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:19.433676 10408 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:19.429786 10292 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.434258 10408 raft_consensus.cc:697] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 7 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:19.408010  8400 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:19.411512  8403 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 4: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:19.411691  8404 rpcz_store.cc:276] 0709 13:02:17.917186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.080328 (+163142us) service_pool.cc:224] Handling call
0709 13:02:19.385249 (+1304921us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.413394 10477 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:19.414796 10260 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:19.412189  9000 rpcz_store.cc:276] 0709 13:02:16.965996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.975155 (+  9159us) service_pool.cc:224] Handling call
0709 13:02:17.963481 (+988326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:18.083962 (+120481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.315995 (+232033us) spinlock_profiling.cc:243] Waited 921 ms on lock 0x7b54006c07d0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:18.316179 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2119328000}
I20260709 13:02:19.415485 10469 raft_consensus.cc:697] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:19.414844 10474 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:19.416340  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 618481us
I20260709 13:02:18.773993 10433 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.405647 10439 raft_consensus.cc:740] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:19.395572  8397 rpcz_store.cc:276] 0709 13:02:18.003820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.692571 (+688751us) service_pool.cc:224] Handling call
0709 13:02:19.363341 (+670770us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.421093  8406 raft_consensus.cc:2410] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:02:19.413468 10452 raft_consensus.cc:740] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:19.421976 10442 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:19.422832 10431 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.423029 10325 consensus_queue.cc:237] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.424698 10219 consensus_queue.cc:260] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.426017  8295 leader_election.cc:400] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5.
I20260709 13:02:19.427183 10463 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:19.426920 10299 consensus_queue.cc:260] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.427240 10335 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 5. Current term is 6.
I20260709 13:02:19.427688  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1862) took 1404 ms. Trace:
I20260709 13:02:19.392536 10438 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.428056 10329 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.428426  8328 raft_consensus.cc:606] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:02:19.433238 10381 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:19.433543  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.438987  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1857) took 1436 ms. Trace:
I20260709 13:02:19.456167  8295 leader_election.cc:403] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:19.461410  8399 raft_consensus.cc:2410] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:19.458434  8402 rpcz_store.cc:276] 0709 13:02:18.004281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.144644 (+140363us) service_pool.cc:224] Handling call
0709 13:02:19.405939 (+1261295us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.457104  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1858) took 1453 ms. Trace:
I20260709 13:02:19.429344  9971 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:19.433207 10072 raft_consensus.cc:3055] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:19.482264  8328 raft_consensus.cc:614] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:19.464195  8406 rpcz_store.cc:276] 0709 13:02:18.003240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.182811 (+179571us) service_pool.cc:224] Handling call
0709 13:02:19.442239 (+1259428us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.481081  8324 raft_consensus.cc:606] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Received request to transfer leadership
I20260709 13:02:19.455724 10474 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:19.483633  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1864) took 1450 ms. Trace:
I20260709 13:02:19.487859  8399 rpcz_store.cc:276] 0709 13:02:18.032877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.379339 (+1346462us) service_pool.cc:224] Handling call
0709 13:02:19.483553 (+104214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.435761 10408 consensus_queue.cc:237] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.463630  8400 rpcz_store.cc:276] 0709 13:02:17.999571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.353525 (+1353954us) service_pool.cc:224] Handling call
0709 13:02:19.436009 (+ 82484us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.441735 10260 raft_consensus.cc:697] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:02:19.492197  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 1.48805e+06us
I20260709 13:02:19.492964 10469 consensus_queue.cc:237] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.483592 10072 raft_consensus.cc:740] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:19.481670 10454 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:19.501108 10329 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.504287 10510 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.504992 10510 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:19.507114  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 81312.4us
I20260709 13:02:19.506733 10260 consensus_queue.cc:237] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.508224 10198 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:19.507548 10510 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:19.503804 10335 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5.
I20260709 13:02:19.417791 10498 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.482015 10320 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:19.519557 10219 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:19.520112 10451 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:19.535295  8403 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:02:19.521065  8397 raft_consensus.cc:1275] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:19.537403  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1853) took 1540 ms. Trace:
I20260709 13:02:19.537837  8397 rpcz_store.cc:276] 0709 13:02:17.997321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.457399 (+1460078us) service_pool.cc:224] Handling call
0709 13:02:19.520536 (+ 63137us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.536033 (+ 15497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.537310 (+  1277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.538343  8404 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:19.539057  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1854) took 1540 ms. Trace:
I20260709 13:02:19.539407  8397 raft_consensus.cc:1275] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:02:19.540652  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 1.3495e+06us
I20260709 13:02:19.541790 10339 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.542140  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:19.542691 10339 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.543848 10498 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:19.545027  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:19.545542  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:19.545809  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 410491us
W20260709 13:02:19.546222  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.546068  8324 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:19.548930 10313 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:19.549757 10313 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.549916 10320 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:02:19.552178 10313 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:19.552459 10320 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:19.552773 10313 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.512521 10292 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:19.554669 10439 consensus_queue.cc:260] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.555447 10451 raft_consensus.cc:697] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:19.555999 10439 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:19.521754 10329 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:19.556893 10451 consensus_queue.cc:237] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.525661 10438 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:19.505033  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:19.558938  9000 raft_consensus.cc:2376] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6. Re-sending same reply.
I20260709 13:02:19.543344  8323 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:19.533571  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:37920 (request call id 12) took 1440 ms. Trace:
W20260709 13:02:19.561091  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 66961.5us
I20260709 13:02:19.561510  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1574) took 1621 ms. Trace:
I20260709 13:02:19.562124  8403 rpcz_store.cc:276] 0709 13:02:17.916320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.460924 (+1544604us) service_pool.cc:224] Handling call
0709 13:02:19.522045 (+ 61121us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:19.537005 (+ 14960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.537203 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.539419  8404 rpcz_store.cc:276] 0709 13:02:17.998694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.483262 (+1484568us) service_pool.cc:224] Handling call
0709 13:02:19.535209 (+ 51947us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.538769 (+  3560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.538991 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2304}
I20260709 13:02:19.561601  8328 rpcz_store.cc:276] 0709 13:02:18.062024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.062314 (+   290us) service_pool.cc:224] Handling call
0709 13:02:19.499025 (+1436711us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.510219  9971 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.533195 10428 raft_consensus.cc:697] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 5 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:02:19.581959  8401 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 1551) took 5240 ms (client timeout 3131 ms). Trace:
W20260709 13:02:19.582543  8401 rpcz_store.cc:269] 0709 13:02:14.341665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.215113 (+1873448us) service_pool.cc:224] Handling call
0709 13:02:19.581659 (+3366546us) spinlock_profiling.cc:243] Waited 2.76 s on lock 0x7b54003e8354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:19.581828 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6344364288}
I20260709 13:02:19.594566  8405 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
W20260709 13:02:19.599080  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 89656.6us
I20260709 13:02:19.511757 10072 consensus_queue.cc:260] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.607818  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:19.608798  8406 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:19.564066 10313 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.582044  9971 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.500428 10504 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.623853  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:19.614806 10299 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:19.613065 10498 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:02:19.630263  8295 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:19.630890 10459 consensus_queue.cc:1059] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:02:19.582810 10428 consensus_queue.cc:237] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.632297 10407 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:19.632843 10407 raft_consensus.cc:697] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:19.633958 10407 consensus_queue.cc:237] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.652832  8974 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:19.655148 10313 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:19.657927  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:19.660651  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:19.358598  9009 rpcz_store.cc:276] 0709 13:02:17.995928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.996106 (+   178us) service_pool.cc:224] Handling call
0709 13:02:17.996443 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.996636 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:18.192435 (+195799us) log.cc:844] Serialized 14 byte log entry
0709 13:02:18.908073 (+715638us) spinlock_profiling.cc:243] Waited 25.8 ms on lock 0x7b18003bc080. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882569f2a6 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1
0709 13:02:18.914046 (+  5973us) spinlock_profiling.cc:243] Waited 9.16 ms on lock 0x7b18003bc080. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:02:18.992519 (+ 78473us) spinlock_profiling.cc:243] Waited 26.5 ms on lock 0x7b18003bc080. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256ce455 00007f882c1b63c7 00007f88287c89a7 00007f88287d80bf 00007f88287d8038
0709 13:02:19.221587 (+229068us) log.cc:844] Serialized 14 byte log entry
0709 13:02:19.222249 (+   662us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:19.222395 (+   146us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:19.222572 (+   177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.222769 (+   197us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:19.223090 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":188491,"raft.queue_time_us":249531,"raft.run_cpu_time_us":529,"raft.run_wall_time_us":34422,"spinlock_wait_cycles":141353216,"thread_start_us":684311,"threads_started":2,"wal-append.queue_time_us":637212}
I20260709 13:02:19.632300 10498 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:02:19.662794  8222 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:19.664907  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:19.676829  8406 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:19.677585  8403 raft_consensus.cc:2393] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 7: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:02:19.677471  9008 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56954 (request call id 448) took 1588 ms (client timeout 1793 ms). Trace:
W20260709 13:02:19.677482  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:19.708894  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.714362  8397 raft_consensus.cc:2393] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 5: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:19.717672  8398 raft_consensus.cc:2410] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:02:19.718250  8330 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:19.719298 10431 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:19.719504 10443 consensus_meta.cc:287] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.904s	user 0.010s	sys 0.007s
I20260709 13:02:19.723573 10388 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:19.713768  9008 rpcz_store.cc:269] 0709 13:02:18.089024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.089194 (+   170us) service_pool.cc:224] Handling call
0709 13:02:19.677154 (+1587960us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b54004132d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:19.677366 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37974272}
I20260709 13:02:19.715339  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:19.726452  8400 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:19.706888  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:19.730062  8328 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:19.733469 10384 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:19.733605  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 66781.7us
I20260709 13:02:19.734149 10384 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.734472  8972 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:19.734629  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:19.716392 10504 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:19.736936  8400 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
W20260709 13:02:19.738700  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.745923 10431 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:19.739280 10476 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.747745 10505 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.748143 10476 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:19.748312 10505 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:19.750177  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 133022us
I20260709 13:02:19.748773 10467 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:19.747677  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 85072.1us
I20260709 13:02:19.710332 10339 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:19.725449 10072 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:19.753670  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:19.751349 10433 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:19.754153 10469 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:19.768834  8397 raft_consensus.cc:1240] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:19.769388 10469 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.770457  8397 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:02:19.771538 10469 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:19.754906  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:19.772344 10384 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:19.760123 10523 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:19.761973 10198 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:02:19.773487  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 1.51864e+06us
I20260709 13:02:19.775460  8220 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:19.775929 10505 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:19.777650 10198 consensus_queue.cc:1059] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:19.763622  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 1872) took 2278 ms. Trace:
W20260709 13:02:19.763840  8972 connection.cc:256] RPC call timeout handler was delayed by -1.843304250000017s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1544s for a response
I20260709 13:02:19.781749 10313 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:19.782017  8326 rpcz_store.cc:276] 0709 13:02:17.484936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.493762 (+  8826us) service_pool.cc:224] Handling call
0709 13:02:17.534054 (+ 40292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.324510 (+1790456us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.763201 (+438691us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400064d54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:19.763434 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2971389824}
I20260709 13:02:19.783124 10431 raft_consensus.cc:697] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:19.784263 10219 consensus_queue.cc:1048] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:19.784381 10431 consensus_queue.cc:237] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.786314 10198 raft_consensus.cc:3055] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:19.786749 10198 raft_consensus.cc:740] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:19.787654 10198 consensus_queue.cc:260] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.788848 10198 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:19.791628 10442 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.765035  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:19.793799  8403 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 5: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:02:19.769781  8369 connection.cc:256] RPC call timeout handler was delayed by -2.5029939350000063s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18030000000000002s for a response
I20260709 13:02:19.772307  8327 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
W20260709 13:02:19.755920  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 136573us
W20260709 13:02:19.777627  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.777180  8220 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:19.761821  8222 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:19.792057 10299 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:19.766408  8295 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:19.792662 10511 consensus_queue.cc:1059] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:02:19.794188  8369 connection.cc:256] RPC call timeout handler was delayed by -2.4966939349999961s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18100000000000002s for a response
W20260709 13:02:19.798893  8369 connection.cc:256] RPC call timeout handler was delayed by -2.4840939350000326s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1824s for a response
W20260709 13:02:19.799263  8369 connection.cc:256] RPC call timeout handler was delayed by -2.4219939349999891s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18930000000000002s for a response
W20260709 13:02:19.799567  8369 connection.cc:256] RPC call timeout handler was delayed by -2.4075939350000226s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19090000000000001s for a response
W20260709 13:02:19.799885  8369 connection.cc:256] RPC call timeout handler was delayed by -2.343693935000033s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.198s for a response
I20260709 13:02:19.799729  8974 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:19.800997 10433 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:19.802249  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 78243.2us
I20260709 13:02:19.804581  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:19.804922  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:19.806370 10439 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.806989 10439 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.807411  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:19.807834 10433 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:19.808538  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:19.808439 10433 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.810542 10433 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:19.810702 10474 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.811218 10514 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.811873 10514 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:19.812575  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.812808 10474 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:19.814155 10514 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:19.813992  8297 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:19.815303 10463 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:19.815681  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 53963.7us
I20260709 13:02:19.815964 10463 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.816413 10463 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:19.817407  8222 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:19.818267 10451 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:19.815474 10313 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:02:19.819998 10451 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:19.820775 10451 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:19.821079 10451 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.821836 10428 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:19.804643  8295 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:19.814970 10514 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.797456  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:19.809137  8403 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:19.829916 10511 raft_consensus.cc:3055] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:19.830416 10511 raft_consensus.cc:740] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:19.829743 10514 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.831511 10511 consensus_queue.cc:260] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.833540 10219 consensus_queue.cc:1048] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:19.835263 10446 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:19.835595 10442 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:19.809190 10439 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:19.821375 10451 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:19.841879  8328 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:19.843159  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:19.844676  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:19.845793  8972 connection.cc:256] RPC call timeout handler was delayed by -1.7725727839999763s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1767s for a response
W20260709 13:02:19.846415  8972 connection.cc:256] RPC call timeout handler was delayed by -1.71261417300002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17880000000000001s for a response
W20260709 13:02:19.846832  8972 connection.cc:256] RPC call timeout handler was delayed by -1.5866141729999867s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1928s for a response
W20260709 13:02:19.847572  8972 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 3.204s (SENT) [suppressed 37 similar messages]
I20260709 13:02:19.848843 10292 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:19.849202 10476 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:19.849818 10476 raft_consensus.cc:697] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 5 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:19.849552 10292 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:19.850337  8369 connection.cc:256] RPC call timeout handler was delayed by -1.9818939350000164s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23820000000000002s for a response
I20260709 13:02:19.851285 10384 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:19.850795  8369 connection.cc:256] RPC call timeout handler was delayed by -1.7673015439999631s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1542s for a response
W20260709 13:02:19.852026  8369 connection.cc:256] RPC call timeout handler was delayed by -1.645801543999994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16770000000000002s for a response
I20260709 13:02:19.851861 10384 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:19.852512  8369 connection.cc:256] RPC call timeout handler was delayed by -1.5423015439999972s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17920000000000003s for a response
W20260709 13:02:19.852926  8369 connection.cc:256] RPC call timeout handler was delayed by -1.3680939349999903s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3064s for a response
W20260709 13:02:19.853298  8369 connection.cc:256] RPC call timeout handler was delayed by -1.2615015439999979s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21040000000000003s for a response
W20260709 13:02:19.853783  8369 connection.cc:256] RPC call timeout handler was delayed by -1.1178939350000405s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.33420000000000005s for a response
I20260709 13:02:19.854115 10463 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.854815  8369 leader_election.cc:400] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5.
I20260709 13:02:19.855279  8369 leader_election.cc:403] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:19.856112  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:19.857227 10511 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:19.857652 10292 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:19.850793 10476 consensus_queue.cc:237] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.856315 10463 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:19.862390  8397 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:19.858385  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:19.868680  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 69094us
I20260709 13:02:19.868880  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:19.869858 10292 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:19.869989 10477 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.870364  8401 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 1. Current term is 2. Ops: []
W20260709 13:02:19.866390  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 51062.7us
I20260709 13:02:19.871924  8406 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:19.882010 10384 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:19.885504  9971 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:19.885871  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 71974.5us
I20260709 13:02:19.886857  8297 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:19.889017  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 84493.3us
I20260709 13:02:19.889942  8295 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:19.867748 10439 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5.
I20260709 13:02:19.891319 10428 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:02:19.891405  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.891963  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 1882) took 2324 ms. Trace:
I20260709 13:02:19.892206 10428 consensus_queue.cc:1059] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:19.892465  8332 rpcz_store.cc:276] 0709 13:02:17.567575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.707761 (+140186us) service_pool.cc:224] Handling call
0709 13:02:19.891626 (+2183865us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b5400090bd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:19.891789 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3577194368}
I20260709 13:02:19.875982 10433 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:19.895080  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 1867) took 2464 ms. Trace:
I20260709 13:02:19.895390 10279 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:19.895556  8331 rpcz_store.cc:276] 0709 13:02:17.430522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.438788 (+  8266us) service_pool.cc:224] Handling call
0709 13:02:17.439071 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.416308 (+1977237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.894826 (+478518us) spinlock_profiling.cc:243] Waited 1.91 s on lock 0x7b54003c9e54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:19.894995 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4400266496}
I20260709 13:02:19.896345 10279 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.897034 10260 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:19.905133 10433 consensus_queue.cc:1048] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:19.891319 10083 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:19.904978 10260 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.906250 10279 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:19.906849 10382 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.907424 10382 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.907842  8401 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:19.909022  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:19.909644 10475 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:19.909709 10138 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.910218  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:19.910305 10475 raft_consensus.cc:697] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:19.910416 10138 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.910933  8401 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:19.911847  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:19.911582 10475 consensus_queue.cc:237] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.912755 10138 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:19.912948  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:19.916188  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 50203.1us
I20260709 13:02:19.875036 10514 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:19.921350  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 66806.9us
I20260709 13:02:19.913590 10260 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:19.921653  8401 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:19.920300 10463 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:02:19.924150 10382 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:19.930423 10477 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:19.934543  8257 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:19.934582  8222 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:02:19.940442  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 92865.1us
I20260709 13:02:19.944908  8222 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:19.943008  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:19.952419  8972 connection.cc:256] RPC call timeout handler was delayed by -1.3906042499999671s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20470000000000002s for a response
I20260709 13:02:19.935966 10514 consensus_queue.cc:1059] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:02:19.938666 10477 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
W20260709 13:02:19.954828  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.962218  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:19.963212  8255 raft_consensus.cc:2393] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 5: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:19.951902 10198 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:19.973757  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1863) took 1941 ms. Trace:
I20260709 13:02:19.974208  8402 rpcz_store.cc:276] 0709 13:02:18.031963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.484684 (+1452721us) service_pool.cc:224] Handling call
0709 13:02:19.536453 (+ 51769us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:19.601451 (+ 64998us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:19.601497 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:19.624390 (+ 22893us) log.cc:844] Serialized 26 byte log entry
0709 13:02:19.726573 (+102183us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:19.726605 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.726674 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:19.973538 (+246864us) raft_consensus.cc:1685] finished
0709 13:02:19.973557 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:19.973695 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20864,"thread_start_us":48321,"threads_started":1,"wal-append.queue_time_us":180066}
I20260709 13:02:19.975797  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:19.976472  8402 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:19.955996 10138 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.952334 10382 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:19.978296 10382 raft_consensus.cc:697] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:19.978168 10138 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.979485  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:19.979398 10382 consensus_queue.cc:237] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.980175  9003 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
W20260709 13:02:19.980576  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.980983 10138 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:19.981488  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:19.982457  9005 raft_consensus.cc:2410] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:02:19.984790  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:19.985018  8250 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:19.985517  8252 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:19.953888 10083 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.986899 10083 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:19.958341 10431 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:19.988180  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:19.989125 10083 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:19.991904  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:19.988498 10431 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:19.995105  8297 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:19.991995  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:19.991786 10443 consensus_queue.cc:1059] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:19.992604  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:19.992694  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:19.998531  8255 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 7: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:19.996438  8257 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 4: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:02:19.996901  8972 leader_election.cc:336] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.767s (SENT)
I20260709 13:02:19.999228  8974 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:19.994546  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:02:20.000083  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:20.003368  8398 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 275) took 3955 ms (client timeout 3305 ms). Trace:
W20260709 13:02:20.003795  8398 rpcz_store.cc:269] 0709 13:02:16.047799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.476270 (+428471us) service_pool.cc:224] Handling call
0709 13:02:20.003097 (+3526827us) spinlock_profiling.cc:243] Waited 2.94 s on lock 0x7b54001cc354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:20.003302 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6764555136}
I20260709 13:02:20.004911  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:20.008565  8398 raft_consensus.cc:2393] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 4: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:19.966256 10198 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.012081  8397 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 13:02:20.012629  8397 pending_rounds.cc:85] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 4
I20260709 13:02:20.015581  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:20.016433  8400 raft_consensus.cc:2410] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:20.012899  8397 pending_rounds.cc:107] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:02:20.020491  8397 raft_consensus.cc:2889] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:20.008276 10292 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:20.026988  8397 pending_rounds.cc:107] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:02:20.026633  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:20.027292  8397 raft_consensus.cc:2889] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:20.027645  8398 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:20.014657 10083 raft_consensus.cc:2764] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:02:20.014648 10198 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:20.014505 10475 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:20.031880  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:20.034890 10219 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:20.036809  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.007073 10476 consensus_queue.cc:1059] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:02:20.040290  8402 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
W20260709 13:02:20.041723  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.026916 10292 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.030907  9004 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:20.046298  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:20.045360  8974 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:20.030129 10523 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:20.050925  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:20.051918  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 55069.3us
I20260709 13:02:20.040776 10477 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:20.055450  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 64923.8us
I20260709 13:02:20.056381  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:20.054304  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:20.056418 10477 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:20.059664  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:20.067178  8331 raft_consensus.cc:2393] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 4: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:20.067284  8257 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:20.060761 10431 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:20.061224  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:20.061733  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:20.062439  8972 leader_election.cc:336] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.788s (SENT)
I20260709 13:02:20.063011  9009 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:20.048523 10292 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:20.047057 10475 raft_consensus.cc:697] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:20.054612 10476 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:20.070770  8252 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:20.090355  8257 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:02:20.090484 10476 raft_consensus.cc:697] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 6 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:20.091564 10476 consensus_queue.cc:237] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:20.094287 10475 consensus_queue.cc:237] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:20.058017 10219 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:20.074026  8328 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:20.078449  8331 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:02:20.078727  9971 raft_consensus.cc:3055] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:20.103574  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:20.079557 10451 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:20.079784  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:20.082271  8332 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:20.082655 10204 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:20.083415  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:20.086094  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:20.111233  8330 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:02:20.111775  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:20.112020  8252 raft_consensus.cc:1240] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:20.087172  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:20.113741  8324 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:20.087816  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:20.088459  8329 raft_consensus.cc:1275] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:20.088564  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:20.089669  8369 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:20.072055 10477 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:20.072504  8972 leader_election.cc:336] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.851s (SENT)
W20260709 13:02:20.098881  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 43881.7us
I20260709 13:02:20.105032  8256 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
W20260709 13:02:20.105901  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.107132  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:20.107299  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:20.107399  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:20.108434  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:20.109512 10381 consensus_queue.cc:1059] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:02:20.109854  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:20.134110  8331 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:20.104241  9971 raft_consensus.cc:740] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:20.113101  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:20.128275  8220 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:20.144364  8257 raft_consensus.cc:1240] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:20.145551  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:20.146342  8329 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
W20260709 13:02:20.148856  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 76348.5us
W20260709 13:02:20.149711  8972 leader_election.cc:336] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.928s (SENT)
I20260709 13:02:20.150040  8972 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:20.150660 10451 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:20.151237  8972 leader_election.cc:336] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.037s (SENT)
I20260709 13:02:20.151696 10452 consensus_queue.cc:260] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:20.153580  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:20.154170  8328 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:20.155519  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:20.155820 10498 raft_consensus.cc:3055] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:20.156353  8402 raft_consensus.cc:2410] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:02:20.156750  8327 raft_consensus.cc:2393] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 3: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:20.156275 10498 raft_consensus.cc:740] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:02:20.158605  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.159755  8330 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:20.161185  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:20.147519  8326 raft_consensus.cc:2376] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6. Re-sending same reply.
I20260709 13:02:20.142325  8325 raft_consensus.cc:2376] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5. Re-sending same reply.
W20260709 13:02:20.164202  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.170054  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:20.144052 10204 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:20.142714  9005 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:20.169503 10452 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:20.178951  8398 raft_consensus.cc:2410] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:02:20.179370  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:20.185949  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 57746.4us
I20260709 13:02:20.187804  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:20.188887  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:20.189636  8329 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:20.185971  8332 raft_consensus.cc:2376] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4. Re-sending same reply.
W20260709 13:02:20.194377  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 138334us
I20260709 13:02:20.194653  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:20.194978  9004 raft_consensus.cc:2410] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 5 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:02:20.195281  8332 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:20.195968  8369 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:20.196188  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:20.196254  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:20.197228  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:20.197464  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:20.197679 10325 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:20.197827  8252 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:20.198046  8326 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:20.198989  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:20.199559  8326 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:02:20.200371  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:20.209851  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:20.210409  8398 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:20.213268  9008 raft_consensus.cc:1240] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:20.176822 10204 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:20.215768  8298 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:20.201704 10498 consensus_queue.cc:260] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:20.235508 10219 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:20.209776 10459 consensus_queue.cc:1059] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:02:20.237231 10459 raft_consensus.cc:3055] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:20.237099  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:20.237587 10204 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:20.237682 10459 raft_consensus.cc:740] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:20.238723  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:20.239338  9002 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:20.245762  8404 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:20.238901 10459 consensus_queue.cc:260] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:20.252457  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:20.216490  8325 raft_consensus.cc:2376] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4. Re-sending same reply.
W20260709 13:02:20.261886  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:20.264837  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.271646  8329 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:20.261494  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:20.287415  8324 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:20.201856 10431 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:20.179955 10451 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:20.308524  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 52335.8us
I20260709 13:02:20.311389  8331 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:20.274345 10433 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:20.344360  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:20.345110  8252 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:20.335602 10279 consensus_queue.cc:1059] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:02:20.354802  8326 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:02:20.375473  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:20.381055  9008 raft_consensus.cc:2393] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 2: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:20.398356 10279 raft_consensus.cc:3055] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:20.398854 10279 raft_consensus.cc:740] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:20.399698 10279 consensus_queue.cc:260] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:20.402412  9002 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:20.405642  9003 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:20.429689  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.321386 10198 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:20.485982  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:20.523599  8403 raft_consensus.cc:1240] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:20.531656 10428 raft_consensus.cc:3055] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:20.532313 10428 raft_consensus.cc:740] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:20.534181 10477 raft_consensus.cc:3055] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:20.534789 10477 raft_consensus.cc:740] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:20.536010 10477 consensus_queue.cc:260] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:20.382542 10292 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:20.614434 10213 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:20.616914  9971 consensus_queue.cc:260] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:20.619884  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 422373us
I20260709 13:02:20.602170 10477 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:20.622309  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:20.622944 10198 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:20.625519 10459 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:20.625916  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 313895us
I20260709 13:02:20.626669  8220 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:20.627928  8220 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:20.628664 10451 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:20.629065  8220 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:20.629254 10451 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:20.629492  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 478054us
I20260709 13:02:20.629650  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:20.630139 10498 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:20.635897  8400 raft_consensus.cc:1275] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:20.620301  9971 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:20.646176  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.648355  8331 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:02:20.645255  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:20.667096  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.675024  8257 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:20.655722  9005 raft_consensus.cc:1240] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 7. Ops: []
I20260709 13:02:20.571763 10428 consensus_queue.cc:260] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:20.644789 10279 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:20.682912  8369 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:20.686717  8972 leader_election.cc:336] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.047s (SENT)
I20260709 13:02:20.668898 10198 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:20.691224  8972 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:20.692649  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:20.692885  8401 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.010s	sys 0.025s
I20260709 13:02:20.693323  8400 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
W20260709 13:02:20.693326  8401 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.010s	sys 0.025s
W20260709 13:02:20.692649 10476 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.014s	sys 0.039s
I20260709 13:02:20.687911  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:20.696455  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.665573 10523 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:02:20.699121  8220 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:20.701411  8220 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:20.688371  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:02:20.705870  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 84384.2us
W20260709 13:02:20.706605  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:20.707197  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:20.707619  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.707039  8220 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:20.708932  8220 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:20.699023 10476 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.014s	sys 0.039s
I20260709 13:02:20.697078 10439 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:20.718879  8256 raft_consensus.cc:1275] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:20.719563  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:20.688830 10381 consensus_queue.cc:1048] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:02:20.703471  8257 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:55262
I20260709 13:02:20.725279  8257 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:20.725718  8257 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:20.726293  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 49193.9us
I20260709 13:02:20.703944 10219 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:20.711136 10382 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:20.727916  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:20.711380 10198 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:20.730319  9009 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:20.717496 10428 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:20.734931  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 78835.4us
I20260709 13:02:20.704938 10204 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:20.711705  9003 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:20.722594 10387 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:20.716766 10213 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:20.728312  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:20.729269  9001 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:20.724624 10299 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:20.744241  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:20.734043 10382 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.751711 10382 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:20.727922 10219 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.737836 10458 raft_consensus.cc:3055] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:20.756556  8400 raft_consensus.cc:1275] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:02:20.757638  8400 pending_rounds.cc:85] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 3
I20260709 13:02:20.757927  8400 pending_rounds.cc:107] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:02:20.758214  8400 raft_consensus.cc:2889] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:20.749892  8402 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:20.750306  8369 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:20.733377 10198 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:20.759533 10198 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:20.760284  8402 raft_consensus.cc:1275] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:20.761598  8402 raft_consensus.cc:1275] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:20.750917  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:20.753791  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:20.768428  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:20.749922  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:20.829917  8257 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:20.763468 10458 raft_consensus.cc:740] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:20.767469 10511 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:20.849709  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 107123us
I20260709 13:02:20.763028 10219 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:20.847865  8257 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:20.865341  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:20.829771 10387 raft_consensus.cc:697] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:20.873842  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:20.886942  8255 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:20.852346  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:20.877832  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:20.857890 10511 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:20.881961 10204 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:20.901051  8252 raft_consensus.cc:2393] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 3: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:20.903869  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:20.901242  8323 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 2.
I20260709 13:02:20.937443  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:20.937898  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.901130 10511 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:20.914762 10533 log.cc:927] Time spent T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 13:02:20.959141  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 220585us
I20260709 13:02:20.960937  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:20.961787  8257 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:20.963408  8255 raft_consensus.cc:2410] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:02:20.964367  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 255390us
I20260709 13:02:20.965175  8220 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:20.969892 10477 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:20.965790  8972 leader_election.cc:336] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.094s (SENT)
I20260709 13:02:20.976913  8972 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:20.979647  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 100387us
I20260709 13:02:20.984364  8220 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:20.985575  8255 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:20.987141 10219 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:20.989621 10219 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:20.990325 10339 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:20.991835  8256 raft_consensus.cc:2410] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:02:20.980157 10387 consensus_queue.cc:237] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:21.000152  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:21.000974  8331 raft_consensus.cc:2393] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 4: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:21.001154 10451 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:21.001874  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:21.020347  8329 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
W20260709 13:02:21.005864  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 789797us
I20260709 13:02:21.021801  8298 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:21.012873  8250 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:21.013517  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:20.993206 10458 consensus_queue.cc:260] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:21.023710  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:20.984642  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:21.044342  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:21.016927 10198 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:21.047796 10475 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:21.048547  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:21.063259  8329 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
W20260709 13:02:21.063400  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 85862.3us
I20260709 13:02:21.053977  9006 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:21.049829  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:21.050194  8331 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:21.067184  8326 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:21.052644 10477 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:21.054543  8257 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 5: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:21.045300  8298 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:21.071038  8298 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:21.072046  8252 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:21.072685  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:21.073527  8253 raft_consensus.cc:2393] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 5: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:21.073588  8251 raft_consensus.cc:1240] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:21.073913  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:21.068465 10474 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:21.074354 10452 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:21.074501  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:21.074678 10474 raft_consensus.cc:697] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:21.075124 10452 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:21.075507 10452 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:21.075894 10452 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:21.075667 10474 consensus_queue.cc:237] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:21.074505  8248 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:21.077961  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:21.069670  8369 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:21.065261 10475 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:21.079527 10475 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:21.079936  8369 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:21.071951  9007 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:21.081669 10562 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:21.082387 10562 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:21.083217  9002 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:21.082814  8298 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:21.068512  9971 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.091341  9971 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:21.092743  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.093801 10408 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:21.094276 10408 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:21.094656 10408 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:21.096704  9005 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 5 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:21.097112 10292 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:21.099109  9971 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:21.079581  8255 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:02:21.100569  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 385395us
I20260709 13:02:21.084744  8369 leader_election.cc:400] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4.
I20260709 13:02:21.085022  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:21.101606  8220 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:21.087180 10439 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:21.090165  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:21.102449  8257 raft_consensus.cc:2368] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 4. Current term is 5.
I20260709 13:02:21.078024  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:21.057936  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:21.104067  8324 raft_consensus.cc:2368] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 5. Current term is 6.
W20260709 13:02:21.105885  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.107103  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.107429  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.108098  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 147052us
I20260709 13:02:21.105708  8254 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:21.106173  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:21.111171  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:21.113511  8297 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:21.113674  8331 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:57170
W20260709 13:02:21.115214  8972 leader_election.cc:336] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.153s (SENT)
I20260709 13:02:21.114424  8331 raft_consensus.cc:606] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Received request to transfer leadership
I20260709 13:02:21.115536  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:21.115820  8331 raft_consensus.cc:614] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:21.115803  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:21.117251  8324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:02:21.116344  9007 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:21.124650  8324 raft_consensus.cc:606] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:21.126636  8222 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:21.113035 10477 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:21.128631  8324 raft_consensus.cc:614] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:21.132872  8972 leader_election.cc:336] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.186s (SENT)
W20260709 13:02:21.134205  8972 leader_election.cc:336] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.231s (SENT)
I20260709 13:02:21.134665  8972 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:21.137357 10292 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:21.138756 10523 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:21.139559 10523 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:21.140311 10198 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:21.141008  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:21.141585 10198 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:21.142590 10545 log.cc:927] Time spent T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.407s	user 0.000s	sys 0.001s
I20260709 13:02:21.143231 10498 consensus_queue.cc:1059] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:21.144021 10292 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:21.145596 10523 raft_consensus.cc:3055] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:21.145920 10523 raft_consensus.cc:740] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:21.145973  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1884) took 1187 ms. Trace:
W20260709 13:02:21.137524  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.146412  8406 rpcz_store.cc:276] 0709 13:02:19.958824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.154330 (+195506us) service_pool.cc:224] Handling call
0709 13:02:20.154654 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:20.155015 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:20.155042 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:20.155498 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 13:02:20.290074 (+134576us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:20.290129 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.290221 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:21.142045 (+851824us) spinlock_profiling.cc:243] Waited 1.7 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:21.145674 (+  3629us) raft_consensus.cc:1685] finished
0709 13:02:21.145697 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:21.145911 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3270,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"spinlock_wait_cycles":3920128,"thread_start_us":134130,"threads_started":1,"wal-append.queue_time_us":547995}
W20260709 13:02:21.149634  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 65010us
I20260709 13:02:21.110437  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:21.155884  8972 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:21.164021  8972 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:21.126243  9971 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:21.123100 10433 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:02:21.168311 10498 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:02:21.201601  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.171346  9001 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:21.171469  8369 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:21.114800 10451 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:02:21.156481 10571 consensus_queue.cc:1059] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:02:21.173501 10477 raft_consensus.cc:697] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:21.182049 10213 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:21.182557 10339 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:21.183185 10382 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:21.190616  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:21.192430 10511 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:21.193099  8203 auto_rebalancer.cc:893] Removing replace marker failed: Replica ff7a6fe533e94c1bae555b4cae427728 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:02:21.226325  8203 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:02:21.212966 10553 log.cc:927] Time spent T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.451s	user 0.008s	sys 0.000s
I20260709 13:02:21.196579  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:21.197275  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:21.198673  8326 raft_consensus.cc:1240] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260709 13:02:21.169415 10523 consensus_queue.cc:260] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:21.210412  8324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:37920
W20260709 13:02:21.213891  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 92616.9us
I20260709 13:02:21.230433  8324 raft_consensus.cc:606] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:21.230903  8324 raft_consensus.cc:614] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:21.221745  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 95143.7us
I20260709 13:02:21.231112  8332 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 7: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:02:21.223157  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 51792.6us
I20260709 13:02:21.193878  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 1900) took 1101 ms. Trace:
I20260709 13:02:21.231614  8222 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:21.226625  8203 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:02:21.216020 10213 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:21.245407  8257 raft_consensus.cc:1240] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:21.232468 10580 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:21.231709  8330 rpcz_store.cc:276] 0709 13:02:20.092344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.301248 (+208904us) service_pool.cc:224] Handling call
0709 13:02:20.623922 (+322674us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:20.624641 (+   719us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:20.624669 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:20.999011 (+374342us) spinlock_profiling.cc:243] Waited 5.78 ms on lock 0x7b2c000d4468. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:02:20.999403 (+   392us) log.cc:844] Serialized 26 byte log entry
0709 13:02:21.125418 (+126015us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:21.125487 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.126258 (+   771us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:21.193587 (+ 67329us) raft_consensus.cc:1685] finished
0709 13:02:21.193611 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:21.193816 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":145,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":72,"spinlock_wait_cycles":13284224,"thread_start_us":125629,"threads_started":1,"wal-append.queue_time_us":125946}
I20260709 13:02:21.259052  8255 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:02:21.259404  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:21.260051  8330 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:21.255201  8222 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:21.256261 10571 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.263715 10511 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:21.264658 10570 log.cc:927] Time spent T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:02:21.266491  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:21.267135  8324 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:21.225042 10428 raft_consensus.cc:3055] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:21.269789  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:21.273869  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1865) took 2474 ms. Trace:
I20260709 13:02:21.273976  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:21.273921  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 1897) took 1184 ms. Trace:
I20260709 13:02:21.277452  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:21.279021  8222 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:21.279772  8330 raft_consensus.cc:1275] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:21.274398  8399 rpcz_store.cc:276] 0709 13:02:18.794601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.499076 (+704475us) service_pool.cc:224] Handling call
0709 13:02:19.597240 (+ 98164us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:19.615339 (+ 18099us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:19.615377 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:20.415016 (+799639us) spinlock_profiling.cc:243] Waited 53.1 ms on lock 0x7b4c0019a7c0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8826a8aa20 00007f8826a9ba29 00007f8826a61644 00007f8826a88f2d 00007f8826a88b93 00007f88287b918b 00007f88287b9cb5 00007f88287bb004 00007f88287ba729 00007f88287bfd96 00007f88287b4adb
0709 13:02:20.416106 (+  1090us) log.cc:844] Serialized 548 byte log entry
0709 13:02:20.710076 (+293970us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:20.710127 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.710199 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:21.189467 (+479268us) spinlock_profiling.cc:243] Waited 5.14 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:21.268634 (+ 79167us) raft_consensus.cc:1685] finished
0709 13:02:21.268653 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:21.268764 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":134034176,"thread_start_us":293579,"threads_started":1,"wal-append.queue_time_us":320226}
I20260709 13:02:21.260540 10213 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:21.286540 10477 consensus_queue.cc:237] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:21.287801  8323 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 1638) took 1688 ms (client timeout 2158 ms). Trace:
I20260709 13:02:21.272307 10428 raft_consensus.cc:740] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:02:21.288220  8323 rpcz_store.cc:269] 0709 13:02:19.601139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.607689 (+  6550us) service_pool.cc:224] Handling call
0709 13:02:21.287610 (+1679921us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540014c654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:21.287710 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2414983424}
I20260709 13:02:21.289175 10428 consensus_queue.cc:260] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:21.275437  8325 rpcz_store.cc:276] 0709 13:02:20.089197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.288570 (+199373us) service_pool.cc:224] Handling call
0709 13:02:20.623698 (+335128us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:20.624332 (+   634us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:20.624359 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:21.003935 (+379576us) spinlock_profiling.cc:243] Waited 5.53 ms on lock 0x7b2c000d4468. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:02:21.004285 (+   350us) log.cc:844] Serialized 26 byte log entry
0709 13:02:21.127910 (+123625us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:21.127956 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.128018 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:21.268445 (+140427us) raft_consensus.cc:1685] finished
0709 13:02:21.268465 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:21.273852 (+  5387us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47,"raft.queue_time_us":131,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"spinlock_wait_cycles":12720384,"thread_start_us":118501,"threads_started":1,"wal-append.queue_time_us":118039}
I20260709 13:02:21.260221 10523 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:21.289639 10498 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:21.297950 10498 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:21.300050 10498 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:21.289403  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:21.301007  8323 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:21.303972  9003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56940 (request call id 559) took 1072 ms. Trace:
I20260709 13:02:21.304425  9003 rpcz_store.cc:276] 0709 13:02:20.231958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.236807 (+  4849us) service_pool.cc:224] Handling call
0709 13:02:21.303765 (+1066958us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b540057d5d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:21.303905 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":385452416}
I20260709 13:02:21.290217 10452 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:21.307543 10452 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:21.307658 10559 log.cc:927] Time spent T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.341s	user 0.001s	sys 0.000s
I20260709 13:02:21.290592 10382 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:21.290498 10428 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:21.291392 10408 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:21.309569 10546 log.cc:927] Time spent T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.548s	user 0.005s	sys 0.003s
I20260709 13:02:21.310007 10475 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:21.310971  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 292) took 1316 ms. Trace:
I20260709 13:02:21.311184 10408 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:21.312141 10475 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:21.293778 10339 consensus_queue.cc:1059] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:02:21.312964  8327 raft_consensus.cc:2368] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 3. Current term is 4.
I20260709 13:02:21.291494 10575 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:21.274930 10571 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:21.261186 10580 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:21.314113 10498 raft_consensus.cc:3055] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:21.304917 10198 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:21.294505  8253 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:21.294049  8252 raft_consensus.cc:3055] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:21.276126  8369 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:21.316015  8326 raft_consensus.cc:2393] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 5: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:02:21.294670  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.355607  8252 raft_consensus.cc:740] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:21.355777  8324 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:21.356863  8323 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
W20260709 13:02:21.301018  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.358178  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1593) took 1313 ms. Trace:
I20260709 13:02:21.360409  8398 rpcz_store.cc:276] 0709 13:02:20.044844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.211457 (+166613us) service_pool.cc:224] Handling call
0709 13:02:20.641525 (+430068us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:20.674993 (+ 33468us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:20.675022 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:20.675444 (+   422us) log.cc:844] Serialized 26 byte log entry
0709 13:02:20.858862 (+183418us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:20.858918 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.858987 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:21.307067 (+448080us) spinlock_profiling.cc:243] Waited 7.66 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:21.315459 (+  8392us) raft_consensus.cc:1685] finished
0709 13:02:21.315483 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:21.358117 (+ 42634us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":21135,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":13425,"spinlock_wait_cycles":17619712,"thread_start_us":143417,"threads_started":1,"wal-append.queue_time_us":213969}
I20260709 13:02:21.323103  8325 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:21.294399  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:21.364524  8251 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:21.314239 10580 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:21.366393  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:21.366835  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 96136.3us
I20260709 13:02:21.367327  8257 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:21.357056  8252 consensus_queue.cc:260] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:21.445096  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 85079.7us
I20260709 13:02:21.311394  8401 rpcz_store.cc:276] 0709 13:02:19.995029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.156474 (+161445us) service_pool.cc:224] Handling call
0709 13:02:20.156817 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:20.157168 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:20.157197 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:20.157654 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 13:02:20.695686 (+538032us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:20.695745 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.695844 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:21.308840 (+612996us) spinlock_profiling.cc:243] Waited 4.24 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:21.310660 (+  1820us) raft_consensus.cc:1685] finished
0709 13:02:21.310694 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:21.310905 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":590,"raft.queue_time_us":1341,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":8235,"spinlock_wait_cycles":9743232,"thread_start_us":537555,"threads_started":1,"wal-append.queue_time_us":574905}
I20260709 13:02:21.390997  8406 raft_consensus.cc:1275] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:21.314664 10387 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:02:21.413128  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 45659.8us
I20260709 13:02:21.398659  8332 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:02:21.427824  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:21.448429  8252 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:21.314543 10498 raft_consensus.cc:740] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:21.386770  8399 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:21.402096  8403 raft_consensus.cc:1275] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:21.280396 10566 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:21.453418 10387 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:21.457787  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:21.458005 10387 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:21.448204  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:21.461820  8330 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:21.462016 10565 log.cc:927] Time spent T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.324s	user 0.002s	sys 0.000s
W20260709 13:02:21.427089  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 155135us
W20260709 13:02:21.456152  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.470539  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.508370  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 59615.9us
I20260709 13:02:21.435395  8251 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:21.380662  9002 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:21.534938  8328 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:02:21.535957  8332 raft_consensus.cc:606] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:21.538462  8332 raft_consensus.cc:614] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:21.466854  8372 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:21.358943 10563 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:21.539572  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 72791.2us
I20260709 13:02:21.541262  8324 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:21.295151  9005 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:21.379779 10561 raft_consensus.cc:2764] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:02:21.544556  8323 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:21.545476  8330 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:21.546315  8406 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:21.402500 10571 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:21.449905 10279 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.400847 10458 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:21.556658  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 105362us
I20260709 13:02:21.547176 10562 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:21.398337 10408 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.358249 10475 consensus_queue.cc:1048] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.181s
I20260709 13:02:21.554634  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1595) took 1506 ms. Trace:
W20260709 13:02:21.571807  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 106866us
I20260709 13:02:21.383004 10213 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:21.570058  8372 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:21.551959  8220 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:21.573984  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:21.560896 10279 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:21.586601  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:21.571714  8404 rpcz_store.cc:276] 0709 13:02:20.047980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.246532 (+198552us) service_pool.cc:224] Handling call
0709 13:02:20.246815 (+   283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:20.543362 (+296547us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:20.543409 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:20.584458 (+ 41049us) log.cc:844] Serialized 26 byte log entry
0709 13:02:20.822684 (+238226us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:20.822738 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.464036 (+641298us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b54000fd7d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:21.464081 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:21.464136 (+    55us) raft_consensus.cc:1685] finished
0709 13:02:21.464151 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:21.554558 (+ 90407us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":680999680,"thread_start_us":148622,"threads_started":1,"wal-append.queue_time_us":290321}
I20260709 13:02:21.407824 10580 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:21.590121 10451 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:21.590549 10451 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:21.590915 10451 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:21.591948 10580 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:21.576011  8972 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:21.464851 10299 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.114s
I20260709 13:02:21.563093  8369 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:21.593669 10581 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:21.593878 10498 consensus_queue.cc:260] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:21.631983  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 61971us
W20260709 13:02:21.598230  8972 leader_election.cc:336] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 3.197s (SENT)
I20260709 13:02:21.632710  8972 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:21.598402 10213 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:21.568913 10562 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:02:21.621261  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.574900 10408 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:21.640229  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 76711.1us
I20260709 13:02:21.638382  8220 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:21.639871  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:21.643193  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:21.643563  8399 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:21.642222  8252 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:02:21.646263  8252 pending_rounds.cc:85] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 5
I20260709 13:02:21.642079 10562 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:21.648413 10408 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:21.639389 10580 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:21.640034  8372 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:21.651657 10578 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:21.563701 10563 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:21.656823  8252 pending_rounds.cc:107] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:02:21.695080  8252 raft_consensus.cc:2889] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:21.647899 10451 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:21.665980  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 128918us
W20260709 13:02:21.666823  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.639948 10584 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:21.672390  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:21.699460  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 59486.3us
I20260709 13:02:21.699786  8398 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:21.700054  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 418173us
I20260709 13:02:21.700304  8372 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:21.643668 10560 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.027s	sys 0.066s
I20260709 13:02:21.654634 10576 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:02:21.684716  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.642356 10198 consensus_queue.cc:1048] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:02:21.698341  8223 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:21.656823 10498 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:21.655871 10299 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:21.712697  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 270717us
W20260709 13:02:21.707770  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 109542us
I20260709 13:02:21.708868  8399 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:21.681016 10580 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:21.714396  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.705053 10560 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.028s	sys 0.067s
I20260709 13:02:21.719884  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:21.720505  8324 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:21.720672 10590 consensus_queue.cc:1048] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:21.722674 10590 consensus_queue.cc:1059] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:21.723745 10576 raft_consensus.cc:697] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:21.660933 10381 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:02:21.658308 10477 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
W20260709 13:02:21.726930  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.716276  8297 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:21.724686 10576 consensus_queue.cc:237] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:21.733669 10359 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:21.736891  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.713217 10578 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:21.739545  8972 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:21.741144  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 97988.6us
I20260709 13:02:21.749351  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:21.712570 10299 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:21.726044 10580 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:21.734074 10359 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:21.759172  8369 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:21.759928  8332 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 13:02:21.764328  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.771176  8399 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:02:21.768276  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.771865  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.717795 10382 consensus_queue.cc:1048] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 13:02:21.712831 10475 consensus_queue.cc:1059] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:02:21.713361 10574 consensus_queue.cc:1048] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:02:21.717887 10459 consensus_queue.cc:1048] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 13:02:21.718047 10452 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:02:21.745647 10204 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:21.790027  8398 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:21.826494  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:21.772720 10359 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:21.776283  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 67616.6us
I20260709 13:02:21.761904 10584 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:21.786576  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:21.833132  8397 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:21.786818  8399 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:21.766402 10299 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:21.762735 10511 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.827330  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:02:21.788530  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 88251.7us
I20260709 13:02:21.743927 10578 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:21.829463  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:21.842389  8402 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:21.828634  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 126366us
I20260709 13:02:21.775483 10439 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.780310  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:21.828269 10585 log.cc:927] Time spent T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 13:02:21.829056  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:21.836436 10566 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:21.821655 10561 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:21.857568  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 84636.8us
I20260709 13:02:21.843667  8220 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:21.789103 10213 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:02:21.879249  8332 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:37920
W20260709 13:02:21.842717 10428 consensus_meta.cc:287] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.534s	user 0.008s	sys 0.002s
I20260709 13:02:21.834002 10204 raft_consensus.cc:697] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:21.837304 10584 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:21.787783 10591 raft_consensus.cc:3055] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:21.929559 10591 raft_consensus.cc:740] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:21.789197 10292 consensus_queue.cc:1059] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:02:21.841862 10381 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.880012  8332 raft_consensus.cc:606] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:21.938339  8332 raft_consensus.cc:614] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:21.883672  8298 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:21.871541 10571 consensus_queue.cc:1059] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:02:21.849696 10563 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:21.885031 10451 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:21.885406 10561 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:21.941650 10381 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:21.842207  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:21.827319 10476 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
W20260709 13:02:21.910781  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 212478us
I20260709 13:02:21.911608 10475 consensus_queue.cc:1243] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 5.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:02:21.912509  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:21.916217  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 136921us
I20260709 13:02:21.917240 10566 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:21.901019 10604 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:02:21.918761 10594 log.cc:927] Time spent T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:02:21.903883 10408 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:02:21.928056  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:21.846215 10578 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:21.850785 10597 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:21.944867  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 61401.8us
I20260709 13:02:21.860339 10523 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 13:02:21.849817 10439 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:21.846392 10511 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:21.930505 10591 consensus_queue.cc:260] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:21.985276  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 109949us
W20260709 13:02:21.947109  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 196506us
I20260709 13:02:22.060457  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:22.144665  8402 raft_consensus.cc:2410] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:02:22.097456  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:22.164505  8298 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:22.179297  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:22.162515  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.172776  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:22.180634  8326 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:22.162889  8974 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:22.063015  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:22.179865  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:22.156631  8401 raft_consensus.cc:2376] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5. Re-sending same reply.
I20260709 13:02:21.931567 10584 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:22.104174 10596 log.cc:927] Time spent T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.187s	user 0.003s	sys 0.000s
I20260709 13:02:22.130472 10605 raft_consensus.cc:3055] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:02:22.020570 10595 log.cc:927] Time spent T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
W20260709 13:02:22.210207  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 45810.2us
I20260709 13:02:22.195904  8398 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:22.206869  8397 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:22.031968  8972 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:22.198504  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:22.106356 10561 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:22.102442 10458 consensus_meta.cc:287] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.541s	user 0.008s	sys 0.002s
W20260709 13:02:22.016937 10588 log.cc:927] Time spent T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
I20260709 13:02:22.114223 10381 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:22.208853  8399 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:22.183059  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.183059  8223 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:02:22.168649  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:22.048161 10601 log.cc:927] Time spent T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
W20260709 13:02:22.015425  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 174583us
I20260709 13:02:22.211774  8332 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 7: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:22.198457 10604 raft_consensus.cc:697] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:22.120383 10581 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:22.102160 10387 consensus_meta.cc:287] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.643s	user 0.010s	sys 0.000s
I20260709 13:02:21.982240  9003 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:21.947829 10593 log.cc:927] Time spent T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.204s	user 0.001s	sys 0.001s
I20260709 13:02:22.219638  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:22.174090 10511 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:22.128845 10279 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:22.221154  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:22.221189  8405 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:22.098733 10598 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:02:22.223189  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 191249us
I20260709 13:02:22.186147  8222 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:22.219897 10571 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:02:22.227349  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.183734 10339 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:22.178308 10475 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:22.229943  8324 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:22.180240 10597 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:22.231974  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:22.329171  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:22.329811  8400 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 2.
I20260709 13:02:22.232192  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:22.330813  8329 raft_consensus.cc:2393] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 4: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:21.986054 10452 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:22.172353 10579 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:02:22.210402 10605 raft_consensus.cc:740] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:22.247530  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:22.121357 10562 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:22.249742  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:22.343499  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 157394us
I20260709 13:02:22.248236  8974 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:22.103022 10359 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:22.229171 10523 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:22.345218 10581 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.222258 10566 raft_consensus.cc:697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:02:22.363608  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 115422us
W20260709 13:02:22.318770  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 135723us
I20260709 13:02:22.231880 10616 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.108s
I20260709 13:02:22.236517 10591 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:22.060992 10204 consensus_queue.cc:237] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:22.338892  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:22.338536  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:22.390480  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.327085 10428 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:22.396559  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:22.347738  8324 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:22.334697 10563 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:22.319190  8369 connection.cc:256] RPC call timeout handler was delayed by -1.4170124470000474s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1545s for a response
I20260709 13:02:22.349601  8405 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:22.365204  8223 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:22.349013 10475 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:22.366532  8399 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:22.352196  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:22.328109 10511 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:22.337164 10597 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:22.340691  8398 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:22.259362 10387 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:22.435887  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:22.388418 10580 consensus_meta.cc:287] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.584s	user 0.008s	sys 0.003s
I20260709 13:02:22.436936  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:22.486657  8328 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:22.397394  8327 raft_consensus.cc:2410] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:22.351888 10381 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:22.492080  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:22.492499  8297 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:22.352900 10612 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.499085  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:22.354882 10359 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:22.358853 10439 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:22.353289 10562 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:22.361564 10339 raft_consensus.cc:697] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:02:22.437150  8369 connection.cc:256] RPC call timeout handler was delayed by -1.21811244700001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1766s for a response
I20260709 13:02:22.435592 10459 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:22.486912 10563 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:02:22.524919  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 308751us
W20260709 13:02:22.523115 10299 consensus_meta.cc:287] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.677s	user 0.012s	sys 0.000s
I20260709 13:02:22.553133  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:22.352837 10603 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:02:22.539841  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 174628us
I20260709 13:02:22.540922  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:22.556391  8329 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:22.559798  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:22.338302 10598 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:22.544180  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:22.544907  8248 raft_consensus.cc:1240] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:22.568775  8324 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:22.570032  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:22.570727  8324 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:22.548064  8369 connection.cc:256] RPC call timeout handler was delayed by -1.0543124470000294s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1948s for a response
W20260709 13:02:22.572881  8369 connection.cc:256] RPC call timeout handler was delayed by -1.0372124470000017s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1967s for a response
I20260709 13:02:22.520488  9009 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:22.358186 10581 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:22.561280  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:22.561893  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:22.394927 10198 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.535362 10452 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.436203 10428 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:22.545949 10571 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:22.551491 10477 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.558970 10579 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.586994 10428 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:22.551293 10607 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.551793 10599 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:02:22.589092  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.560470 10563 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:22.518626 10511 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:22.593369  9007 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 6 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:22.586195  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:22.360101 10604 consensus_queue.cc:237] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:22.522578 10580 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:22.360101 10566 consensus_queue.cc:237] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:22.621542  8324 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:22.588316 10571 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.598770 10279 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.561551 10299 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:22.641396  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:22.561882 10459 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:22.637497  8329 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:22.640501  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:22.640641 10599 raft_consensus.cc:697] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:22.599397 10452 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:22.618340 10561 consensus_queue.cc:1059] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:02:22.543732 10612 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:22.641407 10571 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:22.688212  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:22.712357  8324 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:22.618611 10439 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.066s
I20260709 13:02:22.540892 10605 consensus_queue.cc:260] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:22.568216 10603 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:22.631628 10619 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:22.607817 10477 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:22.614332 10198 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:22.708212  8401 raft_consensus.cc:1240] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:22.643648 10279 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:22.728461  9008 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:22.721115  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:22.729908  9007 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:22.719543  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 92822.3us
W20260709 13:02:22.715919 10606 log.cc:927] Time spent T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.383s	user 0.000s	sys 0.001s
I20260709 13:02:22.731767  8369 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:22.733112  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.625306 10607 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:22.735839  8324 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:22.731767  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:22.611714 10579 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:22.746089  8324 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:22.755116  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:22.761566 10611 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.395s	user 0.001s	sys 0.000s
I20260709 13:02:22.761917 10387 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:22.762319  9000 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:22.764861  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 300) took 1063 ms. Trace:
I20260709 13:02:22.766675  8324 raft_consensus.cc:1240] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:22.783501  8324 raft_consensus.cc:1240] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:22.694239 10339 consensus_queue.cc:237] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:22.765456  8404 rpcz_store.cc:276] 0709 13:02:21.701925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.704127 (+  2202us) service_pool.cc:224] Handling call
0709 13:02:21.718657 (+ 14530us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.829751 (+111094us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.829785 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:21.830248 (+   463us) log.cc:844] Serialized 26 byte log entry
0709 13:02:22.070100 (+239852us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:22.070163 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.070264 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:22.764582 (+694318us) raft_consensus.cc:1685] finished
0709 13:02:22.764606 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:22.764775 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47219,"raft.queue_time_us":105568,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":12837,"thread_start_us":285038,"threads_started":2,"wal-append.queue_time_us":367405}
W20260709 13:02:22.796921  8326 consensus_meta.cc:287] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.603s	user 0.009s	sys 0.009s
I20260709 13:02:22.797875  8326 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
W20260709 13:02:22.803898  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 172480us
I20260709 13:02:22.763334 10625 consensus_queue.cc:1059] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:02:22.782550  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:22.767500 10475 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:22.789562 10619 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.808633  8326 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:02:22.827992  8326 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:02:22.844772  8326 raft_consensus.cc:1240] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:22.882815  8222 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:22.883201  9000 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:02:22.883447 10474 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.884025  8222 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:22.883980 10474 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:22.884744  9008 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:22.885648 10381 consensus_queue.cc:1059] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:22.886420 10474 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:22.887212 10614 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:22.887074 10575 consensus_queue.cc:1059] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:22.887653 10614 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.888088 10614 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:22.890406 10598 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:22.891122 10598 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.891747 10598 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:22.895011  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:22.895843  9008 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:22.897045 10617 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:22.897790 10617 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.898255 10617 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:22.882884 10581 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:22.905692 10474 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:22.910559  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:22.911474  9008 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:22.912478 10598 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:22.920765  9005 raft_consensus.cc:2393] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 2: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:22.922260 10627 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:22.922821 10477 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:22.924415 10627 consensus_queue.cc:1048] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:22.878741 10387 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.925948 10615 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:22.926548 10615 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:22.926030 10387 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:22.926950 10615 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:22.815120 10439 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:22.930173 10634 log.cc:927] Time spent T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.215s	user 0.000s	sys 0.000s
W20260709 13:02:22.931836 10640 log.cc:927] Time spent T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:02:22.768540 10599 consensus_queue.cc:237] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:22.878968 10574 raft_consensus.cc:3055] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:22.933144 10574 raft_consensus.cc:740] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:22.934038 10574 consensus_queue.cc:260] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:22.935106 10574 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:22.936043 10631 log.cc:927] Time spent T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:02:22.936898 10452 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:22.938066 10635 log.cc:927] Time spent T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
W20260709 13:02:22.939446 10633 log.cc:927] Time spent T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
W20260709 13:02:22.940881 10643 log.cc:927] Time spent T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
W20260709 13:02:22.946187 10642 log.cc:927] Time spent T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 13:02:22.947690  8326 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:22.963716  9007 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:22.975783  9009 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:22.980700  8326 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:02:22.991907  8402 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:02:22.995160  9006 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:23.001138  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:22.879374 10299 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:22.965646 10581 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.990154 10387 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:22.998836 10580 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:23.037467  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 152483us
I20260709 13:02:23.040282  8222 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:23.040346  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 285174us
I20260709 13:02:23.027643 10459 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:23.030671 10359 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:23.057937  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.062345  8369 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:23.053385  8402 raft_consensus.cc:1275] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:23.066454  8405 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:23.042296 10213 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.822994 10619 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:23.073524  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 231) took 1345 ms. Trace:
I20260709 13:02:23.074747  8369 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:23.074074  8323 rpcz_store.cc:276] 0709 13:02:21.719120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.719246 (+   126us) service_pool.cc:224] Handling call
0709 13:02:21.721477 (+  2231us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.721781 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.721812 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:21.722267 (+   455us) log.cc:844] Serialized 26 byte log entry
0709 13:02:21.838349 (+116082us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:21.838389 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.838465 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:22.998767 (+1160302us) spinlock_profiling.cc:243] Waited 9.83 ms on lock 0x7b240012d658. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255579c4 00007f8825555acb 00007f8825710dc9 00007f8825710a55 00007f882570ffb0 00007f8828386488 00007f8828386be8 00007f882835e398 00007f882835976a 00007f8828358bb6
0709 13:02:23.050708 (+ 51941us) raft_consensus.cc:1685] finished
0709 13:02:23.050737 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.064587 (+ 13850us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1009,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":116,"spinlock_wait_cycles":22599424,"thread_start_us":114500,"threads_started":1,"wal-append.queue_time_us":164578}
W20260709 13:02:23.084587 10630 log.cc:927] Time spent T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.402s	user 0.000s	sys 0.001s
I20260709 13:02:23.120854  8220 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:23.121948  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:23.123669  8254 raft_consensus.cc:2410] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:02:23.125591  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:23.126477  8252 raft_consensus.cc:2410] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:23.133584  8252 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:23.093532 10381 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:23.134989  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:23.135465 10381 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:23.136430  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:23.093068 10562 raft_consensus.cc:3055] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:23.137636  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:23.138024  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.037182 10581 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:23.063515 10198 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:23.141456 10575 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.091820 10603 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:23.094558  9003 consensus_meta.cc:287] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.758s	user 0.008s	sys 0.012s
W20260709 13:02:23.091102 10584 consensus_meta.cc:287] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.864s	user 0.006s	sys 0.005s
W20260709 13:02:23.096352 10597 consensus_meta.cc:287] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.551s	user 0.005s	sys 0.000s
I20260709 13:02:23.120221 10579 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:23.181999  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 107279us
I20260709 13:02:23.075208 10299 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:23.145087  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:23.187767  8402 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:23.187796  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.093300 10571 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:23.188373  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.182113  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1903) took 1266 ms. Trace:
I20260709 13:02:23.188786  8400 rpcz_store.cc:276] 0709 13:02:21.915401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.330399 (+414998us) service_pool.cc:224] Handling call
0709 13:02:22.353255 (+ 22856us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:22.353626 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:22.353651 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:22.354120 (+   469us) log.cc:844] Serialized 26 byte log entry
0709 13:02:22.588901 (+234781us) spinlock_profiling.cc:243] Waited 31.6 ms on lock 0x7b18001d6260. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88283664bc 00007f88256b448b 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e0d1
0709 13:02:22.589047 (+   146us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:22.589079 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.589152 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:23.083987 (+494835us) spinlock_profiling.cc:243] Waited 35.8 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:23.137674 (+ 53687us) raft_consensus.cc:1685] finished
0709 13:02:23.137696 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.182047 (+ 44351us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47761,"raft.queue_time_us":31017,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":46026,"spinlock_wait_cycles":154982912,"thread_start_us":102532,"threads_started":1,"wal-append.queue_time_us":248875}
I20260709 13:02:23.184834  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:23.089609 10612 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:23.193673  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:23.194309  8249 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:23.174595 10654 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:23.174506  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:23.186995  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:23.198839  8254 raft_consensus.cc:2393] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 4: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:02:23.169165  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 126380us
W20260709 13:02:23.189962  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 69165.4us
I20260709 13:02:23.185295 10279 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:23.192807 10381 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:23.200145  8255 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:23.200750  8220 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:23.202814 10566 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:23.203325 10566 raft_consensus.cc:697] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:23.063262 10359 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:23.137815 10562 raft_consensus.cc:740] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:23.206290  9009 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:23.202682 10654 consensus_queue.cc:1048] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:23.212074  8251 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:23.212683  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:23.212987  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:23.213089  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:23.213644  8256 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
W20260709 13:02:23.213718 10620 log.cc:927] Time spent T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.525s	user 0.002s	sys 0.000s
I20260709 13:02:23.213814  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:23.214288  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:23.087705 10647 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.146s
I20260709 13:02:23.204219 10566 consensus_queue.cc:237] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:23.121383 10213 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:23.085376 10615 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
W20260709 13:02:23.257524  8332 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.005s	sys 0.014s
W20260709 13:02:23.204695 10591 consensus_meta.cc:287] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.809s	user 0.007s	sys 0.000s
I20260709 13:02:23.155972  9003 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:23.227870  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:02:23.265103  8332 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.005s	sys 0.015s
W20260709 13:02:23.234743  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 49953.2us
I20260709 13:02:23.261955  8324 raft_consensus.cc:1240] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:02:23.263530  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:23.264214  8406 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.004s	sys 0.016s
W20260709 13:02:23.264487  8972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.869s	user 0.000s	sys 0.020s
W20260709 13:02:23.181246 10578 consensus_meta.cc:287] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 1.000s	user 0.004s	sys 0.016s
W20260709 13:02:23.271729  8406 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.004s	sys 0.016s
I20260709 13:02:23.227870  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:23.235302  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:23.228009 10648 log.cc:927] Time spent T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:02:23.274231  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:23.183275 10198 consensus_queue.cc:1059] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.084s
W20260709 13:02:23.233042 10563 consensus_meta.cc:287] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.607s	user 0.005s	sys 0.002s
I20260709 13:02:23.273154  8251 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
W20260709 13:02:23.271922  8972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.000s	sys 0.020s
I20260709 13:02:23.271782  8254 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:23.272452  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:23.306838  8326 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:23.235034 10580 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:23.140708 10617 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:23.306808  8369 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:23.315496  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:23.209333 10279 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.096s
I20260709 13:02:23.185905 10597 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:23.304636  9003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56954 (request call id 464) took 1531 ms. Trace:
I20260709 13:02:23.358719 10657 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:23.358857  9003 rpcz_store.cc:276] 0709 13:02:21.772850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.775276 (+  2426us) service_pool.cc:224] Handling call
0709 13:02:23.278558 (+1503282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:23.331800  8255 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:23.272135 10614 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:23.337962 10653 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:23.362704 10213 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:23.362896 10614 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:23.335899  8369 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:23.340749  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:23.363950  8400 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:23.273026 10561 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:23.364774 10523 consensus_meta.cc:287] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 1.010s	user 0.010s	sys 0.000s
I20260709 13:02:23.365272 10523 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:23.366035 10619 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:23.366377 10656 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:23.366684  9001 raft_consensus.cc:1240] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:23.367102 10656 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:23.367921  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:23.368397  9001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 543) took 1121 ms. Trace:
I20260709 13:02:23.347867 10580 raft_consensus.cc:697] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:23.370234  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 226) took 2174 ms. Trace:
I20260709 13:02:23.370631 10580 consensus_queue.cc:237] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:23.371465  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:23.372602  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:23.373106 10597 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:23.373780  8398 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:23.374229 10653 raft_consensus.cc:697] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:23.375130  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:23.375211 10653 consensus_queue.cc:237] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:23.308588 10591 consensus_queue.cc:1059] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:02:23.376586 10614 raft_consensus.cc:3055] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:23.376935 10614 raft_consensus.cc:740] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:23.188760 10575 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:23.370707  8331 rpcz_store.cc:276] 0709 13:02:21.195491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.198409 (+  2918us) service_pool.cc:224] Handling call
0709 13:02:21.198693 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.395927 (+1197234us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:22.395958 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:22.395998 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:22.396036 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.369964 (+973928us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54003f0cd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:23.370020 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.370163 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":36741,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":23014,"spinlock_wait_cycles":2364059520,"thread_start_us":168489,"threads_started":1}
I20260709 13:02:23.344799  8326 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:23.379835  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:23.324141  9009 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:23.228242 10644 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:23.186405 10584 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:23.326869 10579 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.388060 10598 consensus_queue.cc:1059] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:23.388490 10579 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:23.389135 10598 consensus_queue.cc:1059] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:23.389554 10605 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:23.390218 10598 consensus_queue.cc:1059] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:23.339583 10498 raft_consensus.cc:3055] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:23.357136 10359 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:23.358351 10652 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.364807  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:23.381789  8257 raft_consensus.cc:2393] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 4: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:23.380415  8331 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:23.396802  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:23.385311  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:23.386000  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:23.355372  8402 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:23.386677 10613 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:02:23.387651 10637 log.cc:927] Time spent T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.747s	user 0.002s	sys 0.000s
I20260709 13:02:23.387339 10655 consensus_queue.cc:1059] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:23.387823 10575 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:23.368767  9001 rpcz_store.cc:276] 0709 13:02:22.247186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.249922 (+  2736us) service_pool.cc:224] Handling call
0709 13:02:22.365238 (+115316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:23.367120 (+1001882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.368177 (+  1057us) spinlock_profiling.cc:243] Waited 982 ms on lock 0x7b54004c2554. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:23.368342 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2258051840}
I20260709 13:02:23.382773  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:23.380689  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:23.228674 10654 consensus_queue.cc:1059] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.088s
I20260709 13:02:23.397472  8400 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:23.467468  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:23.400281 10299 raft_consensus.cc:3055] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:23.469250  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 305) took 1233 ms. Trace:
I20260709 13:02:23.492264  8406 rpcz_store.cc:276] 0709 13:02:22.235446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.642330 (+406884us) service_pool.cc:224] Handling call
0709 13:02:22.708241 (+ 65911us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:22.718988 (+ 10747us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:22.719022 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:22.719451 (+   429us) log.cc:844] Serialized 26 byte log entry
0709 13:02:23.272632 (+553181us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:23.272671 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.272751 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:23.469041 (+196290us) raft_consensus.cc:1685] finished
0709 13:02:23.469060 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.469193 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2856,"raft.queue_time_us":6526,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":5789,"thread_start_us":532025,"threads_started":1,"wal-append.queue_time_us":739173}
I20260709 13:02:23.497841  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:23.498916  8249 raft_consensus.cc:2393] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 5: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:23.358752 10571 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:23.332327 10563 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:23.419422  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:23.308354 10578 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:23.399055  8255 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:02:23.470487  8369 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:23.399657  8251 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:23.450482 10603 raft_consensus.cc:3055] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:23.377926 10657 raft_consensus.cc:697] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:23.491827  8253 raft_consensus.cc:2393] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 5: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:02:23.515880  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 45456.5us
I20260709 13:02:23.396833  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:23.502167  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:23.450764 10647 raft_consensus.cc:3055] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:23.462918 10597 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.412897 10579 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:23.451879 10619 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:23.395799 10498 raft_consensus.cc:740] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:02:23.492519 10428 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.010s	sys 0.041s
I20260709 13:02:23.515839  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:23.518172  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:23.487373  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:23.520901  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:23.464094 10627 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.348058 10562 consensus_queue.cc:260] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:23.527334  8249 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:23.526388  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:23.492260 10299 raft_consensus.cc:740] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:23.637840  8251 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:23.637775  8402 raft_consensus.cc:2393] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 4: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:23.637899  8255 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:23.520275 10647 raft_consensus.cc:740] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:02:23.546597  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 344102us
I20260709 13:02:23.491284 10523 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:02:23.511256 10656 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:23.513805 10603 raft_consensus.cc:740] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:23.403110 10644 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:23.521255 10581 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:23.509356 10213 consensus_queue.cc:1059] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:02:23.533721 10604 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:23.632617  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:23.526542 10613 raft_consensus.cc:697] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:23.470456 10614 consensus_queue.cc:260] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:23.508304 10652 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:23.643571  8400 raft_consensus.cc:3055] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:02:23.629323 10428 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.831s	user 0.012s	sys 0.043s
W20260709 13:02:23.503621  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:23.688753  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 45986.7us
I20260709 13:02:23.690881  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 1924) took 1299 ms. Trace:
I20260709 13:02:23.691475  8255 raft_consensus.cc:1275] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:23.638276 10627 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:23.633330 10663 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:02:23.577723 10597 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:24.104272  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1615) took 1923 ms. Trace:
W20260709 13:02:24.108018  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.996371  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:24.110658  8329 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:23.998469  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:23.710137  8249 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:24.108989  8400 raft_consensus.cc:740] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:23.708645  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:02:23.755506  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.708307  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:23.871829  8251 raft_consensus.cc:1275] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:24.060622  9009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56948 (request call id 549) took 1060 ms. Trace:
I20260709 13:02:24.009931  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:24.115236  9009 rpcz_store.cc:276] 0709 13:02:23.000554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.000811 (+   257us) service_pool.cc:224] Handling call
0709 13:02:24.060369 (+1059558us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b54001b3fd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:24.060540 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":469647616}
I20260709 13:02:24.087122  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1614) took 1906 ms. Trace:
I20260709 13:02:24.107939  8330 rpcz_store.cc:276] 0709 13:02:22.391795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.391923 (+   128us) service_pool.cc:224] Handling call
0709 13:02:22.392203 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.543079 (+150876us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:22.558047 (+ 14968us) log.cc:844] Serialized 14 byte log entry
0709 13:02:22.580780 (+ 22733us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:22.580842 (+    62us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:22.580902 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.386999 (+806097us) spinlock_profiling.cc:243] Waited 76.8 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:23.690598 (+303599us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54003c9954. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:23.690673 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.690820 (+   147us) inbound_call.cc:177] Queueing success response
0709 13:02:23.994532 (+303712us) spinlock_profiling.cc:243] Waited 8.94 ms on lock 0x7b18003680e0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
Metrics: {"mutex_wait_us":14270,"prepare.queue_time_us":17796,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":32162,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":125,"spinlock_wait_cycles":543459712,"thread_start_us":8024,"threads_started":1,"wal-append.queue_time_us":68623}
I20260709 13:02:24.116084  8253 raft_consensus.cc:1240] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:23.995678  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1901) took 2080 ms. Trace:
I20260709 13:02:24.117172  8401 rpcz_store.cc:276] 0709 13:02:21.914872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.215464 (+300592us) service_pool.cc:224] Handling call
0709 13:02:22.232133 (+ 16669us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.724786 (+492653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.995321 (+1270535us) spinlock_profiling.cc:243] Waited 476 ms on lock 0x7b54000c8854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:23.995591 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1093978752}
W20260709 13:02:24.118453  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.946801  8298 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:24.120178  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 173425us
I20260709 13:02:23.708956  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:24.116313  8397 rpcz_store.cc:276] 0709 13:02:22.180639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.354528 (+173889us) service_pool.cc:224] Handling call
0709 13:02:22.354823 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.604521 (+249698us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:22.619902 (+ 15381us) log.cc:844] Serialized 14 byte log entry
0709 13:02:22.809110 (+189208us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:22.809157 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:22.809202 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.088562 (+279360us) spinlock_profiling.cc:243] Waited 3.93 ms on lock 0x7b480002cc58. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882d6e7aa1 00007f882d6deb37 00007f882d6de9ca 00007f882d7d626f 00007f882d7d5f37 00007f882d7ea55c 00007f882d7ea508 00007f882d7ea488 00007f882d7ea443 00007f882d7e96ec 00007f882c713a96
0709 13:02:24.086765 (+998203us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b54000fd7d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:24.086851 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:24.087039 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8548,"prepare.queue_time_us":16208,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":5722,"raft.run_cpu_time_us":1955,"raft.run_wall_time_us":360823,"spinlock_wait_cycles":582361856,"thread_start_us":183041,"threads_started":2,"wal-append.queue_time_us":138274,"wal-append.run_cpu_time_us":1305,"wal-append.run_wall_time_us":1075678}
I20260709 13:02:24.111809  8326 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:24.123508  8220 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:24.129050  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:24.116021  8323 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:24.131538  8330 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:24.125514 10574 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:23.834542 10562 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:24.132802 10574 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:24.132332  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 1671) took 1083 ms. Trace:
I20260709 13:02:24.134402  8330 rpcz_store.cc:276] 0709 13:02:23.049112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.128773 (+1079661us) service_pool.cc:224] Handling call
0709 13:02:24.132257 (+  3484us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:24.135802  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:24.137272  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:24.138062  8330 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:24.138958 10574 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:23.705559 10669 log.cc:927] Time spent T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 13:02:24.108229  8403 rpcz_store.cc:276] 0709 13:02:22.180923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.358517 (+177594us) service_pool.cc:224] Handling call
0709 13:02:22.358834 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.692491 (+333657us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:22.716799 (+ 24308us) log.cc:844] Serialized 14 byte log entry
0709 13:02:23.033878 (+317079us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:23.033932 (+    54us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:23.033979 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.103882 (+1069903us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b54003a1954. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:24.103949 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:24.104179 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22106,"prepare.queue_time_us":21965,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":35,"raft.queue_time_us":58560,"raft.run_cpu_time_us":61,"raft.run_wall_time_us":61,"spinlock_wait_cycles":654441216,"thread_start_us":289773,"threads_started":2,"wal-append.queue_time_us":315532}
I20260709 13:02:24.109795 10666 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.688037  8250 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 5: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:24.112903  8400 consensus_queue.cc:260] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:24.143749  8400 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:24.144711  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1619) took 1590 ms. Trace:
I20260709 13:02:24.145068  8404 rpcz_store.cc:276] 0709 13:02:22.554788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.788193 (+233405us) service_pool.cc:224] Handling call
0709 13:02:22.788630 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:23.014078 (+225448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:23.014127 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:23.014789 (+   662us) log.cc:844] Serialized 26 byte log entry
0709 13:02:23.143990 (+129201us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:23.144043 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.144138 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:24.144517 (+1000379us) raft_consensus.cc:1685] finished
0709 13:02:24.144536 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:24.144657 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":89354,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":138,"thread_start_us":351836,"threads_started":2,"wal-append.queue_time_us":332586}
W20260709 13:02:23.706205 10667 log.cc:927] Time spent T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.242s	user 0.000s	sys 0.000s
I20260709 13:02:24.152798  8405 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
W20260709 13:02:23.706910 10668 log.cc:927] Time spent T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:02:23.976131 10339 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.014s	sys 0.047s
I20260709 13:02:23.694415 10617 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:24.128561 10605 consensus_meta.cc:287] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.735s	user 0.011s	sys 0.000s
I20260709 13:02:24.154953  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:24.170081  8249 raft_consensus.cc:2376] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4. Re-sending same reply.
I20260709 13:02:24.128494 10624 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:23.693203 10674 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.467s
W20260709 13:02:23.912222 10677 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.270s	user 0.000s	sys 0.000s
W20260709 13:02:24.048251 10566 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.030s	sys 0.052s
I20260709 13:02:24.157717  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:23.702802 10578 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:24.155762 10279 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:24.159559  8406 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:23.911823 10612 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.163151  8329 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:23.633313 10619 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:24.043812 10678 log.cc:927] Time spent T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:02:23.871004 10523 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.164444  8397 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:24.103027 10607 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:24.123047 10580 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.014s	sys 0.055s
I20260709 13:02:23.911168 10626 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.911548 10433 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.159085  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:24.122292 10581 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.166249  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:24.166551  8256 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:23.694711 10615 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:24.196908  8332 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:24.196990  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:24.197521  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:24.197906  8251 raft_consensus.cc:2376] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3. Re-sending same reply.
I20260709 13:02:24.198331  8256 raft_consensus.cc:2376] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4. Re-sending same reply.
I20260709 13:02:24.142649 10603 consensus_queue.cc:260] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:23.965960 10575 raft_consensus.cc:3055] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:24.142676 10656 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.143533 10614 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:24.143500 10604 raft_consensus.cc:697] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:24.178607  8400 raft_consensus.cc:1275] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:02:24.213342  8400 pending_rounds.cc:85] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 2
I20260709 13:02:24.213063  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:24.213645  8400 pending_rounds.cc:107] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:02:24.181564  8254 raft_consensus.cc:2376] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4. Re-sending same reply.
I20260709 13:02:23.873271 10477 raft_consensus.cc:3055] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:24.189620 10563 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:24.179718  8972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.007s	sys 0.038s
W20260709 13:02:24.221417  8972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.007s	sys 0.038s
W20260709 13:02:24.221840  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 718545us
I20260709 13:02:24.222110  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:24.194046 10584 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:24.186818  8326 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
W20260709 13:02:24.166239 10339 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.015s	sys 0.048s
I20260709 13:02:24.166875 10213 raft_consensus.cc:3055] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:24.192257  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 1663) took 1606 ms. Trace:
W20260709 13:02:24.198549  8402 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.004s	sys 0.014s
W20260709 13:02:24.230507  8402 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.004s	sys 0.014s
W20260709 13:02:24.184382 10566 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.845s	user 0.031s	sys 0.053s
I20260709 13:02:24.167264  8220 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:24.200930  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 1929) took 1807 ms. Trace:
I20260709 13:02:24.201697  8249 raft_consensus.cc:2410] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:02:24.203339  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 307) took 2904 ms. Trace:
I20260709 13:02:24.182209 10359 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:24.184686 10381 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.010s	sys 0.010s
I20260709 13:02:23.696349 10299 consensus_queue.cc:260] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:23.701759 10498 consensus_queue.cc:260] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:24.296662 10498 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:24.185920 10279 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.301673 10279 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:24.195955 10580 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.014s	sys 0.056s
I20260709 13:02:24.213939  8400 raft_consensus.cc:2889] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:24.306485  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1915) took 1267 ms. Trace:
I20260709 13:02:24.306867  8400 rpcz_store.cc:276] 0709 13:02:23.038542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.643145 (+604603us) service_pool.cc:224] Handling call
0709 13:02:23.643443 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.306176 (+662733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.306415 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:24.213922  8256 raft_consensus.cc:2376] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6. Re-sending same reply.
I20260709 13:02:24.215677  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:24.183974  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:24.312261  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:24.313056  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 1942) took 1066 ms. Trace:
I20260709 13:02:24.313380  8323 rpcz_store.cc:276] 0709 13:02:23.246591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.311992 (+1065401us) service_pool.cc:224] Handling call
0709 13:02:24.312997 (+  1005us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:24.189296 10624 raft_consensus.cc:697] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:24.174007  8330 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:24.164239 10666 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:24.315786  8330 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:24.315526 10624 consensus_queue.cc:237] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:24.317036 10666 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:24.317946 10666 consensus_queue.cc:1059] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:24.202512 10607 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.319137 10607 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:23.869055 10657 consensus_queue.cc:237] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:24.202453 10575 raft_consensus.cc:740] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:02:24.231349  8972 leader_election.cc:336] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.589s (SENT)
I20260709 13:02:24.321867  8972 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:24.322144 10575 consensus_queue.cc:260] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:24.326354 10607 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:24.327561 10666 raft_consensus.cc:3055] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:24.328128 10666 raft_consensus.cc:740] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:24.329677 10666 consensus_queue.cc:260] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:24.331565 10666 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:24.334448 10279 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:24.338387 10279 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:24.377292  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:24.232102  8325 rpcz_store.cc:276] 0709 13:02:22.393335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.580029 (+186694us) service_pool.cc:224] Handling call
0709 13:02:22.584955 (+  4926us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:23.027982 (+443027us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:23.028020 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:23.035278 (+  7258us) log.cc:844] Serialized 26 byte log entry
0709 13:02:23.189470 (+154192us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:23.189520 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.189610 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:24.200744 (+1011134us) raft_consensus.cc:1685] finished
0709 13:02:24.200765 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:24.200876 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14316,"raft.queue_time_us":136109,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":10889,"thread_start_us":527105,"threads_started":2,"wal-append.queue_time_us":265184}
I20260709 13:02:24.233356  8248 rpcz_store.cc:276] 0709 13:02:21.291772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.296157 (+  4385us) service_pool.cc:224] Handling call
0709 13:02:21.296503 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.577054 (+1280551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.196260 (+1619206us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b540033ed54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:24.196420 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2869511424}
I20260709 13:02:24.199021 10612 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:24.230361  8329 rpcz_store.cc:276] 0709 13:02:22.586225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.659408 (+ 73183us) service_pool.cc:224] Handling call
0709 13:02:23.996178 (+1336770us) spinlock_profiling.cc:243] Waited 1 ms on lock 0x7b58000e3498. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:02:24.192198 (+196020us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2300160}
I20260709 13:02:23.966955 10647 consensus_queue.cc:260] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:24.233991  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:24.204064 10523 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:24.274612 10563 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.210543 10433 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:24.403891  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 220777us
I20260709 13:02:24.294210  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:24.295666  8222 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:24.407737  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 112101us
I20260709 13:02:24.317164  8330 raft_consensus.cc:1275] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:24.409667  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1673) took 1025 ms. Trace:
I20260709 13:02:24.410120  8330 rpcz_store.cc:276] 0709 13:02:23.383814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.316596 (+932782us) service_pool.cc:224] Handling call
0709 13:02:24.316853 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.409315 (+ 92462us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.409592 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:24.411947  8330 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:24.225517  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:24.216491 10477 raft_consensus.cc:740] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:24.416280 10477 consensus_queue.cc:260] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:24.202414 10626 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:24.230023 10213 raft_consensus.cc:740] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:24.421259 10477 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:24.422039 10213 consensus_queue.cc:260] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:24.423646 10213 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:24.211287 10581 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:24.426965 10581 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:24.350534  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 119164us
W20260709 13:02:24.427747  8972 leader_election.cc:336] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.624s (SENT)
I20260709 13:02:24.428110  8972 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:24.219866 10656 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:24.282156 10678 log.cc:927] Time spent T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:02:24.360334  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 1630) took 1167 ms. Trace:
I20260709 13:02:24.429827  8405 rpcz_store.cc:276] 0709 13:02:23.193250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.373384 (+180134us) service_pool.cc:224] Handling call
0709 13:02:24.360092 (+986708us) spinlock_profiling.cc:243] Waited 664 ms on lock 0x7b54001dfe54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:24.360253 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1527741696}
I20260709 13:02:24.364897  8331 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:02:24.431120  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1674) took 1047 ms. Trace:
I20260709 13:02:24.431454  8331 rpcz_store.cc:276] 0709 13:02:23.384075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.364122 (+980047us) service_pool.cc:224] Handling call
0709 13:02:24.364575 (+   453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.430875 (+ 66300us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.431067 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:24.347270 10381 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.011s	sys 0.010s
I20260709 13:02:24.432979 10381 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.433627 10381 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:24.436349 10381 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:24.357151 10359 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:24.376199 10680 log.cc:927] Time spent T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:02:24.439569 10626 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:24.378189  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:24.440717  8254 raft_consensus.cc:2376] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7. Re-sending same reply.
I20260709 13:02:24.293735 10566 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:24.441619 10656 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:24.441780 10566 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.442301 10566 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:24.442214 10656 consensus_queue.cc:1059] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:24.444041  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1631) took 1177 ms. Trace:
I20260709 13:02:24.444453  8404 rpcz_store.cc:276] 0709 13:02:23.266461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.155604 (+889143us) service_pool.cc:224] Handling call
0709 13:02:24.155911 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.156110 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:24.368400 (+212290us) log.cc:844] Serialized 14 byte log entry
0709 13:02:24.443685 (+ 75285us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:24.443740 (+    55us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:24.443784 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.443851 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:24.443982 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":251529,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"raft.queue_time_us":93,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":125,"thread_start_us":260455,"threads_started":2,"wal-append.queue_time_us":73605}
I20260709 13:02:24.379948  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:24.437067 10381 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:24.450255 10381 raft_consensus.cc:3055] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:24.450722 10381 raft_consensus.cc:740] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:24.365703 10607 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:24.268249 10663 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.101s
I20260709 13:02:24.224646 10617 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:02:24.266093  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 98868.3us
W20260709 13:02:24.464905  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.348799 10574 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:24.453049  8249 raft_consensus.cc:2410] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:02:24.467991  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:24.467008  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:24.469853  8331 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:24.453116  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1624) took 1267 ms. Trace:
I20260709 13:02:24.377974 10299 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:24.454972  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:24.455219  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 1937) took 1210 ms. Trace:
I20260709 13:02:24.211742 10613 consensus_queue.cc:237] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:24.592303 10681 log.cc:927] Time spent T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.235s	user 0.000s	sys 0.000s
I20260709 13:02:24.594192 10498 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:24.458958  8329 raft_consensus.cc:1240] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:24.387311  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 280077us
I20260709 13:02:24.595741  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1634) took 1229 ms. Trace:
I20260709 13:02:24.470836  8402 rpcz_store.cc:276] 0709 13:02:23.185709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.694830 (+509121us) service_pool.cc:224] Handling call
0709 13:02:23.695122 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:23.695329 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:24.231077 (+535748us) log.cc:844] Serialized 14 byte log entry
0709 13:02:24.452502 (+221425us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:24.452584 (+    82us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:24.452659 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.452749 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:24.453044 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":578945,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":97,"thread_start_us":756360,"threads_started":2,"wal-append.queue_time_us":204338}
I20260709 13:02:24.604841  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:24.605681  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:24.606330  8325 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:24.452304 10666 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:24.587343  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 128052us
W20260709 13:02:24.587515  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 159717us
W20260709 13:02:24.588097  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.588966  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:24.613695  8400 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
W20260709 13:02:24.589854 10686 log.cc:927] Time spent T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.314s	user 0.000s	sys 0.001s
I20260709 13:02:24.640015  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:24.585913  8220 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:24.614770  9009 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:24.613691  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.460489 10561 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:24.454440 10691 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:24.636900  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 228249us
I20260709 13:02:24.457194 10563 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:24.346333 10579 consensus_queue.cc:1175] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8c42259d7c4b463a9f485f42c320c81b" responder_term: 7 status { last_received { term: 4 index: 6 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 6. Current term is 7. Ops: [6.7-6.7]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:24.460924 10604 consensus_queue.cc:237] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:24.451815 10381 consensus_queue.cc:260] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:24.640750  9007 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:24.610376 10617 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:24.814431  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:24.819680  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:24.820333  8249 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:24.820614  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1555) took 1111 ms. Trace:
I20260709 13:02:24.643965  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:24.822124  8400 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 5: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:24.822177  8404 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:24.729226  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:24.816203  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:24.666990  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:24.644399  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.705889  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:24.666487  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:24.641105  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 55258.9us
I20260709 13:02:24.826756  8400 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:24.821532  8222 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:02:24.819831  8972 leader_election.cc:336] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.706s (SENT)
I20260709 13:02:24.706353  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:24.828284  8972 leader_election.cc:336] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.754s (SENT)
I20260709 13:02:24.828764  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:24.829259  8972 leader_election.cc:336] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.909s (SENT)
I20260709 13:02:24.829715  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:24.829756  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:24.830426  8254 raft_consensus.cc:2410] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:24.830607 10663 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:24.830864  8972 leader_election.cc:336] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 2.019s (SENT)
I20260709 13:02:24.831187  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:24.831297  8972 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:24.832034 10213 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:24.832372 10687 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:24.821089  8255 rpcz_store.cc:276] 0709 13:02:23.709215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.112469 (+403254us) service_pool.cc:224] Handling call
0709 13:02:24.819542 (+707073us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b5800002a98. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:02:24.820558 (+  1016us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33607424}
I20260709 13:02:24.822803  8325 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:24.833940  8298 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:24.834084  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:24.832919 10687 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:24.835779  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:24.699190  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.825201  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:24.840390  8222 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:24.603134 10702 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:02:24.815327  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:24.773725 10699 log.cc:927] Time spent T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:02:24.843681  8254 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:24.843856  8369 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:24.700512 10689 log.cc:927] Time spent T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.230s	user 0.000s	sys 0.000s
I20260709 13:02:24.699198  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:24.847401  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:24.856985  9007 raft_consensus.cc:2410] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:24.792364 10619 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:24.850203  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:24.850574  9008 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:24.863478 10563 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:24.645913 10666 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:24.865155  8254 raft_consensus.cc:2393] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 6: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:24.865662 10666 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:24.867637  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:24.870784  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:24.701977 10566 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:24.872732  9005 raft_consensus.cc:2410] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 7 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:24.837508 10691 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:24.855697 10657 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:24.873967  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:24.875442  8369 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:24.875702  8254 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
W20260709 13:02:24.877245 10614 consensus_meta.cc:287] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.659s	user 0.007s	sys 0.004s
I20260709 13:02:24.878371  8252 raft_consensus.cc:2410] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:24.878194  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:24.879454  8222 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:24.852062  8255 raft_consensus.cc:2376] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7. Re-sending same reply.
I20260709 13:02:24.854149  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 1925) took 2462 ms. Trace:
I20260709 13:02:24.774967 10523 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:24.774252 10652 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:24.638628 10647 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:24.882604 10477 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:24.652716 10674 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:24.699776 10683 log.cc:927] Time spent T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:02:24.643939 10626 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:24.652495 10597 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:24.893772 10597 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:24.894343 10597 raft_consensus.cc:697] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:24.895164 10597 consensus_queue.cc:237] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:24.773569 10612 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:24.793800 10433 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:24.856786  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:24.898092  8249 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
I20260709 13:02:24.898413 10566 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:24.899012  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:24.900120  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:24.901098  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:24.886675  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:24.887293  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:24.844688 10624 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:24.888698  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:24.875820 10691 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:24.891361  8298 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:24.891620 10614 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:24.892827 10674 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.893414 10626 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:24.905309  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:24.909041 10674 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:24.909241  8254 raft_consensus.cc:2393] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 5: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:24.909603 10624 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.910041 10624 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:24.878619 10658 consensus_queue.cc:1059] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:02:24.910076  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:24.610756 10574 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:24.883641  8327 rpcz_store.cc:276] 0709 13:02:22.392149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.580796 (+188647us) service_pool.cc:224] Handling call
0709 13:02:22.614395 (+ 33599us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.720475 (+106080us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:22.752598 (+ 32123us) log.cc:844] Serialized 14 byte log entry
0709 13:02:23.176233 (+423635us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:23.176335 (+   102us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:23.176405 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.853815 (+1677410us) spinlock_profiling.cc:243] Waited 89.1 ms on lock 0x7b540011eed4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:24.853886 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:24.854069 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61769,"prepare.queue_time_us":26922,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"raft.queue_time_us":27384,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":6597,"spinlock_wait_cycles":205038464,"thread_start_us":368254,"threads_started":1,"wal-append.queue_time_us":567440,"wal-append.run_cpu_time_us":1099,"wal-append.run_wall_time_us":1043008}
I20260709 13:02:24.923137 10658 raft_consensus.cc:3055] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:24.923612 10658 raft_consensus.cc:740] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:24.924526 10658 consensus_queue.cc:260] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:24.878239 10617 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:24.925789 10523 consensus_queue.cc:1059] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:02:24.926609  8295 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.718s (SENT) [suppressed 27 similar messages]
I20260709 13:02:24.926789  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:24.888811 10695 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:24.911448  8249 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:02:24.911578 10657 raft_consensus.cc:697] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:24.837414 10561 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:24.874064 10613 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:24.927829 10626 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:24.937806  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:24.906996 10687 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:24.911036 10614 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.949247  8248 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:02:24.905623 10652 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.118s
I20260709 13:02:25.037971  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:25.039156  8223 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:24.920192 10433 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:25.040557 10433 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.040952 10433 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:24.905179 10299 consensus_queue.cc:1048] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:02:24.888008 10279 consensus_queue.cc:1048] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
W20260709 13:02:24.931360  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.042747  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1628) took 1850 ms. Trace:
I20260709 13:02:25.053790  9009 raft_consensus.cc:2393] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 5: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:25.055985  8403 rpcz_store.cc:276] 0709 13:02:23.192718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.142234 (+949516us) service_pool.cc:224] Handling call
0709 13:02:24.142547 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.142745 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:24.609244 (+466499us) log.cc:844] Serialized 14 byte log entry
0709 13:02:25.042398 (+433154us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:25.042446 (+    48us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:25.042487 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.042544 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.042655 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2711,"prepare.queue_time_us":264673,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":32148,"raft.run_cpu_time_us":69,"raft.run_wall_time_us":70,"thread_start_us":884513,"threads_started":3,"wal-append.queue_time_us":316049}
I20260709 13:02:25.022987  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:02:25.066728  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 140094us
I20260709 13:02:25.083591  8253 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 3. Current term is 4. Ops: []
W20260709 13:02:25.080751  8257 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1578) took 1811 ms (client timeout 1907 ms). Trace:
I20260709 13:02:25.181159  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:25.193993  8257 rpcz_store.cc:269] 0709 13:02:23.268972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.269090 (+   118us) service_pool.cc:224] Handling call
0709 13:02:25.080521 (+1811431us) spinlock_profiling.cc:243] Waited 46.3 ms on lock 0x7b540016f654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:25.080667 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":106388736}
I20260709 13:02:25.146235  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 1935) took 2657 ms. Trace:
W20260709 13:02:25.077507  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:25.112097  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:25.138983  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 168991us
I20260709 13:02:25.084452  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:25.201468  9009 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
W20260709 13:02:25.153525  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:25.186308  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 309475us
W20260709 13:02:25.202940  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.085791  8223 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:25.075311  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 1928) took 2682 ms. Trace:
W20260709 13:02:25.204452  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 118706us
I20260709 13:02:25.124337  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:25.206107  9007 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
W20260709 13:02:25.193158  8295 connection.cc:256] RPC call timeout handler was delayed by -1.661015266999982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17220000000000002s for a response
W20260709 13:02:25.207267  8295 connection.cc:256] RPC call timeout handler was delayed by -1.6397826949999512s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1532s for a response
W20260709 13:02:25.208014  8295 connection.cc:256] RPC call timeout handler was delayed by -1.5773343169999521s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15660000000000002s for a response
W20260709 13:02:25.211203  8295 connection.cc:256] RPC call timeout handler was delayed by -1.5587678869999877s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25070000000000003s for a response
W20260709 13:02:25.211881  8295 connection.cc:256] RPC call timeout handler was delayed by -1.5289133559999755s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22240000000000004s for a response
W20260709 13:02:25.212484  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:25.214630  8295 connection.cc:256] RPC call timeout handler was delayed by -1.3550343169999337s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18130000000000002s for a response
W20260709 13:02:25.215833  8295 connection.cc:256] RPC call timeout handler was delayed by -1.3496343169999818s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1819s for a response
I20260709 13:02:25.213724  8223 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:25.033157 10695 raft_consensus.cc:697] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:25.207458  8328 rpcz_store.cc:276] 0709 13:02:22.393067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.223209 (+1830142us) service_pool.cc:224] Handling call
0709 13:02:24.233279 (+ 10070us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.265964 (+ 32685us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:24.266000 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:24.294975 (+ 28975us) log.cc:844] Serialized 26 byte log entry
0709 13:02:24.842060 (+547085us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:24.842104 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.842166 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:25.075074 (+232908us) raft_consensus.cc:1685] finished
0709 13:02:25.075096 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.075238 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20163,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":14906,"thread_start_us":381973,"threads_started":1,"wal-append.queue_time_us":656002}
I20260709 13:02:25.034677 10612 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.147358  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:25.032775 10658 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:25.081058 10387 raft_consensus.cc:3055] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:25.056077 10614 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:25.152259 10575 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:25.074669  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:25.085562 10662 log.cc:927] Time spent T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 1.851s	user 0.003s	sys 0.011s
W20260709 13:02:25.225795  8295 connection.cc:256] RPC call timeout handler was delayed by -1.153334129999962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1912s for a response
I20260709 13:02:25.248950  8326 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 13:02:25.249074  8295 connection.cc:256] RPC call timeout handler was delayed by -1.1317341299999839s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1936s for a response
W20260709 13:02:25.249447  8295 connection.cc:256] RPC call timeout handler was delayed by -1.0210341299999754s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20590000000000003s for a response
W20260709 13:02:25.249713  8295 connection.cc:256] RPC call timeout handler was delayed by -1.010234129999958s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20710000000000003s for a response
I20260709 13:02:25.250128  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:25.220371  8972 leader_election.cc:336] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 2.248s (SENT)
I20260709 13:02:25.085865 10644 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:25.075999 10711 log.cc:927] Time spent T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 13:02:25.196102  8324 rpcz_store.cc:276] 0709 13:02:22.488655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.789177 (+300522us) service_pool.cc:224] Handling call
0709 13:02:22.789564 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:23.271634 (+482070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.145875 (+1874241us) spinlock_profiling.cc:243] Waited 472 ms on lock 0x7b54003ca354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:25.146158 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1085278080}
I20260709 13:02:25.070149 10613 raft_consensus.cc:697] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:25.235229  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:25.256109  8402 raft_consensus.cc:1240] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:02:25.256392  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:25.256548  8398 raft_consensus.cc:2410] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:02:25.236555  8372 connection.cc:256] RPC call timeout handler was delayed by -1.2212485300000253s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16620000000000001s for a response
I20260709 13:02:25.258836  8372 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:25.241226  8369 connection.cc:256] RPC call timeout handler was delayed by -1.0973483579999765s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2202s for a response
I20260709 13:02:25.243588  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:25.260510  8400 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
I20260709 13:02:25.261780  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:25.073346 10511 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:25.110590 10626 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.084717 10566 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:25.247223  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:25.070384 10704 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.194913  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:25.251551  8972 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:25.223850  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:25.322479  9006 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:25.083649 10574 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:25.221029 10477 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:02:25.263055  8369 connection.cc:256] RPC call timeout handler was delayed by -1.0874483580000174s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22130000000000002s for a response
I20260709 13:02:25.227092 10474 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:25.254312  9003 raft_consensus.cc:1813] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Handling vote request from an unknown peer 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:25.226892 10719 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.116564 10617 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.129s
I20260709 13:02:25.265259  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:25.226935 10387 raft_consensus.cc:740] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:25.246104 10717 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.084949 10665 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.170s
I20260709 13:02:25.085182 10560 consensus_queue.cc:1059] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.168s
I20260709 13:02:25.085413 10691 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.166s
I20260709 13:02:25.268666  8372 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:25.249804 10299 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:02:25.329610  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 449094us
I20260709 13:02:25.335879  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:25.344271  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 123929us
I20260709 13:02:25.344883  9000 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:25.273699  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:25.375007 10721 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:02:25.371518  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:25.375280  8327 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:25.375882  9009 raft_consensus.cc:2410] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:25.085631 10657 consensus_queue.cc:237] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:25.370867  9003 raft_consensus.cc:2368] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 4. Current term is 5.
I20260709 13:02:25.377467 10561 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:25.378206 10561 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:25.230692 10612 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:25.378578 10561 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.378939 10561 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:25.379280  9003 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:02:25.380290 10612 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:25.257524 10605 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.095s
I20260709 13:02:25.380990  9003 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:25.342114 10666 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:25.368857  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:25.374197  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.257705  9971 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.097s
I20260709 13:02:25.382517  9009 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:02:25.411829  9009 pending_rounds.cc:85] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: Aborting all ops after (but not including) 6
I20260709 13:02:25.385725 10613 consensus_queue.cc:237] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:25.341975 10713 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.269526 10644 consensus_queue.cc:1048] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.093s
I20260709 13:02:25.322643 10511 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:25.258105 10716 consensus_queue.cc:1059] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.114s
I20260709 13:02:25.437544 10716 raft_consensus.cc:3055] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:25.437968 10716 raft_consensus.cc:740] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:24.586725  8323 rpcz_store.cc:276] 0709 13:02:23.245096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.314062 (+1068966us) service_pool.cc:224] Handling call
0709 13:02:24.314362 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.314578 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:24.318822 (+  4244us) log.cc:844] Serialized 14 byte log entry
0709 13:02:24.454789 (+135967us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:24.454857 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:24.454919 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.455005 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:24.455148 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3755,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":427,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":126316,"thread_start_us":135157,"threads_started":1,"wal-append.queue_time_us":112585}
I20260709 13:02:25.406818  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:25.441016  8398 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
W20260709 13:02:25.411986  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.412106  9009 pending_rounds.cc:107] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: Aborting uncommitted NO_OP operation due to leader change: 6.7
W20260709 13:02:25.400758 10720 log.cc:927] Time spent T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 13:02:25.450863  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1616) took 3226 ms. Trace:
I20260709 13:02:25.384833 10433 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:25.361362 10687 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:25.428758  8257 raft_consensus.cc:1275] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:25.428925  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:25.458061  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:25.458683  8255 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:25.459758  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:25.460354  8255 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:25.429080  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:25.462210  8402 raft_consensus.cc:2410] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:25.429577 10713 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:25.431730  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.436640 10511 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:25.351963 10566 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.442229  9009 raft_consensus.cc:2889] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:25.323048 10626 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:25.444927  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:25.451354  8399 rpcz_store.cc:276] 0709 13:02:22.197184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.486814 (+289630us) service_pool.cc:224] Handling call
0709 13:02:22.487131 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:22.536453 (+ 49322us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:02:22.536503 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:22.544494 (+  7991us) log.cc:844] Serialized 26 byte log entry
0709 13:02:23.026286 (+481792us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:23.026329 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.026402 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:25.085023 (+2058621us) spinlock_profiling.cc:243] Waited 4.87 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:25.392966 (+307943us) raft_consensus.cc:1685] finished
0709 13:02:25.393024 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.422991 (+ 29967us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":59020,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":9218,"spinlock_wait_cycles":11209600,"thread_start_us":418613,"threads_started":1,"wal-append.queue_time_us":651687}
I20260709 13:02:25.370219  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1560) took 2240 ms. Trace:
W20260709 13:02:25.455228  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.338555 10704 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:25.487733  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:25.489046  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:25.468554  8972 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:25.357612 10719 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:25.487413  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.496136  8255 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:25.498059  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:25.438972 10716 consensus_queue.cc:260] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:25.452260 10607 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:25.488091 10730 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.091s	user 0.000s	sys 0.002s
I20260709 13:02:25.480628  8250 rpcz_store.cc:276] 0709 13:02:23.121221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.136158 (+ 14937us) service_pool.cc:224] Handling call
0709 13:02:25.329506 (+2193348us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b540009ecd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:25.358743 (+ 29237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":947793280}
I20260709 13:02:25.480386 10511 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:25.486333 10279 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.510524  8254 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:25.511183 10477 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.511346  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:25.486496 10523 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:25.501317 10581 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:02:25.566282  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 315058us
I20260709 13:02:25.525789  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:25.535033 10626 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:25.422405 10666 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:25.373836 10695 consensus_queue.cc:237] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:25.540627  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:25.541699 10650 raft_consensus.cc:3055] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:25.545509 10433 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:25.545200  8252 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1553) took 1836 ms (client timeout 1718 ms). Trace:
I20260709 13:02:25.549808 10603 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:25.516309 10714 log.cc:927] Time spent T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 13:02:25.368010 10717 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:25.559201  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 290538us
I20260709 13:02:25.412261 10299 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:25.454895 10387 consensus_queue.cc:260] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:25.564164  9006 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:25.519416 10719 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:25.569360  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 234075us
I20260709 13:02:25.486328 10566 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:25.576116  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.721561  8372 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:25.736917  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:25.808562  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:25.817296  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 96268.8us
W20260709 13:02:25.720348  8252 rpcz_store.cc:269] 0709 13:02:23.708573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.708701 (+   128us) service_pool.cc:224] Handling call
0709 13:02:25.544858 (+1836157us) spinlock_profiling.cc:243] Waited 751 ms on lock 0x7b54003471d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:25.545082 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1727037696}
I20260709 13:02:25.722681  8295 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:02:25.820278  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 97636.1us
I20260709 13:02:25.821200  8295 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:25.822682  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:25.821247  8372 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:25.824613  8402 raft_consensus.cc:2393] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 5: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:02:25.568406 10674 consensus_meta.cc:287] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.659s	user 0.012s	sys 0.000s
W20260709 13:02:25.825208  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:25.825649  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.828972  8328 raft_consensus.cc:2410] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:02:25.567977 10581 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.831969  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:25.833376  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:25.834692  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.562960 10523 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:25.833371  8223 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:25.837548  8398 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:25.566172 10279 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:25.828353 10560 consensus_queue.cc:1175] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e64761f43e4240319c41117cf886d22f" responder_term: 4 status { last_received { term: 4 index: 3 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:25.565049 10477 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:25.615761 10716 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:25.833726  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:25.558039 10607 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:25.823022 10613 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:02:25.867575  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.828415 10387 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:25.871948  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 50815.2us
I20260709 13:02:25.886014  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:25.833590 10581 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:25.883559  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:25.864617  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.819885  9971 consensus_queue.cc:1048] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:02:25.895479 10279 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:25.896660  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:25.897678 10602 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:25.830750 10650 raft_consensus.cc:740] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:25.833869 10674 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:25.899751 10561 consensus_meta.cc:287] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.520s	user 0.002s	sys 0.007s
W20260709 13:02:25.901995 10624 consensus_meta.cc:287] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.968s	user 0.008s	sys 0.003s
I20260709 13:02:25.876192 10737 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:02:25.897693  9002 consensus_meta.cc:287] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.673s	user 0.002s	sys 0.009s
W20260709 13:02:25.907869  8297 connection.cc:256] RPC call timeout handler was delayed by -1.1119151599999668s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18910000000000002s for a response
I20260709 13:02:25.908630 10732 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.911278  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:25.911741 10607 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:25.914288 10717 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:25.820052 10605 consensus_queue.cc:1059] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:02:25.927351  8295 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:25.835016 10626 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:25.929010  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.844424 10523 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:25.899597 10477 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:25.848295 10598 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.838771 10719 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:25.849541 10474 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:25.842164 10734 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:25.944476  8400 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.023s	sys 0.066s
I20260709 13:02:25.932942 10591 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.947430  8252 raft_consensus.cc:2368] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5.
I20260709 13:02:25.932574 10602 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:02:25.960758  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 139526us
I20260709 13:02:25.941768 10433 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.946797  8250 raft_consensus.cc:2368] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6.
I20260709 13:02:25.939663  9002 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
W20260709 13:02:25.972393  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.942867 10624 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:25.957906 10474 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:25.943182 10657 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.020s	sys 0.031s
W20260709 13:02:25.973321  8154 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45448 (request call id 282) took 15015 ms (client timeout 15000 ms). Trace:
I20260709 13:02:25.956727  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:25.943399 10561 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:25.963089  9971 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:25.963580 10602 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:25.957319  8400 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.023s	sys 0.067s
I20260709 13:02:25.944029 10737 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:25.962990 10511 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:25.966109 10719 raft_consensus.cc:697] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:25.946099 10732 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:25.958132 10598 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:25.966876  9004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56940 (request call id 579) took 1106 ms. Trace:
I20260709 13:02:25.981020 10474 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:26.017226  9971 raft_consensus.cc:697] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:25.981151  9002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56940 (request call id 585) took 1088 ms. Trace:
I20260709 13:02:25.963361 10591 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:26.017366 10719 consensus_queue.cc:237] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:26.018738 10213 raft_consensus.cc:3055] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:25.961694 10734 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:26.002434  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:26.045444  8404 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
W20260709 13:02:25.981420 10657 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.020s	sys 0.031s
I20260709 13:02:26.027330 10299 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:26.050843 10511 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:26.052117  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:26.019999 10737 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:25.973773 10433 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:26.056344 10433 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:26.062511  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:26.062985  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1554) took 2354 ms. Trace:
I20260709 13:02:26.063174  8404 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:02:26.063355  8253 rpcz_store.cc:276] 0709 13:02:23.708884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.116974 (+408090us) service_pool.cc:224] Handling call
0709 13:02:25.083405 (+966431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:25.195037 (+111632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.062765 (+867728us) spinlock_profiling.cc:243] Waited 928 ms on lock 0x7b540016f650. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:26.062905 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2133501184}
I20260709 13:02:26.064211 10624 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:25.981633  8154 rpcz_store.cc:269] 0709 13:02:10.947988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.948163 (+   175us) service_pool.cc:224] Handling call
0709 13:02:25.962934 (+15014771us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:17.641457 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:17.642081 (+   624us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:17.642127 (+    46us) write_op.cc:435] Acquired schema lock
0709 13:02:17.642141 (+    14us) tablet.cc:662] Decoding operations
0709 13:02:17.785021 (+142880us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:17.789670 (+  4649us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:17.791924 (+  2254us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:02:17.867922 (+ 75998us) tablet.cc:701] Row locks acquired
0709 13:02:17.867937 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:02:17.868150 (+   213us) write_op.cc:270] Start()
0709 13:02:17.868293 (+   143us) write_op.cc:276] Timestamp: P: 1783602137868047 usec, L: 0
0709 13:02:17.868318 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:02:18.004344 (+136026us) log.cc:844] Serialized 6347 byte log entry
0709 13:02:19.492198 (+1487854us) op_driver.cc:464] REPLICATION: finished
0709 13:02:19.708625 (+216427us) write_op.cc:301] APPLY: starting
0709 13:02:19.708749 (+   124us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:19.709231 (+   482us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:19.709446 (+   215us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:19.734434 (+ 24988us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:19.734863 (+   429us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 13:02:19.734909 (+    46us) write_op.cc:312] APPLY: finished
0709 13:02:19.735478 (+   569us) log.cc:844] Serialized 113 byte log entry
0709 13:02:19.735680 (+   202us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:19.735986 (+   306us) write_op.cc:454] Released schema lock
0709 13:02:19.736071 (+    85us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:19.768302 (+ 32231us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:19.768331 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6638,"reported_tablets":29,"child_traces":[["op",{"apply.queue_time_us":216016,"apply.run_cpu_time_us":5850,"apply.run_wall_time_us":59635,"num_ops":16,"prepare.queue_time_us":229263,"prepare.run_cpu_time_us":87137,"prepare.run_wall_time_us":729465,"raft.queue_time_us":130644,"raft.run_cpu_time_us":15582,"raft.run_wall_time_us":164165,"replication_time_us":1614356,"thread_start_us":559860,"threads_started":4,"wal-append.queue_time_us":1258954,"wal-append.run_cpu_time_us":36967,"wal-append.run_wall_time_us":1438419}]]}
I20260709 13:02:26.028481 10717 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:26.018784 10742 log.cc:927] Time spent T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:02:26.066826 10717 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:26.029594 10674 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:26.068930 10717 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:24.596294  8397 rpcz_store.cc:276] 0709 13:02:23.366349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.200181 (+833832us) service_pool.cc:224] Handling call
0709 13:02:24.216805 (+ 16624us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.217006 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:24.372350 (+155344us) log.cc:844] Serialized 14 byte log entry
0709 13:02:24.595276 (+222926us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:24.595344 (+    68us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:24.595415 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.595522 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:24.595671 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":190813,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":32,"raft.queue_time_us":101,"thread_start_us":367528,"threads_started":2}
W20260709 13:02:26.070943  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.975718  8245 tablet_service.cc:1558] Processing DeleteTablet for tablet ae82ced6ff014453bbcfd5f9f14b1f8d with delete_type TABLET_DATA_TOMBSTONED (TS 8c42259d7c4b463a9f485f42c320c81b not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:55248
I20260709 13:02:26.072566  8222 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:26.044040 10213 raft_consensus.cc:740] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:26.040975  9002 rpcz_store.cc:276] 0709 13:02:24.890614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.929466 (+ 38852us) service_pool.cc:224] Handling call
0709 13:02:25.978875 (+1049409us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:26.070727  8252 raft_consensus.cc:1240] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 2. Current term is 3. Ops: [2.5-2.5]
W20260709 13:02:26.019342 10739 log.cc:927] Time spent T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:02:26.023759  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:26.031901 10666 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:26.077111  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 149772us
W20260709 13:02:26.084645  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 142350us
I20260709 13:02:26.084793  8295 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:02:26.073459 10658 consensus_meta.cc:287] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.828s	user 0.006s	sys 0.006s
I20260709 13:02:26.089622  8369 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:26.072943 10566 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:26.098981  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:26.077171 10674 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:26.097656  8295 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:25.981757 10650 consensus_queue.cc:260] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:26.101847  8369 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:26.102782  8397 raft_consensus.cc:2410] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:26.105458  8295 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:26.106549  8295 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:26.030947  9004 rpcz_store.cc:276] 0709 13:02:24.860504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.882042 (+ 21538us) service_pool.cc:224] Handling call
0709 13:02:25.911155 (+1029113us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b5800175898. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699eee 0000000000375939 00007f882d70d1e0 00007f882d7d60c4 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:02:25.965250 (+ 54095us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24374528}
I20260709 13:02:26.094854 10511 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.123818  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:26.124488  8397 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:26.100688 10674 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.134862 10523 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:26.135402 10674 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:26.136406 10626 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:26.138134 10523 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:26.105723 10613 raft_consensus.cc:697] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:26.139189 10713 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:26.139251 10613 consensus_queue.cc:237] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:26.140511 10756 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:26.143486 10732 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:26.144837 10598 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:26.146427 10750 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:26.146984 10750 raft_consensus.cc:697] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:26.148948  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:26.098927 10737 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:26.129671 10751 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:26.190771 10751 raft_consensus.cc:697] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:26.191779  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 328) took 1337 ms. Trace:
W20260709 13:02:26.130462 10745 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:02:26.130744  8357 heartbeater.cc:646] Failed to heartbeat to 127.6.255.62:40337 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.6.255.62:40337 timed out after 15.000s (SENT)
I20260709 13:02:26.131336 10627 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:26.191634 10751 consensus_queue.cc:237] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:26.132342 10575 consensus_meta.cc:287] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.884s	user 0.002s	sys 0.009s
I20260709 13:02:26.133360  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:26.134088 10614 consensus_meta.cc:287] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.886s	user 0.009s	sys 0.002s
I20260709 13:02:26.134236  8222 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:02:26.228760  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 94553us
I20260709 13:02:26.111768 10650 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:26.134974 10734 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:26.154767 10359 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:26.154697 10561 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:26.111943 10213 consensus_queue.cc:260] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:26.235656 10757 tablet_replica.cc:333] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:02:26.236137 10213 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:26.163421  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:26.169982  9971 consensus_queue.cc:237] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:26.239076  8403 raft_consensus.cc:2410] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:02:26.239173  8400 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 8. Ops: []
I20260709 13:02:26.175218  8404 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:26.175915 10511 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:26.178217  8402 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:26.125530  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:26.187085  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.188025 10474 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:26.147933 10750 consensus_queue.cc:237] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:26.160739 10581 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:26.229496 10737 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.231230 10675 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:26.226416  8399 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:26.230680 10749 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:02:26.229514  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.235140 10382 raft_consensus.cc:3055] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:26.229596 10603 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.171005  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:26.236903 10757 raft_consensus.cc:2243] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:26.262866  8397 raft_consensus.cc:2410] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 8 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
W20260709 13:02:26.243745  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.192312  8257 rpcz_store.cc:276] 0709 13:02:24.853938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.220853 (+366915us) service_pool.cc:224] Handling call
0709 13:02:25.428428 (+207575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:25.457160 (+ 28732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.191509 (+734349us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b54003fdc50. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:26.191706 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":450472704}
I20260709 13:02:26.246116 10474 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:26.246769  8249 raft_consensus.cc:2393] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 4: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:02:26.247677  8406 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 1626) took 3055 ms (client timeout 2898 ms). Trace:
W20260709 13:02:26.265000  8406 rpcz_store.cc:269] 0709 13:02:23.192258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.493968 (+301710us) service_pool.cc:224] Handling call
0709 13:02:26.247483 (+2753515us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b54003a1bd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:26.247601 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1216478976}
I20260709 13:02:26.227798  8251 raft_consensus.cc:2410] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:26.253121  8403 raft_consensus.cc:1240] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:26.230631 10614 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:26.254571  8222 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:26.258023  8399 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:26.260356  8402 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
W20260709 13:02:26.261058  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.238782 10604 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:26.242797  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 117320us
I20260709 13:02:26.278090  8369 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:26.278797  8399 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:26.279259  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:26.280432  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:26.261114 10382 raft_consensus.cc:740] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:02:26.283936  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 177780us
W20260709 13:02:26.254619  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 148079us
I20260709 13:02:26.249706 10675 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.248741 10561 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:26.288858  8295 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:26.285005  8222 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:26.316282 10734 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:26.316818 10734 raft_consensus.cc:697] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:02:26.298640  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.317826 10734 consensus_queue.cc:237] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:26.292256  8248 raft_consensus.cc:1240] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:02:26.290283  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 1945) took 2144 ms. Trace:
I20260709 13:02:26.289353  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:26.253664 10737 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:26.310717  8297 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:26.277695 10647 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.268074 10603 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:26.340317  8330 rpcz_store.cc:276] 0709 13:02:24.145576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.419690 (+274114us) service_pool.cc:224] Handling call
0709 13:02:24.428649 (+  8959us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:02:25.042331 (+613682us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:25.042468 (+   137us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:02:25.045105 (+  2637us) log.cc:844] Serialized 78 byte log entry
0709 13:02:25.443549 (+398444us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:25.481093 (+ 37544us) log.cc:844] Serialized 14 byte log entry
0709 13:02:25.481335 (+   242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.282402 (+801067us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b540037b254. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:26.282454 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:26.282504 (+    50us) raft_consensus.cc:1685] finished
0709 13:02:26.290001 (+  7497us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.290201 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10549,"prepare.queue_time_us":37904,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":53309,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":54962,"spinlock_wait_cycles":383807360,"thread_start_us":832726,"threads_started":2,"wal-append.queue_time_us":541257}
W20260709 13:02:26.339699  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 50906.5us
I20260709 13:02:26.283840 10716 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:26.338201  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.344106  8295 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:26.311095 10675 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:26.283728 10613 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:26.283941 10753 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:26.354784  8297 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:26.289682 10474 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:26.312976 10359 consensus_queue.cc:1059] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:02:26.359134 10511 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:26.338683 10602 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:26.361254 10602 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:26.363149 10704 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:26.363333 10561 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:26.262764 10749 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:26.366451  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:26.353833 10758 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:26.367319  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.367063  8403 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:26.367749  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1625) took 3175 ms. Trace:
W20260709 13:02:26.369292  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 79976.3us
I20260709 13:02:26.372512  8295 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:26.372994  8369 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:26.375020  8403 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:02:26.376487  8403 pending_rounds.cc:85] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 3
I20260709 13:02:26.376576  8369 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:26.376765  8403 pending_rounds.cc:107] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:02:26.309998 10691 consensus_queue.cc:1059] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:02:26.353754 10757 raft_consensus.cc:2272] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:26.353789 10716 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:26.355980 10751 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:26.383543  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:26.377138  8403 raft_consensus.cc:2889] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:26.357378 10474 raft_consensus.cc:697] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:26.356729 10613 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:26.414366 10674 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:26.414983 10613 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:26.400599 10359 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:26.400332  8295 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:26.377985 10732 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:26.355340 10647 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:26.399462  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:26.413007  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:26.427991  8257 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:26.386920 10602 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:26.399644 10756 raft_consensus.cc:3055] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:26.382807  8369 leader_election.cc:400] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6.
I20260709 13:02:26.418290  8295 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:26.418996 10751 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:26.378233 10704 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.420258  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:26.439021  8403 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 5
W20260709 13:02:26.387374  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.382833 10511 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:26.380657 10624 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:26.414732 10591 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:26.410975 10716 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:26.432890  8369 leader_election.cc:403] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:02:26.434788  8295 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:02:26.418911  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.426304 10674 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:26.371976  8401 rpcz_store.cc:276] 0709 13:02:23.191923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.132335 (+940412us) service_pool.cc:224] Handling call
0709 13:02:24.132669 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.133104 (+   435us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:26.165005 (+2031901us) spinlock_profiling.cc:243] Waited 46.3 ms on lock 0x7b180037eac0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88283664bc 00007f88256b448b 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256ce455 00007f882c1b63c7 00007f88287c89a7 00007f88287d80bf
0709 13:02:26.165166 (+   161us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.367431 (+202265us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:26.367482 (+    51us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:26.367524 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.367585 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.367700 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10567,"prepare.queue_time_us":224917,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":10680,"spinlock_wait_cycles":106424576,"thread_start_us":461160,"threads_started":2,"wal-append.queue_time_us":187715}
W20260709 13:02:26.417923 10762 log.cc:927] Time spent T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:02:26.435971 10644 raft_consensus.cc:2764] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:02:26.461457 10644 consensus_queue.cc:1059] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:26.462467 10644 raft_consensus.cc:3055] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:26.445084 10614 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:26.427907 10359 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:26.442533 10704 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:26.414790 10474 consensus_queue.cc:237] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:26.449620  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:26.450176  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 67473.1us
I20260709 13:02:26.450505  8223 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:26.432607 10756 raft_consensus.cc:740] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:26.440333 10751 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:26.447046 10511 raft_consensus.cc:697] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:26.451669 10719 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:26.490844  8324 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:02:26.491209  8223 leader_election.cc:400] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 4. Current term is 5.
I20260709 13:02:26.455416  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:26.446995 10746 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6.
I20260709 13:02:26.456852 10213 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.457708  9002 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:26.458071 10695 consensus_queue.cc:1059] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:26.456882  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:26.504504  8324 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:26.460673  8295 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:26.505317  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.505949  8323 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:26.464232  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:26.467710  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.462787 10644 raft_consensus.cc:740] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:26.477270  9005 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:26.476447  8257 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
I20260709 13:02:26.482997  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.484763  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1935) took 1065 ms. Trace:
W20260709 13:02:26.510788  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 50167us
I20260709 13:02:26.511741  8295 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:26.511066  8404 rpcz_store.cc:276] 0709 13:02:25.418990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.239926 (+820936us) service_pool.cc:224] Handling call
0709 13:02:26.241674 (+  1748us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:26.244639 (+  2965us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:26.244669 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:26.245144 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 13:02:26.322354 (+ 77210us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:26.322394 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.322458 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:26.484534 (+162076us) raft_consensus.cc:1685] finished
0709 13:02:26.484556 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.484695 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":321,"raft.queue_time_us":1066,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":75,"thread_start_us":76688,"threads_started":1,"wal-append.queue_time_us":172132}
I20260709 13:02:26.446422 10602 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:26.451200 10581 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:26.492638  8223 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:26.517462  8332 raft_consensus.cc:1275] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:26.517915  8223 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:26.492789  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:26.454798 10523 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.454370 10579 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:26.498791  8331 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:02:26.500949  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 122096us
I20260709 13:02:26.501691  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.542404  8297 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:02:26.506871  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.480937  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:26.547057  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:26.513957  8323 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:02:26.515801  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.547717  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.494877 10674 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:26.493911 10603 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:26.556602  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:26.542637  8328 raft_consensus.cc:2410] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:02:26.509052  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.543996  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:26.546142  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:26.548278  8326 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
I20260709 13:02:26.548353  8325 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
I20260709 13:02:26.552273  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.553532  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.554188  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:26.553668  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.555038  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:26.556402 10767 log.cc:927] Time spent T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
W20260709 13:02:26.530112  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.497088 10614 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:02:26.510026  9004 raft_consensus.cc:1275] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:26.578470  8324 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 7: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:26.556708 10624 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:26.501318 10691 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:02:26.513985  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:26.552099 10575 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:26.573356  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:26.572163 10760 log.cc:927] Time spent T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:02:26.595180  8369 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:26.520642 10759 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:26.586367  8257 raft_consensus.cc:2368] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5.
I20260709 13:02:26.587532  8330 raft_consensus.cc:2393] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 3: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:26.589680  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.599637  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.600411  8324 raft_consensus.cc:2410] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:26.601116  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.603489  8324 raft_consensus.cc:2393] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 4: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:26.541478 10734 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:26.503499 10213 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:26.591593  8331 raft_consensus.cc:2393] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 5: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:26.587920  8253 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:26.549322 10746 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:26.582866  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.593529  8250 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:26.583606  8329 raft_consensus.cc:2393] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 3: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:26.585379  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.613453  8328 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 3.
I20260709 13:02:26.601701  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:26.578367  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.605036  8369 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:26.577557 10749 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:26.541695  9971 consensus_queue.cc:1059] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 13:02:26.520466 10359 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:26.614056  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.625766  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:26.613483  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.614548  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.626362  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.588498 10624 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.520030 10602 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:26.594626 10603 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:26.619257  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:26.630169  8253 raft_consensus.cc:2393] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 5: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:26.619917  8323 raft_consensus.cc:2410] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:02:26.633813  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:26.553072 10523 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:26.635842  8252 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:26.636775  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:26.637072  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:26.595568  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:26.625006  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.615911  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.615269 10747 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:26.627581  8330 raft_consensus.cc:2393] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 5: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:26.616835  9001 raft_consensus.cc:2410] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:02:26.629104  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.637526  8253 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:26.651504  8331 raft_consensus.cc:1275] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:26.639264  8329 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
I20260709 13:02:26.652820  8329 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:26.652850  8331 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:26.654070  8329 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:26.612520 10737 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:26.552894 10579 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:26.639910  8252 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:26.628415  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:26.645304  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.658491  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:26.659027  8328 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:26.659978  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:26.660251  8328 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:26.626372  8257 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:26.662613  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:26.620718  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:26.619273 10752 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:26.664036  8324 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:02:26.572993 10511 consensus_queue.cc:237] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:26.651535  8326 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
I20260709 13:02:26.602286 10695 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:02:26.673040 10695 consensus_queue.cc:1243] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:02:26.653607  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:26.675619  8330 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:26.677167  8330 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:26.656746  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:26.679342  8250 raft_consensus.cc:1275] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:26.680601  8250 raft_consensus.cc:1275] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:26.682273  8250 raft_consensus.cc:1275] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:26.684777  9006 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:26.625000 10734 raft_consensus.cc:697] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:26.635310 10624 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:26.692731  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.630467 10749 consensus_queue.cc:1059] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:02:26.689607  8327 raft_consensus.cc:2410] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:02:26.592552 10644 consensus_queue.cc:260] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:26.629456 10746 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:26.645063 10747 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:26.702293  8325 pending_rounds.cc:85] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 5
I20260709 13:02:26.719195  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.704034  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:26.696496  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.734053  8331 raft_consensus.cc:2410] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:02:26.737061  8403 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:26.740104  8324 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:26.741299  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:26.742197  8324 raft_consensus.cc:2393] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 3: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:02:26.742157  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 120953us
I20260709 13:02:26.742839  8325 pending_rounds.cc:107] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 4.6
W20260709 13:02:26.741570  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 123364us
I20260709 13:02:26.743358  8325 raft_consensus.cc:2889] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:26.743638  8369 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:26.752655  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.754802  8330 raft_consensus.cc:2393] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 4: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:26.716719  9003 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:26.758884  9007 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:26.759457  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.761505  8404 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 8. Ops: []
I20260709 13:02:26.741636 10747 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:26.764930  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:26.714010  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:26.765612  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:26.754300 10768 log.cc:927] Time spent T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 13:02:26.740985 10581 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.768502  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:26.766341  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 329) took 1912 ms. Trace:
I20260709 13:02:26.754673 10757 ts_tablet_manager.cc:1916] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:26.772365  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:26.747141  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:26.786182  8331 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
W20260709 13:02:26.794533  8295 connection.cc:256] RPC call timeout handler was delayed by -1.7887471010000127s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22599999999999998s for a response
I20260709 13:02:26.763095 10644 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:26.777654 10766 log.cc:927] Time spent T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.263s	user 0.000s	sys 0.001s
I20260709 13:02:26.770015  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:26.773231 10737 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:26.798130 10781 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:26.759516 10704 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:26.800541  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:26.802819  8330 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:02:26.823956 10213 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:26.776885  8255 rpcz_store.cc:276] 0709 13:02:24.854172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.503373 (+649201us) service_pool.cc:224] Handling call
0709 13:02:25.503761 (+   388us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:25.520455 (+ 16694us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:25.520503 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:25.521193 (+   690us) log.cc:844] Serialized 52 byte log entry
0709 13:02:25.827279 (+306086us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:25.879918 (+ 52639us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.453820 (+573902us) spinlock_profiling.cc:243] Waited 25.7 ms on lock 0x7b180011ac40. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256ce455 00007f88287b6060 00007f88287c8716 00007f88287d9287 00007f88287d9208
0709 13:02:26.741954 (+288134us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.748901 (+  6947us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.748986 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:26.766073 (+ 17087us) raft_consensus.cc:1685] finished
0709 13:02:26.766103 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.766262 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15858,"prepare.queue_time_us":238645,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":58998,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":4617,"spinlock_wait_cycles":59174144,"thread_start_us":593534,"threads_started":3,"wal-append.queue_time_us":425788}
I20260709 13:02:26.795729 10602 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:26.817516  8403 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:26.807469  9002 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:26.838562  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:26.769251 10753 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:02:26.791932  9005 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:26.838377  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:26.841526  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 73073.1us
I20260709 13:02:26.846764  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:26.847675  8255 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
W20260709 13:02:26.848224  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.859182  9008 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:26.861848  8331 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:26.864583 10523 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:26.865538  9005 pending_rounds.cc:85] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e: Aborting all ops after (but not including) 4
I20260709 13:02:26.865881  9005 pending_rounds.cc:107] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:02:26.866163  9005 raft_consensus.cc:2889] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:26.801409 10749 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 13:02:26.793642 10581 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:26.820066 10737 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:02:26.875666  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.822249 10781 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.883076 10713 raft_consensus.cc:3055] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:26.883216 10603 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:26.883587 10713 raft_consensus.cc:740] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:26.885267 10433 raft_consensus.cc:3055] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:26.885696 10433 raft_consensus.cc:740] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:26.885842 10713 consensus_queue.cc:260] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:26.886502 10433 consensus_queue.cc:260] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:26.887707 10433 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:26.887993 10713 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:26.893246 10781 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:26.895857  8372 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:26.896822  8372 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:26.897068 10756 consensus_queue.cc:260] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:26.897575 10511 consensus_queue.cc:1048] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:26.897975 10579 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:26.898386 10756 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:26.899626 10511 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:26.899428 10613 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:26.900223 10511 raft_consensus.cc:697] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:26.881096 10737 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:26.901351 10603 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:26.902043 10603 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.902431 10603 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:26.901288 10511 consensus_queue.cc:237] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:26.806365 10775 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.075s
I20260709 13:02:26.834779  9001 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:26.862819  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:26.915848  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 53107.1us
I20260709 13:02:26.819351 10751 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:26.919399  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:02:26.907204 10784 log.cc:927] Time spent T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:02:26.801024 10734 consensus_queue.cc:237] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:26.888080 10647 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:26.950433  8369 leader_election.cc:336] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.628s (SENT)
I20260709 13:02:26.950598  8972 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:26.953238  8972 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:26.954310  8369 leader_election.cc:336] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.666s (SENT)
I20260709 13:02:26.954732  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:26.954887  8972 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:26.956190 10782 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:26.956363 10213 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:26.956449 10579 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:26.958803  8369 leader_election.cc:336] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.710s (SENT)
I20260709 13:02:26.959255  8369 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:26.962044  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.925387 10613 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:26.964700 10647 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:26.964968 10782 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.975060 10579 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.998574 10782 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:26.998857 10579 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:26.975176 10561 consensus_meta.cc:287] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.611s	user 0.009s	sys 0.003s
I20260709 13:02:27.001323 10561 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:27.014854  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:27.009756 10785 log.cc:927] Time spent T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:02:26.956391 10746 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:26.976285 10511 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:26.975123  9003 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:26.965139 10213 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:27.034943 10675 consensus_meta.cc:287] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.665s	user 0.007s	sys 0.004s
I20260709 13:02:27.042182  9000 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:26.971612 10704 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:02:27.015455 10758 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:27.073841  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 272986us
I20260709 13:02:27.073148  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:27.074648  8254 raft_consensus.cc:2376] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4. Re-sending same reply.
W20260709 13:02:27.075487  8254 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1590) took 1243 ms (client timeout 1628 ms). Trace:
I20260709 13:02:27.080372  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:27.075467  8295 leader_election.cc:336] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 2.260s (SENT)
W20260709 13:02:27.081246  8972 leader_election.cc:336] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.515s (SENT)
W20260709 13:02:27.079206  8254 rpcz_store.cc:269] 0709 13:02:25.832253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.072877 (+1240624us) service_pool.cc:224] Handling call
0709 13:02:27.075400 (+  2523us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:27.081676  8295 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:27.084074  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:27.084378  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.086642  8254 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:27.071127 10523 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:27.087808  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:27.070319 10581 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:27.070214 10604 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:02:27.094129 10783 log.cc:927] Time spent T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:02:27.091141  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:27.091214  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:27.090343  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:27.103624  8324 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:27.103808  8328 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:27.113487  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:27.113615  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:27.114439  8330 raft_consensus.cc:2393] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 3: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:27.086506 10627 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:27.072239 10675 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:27.089726 10511 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:27.080880 10781 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:27.107877 10581 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:27.104743 10213 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:27.076393 10758 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:27.104964 10604 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:27.135092  8250 raft_consensus.cc:2368] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5.
I20260709 13:02:27.138104  9002 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:27.140636 10603 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:27.127095 10561 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
W20260709 13:02:27.146349 10786 log.cc:927] Time spent T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
W20260709 13:02:27.146860 10790 log.cc:927] Time spent T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:02:27.165575  8398 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1936) took 1743 ms (client timeout 1519 ms). Trace:
W20260709 13:02:27.166210  8398 rpcz_store.cc:269] 0709 13:02:25.422120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.499603 (+ 77483us) service_pool.cc:224] Handling call
0709 13:02:27.165269 (+1665666us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b5400067f54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:27.165465 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":638391040}
I20260709 13:02:27.167212  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:27.150738 10213 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:27.172642 10213 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:27.174307 10781 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:27.176717  8251 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1569) took 2717 ms (client timeout 2260 ms). Trace:
I20260709 13:02:27.178028  8249 raft_consensus.cc:1275] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:02:27.177104  8251 rpcz_store.cc:269] 0709 13:02:24.459569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.462786 (+  3217us) service_pool.cc:224] Handling call
0709 13:02:27.176396 (+2713610us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54002eead4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:27.176643 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3728162176}
I20260709 13:02:27.181383  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:27.182044  8251 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:27.182849  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:27.183560  8251 raft_consensus.cc:2393] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 3: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:27.184291  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:27.184841  8251 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:27.185971 10381 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:27.188104 10624 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:27.189220  8326 raft_consensus.cc:1240] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:27.193163  8298 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:27.133869 10511 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:27.198153  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:27.150257 10581 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:27.213967  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:27.214740  9008 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:27.215235  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:27.219039  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:27.220090  8251 raft_consensus.cc:2410] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:27.221449  8251 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:02:27.223544  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:27.225147  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:27.225945  8252 raft_consensus.cc:2393] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 3: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:27.224229  9009 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:27.226965  9006 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 8. Ops: [6.6-6.6]
I20260709 13:02:27.229161  8369 leader_election.cc:400] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5.
I20260709 13:02:27.229610  8369 leader_election.cc:403] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:27.230795  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:27.203632 10782 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:27.237560  8974 connection.cc:256] RPC call timeout handler was delayed by -2.0323632829999951s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1683s for a response
W20260709 13:02:27.238168  8974 connection.cc:256] RPC call timeout handler was delayed by -1.8901632829999926s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1841s for a response
W20260709 13:02:27.238803  8974 connection.cc:256] RPC call timeout handler was delayed by -1.1124689100000182s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1583s for a response
W20260709 13:02:27.239205  8974 connection.cc:256] RPC call timeout handler was delayed by -1.018963283000005s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28090000000000004s for a response
I20260709 13:02:27.241806  8974 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:27.244048  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:27.248135  9007 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:27.246595  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:27.249507  9002 raft_consensus.cc:2393] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 4: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:27.234828  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:27.251650  8369 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:27.252295  8974 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:27.256929  8974 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:27.254628  8223 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:27.235432 10759 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:27.260915 10652 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:27.262574  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:27.263749  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 51920.5us
I20260709 13:02:27.266212 10737 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:27.281165  8369 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:27.281773 10652 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.282297 10652 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:27.282913  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:27.283530 10737 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:27.284217 10737 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:27.284358  8369 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:27.285099  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.265656 10579 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:27.285826 10774 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:27.286309 10774 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.286635 10774 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:27.287163 10579 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:27.235433 10647 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5.
I20260709 13:02:27.288376  9008 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 13:02:27.288982 10713 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:27.280506  9009 pending_rounds.cc:85] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: Aborting all ops after (but not including) 4
I20260709 13:02:27.294576  9009 pending_rounds.cc:107] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:02:27.295022  9009 raft_consensus.cc:2889] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:27.267758 10613 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:27.302452  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:27.302973  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1923) took 2841 ms. Trace:
I20260709 13:02:27.303330  8405 rpcz_store.cc:276] 0709 13:02:24.461998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.465791 (+  3793us) service_pool.cc:224] Handling call
0709 13:02:26.248098 (+1782307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:26.249184 (+  1086us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:26.249225 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:26.261578 (+ 12353us) log.cc:844] Serialized 26 byte log entry
0709 13:02:26.370333 (+108755us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:26.370371 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.370432 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:26.922716 (+552284us) raft_consensus.cc:1685] finished
0709 13:02:26.922736 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:27.293701 (+370965us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b54003a1bd0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:27.302915 (+  9214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66,"raft.queue_time_us":76,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":11581,"spinlock_wait_cycles":3814097024,"thread_start_us":87898,"threads_started":1,"wal-append.queue_time_us":205016}
I20260709 13:02:27.306142  8256 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:27.259243 10797 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:27.310034  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 53178.3us
I20260709 13:02:27.296747 10561 consensus_queue.cc:1048] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:02:27.267655 10793 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:27.279927 10474 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:27.314616  8974 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:27.307039 10579 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:27.302193 10613 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.307554 10279 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.308185 10624 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:27.307157 10650 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:02:27.312484 10797 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.313593 10603 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:27.321646 10713 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:27.348654  8255 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:27.334539 10474 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.352981  8255 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:02:27.399760  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:27.357314  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 54911.8us
I20260709 13:02:27.310997 10752 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
W20260709 13:02:27.409557  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.357693  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1942) took 1530 ms. Trace:
I20260709 13:02:27.411777  8400 rpcz_store.cc:276] 0709 13:02:25.827561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.240569 (+413008us) service_pool.cc:224] Handling call
0709 13:02:26.240840 (+   271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:26.369156 (+128316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:26.369185 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:26.369613 (+   428us) log.cc:844] Serialized 26 byte log entry
0709 13:02:26.479860 (+110247us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:26.479902 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.346553 (+866651us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b5400276354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:27.346603 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:27.346654 (+    51us) raft_consensus.cc:1685] finished
0709 13:02:27.346670 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:27.357605 (+ 10935us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5722,"raft.queue_time_us":766,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":9746,"spinlock_wait_cycles":294243072,"thread_start_us":109969,"threads_started":1,"wal-append.queue_time_us":144744}
I20260709 13:02:27.360875  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:27.414001  8255 raft_consensus.cc:1275] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:02:27.414336  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 99330.4us
I20260709 13:02:27.343094 10675 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:27.414808  8405 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:02:27.346552 10279 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:27.351135 10797 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:27.344616 10613 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:27.402302  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:27.415673  8974 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:27.413147 10474 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:27.356483 10713 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:27.401903 10758 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:27.429939 10804 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:27.438652 10650 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:27.438510 10603 consensus_queue.cc:1048] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:02:27.450359 10213 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:27.446249 10716 consensus_meta.cc:287] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.966s	user 0.011s	sys 0.006s
I20260709 13:02:27.452441 10604 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:27.432924 10713 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:27.452265 10774 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:27.439038 10511 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:27.445557 10581 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:27.465361 10213 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:27.484424 10213 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:27.488967 10737 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:27.490981  8405 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:27.464088 10650 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.491889  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:27.483413 10747 consensus_meta.cc:287] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.698s	user 0.008s	sys 0.004s
I20260709 13:02:27.509133  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:27.503250 10800 log.cc:927] Time spent T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:02:27.512454  9008 raft_consensus.cc:2393] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 6: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:27.513329  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:27.503899 10650 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:27.518863 10774 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:27.524464 10717 raft_consensus.cc:3055] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:27.524268 10213 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:27.510185 10675 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.510037 10647 raft_consensus.cc:2764] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:02:27.483059 10716 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:27.547889 10613 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:27.550379 10793 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:27.550501 10675 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:27.553222 10797 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:27.563748 10613 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:27.507893 10747 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:27.570071 10716 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:27.564473 10382 consensus_queue.cc:260] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:27.576207  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:27.580217  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:27.533684 10717 raft_consensus.cc:740] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:27.587405 10652 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:27.588960 10713 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:27.589239  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:27.590931 10213 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:27.593904  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:27.524789 10757 ts_tablet_manager.cc:1929] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:02:27.610366  8332 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:27.611419  9008 raft_consensus.cc:2393] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 5: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:27.609184  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 248) took 1034 ms. Trace:
I20260709 13:02:27.612465 10717 consensus_queue.cc:260] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:27.613708 10717 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:27.613786  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:27.614782  9005 raft_consensus.cc:2393] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 3: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:27.615269 10713 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:27.615724  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:27.616401 10757 log.cc:1199] T ae82ced6ff014453bbcfd5f9f14b1f8d P 8c42259d7c4b463a9f485f42c320c81b: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-0-root/wals/ae82ced6ff014453bbcfd5f9f14b1f8d
I20260709 13:02:27.601773 10753 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:27.617661  8298 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:27.622005 10757 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55248 (request call id 80) took 1705 ms. Trace:
I20260709 13:02:27.622478 10757 rpcz_store.cc:276] 0709 13:02:25.939678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.965078 (+ 25400us) service_pool.cc:224] Handling call
0709 13:02:26.235252 (+270174us) ts_tablet_manager.cc:1151] Deleting tablet ae82ced6ff014453bbcfd5f9f14b1f8d
0709 13:02:26.235360 (+   108us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:02:27.145122 (+909762us) tablet_metadata.cc:665] Metadata flushed
0709 13:02:27.524614 (+379492us) tablet_metadata.cc:665] Metadata flushed
0709 13:02:27.621921 (+ 97307us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":672,"tablet-delete.queue_time_us":154456,"thread_start_us":112140,"threads_started":1}
W20260709 13:02:27.624759 10788 log.cc:927] Time spent T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.551s	user 0.000s	sys 0.000s
I20260709 13:02:27.624923  8139 catalog_manager.cc:5028] TS 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): tablet ae82ced6ff014453bbcfd5f9f14b1f8d (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:02:27.594552 10604 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:27.622205 10797 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:27.622251  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:27.611837  8329 rpcz_store.cc:276] 0709 13:02:26.575063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.654770 (+ 79707us) service_pool.cc:224] Handling call
0709 13:02:26.694323 (+ 39553us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.694521 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:26.741897 (+ 47376us) log.cc:844] Serialized 14 byte log entry
0709 13:02:27.473637 (+731740us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b1800356080. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e23a 00007f88287c8a2a
0709 13:02:27.608822 (+135185us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:27.608876 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:27.608932 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.609001 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:27.609116 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14926,"prepare.queue_time_us":47188,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":3109,"raft.queue_time_us":1194,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":2560,"spinlock_wait_cycles":8040320,"thread_start_us":129315,"threads_started":1}
I20260709 13:02:27.632591  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:27.634562  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:27.635411  9007 raft_consensus.cc:2393] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 6: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:27.636281  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:27.642486  9007 raft_consensus.cc:2393] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 6: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:27.648589  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:27.618892 10382 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:27.613788 10652 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:27.653340 10650 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:27.655592 10650 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:27.629312 10433 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:27.629115 10753 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:27.659794 10753 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:27.660343  9005 raft_consensus.cc:2393] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 3: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:27.661404  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:27.662081  9005 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:02:27.666934 10675 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:27.669919  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:27.637609  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:27.619699 10747 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:27.674505 10474 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:27.675050  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:27.678426  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:27.676817 10474 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:27.683637  9008 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:27.683318  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:27.685005  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:27.682343  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:27.683115  9007 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:27.686851  8327 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:27.675539  8249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1595) took 1132 ms. Trace:
W20260709 13:02:27.694290  8220 connection.cc:256] RPC call timeout handler was delayed by -1.2347212000000241s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2101s for a response
W20260709 13:02:27.695092  8220 connection.cc:256] RPC call timeout handler was delayed by -1.049439297000049s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1952s for a response
W20260709 13:02:27.695436  8220 connection.cc:256] RPC call timeout handler was delayed by -1.0127746090000187s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1529s for a response
I20260709 13:02:27.695497  8326 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:02:27.696338  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.687647  8298 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:27.697271  8220 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:27.798722  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 45580us
I20260709 13:02:27.746321  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:27.736869  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.845687  8326 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:27.720829  8400 raft_consensus.cc:1240] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:27.717022  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:27.698536  8324 raft_consensus.cc:3055] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:27.847311  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:27.847769  8324 raft_consensus.cc:740] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:27.840521  8255 raft_consensus.cc:1240] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:27.830844  9009 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:27.828341  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:27.699492  8328 raft_consensus.cc:1275] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:27.719419  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:27.689649  8249 rpcz_store.cc:276] 0709 13:02:26.542823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.545604 (+  2781us) service_pool.cc:224] Handling call
0709 13:02:27.177703 (+632099us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:27.178397 (+   694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.675278 (+496881us) spinlock_profiling.cc:243] Waited 632 ms on lock 0x7b54002eead0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:27.675461 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1452518144}
I20260709 13:02:27.849023  8330 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:27.697132  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:27.717671  8257 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:02:27.834285  8402 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1940) took 2377 ms (client timeout 2128 ms). Trace:
W20260709 13:02:27.716355  8256 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1570) took 3065 ms (client timeout 1546 ms). Trace:
I20260709 13:02:27.728379  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:27.850759  8402 rpcz_store.cc:269] 0709 13:02:25.456576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.843860 (+387284us) service_pool.cc:224] Handling call
0709 13:02:27.834061 (+1990201us) spinlock_profiling.cc:243] Waited 291 ms on lock 0x7b54000fdcd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:27.834201 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":670376704}
I20260709 13:02:27.799521  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:27.849598  8403 raft_consensus.cc:2393] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 6: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:02:27.852968  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 100613us
I20260709 13:02:27.857255  8295 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:27.800254  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:27.775722  9008 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:27.860822  9002 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:27.721231  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:27.724380  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:27.708902  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:27.739084  8399 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:27.769145  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:27.716902  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:27.709389  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:27.660383  8332 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
I20260709 13:02:27.710929  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:27.863395  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1601) took 1319 ms. Trace:
W20260709 13:02:27.850860  8256 rpcz_store.cc:269] 0709 13:02:24.651081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.651218 (+   137us) service_pool.cc:224] Handling call
0709 13:02:27.716173 (+3064955us) spinlock_profiling.cc:243] Waited 2.48 s on lock 0x7b540034a8d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:27.716291 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5698990464}
I20260709 13:02:27.864254  9008 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:27.851409  8327 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:27.864391  8369 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:27.864265  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:27.864931  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.698588  9005 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:27.715013  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 324) took 2868 ms. Trace:
I20260709 13:02:27.865279  8404 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:27.801363  8972 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:27.865896  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:27.760797  8223 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:27.852389  8324 consensus_queue.cc:260] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:27.878013  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:27.879484  8324 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:27.878013  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:27.879403  8222 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:27.881408  8248 rpcz_store.cc:276] 0709 13:02:24.846755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.038619 (+191864us) service_pool.cc:224] Handling call
0709 13:02:25.038992 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:26.319013 (+1280021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.714702 (+1395689us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b540026b954. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:27.714940 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2880540032}
I20260709 13:02:27.880110  8329 raft_consensus.cc:2410] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:27.879581  8406 raft_consensus.cc:2393] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 6: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:27.878675  8253 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:27.878882  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:27.720695  8398 raft_consensus.cc:1240] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:27.883509  8295 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:27.884617  8400 raft_consensus.cc:2368] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 5. Current term is 6.
W20260709 13:02:27.774950  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.885170  8295 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:27.885468  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:27.885905  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.883836  8257 rpcz_store.cc:276] 0709 13:02:26.544378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.663415 (+119037us) service_pool.cc:224] Handling call
0709 13:02:27.717219 (+1053804us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:27.851611 (+134392us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.863212 (+ 11601us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540034a8d0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:27.863341 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2422087040}
W20260709 13:02:27.886991  8332 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 242) took 1370 ms (client timeout 1583 ms). Trace:
I20260709 13:02:27.883215  8372 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:27.890586  9006 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3. Ops: [2.5-2.5]
W20260709 13:02:27.890031  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.887581  8325 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:27.892129  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 90800.7us
W20260709 13:02:27.880889  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 120155us
I20260709 13:02:27.794865 10433 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.893510 10560 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:27.880780 10717 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.720127 10753 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.893944 10674 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:27.894052 10474 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:27.893966 10691 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:27.932463  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:27.933158  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:27.893936  9008 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:02:27.889470 10749 consensus_queue.cc:1048] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:02:27.896644 10650 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:27.896731 10746 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:27.878963 10204 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:27.878791 10624 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.896687  8222 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:02:27.951071  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 54443.1us
I20260709 13:02:27.896826 10617 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:27.878484 10605 consensus_queue.cc:1059] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:02:27.898124  8369 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:27.898726 10775 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:27.905061 10647 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:02:27.900285 10781 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:27.903182  8324 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:02:27.966293  8324 pending_rounds.cc:85] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 3
I20260709 13:02:27.966063  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:27.966652  8324 pending_rounds.cc:107] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:02:27.966992  8324 raft_consensus.cc:2889] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:27.967154  8406 raft_consensus.cc:2393] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 5: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:27.968092  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:27.968668  8324 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
W20260709 13:02:27.898758  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 201495us
I20260709 13:02:27.969236  8222 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:27.970798  8220 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:27.971557  8222 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:27.906199  8253 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:27.798722 10279 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:27.909111  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:27.909395  8399 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:27.909991  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:27.977631  8401 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:27.888542  8332 rpcz_store.cc:269] 0709 13:02:26.516474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.518126 (+  1652us) service_pool.cc:224] Handling call
0709 13:02:27.589036 (+1070910us) spinlock_profiling.cc:243] Waited 12.6 ms on lock 0x7b58000e3498. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:02:27.886920 (+297884us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28931456}
I20260709 13:02:27.897725 10803 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:27.726871 10751 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:27.883533  8405 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:27.912947  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:27.914744  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:27.739858 10782 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:27.802897  9003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56948 (request call id 560) took 1337 ms. Trace:
I20260709 13:02:27.915840  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:27.918627  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:27.699436 10675 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.913748  8223 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:27.900702 10758 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:27.922351  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:27.917852 10560 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.936498  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:27.938584  8151 catalog_manager.cc:5697] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 2 to 4, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 4 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:27.940320  8326 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:27.945755  8325 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:27.946633  8398 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:27.899689  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:27.963696  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 65616.3us
I20260709 13:02:27.932408 10691 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.974694  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:27.978780  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:27.981992  8323 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:27.983024  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:27.985865  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:27.991344  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:27.987305  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:27.919423 10433 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:27.930178 10717 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:27.963924 10382 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:27.964090 10624 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:28.023025  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 107210us
I20260709 13:02:28.023356  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:27.936007 10747 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:02:27.957173 10650 raft_consensus.cc:697] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:27.981014 10690 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:02:27.976138 10604 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:27.957857 10204 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:27.935509 10753 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:28.179531  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 265763us
I20260709 13:02:28.082024 10474 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:28.187444  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:28.188186 10560 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:28.188217 10433 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:28.186465 10691 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:28.054054  8400 raft_consensus.cc:2393] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 6: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:28.188937 10433 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:28.190224  8329 raft_consensus.cc:2393] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 6: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:28.190481  8404 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:27.982082 10734 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:28.191013 10382 raft_consensus.cc:697] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:28.191089  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:28.208534  8405 raft_consensus.cc:2393] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 6: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:28.029110 10751 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:27.975615 10740 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:28.193225  8323 raft_consensus.cc:1275] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:28.212271  8151 catalog_manager.cc:5697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:28.223184  8323 raft_consensus.cc:1275] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:28.115020  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:28.028682 10781 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:28.193606 10474 consensus_queue.cc:1059] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:27.983146 10803 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.195254 10650 consensus_queue.cc:237] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:28.196148  8399 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 8: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:28.253474  8399 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:02:28.253994  8399 pending_rounds.cc:85] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 2
I20260709 13:02:28.254266  8399 pending_rounds.cc:107] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:02:28.254556  8399 raft_consensus.cc:2889] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:28.196921  8401 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:28.028700  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:28.255432  8330 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:28.256036  8401 raft_consensus.cc:1240] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:28.199047  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 313863us
I20260709 13:02:28.201776  8403 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:28.204697  8326 raft_consensus.cc:1275] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:28.257206  8295 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:28.034960  8369 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:28.205928  8331 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:28.032445  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1608) took 1486 ms. Trace:
W20260709 13:02:28.031806 10818 log.cc:927] Time spent T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:02:28.018797 10279 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:28.207310 10433 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.217516 10204 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:28.220788  8405 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:28.222220  8151 catalog_manager.cc:5697] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:28.287945  8399 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 3. Current term is 5. Ops: []
I20260709 13:02:28.288329  8151 catalog_manager.cc:5697] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:28.191711 10604 raft_consensus.cc:697] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:28.240284  8397 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:28.194113  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:28.291292  8400 raft_consensus.cc:2410] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:02:28.258677  8253 rpcz_store.cc:276] 0709 13:02:26.546038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.659718 (+113680us) service_pool.cc:224] Handling call
0709 13:02:28.032227 (+1372509us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400408654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:28.032367 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2800108800}
I20260709 13:02:28.188779  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:28.270696  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:28.207468 10746 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.019266  9003 rpcz_store.cc:276] 0709 13:02:26.465246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.475278 (+ 10032us) service_pool.cc:224] Handling call
0709 13:02:27.802715 (+1327437us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54004b87d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:27.802843 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":283883904}
I20260709 13:02:28.307286  8398 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:28.307890  8406 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:28.311269  8151 catalog_manager.cc:5697] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 2 to 5, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 5 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:28.314316  8400 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:28.028802 10675 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:28.231106 10781 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:28.321547  8151 catalog_manager.cc:5697] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 3 to 6, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 6 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:28.251617 10803 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:28.339129  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:28.307281 10775 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.307338 10804 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:28.287092 10433 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:28.339996  8398 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:28.310460 10740 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.342360  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:28.307053 10746 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:28.338001  8400 raft_consensus.cc:1240] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:28.345935 10604 consensus_queue.cc:237] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:28.348812  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.310667 10204 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:28.335297  8151 catalog_manager.cc:5697] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 2 to 4, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 4 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:28.362140  8151 catalog_manager.cc:5697] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 1 to 6, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 6 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:28.364960  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:28.365751  8400 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:28.363929  8151 catalog_manager.cc:5697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 1 to 2, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 2 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:28.335312  8330 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:28.317245 10279 raft_consensus.cc:697] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:28.336531 10781 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:28.372985  8974 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:28.367874  8151 catalog_manager.cc:5697] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:28.381292  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.382189 10828 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.351222 10775 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:28.389350  8400 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:28.382802  8151 catalog_manager.cc:5697] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 2 to 4, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 4 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:28.363680 10782 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:02:28.343079 10382 consensus_queue.cc:237] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:28.358314 10740 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:28.431535  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:28.379966 10804 consensus_queue.cc:1059] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:02:28.415270  8151 catalog_manager.cc:5697] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e reported cstate change: leader changed from <none> to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 4 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:28.440750  8151 catalog_manager.cc:5697] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 4 to 5, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 5 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:02:28.449661  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 476327us
I20260709 13:02:28.450234  8151 catalog_manager.cc:5697] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 1 to 2, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 2 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:28.467330  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:28.468377  8406 raft_consensus.cc:2410] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:02:28.492434  8222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.018s	sys 0.019s
W20260709 13:02:28.492851  8222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.018s	sys 0.019s
W20260709 13:02:28.493175  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 521516us
I20260709 13:02:28.495138  8222 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:28.420924 10828 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:28.483918 10753 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:28.485922 10733 consensus_queue.cc:1059] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:02:28.452251  8220 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:28.467144 10624 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:28.490579 10804 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:28.530702  8402 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.011s	sys 0.043s
W20260709 13:02:28.531291  8402 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.011s	sys 0.043s
I20260709 13:02:28.507491 10605 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:28.514415 10797 raft_consensus.cc:3055] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:28.438855 10279 consensus_queue.cc:237] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:28.554020 10826 log.cc:927] Time spent T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:02:28.555478  9004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 571) took 1276 ms. Trace:
I20260709 13:02:28.560178 10828 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:28.574757 10560 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:28.577054 10775 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:28.551293 10675 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:28.578408 10560 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:28.579355 10560 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:28.579993 10675 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:28.580543 10675 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.580948 10675 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:28.545704 10832 log.cc:927] Time spent T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
W20260709 13:02:28.541345 10602 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.018s	sys 0.035s
I20260709 13:02:28.555956  9004 rpcz_store.cc:276] 0709 13:02:27.279068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.746145 (+467077us) service_pool.cc:224] Handling call
0709 13:02:27.746613 (+   468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:27.747112 (+   499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:27.747164 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:27.747813 (+   649us) log.cc:844] Serialized 26 byte log entry
0709 13:02:28.150315 (+402502us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:28.150388 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.150514 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:28.555186 (+404672us) raft_consensus.cc:1685] finished
0709 13:02:28.555220 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.555407 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":133,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":117,"thread_start_us":402062,"threads_started":1,"wal-append.queue_time_us":631878}
I20260709 13:02:28.571442 10740 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:28.642222  8298 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.032s	sys 0.040s
W20260709 13:02:28.642729  8298 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.032s	sys 0.040s
I20260709 13:02:28.514528 10691 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:28.548591 10797 raft_consensus.cc:740] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:02:28.615477 10602 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.018s	sys 0.036s
W20260709 13:02:28.652074  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 433456us
W20260709 13:02:28.643102  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 955184us
I20260709 13:02:28.660687  8223 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:28.673022  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 424833us
I20260709 13:02:28.650051 10717 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:28.686127 10624 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:28.707376  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 426061us
I20260709 13:02:28.708458  8295 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:28.650010 10781 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:28.722891  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 687801us
W20260709 13:02:28.723608  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.723886  8369 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:28.739506  8295 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:28.743228  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:28.752329  8369 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:28.759158  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:28.771018  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.759872  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:28.690418 10828 consensus_queue.cc:1059] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:02:28.702457 10837 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:28.705864 10751 raft_consensus.cc:697] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:28.726233  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:28.789038 10674 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.020s	sys 0.018s
W20260709 13:02:28.832636 10674 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.020s	sys 0.019s
I20260709 13:02:28.701035 10624 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:28.836441 10624 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:28.837766 10844 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:28.838585 10844 raft_consensus.cc:697] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:28.702605 10797 consensus_queue.cc:260] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:28.839772 10844 consensus_queue.cc:237] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:28.825327 10647 consensus_queue.cc:1048] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:02:28.808977 10747 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:28.847527  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:28.850447 10647 consensus_queue.cc:1048] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:28.852216 10647 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:28.852828 10647 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:28.761018 10753 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:28.853415 10647 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:28.853783 10753 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:28.854408 10753 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:28.858026 10751 consensus_queue.cc:237] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:28.860833 10803 consensus_meta.cc:287] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.509s	user 0.006s	sys 0.002s
I20260709 13:02:28.848965  8369 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:28.862634 10781 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:28.761214 10775 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:28.863613 10775 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:28.864094 10775 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:28.866535 10775 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:28.867241 10775 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:28.867806 10775 raft_consensus.cc:697] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:28.831882 10204 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:02:28.880652  8974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.004s	sys 0.016s
W20260709 13:02:28.881196  8974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.004s	sys 0.017s
W20260709 13:02:28.881860  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 508746us
I20260709 13:02:28.882705  8974 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:28.890475  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:28.911298  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.912326  8974 leader_election.cc:400] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 5. Current term is 6.
I20260709 13:02:28.912621  8297 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:28.912938  8974 leader_election.cc:403] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:28.723237 10835 consensus_queue.cc:1059] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.176s
I20260709 13:02:28.929004  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 325) took 1158 ms. Trace:
I20260709 13:02:28.911154 10781 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:28.913661 10204 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:28.941704  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 446477us
I20260709 13:02:28.868949 10775 consensus_queue.cc:237] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:28.886622  9004 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:28.861280 10803 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:28.916042 10829 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:28.947897  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 495552us
I20260709 13:02:29.053109  8220 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:29.055512  8220 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:28.953740 10605 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:28.958141 10675 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:28.959571 10836 log.cc:927] Time spent T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 13:02:28.959667 10804 consensus_queue.cc:1059] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:28.961371  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1709) took 1075 ms. Trace:
I20260709 13:02:28.941104 10781 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.963450  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1710) took 1068 ms. Trace:
I20260709 13:02:28.963971  9001 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:02:28.965787 10746 consensus_meta.cc:287] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.610s	user 0.011s	sys 0.000s
I20260709 13:02:29.062783 10674 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:29.062367  8327 rpcz_store.cc:276] 0709 13:02:27.885609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.194520 (+308911us) service_pool.cc:224] Handling call
0709 13:02:28.194785 (+   265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:28.195066 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:28.195090 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:28.195515 (+   425us) log.cc:844] Serialized 26 byte log entry
0709 13:02:28.655629 (+460114us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:28.655683 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.655770 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:28.961162 (+305392us) raft_consensus.cc:1685] finished
0709 13:02:28.961182 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.961316 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":227275,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":5302,"thread_start_us":459816,"threads_started":1,"wal-append.queue_time_us":764246}
I20260709 13:02:29.063221 10746 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:28.943758 10758 raft_consensus.cc:3055] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:28.931324 10602 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:28.929416 10849 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:28.948015 10691 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:28.935330  8402 rpcz_store.cc:276] 0709 13:02:27.770284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.852137 (+ 81853us) service_pool.cc:224] Handling call
0709 13:02:27.889402 (+ 37265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:27.889623 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:28.531787 (+642164us) log.cc:844] Serialized 14 byte log entry
0709 13:02:28.928609 (+396822us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:28.928665 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:28.928721 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.928800 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.928937 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":91217,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":3441,"thread_start_us":1014513,"threads_started":2}
W20260709 13:02:28.974539  8972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.012s	sys 0.012s
I20260709 13:02:29.074632  8328 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 8: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:02:29.074764  8972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.012s	sys 0.012s
W20260709 13:02:29.075093  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 780610us
I20260709 13:02:29.075977  8972 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:29.027565  8400 raft_consensus.cc:1240] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:29.037241  8404 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: []
W20260709 13:02:29.061034 10433 consensus_meta.cc:287] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.701s	user 0.004s	sys 0.007s
I20260709 13:02:29.044804  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:29.045995  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:28.974205 10652 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.059357  8220 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:29.099959  8220 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:02:29.060200  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 399485us
I20260709 13:02:29.065219  8406 raft_consensus.cc:1240] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:02:29.062489  8325 rpcz_store.cc:276] 0709 13:02:27.898878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.202595 (+303717us) service_pool.cc:224] Handling call
0709 13:02:28.203111 (+   516us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:28.203509 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:28.203547 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:28.218723 (+ 15176us) log.cc:844] Serialized 26 byte log entry
0709 13:02:28.684840 (+466117us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:28.684904 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.685015 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:28.963247 (+278232us) raft_consensus.cc:1685] finished
0709 13:02:28.963268 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.963392 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3165,"raft.queue_time_us":255134,"raft.run_cpu_time_us":353,"raft.run_wall_time_us":6379,"thread_start_us":465737,"threads_started":1,"wal-append.queue_time_us":728191}
I20260709 13:02:28.981890 10749 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.102705 10749 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:29.104929 10749 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:28.982875  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:29.073966 10842 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.105643  9003 raft_consensus.cc:2376] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6. Re-sending same reply.
W20260709 13:02:29.052534 10704 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.007s	sys 0.028s
W20260709 13:02:29.106563 10704 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.007s	sys 0.029s
I20260709 13:02:29.039330  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:29.070290 10781 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:29.058717 10734 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:29.058734 10733 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:29.060904 10617 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:02:29.117228 10691 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:29.050654 10204 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:29.086304  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:29.064471 10758 raft_consensus.cc:740] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:29.089201 10851 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 5. Current term is 6.
I20260709 13:02:29.092751  8405 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:29.105808 10842 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:29.087714  8372 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:29.131244  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 43624.6us
I20260709 13:02:29.132597  8372 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:29.068717 10674 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:29.067508 10279 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:02:29.097167 10852 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.099527 10829 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:29.161194  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 61305.8us
I20260709 13:02:29.117630 10856 consensus_queue.cc:1059] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:02:29.171550  8405 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:29.099862 10433 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:29.154753 10749 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:29.137096 10851 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:29.159268 10849 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.115474 10652 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:29.203411  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1704) took 1335 ms. Trace:
W20260709 13:02:29.204461  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 462620us
I20260709 13:02:29.218376  8295 leader_election.cc:400] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5.
I20260709 13:02:29.209358  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:29.211272  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:29.199342  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 59559.3us
I20260709 13:02:29.219205  8326 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
W20260709 13:02:29.229108  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 67002.8us
I20260709 13:02:29.219866  8323 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
W20260709 13:02:29.231282  8151 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.014s	sys 0.014s
W20260709 13:02:29.232864  8151 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.014s	sys 0.015s
I20260709 13:02:29.216820  8324 rpcz_store.cc:276] 0709 13:02:27.868224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.193882 (+325658us) service_pool.cc:224] Handling call
0709 13:02:28.194121 (+   239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:28.194285 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:28.548855 (+354570us) log.cc:844] Serialized 14 byte log entry
0709 13:02:28.934360 (+385505us) log.cc:844] Serialized 14 byte log entry
0709 13:02:29.202992 (+268632us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:29.203061 (+    69us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:29.203120 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.203199 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.203346 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29235,"prepare.queue_time_us":602500,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":90,"raft.queue_time_us":339570,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":8562,"thread_start_us":984955,"threads_started":3,"wal-append.queue_time_us":564875}
I20260709 13:02:29.238941  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:29.240514  8327 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:02:29.172605 10857 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:29.243588  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 315953us
I20260709 13:02:29.246948  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:29.250108  8330 raft_consensus.cc:2393] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 6: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:29.199288 10828 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.201833 10749 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:29.156599 10674 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:29.252504  8332 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 2. Current term is 3. Ops: [2.5-2.5]
W20260709 13:02:29.253564  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 163887us
I20260709 13:02:29.163378 10854 consensus_queue.cc:1059] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.113s
I20260709 13:02:29.264092  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:29.271939  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:29.273182  8257 raft_consensus.cc:2393] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 4: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:29.274781  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:29.275746  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:29.275643  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:29.276424  8254 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:29.276569  8248 raft_consensus.cc:2393] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 5: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:29.278820  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:29.287552  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:29.249318 10857 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:29.287555  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:29.296773  8250 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:29.251569 10604 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:29.160705 10852 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:29.298249  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:29.300194  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:29.301617  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:29.290778  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:29.228030 10624 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:02:29.298468  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:29.258121 10749 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:29.289538  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:29.288378  8257 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:29.308095  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:29.320431  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:29.325884  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 91753.7us
I20260709 13:02:29.327446  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:29.339728  8250 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:29.331367  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:29.341934  8253 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:29.348232  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:29.347079  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:29.336531  8257 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:29.359833  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:29.349197  8248 raft_consensus.cc:2376] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8. Re-sending same reply.
I20260709 13:02:29.373354  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:29.374001  8248 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:29.365995  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1712) took 1466 ms. Trace:
I20260709 13:02:29.230252 10851 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:29.305296 10797 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:29.202364 10758 consensus_queue.cc:260] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:29.431313  9002 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:02:29.431494  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:02:29.371095  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 140590us
I20260709 13:02:29.374792  8332 rpcz_store.cc:276] 0709 13:02:27.899457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.219835 (+320378us) service_pool.cc:224] Handling call
0709 13:02:28.280215 (+ 60380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:29.258110 (+977895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.365633 (+107523us) spinlock_profiling.cc:243] Waited 972 ms on lock 0x7b540014c654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:29.365908 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2234751360}
I20260709 13:02:29.433255  8253 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:29.434603  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:29.435273  8253 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:29.376121  8257 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:29.438619  8257 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:29.440531  8257 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:02:29.337761  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 425088us
I20260709 13:02:29.376494  8329 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:29.254070 10828 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:29.379560  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 1717) took 1479 ms. Trace:
W20260709 13:02:29.380033  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:29.371029  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 295038us
I20260709 13:02:29.452611  8328 rpcz_store.cc:276] 0709 13:02:27.900530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.188548 (+288018us) service_pool.cc:224] Handling call
0709 13:02:29.379353 (+1190805us) spinlock_profiling.cc:243] Waited 779 ms on lock 0x7b540020e754. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:29.379489 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1791715072}
I20260709 13:02:29.268847 10751 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.103s
I20260709 13:02:29.230558 10849 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:29.358359 10826 log.cc:927] Time spent T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:02:29.304941 10857 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:29.326999 10804 raft_consensus.cc:3055] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:29.429207  9008 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:02:29.497031  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 64123.6us
I20260709 13:02:29.304131 10855 consensus_queue.cc:1059] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.199s
I20260709 13:02:29.361558 10575 raft_consensus.cc:3055] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:29.394963  8254 raft_consensus.cc:2393] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 6: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:29.454918  8972 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:29.491482  8253 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:29.492165  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:29.287076 10864 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.200s
I20260709 13:02:29.442032  8297 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:29.412582 10691 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:29.460045 10675 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:29.461315 10733 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:29.523207  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:29.537145  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 277227us
I20260709 13:02:29.543877  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:29.502902 10758 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:29.460340 10803 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:29.559460  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 304698us
I20260709 13:02:29.495971 10804 raft_consensus.cc:740] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:29.497014 10858 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:29.520453 10850 raft_consensus.cc:3055] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:29.498212 10854 consensus_queue.cc:1059] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.089s
I20260709 13:02:29.537086 10575 raft_consensus.cc:740] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:29.591744  8974 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:29.554757  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:29.606938  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 164973us
I20260709 13:02:29.642120  8297 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:02:29.593379  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 221220us
I20260709 13:02:29.616685  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:29.643563  8369 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:29.617625  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:29.576912 10868 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:02:29.603487 10853 log.cc:927] Time spent T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.414s	user 0.000s	sys 0.001s
I20260709 13:02:29.576694 10883 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:29.647228  8323 raft_consensus.cc:2393] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 6: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:29.647053  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:29.665601  8369 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:29.650377  8295 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:29.650377  8297 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:29.574556 10691 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:29.601249 10880 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.670221  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:29.566872 10887 consensus_queue.cc:1048] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.151s
I20260709 13:02:29.671038  8369 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:29.645208 10861 log.cc:927] Time spent T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.411s	user 0.001s	sys 0.000s
I20260709 13:02:29.608320 10850 raft_consensus.cc:740] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:29.528354  8250 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:29.680924  8323 raft_consensus.cc:2393] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 4: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:29.676738  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:29.705092  8332 raft_consensus.cc:2393] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 4: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:29.620584 10433 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:29.663578 10859 log.cc:927] Time spent T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.414s	user 0.000s	sys 0.001s
I20260709 13:02:29.705924  8253 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 3. Current term is 5. Ops: []
I20260709 13:02:29.663882 10652 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:29.681298 10862 log.cc:927] Time spent T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.450s	user 0.001s	sys 0.000s
I20260709 13:02:29.681406 10886 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:02:29.743610  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 72641.1us
W20260709 13:02:29.732815  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 277865us
I20260709 13:02:29.748445  8256 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:29.586649 10733 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:29.694900 10885 raft_consensus.cc:3055] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:29.776850  8369 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:29.676268 10896 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.705355 10674 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:29.749363 10888 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:29.766979 10863 log.cc:927] Time spent T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.435s	user 0.001s	sys 0.000s
I20260709 13:02:29.796607  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:29.747056 10704 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:02:29.741106  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.780581 10864 raft_consensus.cc:3055] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:29.859313 10910 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:29.861790 10675 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:29.861967 10902 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:29.610917 10858 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:29.694510 10847 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.125s
I20260709 13:02:29.665201 10837 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.115s
W20260709 13:02:29.870967  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 74413.5us
I20260709 13:02:29.795635 10885 raft_consensus.cc:740] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:29.897547  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:29.872328  8327 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:29.873574  8248 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:29.663625 10868 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:29.812764 10854 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:29.910982 10878 log.cc:927] Time spent T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.346s	user 0.003s	sys 0.000s
W20260709 13:02:29.894034 10874 log.cc:927] Time spent T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.337s	user 0.000s	sys 0.001s
I20260709 13:02:29.885020 10675 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:29.767113 10575 consensus_queue.cc:260] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:29.894482 10835 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:29.894217 10905 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:02:29.938011 10876 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.938171 10204 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:29.895196 10906 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:29.938578 10876 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:29.938863 10906 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:29.940088 10858 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:29.794656 10886 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:29.940712 10876 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:29.782657 10652 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 13:02:29.942379 10886 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:29.911232 10851 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:29.911830 10852 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:29.912984 10849 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:29.877360 10850 consensus_queue.cc:260] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:29.944144 10850 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:29.943905 10204 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:29.885707 10828 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:29.946393 10560 raft_consensus.cc:3055] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:29.946851 10560 raft_consensus.cc:740] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:29.947468 10675 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:29.948608 10647 consensus_meta.cc:287] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 1.095s	user 0.010s	sys 0.002s
I20260709 13:02:29.947836 10560 consensus_queue.cc:260] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:29.914440 10847 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:29.949049 10647 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:29.949350 10560 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:29.949859 10847 raft_consensus.cc:697] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:29.648874 10804 consensus_queue.cc:260] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:29.950742 10847 consensus_queue.cc:237] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:29.951637  8398 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:29.916417 10868 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:29.917008 10887 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:29.953184 10906 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:29.895196 10918 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:29.917932 10854 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:29.875300 10889 raft_consensus.cc:2764] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:02:29.905606  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:29.693677 10880 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:29.866595 10864 raft_consensus.cc:740] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:29.873003 10704 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:29.966572 10864 consensus_queue.cc:260] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:29.925182 10897 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:29.970579 10864 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:29.970955 10870 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:29.971526 10870 raft_consensus.cc:697] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:29.972287 10746 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:29.972359 10870 consensus_queue.cc:237] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:29.973729 10842 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:29.970605 10897 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:29.974397 10897 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:29.811197 10896 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:29.978866  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:29.925243 10892 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:29.926681 10749 consensus_meta.cc:287] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.565s	user 0.009s	sys 0.001s
I20260709 13:02:29.979718 10892 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:29.980175 10892 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:29.979941 10749 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:29.980672 10875 consensus_queue.cc:1059] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:29.924033 10803 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:29.982059 10803 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:29.982393 10749 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:29.984018 10803 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:29.985756 10857 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:29.987562  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.987769 10857 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:29.988363 10749 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.926726 10891 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:29.876497 10873 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:29.928511 10856 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:29.989495  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:29.989260 10749 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:29.989519 10891 raft_consensus.cc:3055] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:29.927006 10877 consensus_queue.cc:1059] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:29.990262  8397 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:29.990645 10877 raft_consensus.cc:3055] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:29.990988 10877 raft_consensus.cc:740] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:29.991425 10749 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:29.991762 10877 consensus_queue.cc:260] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:29.992934 10877 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:29.930409 10848 raft_consensus.cc:3055] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:29.990263 10891 raft_consensus.cc:740] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:29.993803 10848 raft_consensus.cc:740] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:29.994575 10891 consensus_queue.cc:260] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:29.994895 10848 consensus_queue.cc:260] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:29.933586 10605 raft_consensus.cc:3055] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:29.996956 10605 raft_consensus.cc:740] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:29.957940 10854 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:29.996521 10848 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:29.919075 10733 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:30.000181  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:29.958849 10753 consensus_meta.cc:287] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 1.104s	user 0.011s	sys 0.000s
I20260709 13:02:30.002254 10753 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:29.854785 10751 consensus_queue.cc:1059] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.065s
I20260709 13:02:30.010773 10751 raft_consensus.cc:3055] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:30.011241 10751 raft_consensus.cc:740] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:30.012058 10751 consensus_queue.cc:260] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:30.013118 10753 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:29.959568 10575 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:29.960162 10885 consensus_queue.cc:260] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:29.965539 10880 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:30.013818 10897 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:29.873059 10910 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:29.895192 10602 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:02:29.969156  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 63596.7us
I20260709 13:02:30.041363  8369 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:30.044616  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1970) took 1013 ms. Trace:
I20260709 13:02:29.979445 10896 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:30.045020  8398 rpcz_store.cc:276] 0709 13:02:29.031548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.031704 (+   156us) service_pool.cc:224] Handling call
0709 13:02:29.032062 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.952030 (+919968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.044386 (+ 92356us) spinlock_profiling.cc:243] Waited 919 ms on lock 0x7b5400276354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:30.044558 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2113754368}
I20260709 13:02:30.045764 10896 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:30.046480  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:30.030650 10910 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:29.988718 10918 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:29.989790 10856 raft_consensus.cc:697] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 6 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:02:29.918679 10781 consensus_meta.cc:287] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.805s	user 0.005s	sys 0.004s
I20260709 13:02:29.969822 10647 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:30.016515 10433 consensus_queue.cc:1175] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e64761f43e4240319c41117cf886d22f" responder_term: 5 status { last_received { term: 3 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:30.028707 10704 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:30.013145 10751 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:30.062836  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:30.063604  8332 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:30.064937  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:30.069972 10856 consensus_queue.cc:237] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:30.057267 10918 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:30.079212 10804 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:30.066141 10781 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:30.084326  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.086706  9003 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 8: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
W20260709 13:02:30.087185 10797 consensus_meta.cc:287] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.632s	user 0.008s	sys 0.003s
I20260709 13:02:30.093743 10885 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:30.094727 10829 consensus_meta.cc:287] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.921s	user 0.011s	sys 0.000s
I20260709 13:02:30.095260 10829 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:30.037112  8397 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:29.929029 10691 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:30.050879  8369 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.857s (SENT) [suppressed 29 similar messages]
I20260709 13:02:30.014122 10650 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.051436  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1969) took 1021 ms. Trace:
I20260709 13:02:30.062232 10897 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:30.110805  8403 rpcz_store.cc:276] 0709 13:02:29.029732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.029917 (+   185us) service_pool.cc:224] Handling call
0709 13:02:29.030335 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:29.051831 (+ 21496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.051858 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:29.052387 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 13:02:29.255963 (+203576us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:29.256004 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.256080 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:29.910416 (+654336us) spinlock_profiling.cc:243] Waited 7.33 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:30.050503 (+140087us) raft_consensus.cc:1685] finished
0709 13:02:30.050523 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.051369 (+   846us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":39984,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":16997,"spinlock_wait_cycles":16848640,"thread_start_us":181659,"threads_started":1,"wal-append.queue_time_us":365740}
I20260709 13:02:30.085318 10781 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:30.113368 10892 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:30.136190 10605 consensus_queue.cc:260] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:30.172961 10605 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:30.160419 10896 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:30.165365 10910 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:30.077200 10918 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:30.176690 10910 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:30.110574 10650 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:30.131491 10758 consensus_meta.cc:287] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.543s	user 0.003s	sys 0.005s
I20260709 13:02:30.187798 10892 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:30.250869  8254 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:29.890553  8256 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:30.258396  8252 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:02:30.372119  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:30.415978  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:30.417651  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.282512  8325 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:30.445684  8328 raft_consensus.cc:2410] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:02:30.448559  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1617) took 1173 ms. Trace:
I20260709 13:02:30.450021  8252 rpcz_store.cc:276] 0709 13:02:29.274726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.275474 (+   748us) service_pool.cc:224] Handling call
0709 13:02:29.307562 (+ 32088us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:30.259260 (+951698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.447376 (+188116us) spinlock_profiling.cc:243] Waited 950 ms on lock 0x7b540034c6d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:30.448423 (+  1047us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2184003968}
W20260709 13:02:30.464972  8973 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.009s	user 0.031s	sys 0.088s
W20260709 13:02:30.467672 10747 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.881s	user 0.023s	sys 0.062s
W20260709 13:02:30.470702  8973 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.015s	user 0.032s	sys 0.088s
W20260709 13:02:30.471041 10747 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.894s	user 0.023s	sys 0.063s
W20260709 13:02:30.473901  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:30.472213 10604 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.039s	user 0.030s	sys 0.080s
W20260709 13:02:30.478219 10604 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.098s	user 0.031s	sys 0.081s
W20260709 13:02:30.478799  8251 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.996s	user 0.033s	sys 0.100s
W20260709 13:02:30.466506 10855 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.054s	sys 0.103s
W20260709 13:02:30.480196 10855 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.904s	user 0.054s	sys 0.103s
W20260709 13:02:30.482609  8295 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.022s	sys 0.031s
W20260709 13:02:30.483675  8372 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.960s	user 0.009s	sys 0.047s
I20260709 13:02:30.483973 10382 raft_consensus.cc:3055] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:30.482945 10604 raft_consensus.cc:697] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:02:30.484797  8222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.908s	user 0.008s	sys 0.010s
I20260709 13:02:30.484543 10382 raft_consensus.cc:740] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:02:30.479727  8251 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.997s	user 0.033s	sys 0.101s
W20260709 13:02:30.484166  8372 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.961s	user 0.009s	sys 0.047s
I20260709 13:02:30.487069 10382 consensus_queue.cc:260] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:30.489379 10382 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:30.487685  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 964473us
I20260709 13:02:30.505321 10781 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:30.506397 10781 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:30.483162  8295 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.022s	sys 0.031s
W20260709 13:02:30.527361  8222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.950s	user 0.008s	sys 0.011s
W20260709 13:02:30.546345  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:30.552696  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 901765us
W20260709 13:02:30.607826  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:30.608183  8257 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.094s	user 0.013s	sys 0.021s
W20260709 13:02:30.608791  8257 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.105s	user 0.013s	sys 0.021s
I20260709 13:02:30.610137  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1637) took 1318 ms. Trace:
I20260709 13:02:30.610627  8257 rpcz_store.cc:276] 0709 13:02:29.291880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.441395 (+149515us) service_pool.cc:224] Handling call
0709 13:02:29.468402 (+ 27007us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.609529 (+1141127us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.609588 (+    59us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:30.609664 (+    76us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:30.609732 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.609857 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.610048 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62233,"raft.queue_time_us":139495,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":29645,"thread_start_us":1105583,"threads_started":1}
W20260709 13:02:30.612879 10775 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.928s	user 0.024s	sys 0.035s
W20260709 13:02:30.613410 10775 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.967s	user 0.025s	sys 0.036s
W20260709 13:02:30.614627  8324 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.966s	user 0.005s	sys 0.011s
W20260709 13:02:30.615193  8324 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.067s	user 0.006s	sys 0.011s
W20260709 13:02:30.616495 10717 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.827s	user 0.012s	sys 0.022s
W20260709 13:02:30.617039 10717 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.849s	user 0.012s	sys 0.022s
I20260709 13:02:30.612553  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:30.627203  8369 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.003s	sys 0.013s
I20260709 13:02:30.627327  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:30.627938  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 576874us
I20260709 13:02:30.628059  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:30.629460  9005 raft_consensus.cc:2410] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:30.630306  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:30.631183  8397 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
W20260709 13:02:30.641001  9004 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.007s	sys 0.014s
W20260709 13:02:30.641268  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:30.641786  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.641580  9003 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:30.650473 10835 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.651489 10835 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:30.652226  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:30.652704  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1605) took 1323 ms. Trace:
I20260709 13:02:30.653236  8253 rpcz_store.cc:276] 0709 13:02:29.304359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.581806 (+277447us) service_pool.cc:224] Handling call
0709 13:02:29.705724 (+123918us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.747597 (+ 41873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.523700 (+776103us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b540030a050. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:30.618688 (+ 94988us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":284189056}
I20260709 13:02:30.657933 10650 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:30.658077  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:30.658247  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:30.659106  9007 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
W20260709 13:02:30.665840  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:30.612252 10844 consensus_queue.cc:1175] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ff7a6fe533e94c1bae555b4cae427728" responder_term: 6 status { last_received { term: 3 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:30.672087  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:30.674921 10433 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.675813 10433 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:30.604302 10604 consensus_queue.cc:237] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:30.677768 10624 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.678610 10624 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:02:30.650718  8398 raft_consensus.cc:2393] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 7: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:02:30.641578  9004 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.008s	sys 0.014s
W20260709 13:02:30.689594  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.614650  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:30.722945  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 134525us
I20260709 13:02:30.771118  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:30.733101 10433 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:31.038074  8400 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 6: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:02:30.780941  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:30.743099  8220 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.100s	user 0.000s	sys 0.029s
W20260709 13:02:31.002534  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 186224us
I20260709 13:02:30.870666  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:31.180119  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1619) took 1904 ms. Trace:
I20260709 13:02:31.269227  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:30.920765  8332 consensus_meta.cc:287] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.856s	user 0.012s	sys 0.002s
W20260709 13:02:31.182045  8255 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.591s	user 0.010s	sys 0.013s
W20260709 13:02:30.960171  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 56597.1us
W20260709 13:02:31.281786  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:31.038221  8323 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.000s	sys 0.028s
W20260709 13:02:31.287556  8220 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.781s	user 0.007s	sys 0.044s
I20260709 13:02:31.260151  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1730) took 2021 ms. Trace:
I20260709 13:02:31.287734  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:31.294503  8255 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.817s	user 0.012s	sys 0.022s
W20260709 13:02:30.974684  8330 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.596s	user 0.015s	sys 0.023s
W20260709 13:02:31.137343  8326 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.325s	user 0.006s	sys 0.006s
W20260709 13:02:31.074456  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:31.313180  8323 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.394s	user 0.009s	sys 0.038s
W20260709 13:02:31.180073  8974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.461s	user 0.014s	sys 0.013s
I20260709 13:02:30.827587 10835 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:31.326400  8332 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
W20260709 13:02:31.326795  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 1.8266e+06us
I20260709 13:02:31.327428  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 1737) took 1282 ms. Trace:
I20260709 13:02:31.327826  8332 rpcz_store.cc:276] 0709 13:02:30.045235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.048389 (+  3154us) service_pool.cc:224] Handling call
0709 13:02:31.327290 (+1278901us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:31.137588 10833 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.226s	user 0.008s	sys 0.005s
W20260709 13:02:31.326458  8330 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.021s	user 0.027s	sys 0.034s
W20260709 13:02:31.331153  8326 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.828s	user 0.020s	sys 0.007s
W20260709 13:02:31.176538  8329 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.802s	user 0.012s	sys 0.025s
I20260709 13:02:31.100432  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:31.327462  8297 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.314s	user 0.025s	sys 0.020s
W20260709 13:02:31.363592  8974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.687s	user 0.014s	sys 0.021s
I20260709 13:02:31.330470  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:31.365794 10833 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.754s	user 0.019s	sys 0.009s
W20260709 13:02:31.366070  8297 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.672s	user 0.026s	sys 0.021s
W20260709 13:02:31.366958  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 1.67323e+06us
I20260709 13:02:31.367218  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:31.367867  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:02:31.175887 10900 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.012s	user 0.000s	sys 0.019s
I20260709 13:02:31.282873 10624 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:31.175602 10883 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.075s	user 0.008s	sys 0.005s
W20260709 13:02:31.177258 10871 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.125s	user 0.004s	sys 0.023s
I20260709 13:02:31.274977  8251 rpcz_store.cc:276] 0709 13:02:29.275264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.297877 (+ 22613us) service_pool.cc:224] Handling call
0709 13:02:29.298898 (+  1021us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:30.486900 (+1188002us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.487172 (+   272us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:30.491044 (+  3872us) log.cc:844] Serialized 26 byte log entry
0709 13:02:30.509274 (+ 18230us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:30.509340 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.509545 (+   205us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:31.179798 (+670253us) raft_consensus.cc:1685] finished
0709 13:02:31.179826 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:31.180041 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":199006,"raft.queue_time_us":300275,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":4618,"spinlock_wait_cycles":1349248,"thread_start_us":1020345,"threads_started":2,"wal-append.queue_time_us":421972}
I20260709 13:02:31.288529  8324 rpcz_store.cc:276] 0709 13:02:29.238772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.242432 (+  3660us) service_pool.cc:224] Handling call
0709 13:02:29.242832 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:29.453753 (+210921us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.453793 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:29.454600 (+   807us) log.cc:844] Serialized 26 byte log entry
0709 13:02:30.616002 (+1161402us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:30.616103 (+   101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.616261 (+   158us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:31.183863 (+567602us) raft_consensus.cc:1685] finished
0709 13:02:31.183890 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:31.260000 (+ 76110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9977,"raft.queue_time_us":338953,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":22190,"thread_start_us":1277572,"threads_started":2,"wal-append.queue_time_us":1688882}
I20260709 13:02:31.368391  8398 raft_consensus.cc:2410] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:02:31.432485  8329 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.383s	user 0.012s	sys 0.034s
I20260709 13:02:31.368160  8332 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:31.428079  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 114140us
W20260709 13:02:31.678203  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 310149us
I20260709 13:02:31.674736  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:31.686411  8325 consensus_meta.cc:287] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.013s	user 0.000s	sys 0.014s
I20260709 13:02:31.687909  8325 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:31.688565  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 1742) took 1428 ms. Trace:
W20260709 13:02:31.383773  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.671801  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:31.521498 10717 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.032s	sys 0.016s
W20260709 13:02:31.693041 10717 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.075s	user 0.034s	sys 0.024s
W20260709 13:02:31.383733  8327 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.335s	user 0.040s	sys 0.039s
W20260709 13:02:31.694937  8254 consensus_meta.cc:287] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.443s	user 0.004s	sys 0.039s
I20260709 13:02:31.695593  8254 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:31.368700  9008 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:31.688966  8325 rpcz_store.cc:276] 0709 13:02:30.280114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.671682 (+391568us) service_pool.cc:224] Handling call
0709 13:02:31.688488 (+1016806us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:31.385618  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:02:31.615800  8972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.789s	user 0.031s	sys 0.092s
I20260709 13:02:31.574426  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:31.449687  8328 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:31.177006 10847 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.222s	user 0.007s	sys 0.022s
I20260709 13:02:31.708298  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1639) took 2403 ms. Trace:
W20260709 13:02:31.662241 10382 consensus_meta.cc:287] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.167s	user 0.010s	sys 0.006s
W20260709 13:02:31.510426  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 144758us
W20260709 13:02:31.386117  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 73758.2us
I20260709 13:02:31.681975  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:31.703709  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:31.719696  8327 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.671s	user 0.045s	sys 0.050s
W20260709 13:02:31.545871  8331 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.173s	user 0.006s	sys 0.037s
W20260709 13:02:31.526264 10870 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.944s	user 0.012s	sys 0.009s
W20260709 13:02:31.726691  8972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.948s	user 0.032s	sys 0.100s
I20260709 13:02:31.386849  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:02:31.718091 10871 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.706s	user 0.010s	sys 0.029s
W20260709 13:02:31.734519 10923 log.cc:927] Time spent T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
W20260709 13:02:31.718465 10883 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.946s	user 0.011s	sys 0.014s
I20260709 13:02:31.566056  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:02:31.385948 10856 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.193s	user 0.014s	sys 0.016s
W20260709 13:02:31.598942 10690 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.003s	user 0.018s	sys 0.022s
W20260709 13:02:31.701742  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 2.10929e+06us
W20260709 13:02:31.715643  8401 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.806s	user 0.018s	sys 0.023s
W20260709 13:02:31.708669 10900 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.812s	user 0.006s	sys 0.032s
I20260709 13:02:31.545737 10917 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.386149  9000 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:31.154896 10781 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:31.700392 10650 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.384250 10647 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:31.384019 10903 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.747378  8254 rpcz_store.cc:276] 0709 13:02:29.292469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.580227 (+287758us) service_pool.cc:224] Handling call
0709 13:02:30.249720 (+669493us) spinlock_profiling.cc:243] Waited 26.9 ms on lock 0x7b5800002a98. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:02:30.250310 (+   590us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.696018 (+1445708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.696224 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":61798144}
W20260709 13:02:31.763355  8331 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.959s	user 0.014s	sys 0.041s
I20260709 13:02:31.763866  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:31.384196 10907 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.734282  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:31.546078 10835 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:31.771579  8401 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.485s	user 0.018s	sys 0.024s
W20260709 13:02:31.763453  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 1.98484e+06us
I20260709 13:02:31.674175 10560 consensus_queue.cc:1059] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.235s
I20260709 13:02:31.768352  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:02:31.830746  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.829286  9009 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:31.860100 10605 consensus_meta.cc:287] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.686s	user 0.006s	sys 0.014s
W20260709 13:02:31.830297  8298 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.098s	user 0.014s	sys 0.028s
W20260709 13:02:31.866199  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 538322us
W20260709 13:02:31.782112  9005 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.008s	user 0.042s	sys 0.059s
W20260709 13:02:31.866843  9005 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.093s	user 0.046s	sys 0.070s
W20260709 13:02:31.866164  8298 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.134s	user 0.016s	sys 0.030s
W20260709 13:02:31.867513  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 1.16577e+06us
W20260709 13:02:31.868649  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:31.869025  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.872793  8332 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:31.573114 10833 consensus_queue.cc:1059] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.200s
W20260709 13:02:31.774214 10856 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.701s	user 0.020s	sys 0.028s
W20260709 13:02:31.765502 10870 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.790s	user 0.012s	sys 0.015s
W20260709 13:02:31.781052 10690 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.750s	user 0.021s	sys 0.025s
W20260709 13:02:31.873584  8332 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 1738) took 1828 ms (client timeout 2413 ms). Trace:
W20260709 13:02:31.768133 10847 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.813s	user 0.012s	sys 0.035s
I20260709 13:02:31.777714 10650 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:31.909353 10650 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:31.910017 10650 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:31.910768 10650 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:31.888311  8332 rpcz_store.cc:269] 0709 13:02:30.045649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.329176 (+1283527us) service_pool.cc:224] Handling call
0709 13:02:31.873509 (+544333us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:31.894529  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:31.890527  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 115508us
I20260709 13:02:31.866729  9000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 617) took 1176 ms. Trace:
I20260709 13:02:31.899695  9007 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
I20260709 13:02:31.888751  8222 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:31.778366 10382 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.925946  8257 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:31.925623  8325 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:31.767204 10883 raft_consensus.cc:697] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 3 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:31.972435 10650 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:32.046157 10650 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:31.973348 10907 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:31.935130 10647 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:32.007630  8251 raft_consensus.cc:2393] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 3: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:32.046782 10883 consensus_queue.cc:237] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:32.048353  8399 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
W20260709 13:02:32.023864  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 90896.3us
W20260709 13:02:32.029178  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 253445us
I20260709 13:02:32.031113 10902 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.031647 10896 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:32.005971 10844 raft_consensus.cc:3055] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:32.028297 10930 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.935374 10835 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:31.881721 10917 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:32.037271 10922 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:32.053943 10647 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:32.063035  9008 raft_consensus.cc:2410] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:02:32.054011 10902 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:32.103240 10898 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.064345  8222 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:32.066952 10605 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.067726 10930 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:32.054669  8974 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:32.070022 10921 consensus_queue.cc:1059] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 13:02:32.054287 10896 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:32.057413 10844 raft_consensus.cc:740] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:32.075984  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1731) took 2823 ms. Trace:
W20260709 13:02:32.077597 10936 log.cc:927] Time spent T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:02:32.036093  8220 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:31.926473  9000 rpcz_store.cc:276] 0709 13:02:30.655313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.714768 (+ 59455us) service_pool.cc:224] Handling call
0709 13:02:30.960114 (+245346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.775487 (+815373us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.830083 (+ 54596us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:31.991731  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1644) took 1146 ms. Trace:
I20260709 13:02:32.049304 10907 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:32.081251  8972 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:32.103885 10624 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.021754 10903 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:32.093374 10891 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:32.103871 10898 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:32.110249 10902 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:32.106683  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 1741) took 1777 ms. Trace:
I20260709 13:02:32.117717  9007 raft_consensus.cc:1240] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:32.131705  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 1984) took 1519 ms. Trace:
W20260709 13:02:32.120234  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 189985us
I20260709 13:02:32.115127  8257 rpcz_store.cc:276] 0709 13:02:30.845827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.574071 (+728244us) service_pool.cc:224] Handling call
0709 13:02:31.991653 (+417582us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:32.111893  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 62625.4us
I20260709 13:02:32.125376 10922 raft_consensus.cc:697] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 8 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:32.161863  8220 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:32.163133 10898 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:32.109172  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 45126.4us
I20260709 13:02:32.163913 10898 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:32.112968 10896 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:32.109695 10908 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:32.164476 10898 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:32.165027 10908 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:32.165117 10898 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:32.165560 10908 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:32.157531  8399 rpcz_store.cc:276] 0709 13:02:30.612003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.629690 (+ 17687us) service_pool.cc:224] Handling call
0709 13:02:30.973119 (+343429us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b5800170798. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:02:32.131476 (+1158357us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b540019a5d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:32.131610 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":684617856}
I20260709 13:02:32.113303  8330 rpcz_store.cc:276] 0709 13:02:29.252704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.252857 (+   153us) service_pool.cc:224] Handling call
0709 13:02:29.271671 (+ 18814us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:29.304702 (+ 33031us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.304738 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:29.305208 (+   470us) log.cc:844] Serialized 26 byte log entry
0709 13:02:29.860664 (+555456us) spinlock_profiling.cc:243] Waited 34.6 ms on lock 0x7b48001701d8. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882d6e7aa1 00007f882d6deb37 00007f882d6de9ca 00007f882d7d626f 00007f882d7d5f37 00007f882d7ea55c 00007f882d7ea508 00007f882d7ea488 00007f882d7ea443 00007f882d7e96ec 00007f882c713a96
0709 13:02:31.368987 (+1508323us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:31.369083 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.369172 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:32.075528 (+706356us) raft_consensus.cc:1685] finished
0709 13:02:32.075555 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:32.075880 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56917,"raft.queue_time_us":335674,"raft.run_cpu_time_us":2801,"raft.run_wall_time_us":250042,"spinlock_wait_cycles":79637760,"thread_start_us":2063167,"threads_started":1,"wal-append.queue_time_us":2414802}
I20260709 13:02:32.162662 10922 consensus_queue.cc:237] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:32.114044 10844 consensus_queue.cc:260] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:32.109393 10605 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:32.155644  8325 rpcz_store.cc:276] 0709 13:02:30.259760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.715997 (+1456237us) service_pool.cc:224] Handling call
0709 13:02:32.036290 (+320293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:32.127702  8223 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:32.172389  8326 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.805s	user 0.032s	sys 0.069s
W20260709 13:02:32.180847  8326 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.032s	sys 0.070s
I20260709 13:02:32.131621 10855 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:32.111418 10930 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:32.202702  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:32.204700  8323 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.029s	sys 0.033s
W20260709 13:02:32.205106  8323 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.029s	sys 0.033s
I20260709 13:02:32.196982 10844 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:32.215072  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 134000us
I20260709 13:02:32.182613 10624 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:32.214272  8252 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.239s	user 0.014s	sys 0.053s
W20260709 13:02:32.217338  8252 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.242s	user 0.015s	sys 0.054s
W20260709 13:02:32.217402  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 80466.4us
I20260709 13:02:32.198091 10674 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.180120 10382 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:32.205648 10604 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.525s	user 0.045s	sys 0.094s
W20260709 13:02:32.218992 10935 log.cc:927] Time spent T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
W20260709 13:02:32.231988  8255 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.901s	user 0.027s	sys 0.081s
W20260709 13:02:32.267017  8255 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.936s	user 0.027s	sys 0.081s
W20260709 13:02:32.271339  8371 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.023s	sys 0.020s
W20260709 13:02:32.271973  8371 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.023s	sys 0.020s
W20260709 13:02:32.272468  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:32.273142  9004 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.586s	user 0.075s	sys 0.187s
I20260709 13:02:32.273420 10956 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.274356 10956 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:32.205411 10930 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:02:32.227384 10604 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.547s	user 0.046s	sys 0.094s
W20260709 13:02:32.273589  9004 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.586s	user 0.075s	sys 0.187s
I20260709 13:02:32.220432 10875 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:32.294016  8327 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.013s	sys 0.002s
W20260709 13:02:32.300699  8327 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.014s	sys 0.002s
I20260709 13:02:32.302034 10837 consensus_queue.cc:1048] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.119s
I20260709 13:02:32.302409 10903 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:32.303129 10835 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:32.305729 10674 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:32.222113 10855 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:32.332844 10835 consensus_queue.cc:1059] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:32.334307 10835 raft_consensus.cc:3055] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:32.334714 10835 raft_consensus.cc:740] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:32.335966 10835 consensus_queue.cc:260] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:32.337740 10835 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:32.338960 10674 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:32.339527 10674 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.339970 10674 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:32.340497 10896 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:32.360975  8328 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:32.323246 10917 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:32.367046  8399 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 7. Ops: []
W20260709 13:02:32.371369  8328 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 1740) took 2309 ms (client timeout 1935 ms). Trace:
I20260709 13:02:32.370867  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:32.373152  8328 rpcz_store.cc:269] 0709 13:02:30.051964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.653290 (+601326us) service_pool.cc:224] Handling call
0709 13:02:32.361727 (+1708437us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:32.363938 10875 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:32.365128  9004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 612) took 2938 ms. Trace:
W20260709 13:02:32.394265  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:32.375898 10914 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.016s	sys 0.001s
I20260709 13:02:32.374589 10781 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:02:32.430845  8372 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.010s	sys 0.014s
W20260709 13:02:32.435858  8372 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.011s	sys 0.014s
W20260709 13:02:32.431839  8370 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.009s	sys 0.024s
W20260709 13:02:32.439303  8370 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.009s	sys 0.024s
W20260709 13:02:32.438563  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 668864us
W20260709 13:02:32.434181  8369 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.002s	sys 0.031s
W20260709 13:02:32.445250  8369 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.003s	sys 0.031s
I20260709 13:02:32.445017  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1725) took 3236 ms. Trace:
I20260709 13:02:32.370355 10930 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:32.431274 10914 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.017s	sys 0.002s
W20260709 13:02:32.451148  9003 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.001s	sys 0.026s
W20260709 13:02:32.456749  9003 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.002s	sys 0.026s
I20260709 13:02:32.376296 10855 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
W20260709 13:02:32.453572  8329 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.009s	sys 0.019s
W20260709 13:02:32.463593  8329 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.010s	sys 0.019s
W20260709 13:02:32.450484 10900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.000s	sys 0.026s
W20260709 13:02:32.482565  9005 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.008s	sys 0.012s
W20260709 13:02:32.489943  9005 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.008s	sys 0.012s
I20260709 13:02:32.448884 10674 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:32.488112  9006 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.710s	user 0.016s	sys 0.043s
W20260709 13:02:32.484444 10900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.000s	sys 0.030s
W20260709 13:02:32.510778  9006 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.017s	sys 0.043s
I20260709 13:02:32.457369 10382 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:32.417133 10875 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:32.457504 10717 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.008s	sys 0.030s
I20260709 13:02:32.511791  8297 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:32.445940 10781 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:32.486704 10605 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:32.520399  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 62794.5us
I20260709 13:02:32.481200 10604 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:32.525146 10717 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.831s	user 0.009s	sys 0.031s
W20260709 13:02:32.543116  8256 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.008s	sys 0.024s
W20260709 13:02:32.543509  8256 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.924s	user 0.009s	sys 0.024s
I20260709 13:02:32.525631 10956 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:32.566068  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 1.1803e+06us
I20260709 13:02:32.535636 10624 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:32.571707  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 391764us
I20260709 13:02:32.533565 10921 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:32.594507  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 475573us
I20260709 13:02:32.592101  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1733) took 2548 ms. Trace:
I20260709 13:02:32.541188 10903 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.597600  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:32.568890 10896 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:32.596068  8329 rpcz_store.cc:276] 0709 13:02:30.043757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.048829 (+  5072us) service_pool.cc:224] Handling call
0709 13:02:30.049162 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.049388 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:31.735383 (+1685995us) log.cc:844] Serialized 14 byte log entry
0709 13:02:32.521084 (+785701us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:32.534477 (+ 13393us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:32.568284 (+ 33807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.568384 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:32.592032 (+ 23648us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2014913,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":106854,"thread_start_us":2228917,"threads_started":2}
I20260709 13:02:32.610015  8398 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:32.614187 10624 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:32.614806 10624 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.615396 10624 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:32.616736 10875 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:32.617390 10875 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:32.617995 10875 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:32.620981  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 136102us
I20260709 13:02:32.611981  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:32.585896 10870 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.543301 10604 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:32.542994 10605 consensus_queue.cc:1059] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 13:02:32.626814  9005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 593) took 1980 ms. Trace:
I20260709 13:02:32.620364 10875 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:32.602270 10867 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:32.591647 10908 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:32.621906  8372 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:32.638654  8223 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:32.648218 10976 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.598109 10898 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:32.648950 10976 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:32.651386 10898 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:32.652200 10867 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:32.634691 10747 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:32.653861 10747 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:32.657174 10604 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:32.657377  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:32.658037  8399 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:32.658373  8325 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 6: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:32.658857  8223 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:32.658913 10870 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:32.632682 10896 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.661011 10896 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:32.610119 10921 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:32.663414  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.664330  8220 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:32.624125 10883 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.672644  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:32.673329 10605 raft_consensus.cc:3055] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:32.673599 10833 consensus_queue.cc:1059] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:32.674008 10605 raft_consensus.cc:740] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:32.675542 10833 raft_consensus.cc:2764] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:02:32.677069  8329 raft_consensus.cc:1240] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:32.678171  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:32.678589 10690 raft_consensus.cc:3055] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:32.679198  8399 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 6: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:02:32.680284 10961 log.cc:927] Time spent T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:02:32.680371  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 70190.2us
I20260709 13:02:32.681473 10908 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:32.682444  8369 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:32.682636  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:32.683291  8330 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
I20260709 13:02:32.684161 10900 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:32.685031 10382 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:32.678211 10903 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:32.632623 10956 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.689890 10956 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:32.682457  8297 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:32.692302 10956 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:32.690181  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1646) took 1624 ms. Trace:
I20260709 13:02:32.695834 10970 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:32.696779 10970 consensus_queue.cc:1059] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:32.696867  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:32.407263  9004 rpcz_store.cc:276] 0709 13:02:29.426945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.427077 (+   132us) service_pool.cc:224] Handling call
0709 13:02:30.062045 (+634968us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b5800175898. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:02:30.062268 (+   223us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.062514 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:30.686725 (+624211us) log.cc:844] Serialized 14 byte log entry
0709 13:02:32.328897 (+1642172us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:32.359241 (+ 30344us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:32.364753 (+  5512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.364876 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:32.365048 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1295924,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":1649,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":2457,"spinlock_wait_cycles":50108800,"thread_start_us":2231654,"threads_started":2,"wal-append.queue_time_us":1586481}
I20260709 13:02:32.697999  8398 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 8: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:32.699067 10965 consensus_queue.cc:1059] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:32.699280  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:32.700441 10958 raft_consensus.cc:3055] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:32.700977 10958 raft_consensus.cc:740] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:32.702378 10983 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:32.703510  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 271) took 3086 ms. Trace:
W20260709 13:02:32.707365 10952 log.cc:927] Time spent T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.261s	user 0.001s	sys 0.000s
W20260709 13:02:32.707742 10979 log.cc:927] Time spent T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:02:32.679082 10690 raft_consensus.cc:740] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:02:32.711104 10960 log.cc:927] Time spent T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:02:32.711308 10690 consensus_queue.cc:260] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:32.712270  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 111739us
I20260709 13:02:32.712472 10650 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:32.712689 10921 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:32.712865 10650 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.712965 10690 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:32.714234  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1729) took 3483 ms. Trace:
I20260709 13:02:32.714612  8326 rpcz_store.cc:276] 0709 13:02:29.231160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.233531 (+  2371us) service_pool.cc:224] Handling call
0709 13:02:29.233958 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:31.366522 (+2132564us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:31.366578 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:31.367246 (+   668us) log.cc:844] Serialized 26 byte log entry
0709 13:02:32.181413 (+814167us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:32.181496 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.706342 (+524846us) spinlock_profiling.cc:243] Waited 78.8 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:32.713880 (+  7538us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b54003c9454. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:32.713932 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:32.713992 (+    60us) raft_consensus.cc:1685] finished
0709 13:02:32.714005 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:32.714162 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12622,"raft.queue_time_us":147014,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":13801,"spinlock_wait_cycles":798184960,"thread_start_us":2677087,"threads_started":2,"wal-append.queue_time_us":961665}
W20260709 13:02:32.717104  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 96380.5us
W20260709 13:02:32.695955 10953 log.cc:927] Time spent T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.207s	user 0.002s	sys 0.000s
I20260709 13:02:32.673391  8328 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:32.703102 10983 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:32.675315 10605 consensus_queue.cc:260] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:32.724808  9009 consensus_meta.cc:287] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.835s	user 0.014s	sys 0.000s
I20260709 13:02:32.661190 10870 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:32.632009  9005 rpcz_store.cc:276] 0709 13:02:30.646517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.688937 (+ 42420us) service_pool.cc:224] Handling call
0709 13:02:30.772902 (+ 83965us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.773393 (+   491us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:31.875264 (+1101871us) log.cc:844] Serialized 14 byte log entry
0709 13:02:32.613150 (+737886us) log.cc:844] Serialized 14 byte log entry
0709 13:02:32.625467 (+ 12317us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:32.625544 (+    77us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:32.625610 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.625739 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:32.626708 (+   969us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2681,"prepare.queue_time_us":596404,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":81,"thread_start_us":1750197,"threads_started":3}
W20260709 13:02:32.726586  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 155858us
I20260709 13:02:32.689173 10903 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:32.729024  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:32.729733  8326 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:32.727898  9009 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:32.713349 10650 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:32.725380  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 1974) took 3623 ms. Trace:
I20260709 13:02:32.724326  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:32.739336  9009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56948 (request call id 598) took 1405 ms. Trace:
I20260709 13:02:32.739817  9009 rpcz_store.cc:276] 0709 13:02:31.327803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.386301 (+ 58498us) service_pool.cc:224] Handling call
0709 13:02:32.733579 (+1347278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:32.651386 10976 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:32.735127  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:32.677035 10833 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:02:32.723771 10930 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:32.699340 10781 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:32.704056  8323 rpcz_store.cc:276] 0709 13:02:29.617050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.705066 (+ 88016us) service_pool.cc:224] Handling call
0709 13:02:29.705374 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.917132 (+211758us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:31.572426 (+1655294us) log.cc:844] Serialized 14 byte log entry
0709 13:02:32.266769 (+694343us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:32.291913 (+ 25144us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:32.299991 (+  8078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.694958 (+394967us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:32.703124 (+  8166us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54003f0cd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:32.703182 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:32.703415 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2139127,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":3760,"raft.queue_time_us":115793,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":159,"spinlock_wait_cycles":523646080,"thread_start_us":2028844,"threads_started":2,"wal-append.queue_time_us":804163}
I20260709 13:02:32.692950  8256 rpcz_store.cc:276] 0709 13:02:31.073226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.098404 (+ 25178us) service_pool.cc:224] Handling call
0709 13:02:31.450405 (+352001us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:31.618821 (+168416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:31.618861 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:31.619498 (+   637us) log.cc:844] Serialized 26 byte log entry
0709 13:02:32.543903 (+924405us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:32.592362 (+ 48459us) log.cc:844] Serialized 14 byte log entry
0709 13:02:32.592601 (+   239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.592691 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:32.689948 (+ 97257us) raft_consensus.cc:1685] finished
0709 13:02:32.689970 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:32.690116 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":70170,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":3657,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":5164,"thread_start_us":923920,"threads_started":1,"wal-append.queue_time_us":919993}
W20260709 13:02:32.754338  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 62292.5us
I20260709 13:02:32.756278  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1736) took 2707 ms. Trace:
W20260709 13:02:32.758193  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 52805.9us
W20260709 13:02:32.759044  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 98003.1us
I20260709 13:02:32.759135  9008 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:02:32.751317  8972 connection.cc:256] RPC call timeout handler was delayed by -2.306425521000051s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19730000000000003s for a response
I20260709 13:02:32.736174  8401 rpcz_store.cc:276] 0709 13:02:29.046779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.046918 (+   139us) service_pool.cc:224] Handling call
0709 13:02:29.047316 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:29.254869 (+207553us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.254913 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:29.260091 (+  5178us) log.cc:844] Serialized 26 byte log entry
0709 13:02:32.001174 (+2741083us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:32.512013 (+510839us) log.cc:844] Serialized 14 byte log entry
0709 13:02:32.624890 (+112877us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.633761 (+  8871us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:32.640010 (+  6249us) raft_consensus.cc:1685] finished
0709 13:02:32.640024 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:32.669891 (+ 29867us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":160692,"prepare.queue_time_us":172921,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":154112,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":33099,"thread_start_us":3243092,"threads_started":3,"wal-append.queue_time_us":2331337}
I20260709 13:02:32.760524  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:32.758298  8297 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:32.750075 10382 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:32.765364  8399 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
I20260709 13:02:32.766676  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:32.759862  8972 connection.cc:256] RPC call timeout handler was delayed by -2.1498255210000252s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2147s for a response
W20260709 13:02:32.767493  8972 connection.cc:256] RPC call timeout handler was delayed by -2.0148255210000343s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22970000000000002s for a response
W20260709 13:02:32.767958  8972 connection.cc:256] RPC call timeout handler was delayed by -1.9167255210000462s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24060000000000004s for a response
I20260709 13:02:32.751300 10921 raft_consensus.cc:3055] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:32.770018  9008 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:32.770342  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:32.770673  8222 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:32.772979  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:32.773721 10896 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:32.777560 10981 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.778239 10981 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:32.779866  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 52915.5us
I20260709 13:02:32.780956 10981 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:32.781539 10931 raft_consensus.cc:2764] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 13:02:32.448884  8331 rpcz_store.cc:276] 0709 13:02:29.208338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.208493 (+   155us) service_pool.cc:224] Handling call
0709 13:02:29.246013 (+ 37520us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:29.689228 (+443215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.689274 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:29.703361 (+ 14087us) log.cc:844] Serialized 52 byte log entry
0709 13:02:31.935314 (+2231953us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:32.378015 (+442701us) log.cc:844] Serialized 14 byte log entry
0709 13:02:32.434166 (+ 56151us) log.cc:844] Serialized 14 byte log entry
0709 13:02:32.434390 (+   224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.444717 (+ 10327us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:32.444771 (+    54us) raft_consensus.cc:1685] finished
0709 13:02:32.444788 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:32.444957 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83156,"prepare.queue_time_us":62676,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":88,"raft.queue_time_us":389629,"raft.run_cpu_time_us":380,"raft.run_wall_time_us":55719,"thread_start_us":2710841,"threads_started":3,"wal-append.queue_time_us":2266132}
I20260709 13:02:32.787386 10647 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:32.793772 10647 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.794384 10647 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:32.795658 10978 log.cc:927] Time spent T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.101s	user 0.004s	sys 0.000s
W20260709 13:02:32.796535 10891 consensus_meta.cc:287] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.652s	user 0.010s	sys 0.000s
I20260709 13:02:32.797339  9002 raft_consensus.cc:2410] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:32.798007 10855 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:32.798413 10855 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:32.798772 10855 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:32.798955  8222 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:32.799124 10837 consensus_queue.cc:1059] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:32.799815 10901 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.787415 10983 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:32.788786  8330 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
I20260709 13:02:32.738547 10883 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:32.809944 10844 consensus_meta.cc:287] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.591s	user 0.010s	sys 0.002s
I20260709 13:02:32.759037  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:32.770995 10921 raft_consensus.cc:740] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:32.814968  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1622) took 3535 ms. Trace:
I20260709 13:02:32.804605  8328 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:32.815402  8255 rpcz_store.cc:276] 0709 13:02:29.279977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.280123 (+   146us) service_pool.cc:224] Handling call
0709 13:02:29.280483 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:29.861483 (+581000us) spinlock_profiling.cc:243] Waited 6.82 ms on lock 0x7b4800002dd8. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882d6e7aa1 00007f882d6deb37 00007f882d6de9ca 00007f882d7d626f 00007f882d7d5f37 00007f882d7ea55c 00007f882d7ea508 00007f882d7ea488 00007f882d7ea443 00007f882d7e96ec 00007f882c713a96
0709 13:02:31.330032 (+1468549us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:32.277311 (+947279us) log.cc:844] Serialized 14 byte log entry
0709 13:02:32.630126 (+352815us) log.cc:844] Serialized 14 byte log entry
0709 13:02:32.743301 (+113175us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:32.763864 (+ 20563us) log.cc:844] Serialized 26 byte log entry
0709 13:02:32.769267 (+  5403us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:32.769291 (+    24us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.794883 (+ 25592us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:32.814709 (+ 19826us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:32.814773 (+    64us) raft_consensus.cc:1685] finished
0709 13:02:32.814787 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:32.814911 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":344230,"prepare.queue_time_us":596599,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":88,"raft.queue_time_us":442460,"raft.run_cpu_time_us":1460,"raft.run_wall_time_us":270235,"spinlock_wait_cycles":49764352,"thread_start_us":3049115,"threads_started":3,"wal-append.queue_time_us":390673}
I20260709 13:02:32.820477 10837 raft_consensus.cc:3055] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:32.820968 10837 raft_consensus.cc:740] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:32.821941 10837 consensus_queue.cc:260] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:32.823246 10837 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:32.823832 10647 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:32.824137 10855 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:32.825695 10382 raft_consensus.cc:697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 4 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:32.826365 10926 raft_consensus.cc:3055] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:32.826819 10926 raft_consensus.cc:740] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:32.826858 10382 consensus_queue.cc:237] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:32.804019  9009 raft_consensus.cc:2393] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 5: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:32.806000  8222 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:32.806313 10976 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:32.828255 10976 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.828090 10926 consensus_queue.cc:260] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:32.799851  9002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56940 (request call id 621) took 2142 ms. Trace:
I20260709 13:02:32.829537 10908 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:32.800733 10605 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:32.830049 10908 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.830482 10908 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:32.828675 10976 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:32.829753  9002 rpcz_store.cc:276] 0709 13:02:30.657223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.657519 (+   296us) service_pool.cc:224] Handling call
0709 13:02:32.799630 (+2142111us) spinlock_profiling.cc:243] Waited 2.14 s on lock 0x7b5400598dd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:32.799769 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4916115072}
I20260709 13:02:32.814092 10883 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:32.759132 10930 consensus_queue.cc:1059] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.073s
W20260709 13:02:32.840536 10967 log.cc:927] Time spent T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:02:32.800549 10901 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:32.843638  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:32.844961 10901 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:32.845824  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:32.847110 10958 consensus_queue.cc:260] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:32.848405 10958 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:32.850268 10930 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:32.845953  8400 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:32.850883 10930 raft_consensus.cc:697] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:32.757849  8327 rpcz_store.cc:276] 0709 13:02:30.044856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.047364 (+  2508us) service_pool.cc:224] Handling call
0709 13:02:30.047700 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.047956 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:31.763675 (+1715719us) log.cc:844] Serialized 14 byte log entry
0709 13:02:32.637007 (+873332us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:32.719013 (+ 82006us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:32.738001 (+ 18988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.745269 (+  7268us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:32.752083 (+  6814us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1970,"prepare.queue_time_us":2016109,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":66,"raft.queue_time_us":6428,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":102,"thread_start_us":2544858,"threads_started":3,"wal-append.queue_time_us":497465}
I20260709 13:02:32.827198 10647 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:32.893105 10946 log.cc:927] Time spent T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.400s	user 0.001s	sys 0.003s
I20260709 13:02:32.860373  9003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 595) took 1593 ms. Trace:
I20260709 13:02:32.851718  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:32.926739 10922 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.006s	sys 0.007s
I20260709 13:02:32.927031  9005 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:32.910686  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:32.893949  9007 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:32.910909  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:32.845094  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:32.929853  8398 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 6: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:32.859474 10747 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:32.852311  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:32.855433  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 97176us
W20260709 13:02:32.885493  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.904273  8326 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:32.861766 10835 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:02:32.927404 10922 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.006s	sys 0.007s
I20260709 13:02:32.886453  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:32.934722  9002 raft_consensus.cc:2410] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 13:02:32.935863  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:32.937779  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 166011us
I20260709 13:02:32.851122 10976 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:32.943746  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:32.944419  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.944978  8972 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:32.926177  9003 rpcz_store.cc:276] 0709 13:02:31.266882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.273660 (+  6778us) service_pool.cc:224] Handling call
0709 13:02:31.673517 (+399857us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:31.726818 (+ 53301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:31.726849 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:31.759313 (+ 32464us) log.cc:844] Serialized 26 byte log entry
0709 13:02:32.029525 (+270212us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b480002cdd8. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882d6e7aa1 00007f882d6deb37 00007f882d6de9ca 00007f882d7d626f 00007f882d7d5f37 00007f882d7ea55c 00007f882d7ea508 00007f882d7ea488 00007f882d7ea443 00007f882d7e96ec 00007f882c713a96
0709 13:02:32.480936 (+451411us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:32.480999 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.481121 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:32.839840 (+358719us) spinlock_profiling.cc:243] Waited 61.8 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:32.860131 (+ 20291us) raft_consensus.cc:1685] finished
0709 13:02:32.860158 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:32.860305 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4573,"raft.queue_time_us":157201,"raft.run_cpu_time_us":2785,"raft.run_wall_time_us":176090,"spinlock_wait_cycles":180677248,"thread_start_us":552109,"threads_started":1,"wal-append.queue_time_us":861533}
I20260709 13:02:32.857694 10867 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:32.950596  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.861724 10988 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.953603  8399 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:32.936877 10908 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:32.940608 10896 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:02:32.955796  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:32.957000  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.940868 10690 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.958544  8369 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:32.959123  9007 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:32.857044 10650 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:32.963003  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.964311  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:32.951769 10901 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:32.965154  8248 raft_consensus.cc:2393] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 5: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:32.965288  8254 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:32.814417 10921 consensus_queue.cc:260] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:32.966353  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:32.968575  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:32.968462 10988 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:32.971138  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:32.971611  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:32.971873  8257 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:32.971204  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:32.851971 10930 consensus_queue.cc:237] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:32.973400  8249 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 8: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:32.973137 10988 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:32.975648 10650 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:32.977257 10908 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:32.979866  8255 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:32.978082  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:32.981727  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:32.964982 10896 raft_consensus.cc:697] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:32.986469  8251 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 8: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:32.986729 10690 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:32.988689  8256 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
W20260709 13:02:32.990137  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.959224 10835 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:32.994546  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 87437.3us
I20260709 13:02:32.973141 10976 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:32.998260  9002 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 2. Current term is 3. Ops: []
W20260709 13:02:32.999156  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 90009.4us
I20260709 13:02:32.985939 10855 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:32.993197 10747 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:32.990875 10900 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:33.002194  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:33.001516 10624 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:33.004652 10747 raft_consensus.cc:697] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:33.005867 10747 consensus_queue.cc:237] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:33.006542 10624 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:33.001328 10896 consensus_queue.cc:237] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:33.011734 10969 log.cc:927] Time spent T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.339s	user 0.001s	sys 0.002s
I20260709 13:02:33.057874  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:33.012084  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:33.058830  9003 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
W20260709 13:02:33.066651  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 63094.6us
I20260709 13:02:33.066677  8256 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
I20260709 13:02:33.066815  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:33.069517  9002 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 13:02:33.070683  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.072094  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:33.080478  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 68518.8us
I20260709 13:02:33.081370  8369 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:33.082269  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:33.086632  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:33.087775 10870 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:33.088444 10870 raft_consensus.cc:697] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:33.088804  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:33.090024 10903 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:33.089551 10870 consensus_queue.cc:237] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:33.090126  9002 raft_consensus.cc:2410] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 5 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:02:33.091180  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:33.097899  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.090540 10903 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.100395  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:33.106618  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:33.101971  9003 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:02:33.108949 10901 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:33.114027  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:33.117929  8326 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:02:33.119737  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:33.124635  9005 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
W20260709 13:02:33.107617 10964 log.cc:927] Time spent T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.456s	user 0.003s	sys 0.000s
I20260709 13:02:33.107780 10903 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:33.128903  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:33.107218 10833 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.131093  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:33.132166  8399 raft_consensus.cc:2393] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 7: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:02:33.135421  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 63841.6us
I20260709 13:02:33.133561  9003 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:33.245805  8255 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:02:33.620121  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:33.137470  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:33.427995  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.163744  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:33.627745  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:33.389928 10959 log.cc:927] Time spent T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.737s	user 0.003s	sys 0.001s
I20260709 13:02:33.174508  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:33.671639  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:33.632431  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:33.672014  9001 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56940 (request call id 610) took 4245 ms (client timeout 2093 ms). Trace:
W20260709 13:02:33.643985  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:33.644486  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.295187  8398 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:33.647291  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:33.647243  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:33.647493  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:33.647219  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:33.653199 10903 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:33.675881  8330 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:33.662228 10902 raft_consensus.cc:3055] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:33.663352  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:33.664129  8251 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:33.664639  8327 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:33.622613 10926 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:33.666687 10971 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:33.162811 10833 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:33.624046 10921 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:33.668931 10958 consensus_meta.cc:287] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.820s	user 0.007s	sys 0.001s
I20260709 13:02:33.670264  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:33.670574  8249 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:33.217661 10690 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:33.212268  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:33.670949  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:33.690968  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:33.295492 10835 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:33.673542  8223 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:33.677065  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:33.693799  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:33.340977 10995 log.cc:927] Time spent T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
W20260709 13:02:33.672442  9001 rpcz_store.cc:269] 0709 13:02:29.426104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.427712 (+  1608us) service_pool.cc:224] Handling call
0709 13:02:33.671672 (+4243960us) spinlock_profiling.cc:243] Waited 2.96 s on lock 0x7b54006ce3d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:33.671828 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6797911296}
I20260709 13:02:33.679832  9002 raft_consensus.cc:2410] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:02:33.695180  9007 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 3.
W20260709 13:02:33.603536 10996 log.cc:927] Time spent T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 13:02:33.698905  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:33.699513  8400 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:33.297030 10930 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.682425  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:33.676898 10902 raft_consensus.cc:740] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:33.684640 10931 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:33.688886  8253 raft_consensus.cc:2410] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:02:33.693945  8256 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 8: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:33.678387  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:33.695690  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:33.696163  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:33.695241  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:33.701319  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:33.703367  9003 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:33.708926  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:33.708129  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 44883us
I20260709 13:02:33.710812  8401 raft_consensus.cc:2410] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:02:33.696121  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:33.713928  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:33.716840  8249 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:33.717414  9008 raft_consensus.cc:2393] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 8: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:33.692546 10747 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.715122  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:33.685252 10971 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:33.697999  9004 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
W20260709 13:02:33.722071  8369 connection.cc:256] RPC call timeout handler was delayed by -2.7419967879999945s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3267s for a response
I20260709 13:02:33.719511  8251 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:33.719049  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:33.723675  9009 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:33.723812  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:33.725749  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:33.724372  8256 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:33.727783  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:33.728271  8256 raft_consensus.cc:2393] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 8: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:33.712620 10690 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:33.729598  8369 connection.cc:256] RPC call timeout handler was delayed by -1.425222032000022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18300000000000002s for a response
I20260709 13:02:33.729466  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:02:33.730887  8369 connection.cc:256] RPC call timeout handler was delayed by -1.280322032000015s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19910000000000003s for a response
I20260709 13:02:33.730834  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:33.732903  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:33.720757  9000 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:33.736192  8369 connection.cc:256] RPC call timeout handler was delayed by -1.1309220320000009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2157s for a response
I20260709 13:02:33.737782  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:33.738534  9009 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:33.737251  9006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 600) took 2408 ms. Trace:
I20260709 13:02:33.739015  8255 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 3. Current term is 5. Ops: [3.3-3.3]
I20260709 13:02:33.740638  8253 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:33.741317  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:33.741894  8401 raft_consensus.cc:2393] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 6: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:33.728565  9004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56948 (request call id 601) took 1012 ms. Trace:
I20260709 13:02:33.743388  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:33.745787  8256 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:33.746074  8253 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:33.738636 10855 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.746678  8972 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:33.748188  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:33.748660  8398 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:33.739126  9006 rpcz_store.cc:276] 0709 13:02:31.328560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.366605 (+ 38045us) service_pool.cc:224] Handling call
0709 13:02:31.367080 (+   475us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:31.708055 (+340975us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:31.708087 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:31.771731 (+ 63644us) log.cc:844] Serialized 52 byte log entry
0709 13:02:32.526431 (+754700us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:32.526700 (+   269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.527171 (+   471us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:33.389146 (+861975us) spinlock_profiling.cc:243] Waited 57.1 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:33.732110 (+342964us) raft_consensus.cc:1685] finished
0709 13:02:33.732151 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.737179 (+  5028us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34398,"raft.queue_time_us":440583,"raft.run_cpu_time_us":578,"raft.run_wall_time_us":104299,"spinlock_wait_cycles":131426176,"thread_start_us":737460,"threads_started":1,"wal-append.queue_time_us":875459}
I20260709 13:02:33.735625 10835 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.752707  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:33.753480  9009 raft_consensus.cc:2410] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:02:33.748346  8369 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:33.754197  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:33.745568 10690 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.724967 10930 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:33.733400 10747 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:33.763448  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:33.763834  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:33.764437  8398 raft_consensus.cc:2393] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 6: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:33.765923  8251 raft_consensus.cc:2393] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 8: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:33.765866  8972 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:33.767926  8256 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:33.760650 10925 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:33.770385  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:33.766363 10903 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:33.750545 10855 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:33.755144 10835 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:33.771142  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:33.772053  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:33.772500  8295 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:33.762915 10965 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:33.778728  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:33.745893  9004 rpcz_store.cc:276] 0709 13:02:32.714127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.729718 (+ 15591us) service_pool.cc:224] Handling call
0709 13:02:33.212100 (+482382us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b5800175898. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:02:33.726408 (+514308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4452224}
I20260709 13:02:33.747349 10902 consensus_queue.cc:260] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:33.781710 10965 raft_consensus.cc:697] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:33.782253 10747 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:33.785897  8401 raft_consensus.cc:2368] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 7. Current term is 8.
I20260709 13:02:33.780886  8251 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:33.788156  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:33.779232 10950 log.cc:927] Time spent T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 1.261s	user 0.008s	sys 0.006s
I20260709 13:02:33.790689  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1645) took 2895 ms. Trace:
I20260709 13:02:33.774031 10925 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.791044  8252 rpcz_store.cc:276] 0709 13:02:30.923303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.951920 (+ 28617us) service_pool.cc:224] Handling call
0709 13:02:30.973554 (+ 21634us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:30.974192 (+   638us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.974236 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:30.974859 (+   623us) log.cc:844] Serialized 26 byte log entry
0709 13:02:32.217888 (+1243029us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:32.217930 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.218005 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:33.727122 (+1509117us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:33.790426 (+ 63304us) raft_consensus.cc:1685] finished
0709 13:02:33.790444 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.790633 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":118140,"raft.queue_time_us":1316,"raft.run_cpu_time_us":497,"raft.run_wall_time_us":179172,"spinlock_wait_cycles":43487488,"thread_start_us":1242594,"threads_started":1,"wal-append.queue_time_us":1404342}
I20260709 13:02:33.765534 10690 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:33.791719  8295 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:33.796186  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:33.801489 10902 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:02:33.803216  8256 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:33.804296  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:33.793299 10971 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:33.804955 10999 log.cc:927] Time spent T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:02:33.805855  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:33.791178 10833 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:33.793299 10855 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:33.795317 10925 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:33.806955  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:33.827205  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:33.827863  8401 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 8: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:33.801731 10965 consensus_queue.cc:237] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:33.809497  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:33.811537  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:33.811888 10971 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:33.812415  8399 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:02:33.826148  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:33.827885  8403 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:33.827721  8252 raft_consensus.cc:2376] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5. Re-sending same reply.
I20260709 13:02:33.834925  8251 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:33.836659 11004 consensus_queue.cc:1059] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:33.839040 11004 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:33.839665 11004 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:33.840270 11004 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:33.841380  8398 raft_consensus.cc:2410] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:02:33.845072  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.841970  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:33.847146  8400 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:33.847554  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:33.848178  8405 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:33.852906  8330 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "038cd0877a674c2996410c9f327df911"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:02:33.853883  8330 raft_consensus.cc:606] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:02:33.854288  8330 raft_consensus.cc:614] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:33.855969  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:33.857116  8249 raft_consensus.cc:2393] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 7: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:33.861836  8397 raft_consensus.cc:1275] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:33.863799  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:33.864403  8256 raft_consensus.cc:2376] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7. Re-sending same reply.
I20260709 13:02:33.866178  8369 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:33.863714  8298 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:33.875962  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:33.844156 10971 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:34.014287 11001 log.cc:927] Time spent T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.269s	user 0.001s	sys 0.000s
I20260709 13:02:34.074653 11006 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:34.078884 11006 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:34.080542 11006 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:34.080353 11011 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:34.084363 11011 raft_consensus.cc:697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:34.086001 11011 consensus_queue.cc:237] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:34.098026  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 234303us
I20260709 13:02:34.099572  8298 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:34.099684 11006 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:34.101935 11006 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:34.102850 11004 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:34.106593 11004 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:34.114307  8298 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:34.262089  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:34.262797  8402 raft_consensus.cc:2393] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 5: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:34.263798  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:34.264349  8402 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:34.265192  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:34.265744  8402 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:34.266539  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:34.267632  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:34.268646  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:34.269223  8402 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:34.270165  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:34.271185  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:34.306561 10847 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:34.307238 10847 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:34.308730 10847 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:34.327533 10925 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:34.329727 10925 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:34.343405  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:34.344307  8323 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 3.
I20260709 13:02:34.344648  8324 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:02:34.345358  8332 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:34.343529  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:34.346486  8329 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 8: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:34.347409  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:34.348155  8329 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:02:34.350466  8332 raft_consensus.cc:1240] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:02:34.351497  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:34.352077  8332 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:34.352893  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:34.356732  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:34.361488  8329 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:34.362516  8332 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:34.362649  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:34.363395  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:34.363859  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:34.364120  8332 raft_consensus.cc:2393] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 5: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:34.364912  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:34.365370  8332 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:34.373397  8332 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:34.375248  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:34.375976  8329 raft_consensus.cc:2393] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 5: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:34.378494  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:34.378707  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:34.383636  8249 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:34.384475  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:34.384966  8249 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:34.385913  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:34.387713  8249 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:34.388782  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:34.389300  8249 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:34.390115  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:34.390623  8249 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:34.390789  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1611) took 1426 ms. Trace:
I20260709 13:02:34.391212  8250 rpcz_store.cc:276] 0709 13:02:32.964695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.964809 (+   114us) service_pool.cc:224] Handling call
0709 13:02:32.982192 (+ 17383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:33.001699 (+ 19507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.001737 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:33.011837 (+ 10100us) log.cc:844] Serialized 26 byte log entry
0709 13:02:33.411934 (+400097us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:33.411998 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.390418 (+978420us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b54000762d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:34.390467 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:34.390512 (+    45us) raft_consensus.cc:1685] finished
0709 13:02:34.390524 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.390732 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6206,"raft.queue_time_us":3177,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":20821,"spinlock_wait_cycles":27721600,"thread_start_us":216379,"threads_started":1,"wal-append.queue_time_us":705857}
I20260709 13:02:34.392899  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:34.379231  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:34.395258  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:34.396063  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:34.396796  8972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.003s	sys 0.007s
W20260709 13:02:34.397109  8972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.003s	sys 0.007s
I20260709 13:02:34.397462  8332 raft_consensus.cc:1240] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 13:02:34.397364  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 626837us
I20260709 13:02:34.398684  8332 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 13:02:34.399876  8332 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:02:34.400144  8332 pending_rounds.cc:85] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 7
I20260709 13:02:34.400343  8332 pending_rounds.cc:107] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:02:34.400444 10690 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:34.400611  8332 raft_consensus.cc:2889] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:34.403771 10925 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:34.404335 10925 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:34.406505 10925 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:34.409040 10930 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:34.411887  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:34.413627 10926 consensus_meta.cc:287] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.730s	user 0.009s	sys 0.004s
I20260709 13:02:34.413822  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:34.417325  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:34.419138  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:34.421830  8329 raft_consensus.cc:1275] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:02:34.420284 11012 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.425119  8329 pending_rounds.cc:85] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 2
I20260709 13:02:34.419955  8250 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 7: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:02:34.422499  8327 consensus_meta.cc:287] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.736s	user 0.009s	sys 0.001s
I20260709 13:02:34.429508  8327 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:34.422565  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:34.423339 11011 consensus_queue.cc:1059] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:34.429982 11007 raft_consensus.cc:2764] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:02:34.430414  8250 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:02:34.429999  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 1743) took 1314 ms. Trace:
I20260709 13:02:34.430832 11011 raft_consensus.cc:3055] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:34.431265 11011 raft_consensus.cc:740] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:34.432034 11011 consensus_queue.cc:260] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:34.432593  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:34.432868 11011 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:34.425231 11012 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:34.434376  8332 raft_consensus.cc:1275] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:34.435120 11012 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:34.435461  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 586386us
I20260709 13:02:34.436439  8332 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:02:34.431034  8327 rpcz_store.cc:276] 0709 13:02:33.116079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.123996 (+  7917us) service_pool.cc:224] Handling call
0709 13:02:34.429947 (+1305951us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.437683  8332 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:02:34.428714  8249 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:34.429170  8329 pending_rounds.cc:107] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:02:34.438122  8329 raft_consensus.cc:2889] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:34.438732  8405 consensus_meta.cc:287] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.578s	user 0.008s	sys 0.000s
I20260709 13:02:34.439062  8405 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:34.420663 10925 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.439602 10925 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:34.441408  8401 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:34.442195 10925 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:34.442535 11003 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:34.442866 11010 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:34.443307 11010 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.443723 11010 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:34.445758  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 331382us
I20260709 13:02:34.447016  8298 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:34.448136  8298 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:34.449045 11003 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:34.449576 11003 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.450022 11003 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:34.450766 11021 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:34.452276 11021 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:34.452642 11021 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:34.458665  8397 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:34.459460 11003 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:34.463168 11003 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:34.442417 10902 raft_consensus.cc:3055] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:34.464666 10902 raft_consensus.cc:740] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:34.465466 10902 consensus_queue.cc:260] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:34.466617 10902 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:34.470459 10747 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:02:34.475452  9000 consensus_meta.cc:287] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.723s	user 0.013s	sys 0.000s
I20260709 13:02:34.475880  9000 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:34.476522 11024 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.483747  8298 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:34.484175  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.477035 11024 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:34.487998  8398 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:34.488806  8397 raft_consensus.cc:1275] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:34.489688  8323 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:02:34.490254  8323 raft_consensus.cc:606] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:34.490557  8323 raft_consensus.cc:614] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:34.492183 10847 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:34.491695 11024 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:34.493363 11023 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:34.498658  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:34.499022  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:34.502780 11021 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:34.518066  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.503027 11010 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:34.509914 11021 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:34.529119  8295 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.005s	sys 0.004s
W20260709 13:02:34.529505  8295 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.007s	sys 0.004s
W20260709 13:02:34.529788  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 742842us
I20260709 13:02:34.531536  8295 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:34.532680  8295 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:34.510481  8296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.838s	user 0.018s	sys 0.008s
I20260709 13:02:34.533594 11022 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:34.533860  8296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.018s	sys 0.008s
I20260709 13:02:34.534026 11022 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:34.534403 11022 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:34.535414  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 98404.3us
I20260709 13:02:34.512172  8323 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:02:34.536192  8369 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:34.536671  8323 raft_consensus.cc:606] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:34.537048  8323 raft_consensus.cc:614] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:34.507440  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:34.517061  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.539978  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:34.540722  8329 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:34.542802  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:34.543040  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:34.543558  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1622) took 1246 ms. Trace:
I20260709 13:02:34.543718  8332 raft_consensus.cc:2393] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 6: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:34.543912  8248 rpcz_store.cc:276] 0709 13:02:33.297484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.321709 (+ 24225us) service_pool.cc:224] Handling call
0709 13:02:34.542636 (+1220927us) spinlock_profiling.cc:243] Waited 6 ms on lock 0x7b5800002a98. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:02:34.543470 (+   834us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13795072}
I20260709 13:02:34.544489  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:34.545684  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:34.546278  8325 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:34.547078  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:34.548116  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:34.548599  8325 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:34.548869 11022 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:34.550988  8253 consensus_meta.cc:287] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.803s	user 0.009s	sys 0.000s
I20260709 13:02:34.552309  8253 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:34.554615  8369 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:34.555507 10965 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:34.555912 10965 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.536994 10747 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:34.556324 10965 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:34.559952  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:34.560528  8323 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
W20260709 13:02:34.560681  8974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.783s	user 0.005s	sys 0.001s
W20260709 13:02:34.561080  8974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.005s	sys 0.001s
I20260709 13:02:34.561321  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:34.561906 10925 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:34.562181  8974 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:34.562459 10925 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:34.563340 11012 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:34.563983  8974 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:34.564656 10925 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:34.565191 10965 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:34.551739  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:34.564747 11012 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:34.566495  8329 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:34.567371  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:34.555823  8369 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:34.567866  8974 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:34.567888 10965 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:34.567914  8329 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:34.562280 11011 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:34.570627  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:34.571380  8399 raft_consensus.cc:2393] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 5: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:34.571060 11011 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:34.573125 11012 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:34.573869 11012 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.574188 11012 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:34.574954  8400 consensus_meta.cc:287] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.727s	user 0.010s	sys 0.000s
I20260709 13:02:34.575274  8400 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:34.575989  8402 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:34.569708 10747 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:34.577767 10747 raft_consensus.cc:697] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:34.578800 10747 consensus_queue.cc:237] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:34.580649  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:34.581401  8404 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:34.573140 11011 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:34.570295  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:34.568585  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:34.549319  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:34.585803  8295 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:02:34.588378  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 54985.2us
I20260709 13:02:34.585804  8369 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:34.568585  8327 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:34.593082 11018 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:34.593375  8369 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:34.593662 11018 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.586022 10965 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:34.594152 11018 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:34.594513 10965 raft_consensus.cc:697] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:34.594743  8369 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:34.595944  8369 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:34.596809 10870 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:34.597738 11019 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:34.598359 11019 raft_consensus.cc:697] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:34.598186  8295 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:34.599387 11019 consensus_queue.cc:237] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:34.600600  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:34.591924  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:34.601197  9009 raft_consensus.cc:2376] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5. Re-sending same reply.
I20260709 13:02:34.601951  8331 raft_consensus.cc:2410] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:02:34.602407  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:34.604056  9000 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:34.605103 11024 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:34.595618 10965 consensus_queue.cc:237] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:34.561897  8323 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:34.605324  8327 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:34.551090 11022 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:34.596993  8369 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:34.609637  8323 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:02:34.610330  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:34.605815 10903 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:34.612653 10903 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.612993 10903 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:34.605767 11021 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:34.613879 11021 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.614341 11021 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:34.617115 11010 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:34.617621 11003 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:34.618299 10930 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.569144  8974 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:34.619096  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:34.624897  8331 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:02:34.607030 11024 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:34.626788 11012 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:34.630767  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:34.632311  8329 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:34.655130 11032 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:34.659121  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 48897.7us
I20260709 13:02:34.659399 11035 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:34.665459  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 96316.9us
I20260709 13:02:34.665841 11011 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:02:34.666913 10931 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.946s	user 0.004s	sys 0.013s
W20260709 13:02:34.662324  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 46939.8us
I20260709 13:02:34.723414  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:34.669513 10930 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:34.731472  8256 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
W20260709 13:02:34.732208  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 71232.4us
I20260709 13:02:34.732517 10930 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:34.733595  8372 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:34.734884  8406 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:34.685989 11004 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.737278  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:34.737946  8404 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:34.739171  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:34.742878 11040 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:34.743286  9008 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:34.743362 11040 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.743937 11040 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:34.744619  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:34.745208  9006 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:34.723002 10931 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.004s	user 0.004s	sys 0.014s
I20260709 13:02:34.745769 10931 raft_consensus.cc:697] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 6 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:34.746726 10931 consensus_queue.cc:237] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:34.737305 11036 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:02:34.749663  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:34.749738  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:34.750321  9003 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:34.753696  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:34.754354  9007 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:34.754573  9006 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:34.736289 11004 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:34.755234  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:34.756417  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:34.757005  9004 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:34.726931 10902 raft_consensus.cc:2764] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:02:34.757671  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:34.758235  9009 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:34.758565  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:34.669344 10870 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:34.662629 11020 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:34.727665 11032 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.764829 11036 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:34.757009 11004 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:34.771513 11036 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.771893 11036 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:34.664839 11023 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.765573  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:34.764170  8297 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:34.727504 11035 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.775213 11035 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:34.729768 11011 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.775990 11011 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:34.775964 11021 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:34.780040 11036 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:34.790031  8404 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:34.721747 11022 consensus_queue.cc:1059] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:02:34.791858  8974 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:34.670864 10903 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:34.799394 11006 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:34.800029 11006 raft_consensus.cc:697] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 6 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:34.789088 11011 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:34.800978 11006 consensus_queue.cc:237] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:34.803123 11011 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:34.803876 11012 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:34.806583  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:34.808169  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:34.770181  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:34.810633  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 73795.9us
I20260709 13:02:34.810940  9003 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
I20260709 13:02:34.811754  8372 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:34.785104 11032 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:34.818176  8398 raft_consensus.cc:2393] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 7: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:34.793166 11020 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:34.823603 11020 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:34.787156 11023 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:34.852555  9001 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:02:34.856375  8257 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:34.857254  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1613) took 1886 ms. Trace:
I20260709 13:02:34.860282  8257 rpcz_store.cc:276] 0709 13:02:32.970714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.855728 (+1885014us) service_pool.cc:224] Handling call
0709 13:02:34.856141 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.856950 (+   809us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.857186 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.877415 11036 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:34.878458 11018 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:34.838914 10870 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:34.888710  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 165320us
I20260709 13:02:34.896406 10971 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:34.897147 11023 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:34.881291 11018 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:34.900416  8331 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:34.898000 11021 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:34.893056 10870 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:34.932509 10870 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:34.902312 11024 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:34.902755  8327 raft_consensus.cc:1275] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:34.938478  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:34.939604  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:34.940250  8403 consensus_meta.cc:287] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 1.066s	user 0.006s	sys 0.001s
I20260709 13:02:34.940599  8403 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:34.940871  8327 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:34.941110  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 2002) took 1123 ms. Trace:
I20260709 13:02:34.903438  8325 raft_consensus.cc:3055] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:34.905795  8331 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:34.941250  8405 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7. Ops: []
W20260709 13:02:34.906747  8255 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.119s	user 0.005s	sys 0.004s
I20260709 13:02:34.910298 10903 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:34.910554  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.913861 10934 consensus_queue.cc:1059] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:34.913887  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:34.916356  9005 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56940 (request call id 624) took 1288 ms (client timeout 1623 ms). Trace:
W20260709 13:02:34.924444  8222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.139s	user 0.014s	sys 0.012s
W20260709 13:02:34.942938  8255 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.157s	user 0.006s	sys 0.004s
I20260709 13:02:34.945688 10934 raft_consensus.cc:3055] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:34.948596 10934 raft_consensus.cc:740] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:34.949540 10934 consensus_queue.cc:260] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:34.950872 10934 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:34.945972  8329 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:34.945921  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:34.954885  8327 raft_consensus.cc:2393] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 6: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:02:34.946058  9005 rpcz_store.cc:269] 0709 13:02:33.627495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.627633 (+   138us) service_pool.cc:224] Handling call
0709 13:02:34.916082 (+1288449us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b540057d5d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:34.916268 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2659596032}
I20260709 13:02:34.946374  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:34.941567  8325 raft_consensus.cc:740] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:02:34.946218  8222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.161s	user 0.014s	sys 0.012s
W20260709 13:02:34.958999  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.962343  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:34.963027  8402 raft_consensus.cc:2393] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 7: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:34.941485  8403 rpcz_store.cc:276] 0709 13:02:33.817668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.817801 (+   133us) service_pool.cc:224] Handling call
0709 13:02:34.941058 (+1123257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":200320}
I20260709 13:02:34.972944  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:34.973937  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:34.985594  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 162466us
I20260709 13:02:34.974090  8325 consensus_queue.cc:260] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:34.982252 11035 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:34.995399 11035 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:34.996939  9004 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:34.998176 11040 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:35.002599 11032 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:35.004673 11032 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:35.006839 10835 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:35.015261  8397 raft_consensus.cc:2393] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 7: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:35.019497  8369 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:35.020426 11046 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:35.020565  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:35.021713  8369 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:35.021919 11036 raft_consensus.cc:2764] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:02:35.022594 11036 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:35.022903 11045 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:35.035249  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:35.035836  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:35.036325  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:35.036690  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.045418  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:35.046247  8256 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:35.052417  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:35.055677  8223 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:35.058504 11045 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:35.069860  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:35.046197  8186 debug-util.cc:398] Leaking SignalData structure 0x7b0800064b60 after lost signal to thread 7167
I20260709 13:02:35.035363 11006 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:35.073182  8249 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 2.
I20260709 13:02:35.073230 11045 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:35.073557  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1657) took 1329 ms. Trace:
I20260709 13:02:35.023079 11036 raft_consensus.cc:697] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:35.073932  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:02:35.076493  8186 debug-util.cc:398] Leaking SignalData structure 0x7b08001c6ec0 after lost signal to thread 8189
I20260709 13:02:35.084043  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:35.082369  8250 rpcz_store.cc:276] 0709 13:02:33.744402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.431159 (+686757us) service_pool.cc:224] Handling call
0709 13:02:34.431412 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.073171 (+641759us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:35.073201 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:35.073267 (+    66us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:35.073301 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:35.073362 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:35.073502 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":629095,"raft.queue_time_us":646236}
W20260709 13:02:35.085496  8186 debug-util.cc:398] Leaking SignalData structure 0x7b0800064da0 after lost signal to thread 8281
W20260709 13:02:35.085801  8186 debug-util.cc:398] Leaking SignalData structure 0x7b0800064dc0 after lost signal to thread 8282
W20260709 13:02:35.086632  8186 debug-util.cc:398] Leaking SignalData structure 0x7b0800381320 after lost signal to thread 8356
I20260709 13:02:35.076568  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:35.077180  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:35.078071  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:35.087260  8254 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:35.079638 11006 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:35.088183  8252 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:35.089038  8250 raft_consensus.cc:1275] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:35.089975 10833 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:35.091280 11036 consensus_queue.cc:237] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:35.095398  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:35.096465  8330 raft_consensus.cc:2368] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 7. Current term is 8.
I20260709 13:02:35.078584  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:35.058769 10926 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.098291 10926 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:35.098639  8222 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:35.099524 10855 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:35.099936 10855 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.100138  8222 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:02:35.100440  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.100831 10690 consensus_queue.cc:1048] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:35.081245  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:35.082581  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 96304.4us
I20260709 13:02:35.102306  8256 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:35.100327 10926 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:35.103296  8372 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:35.103523 10926 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:35.103799  8222 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:35.104045 10926 raft_consensus.cc:697] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 8 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:35.104435 11046 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:35.105029 11046 raft_consensus.cc:697] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:35.086449  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:35.123714  8328 raft_consensus.cc:2393] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 5: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:35.100329 10855 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:35.083500  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1652) took 1340 ms. Trace:
I20260709 13:02:35.070780 10931 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.104923 10926 consensus_queue.cc:237] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:35.104049 10921 consensus_meta.cc:287] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.413s	user 0.008s	sys 0.005s
W20260709 13:02:35.169899  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 409177us
I20260709 13:02:35.105808  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:35.172703  8249 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:35.001364 10870 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:35.106112 11008 consensus_queue.cc:1059] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:02:35.106979  8186 debug-util.cc:398] Leaking SignalData structure 0x7b08001d9b80 after lost signal to thread 8430
W20260709 13:02:35.178725  8186 debug-util.cc:398] Leaking SignalData structure 0x7b0800338f40 after lost signal to thread 8431
W20260709 13:02:35.108561 10973 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 2.453s	user 0.024s	sys 0.023s
I20260709 13:02:35.108984  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:35.182324  8323 raft_consensus.cc:606] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:02:35.182634  9006 raft_consensus.cc:2393] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 7: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:35.109747  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:35.183259  9002 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:35.110725  9008 raft_consensus.cc:1240] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:35.105733 11042 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:02:35.187639 11006 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:35.189322  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:35.114075  8253 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:35.113983  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:35.191497  8251 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:35.081820  8248 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:35.115293  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:35.071738 11040 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:35.132563  8222 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:02:35.150271  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 423042us
I20260709 13:02:35.126775  8255 rpcz_store.cc:276] 0709 13:02:33.743356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.743535 (+   179us) service_pool.cc:224] Handling call
0709 13:02:33.751378 (+  7843us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:33.751607 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.751621 (+    14us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:33.761600 (+  9979us) log.cc:844] Serialized 52 byte log entry
0709 13:02:34.947581 (+1185981us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:35.017155 (+ 69574us) log.cc:844] Serialized 14 byte log entry
0709 13:02:35.083147 (+ 65992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:35.083225 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:35.083269 (+    44us) raft_consensus.cc:1685] finished
0709 13:02:35.083284 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:35.083446 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1000038,"prepare.queue_time_us":131748,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":30,"raft.queue_time_us":148307,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":25781,"thread_start_us":1230461,"threads_started":2,"wal-append.queue_time_us":307844}
I20260709 13:02:35.168262 11045 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:35.175539  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:35.172084  8298 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:35.180035  8186 debug-util.cc:398] Leaking SignalData structure 0x7b08002709e0 after lost signal to thread 9034
I20260709 13:02:35.178092  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:35.182736  8323 raft_consensus.cc:614] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:35.128575 11032 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:35.259689  8186 debug-util.cc:398] Leaking SignalData structure 0x7b080037fcc0 after lost signal to thread 9035
I20260709 13:02:35.187237  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:02:35.260308  8186 debug-util.cc:398] Leaking SignalData structure 0x7b08002d3760 after lost signal to thread 10560
W20260709 13:02:35.260591  8186 debug-util.cc:398] Leaking SignalData structure 0x7b0800268ac0 after lost signal to thread 10604
W20260709 13:02:35.193893  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 429626us
W20260709 13:02:35.260831  8186 debug-util.cc:398] Leaking SignalData structure 0x7b080012f2e0 after lost signal to thread 10650
W20260709 13:02:35.261056  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 145810us
W20260709 13:02:35.261564  8186 debug-util.cc:398] Leaking SignalData structure 0x7b0800270360 after lost signal to thread 10837
W20260709 13:02:35.261901  8186 debug-util.cc:398] Leaking SignalData structure 0x7b08001e1c40 after lost signal to thread 10875
W20260709 13:02:35.262174  8186 debug-util.cc:398] Leaking SignalData structure 0x7b08002dd300 after lost signal to thread 10896
W20260709 13:02:35.262413  8186 debug-util.cc:398] Leaking SignalData structure 0x7b08001a2bc0 after lost signal to thread 10898
W20260709 13:02:35.262642  8186 debug-util.cc:398] Leaking SignalData structure 0x7b08002dd4c0 after lost signal to thread 10900
W20260709 13:02:35.262850  8186 debug-util.cc:398] Leaking SignalData structure 0x7b08003b83c0 after lost signal to thread 10901
W20260709 13:02:35.263190  8186 debug-util.cc:398] Leaking SignalData structure 0x7b08002dd340 after lost signal to thread 10908
W20260709 13:02:35.263607  8186 debug-util.cc:398] Leaking SignalData structure 0x7b08002dd2a0 after lost signal to thread 10945
W20260709 13:02:35.263839  8186 debug-util.cc:398] Leaking SignalData structure 0x7b080008f7e0 after lost signal to thread 10946
I20260709 13:02:35.226719  8249 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:35.260793  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:35.266016  9003 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:02:35.263922  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:35.267525  8186 debug-util.cc:398] Leaking SignalData structure 0x7b080037cdc0 after lost signal to thread 10952
W20260709 13:02:35.268141  8186 debug-util.cc:398] Leaking SignalData structure 0x7b08003ae060 after lost signal to thread 10966
W20260709 13:02:35.268689  8186 debug-util.cc:398] Leaking SignalData structure 0x7b08003b0520 after lost signal to thread 10978
W20260709 13:02:35.269001  8186 debug-util.cc:398] Leaking SignalData structure 0x7b08003a5520 after lost signal to thread 10979
W20260709 13:02:35.279379  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 223772us
I20260709 13:02:35.267426  8332 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:35.169745 10690 consensus_queue.cc:1048] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.096s
I20260709 13:02:35.280902  8402 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:35.281792  8223 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:35.261688 10833 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:35.185187  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:35.260026 11032 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.263845 11019 raft_consensus.cc:2764] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:02:35.305267  8253 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
W20260709 13:02:35.278734 10973 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:02:35.181578 11004 consensus_queue.cc:1059] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.104s
I20260709 13:02:35.170795 10931 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:35.270045 11033 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:35.313501  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:35.263916 11018 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:02:35.296562 11014 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.325109  9007 raft_consensus.cc:1275] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:02:35.325474  9007 pending_rounds.cc:85] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e: Aborting all ops after (but not including) 6
I20260709 13:02:35.320659  9008 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
I20260709 13:02:35.325819  9007 pending_rounds.cc:107] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e: Aborting uncommitted NO_OP operation due to leader change: 7.7
W20260709 13:02:35.322708  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 55090.9us
I20260709 13:02:35.293769 10903 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:35.326543  8297 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:02:35.323256  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 55362.9us
I20260709 13:02:35.260092 11034 consensus_queue.cc:1059] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.130s
I20260709 13:02:35.312762 11032 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:35.321185  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 149131us
I20260709 13:02:35.279208 11042 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:02:35.327550  9008 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:35.326145  9007 raft_consensus.cc:2889] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:35.311851 11041 raft_consensus.cc:3055] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:35.325680  9003 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:35.339622  8297 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:35.339659  8372 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:35.311244 11048 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.333961  8298 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:35.285311 11024 consensus_queue.cc:1059] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:02:35.342725  8402 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:02:35.324613 11022 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:35.333289  8295 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.762s (SENT) [suppressed 11 similar messages]
I20260709 13:02:35.350162 11034 raft_consensus.cc:3055] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:35.350518 11057 raft_consensus.cc:3055] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:35.285465 11044 consensus_queue.cc:1059] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.059s
I20260709 13:02:35.335480 10921 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:35.322161 11012 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:35.332665  9000 raft_consensus.cc:2393] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 6: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:35.327060 10903 raft_consensus.cc:697] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:35.359997  8298 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:35.350219 11058 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:02:35.379403  8297 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:35.320490 11039 consensus_queue.cc:1059] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:02:35.359786 11034 raft_consensus.cc:740] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:35.382403  8298 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:35.383450  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 251000us
I20260709 13:02:35.383760 11069 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.392261  8325 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:35.338837 11041 raft_consensus.cc:740] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:35.385231 11065 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:35.336838 11014 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:35.360188 10934 consensus_queue.cc:1059] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:02:35.360705 11022 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.391952 11069 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:35.393507  9002 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:02:35.385004 10833 raft_consensus.cc:697] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:35.394415  8298 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:35.395434 10902 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:35.396086 10971 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:35.394285 11040 raft_consensus.cc:3055] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:02:35.384514  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 103510us
I20260709 13:02:35.403844  8297 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:35.393620 10930 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:35.389894 10965 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:35.359462 11057 raft_consensus.cc:740] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:35.411271 11062 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:35.411718 11039 raft_consensus.cc:3055] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:35.412220 10934 raft_consensus.cc:3055] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:35.411296 11022 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:35.396210 10903 consensus_queue.cc:237] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:35.413990 11045 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:35.413657 11067 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:35.355995 11010 consensus_queue.cc:1059] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:02:35.396595 11023 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:35.398069 11004 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:35.442180  8325 raft_consensus.cc:1275] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:35.360404 11048 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:35.451243 11072 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:35.454114 11066 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:35.392611 11074 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:02:35.390594 11012 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:35.393340 11038 consensus_queue.cc:1059] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.088s
I20260709 13:02:35.427737 10835 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:35.321143 11046 consensus_queue.cc:237] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:35.479811  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 188338us
W20260709 13:02:35.481269  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.482568 11062 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:35.481962  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 78223.2us
I20260709 13:02:35.500346  8297 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:35.443481  8298 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:35.444183 11040 raft_consensus.cc:740] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:35.470752 10970 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:35.466179 10870 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:35.486598  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 1986) took 1140 ms. Trace:
I20260709 13:02:35.470178 11039 raft_consensus.cc:740] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:35.474086 10930 raft_consensus.cc:697] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:35.472779 10934 raft_consensus.cc:740] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:35.478152 11065 raft_consensus.cc:697] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:02:35.575258  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 74982.9us
I20260709 13:02:35.575071 10855 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:35.501824 11066 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:35.478643 10902 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:35.576686  8297 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:35.576013 11065 consensus_queue.cc:237] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:35.499426 11072 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:35.578042 11039 consensus_queue.cc:260] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:35.579383 11039 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:35.579634 11072 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:35.503335 10833 consensus_queue.cc:237] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:35.580312 10902 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:35.580869 11076 log.cc:927] Time spent T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.124s	user 0.000s	sys 0.005s
I20260709 13:02:35.483569 11004 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:35.481313  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 148020us
I20260709 13:02:35.581563 11004 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:35.512367 11069 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:35.488505 11023 raft_consensus.cc:697] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:35.481660 11018 consensus_queue.cc:1048] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:02:35.497570 10965 raft_consensus.cc:697] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:02:35.540633  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.515838  8324 rpcz_store.cc:276] 0709 13:02:34.345805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.345937 (+   132us) service_pool.cc:224] Handling call
0709 13:02:34.346205 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.346407 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:34.368232 (+ 21825us) log.cc:844] Serialized 14 byte log entry
0709 13:02:34.611412 (+243180us) log.cc:844] Serialized 14 byte log entry
0709 13:02:34.681818 (+ 70406us) log.cc:844] Serialized 14 byte log entry
0709 13:02:35.128584 (+446766us) spinlock_profiling.cc:243] Waited 1.65 ms on lock 0x7b1800143020. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882564bc0a 00007f882ed8b219 00007f88256d3a40 00007f88256ce4e4 00007f88256ce455 00007f88287b6060 00007f88287c8716 00007f88287d9287 00007f88287d9208
0709 13:02:35.461616 (+333032us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:35.461690 (+    74us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:35.461760 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:35.461868 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:35.486534 (+ 24666us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":711876,"prepare.queue_time_us":20065,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":4897,"raft.queue_time_us":714234,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":28999,"spinlock_wait_cycles":3792128,"thread_start_us":524523,"threads_started":4,"wal-append.queue_time_us":13272}
I20260709 13:02:35.483336 11010 consensus_queue.cc:1243] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:02:35.572439 11020 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:35.580789 10902 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:35.512818 11024 raft_consensus.cc:3055] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:02:35.582505  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 83992us
I20260709 13:02:35.583606  8297 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:02:35.537621  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 94247.1us
I20260709 13:02:35.502794 10835 raft_consensus.cc:697] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:35.582525 10921 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:35.572820 11008 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.583396 10855 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:35.583292 11066 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:35.637972  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:35.640924  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 57448.6us
I20260709 13:02:35.600646 11007 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.609045 11072 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:35.672508  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 207181us
I20260709 13:02:35.509270 11014 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:35.696651  9005 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:35.697758  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:35.698408  8249 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:35.699537  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:35.700727  8253 raft_consensus.cc:2393] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 5: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:35.618165 10902 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:35.609344 11079 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.618165 11024 raft_consensus.cc:740] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:35.711719  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:35.714246  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:35.715605  8297 leader_election.cc:400] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 7. Current term is 8.
I20260709 13:02:35.715997  8297 leader_election.cc:403] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:35.712188  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:35.673985 11020 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:35.708938  8220 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:35.484462 11041 consensus_queue.cc:260] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:35.630642 10931 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:35.636080 10921 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.765674 10921 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:35.769773  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:35.538683 11057 consensus_queue.cc:260] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:35.770785  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:35.771067 11057 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:35.772132  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:35.772980  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.711483 11065 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:35.773895 11065 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:35.775604 11080 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.776155 11080 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:35.503558 11062 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:35.778549 11080 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:35.715947 11007 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:35.779220 10930 consensus_queue.cc:237] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:35.781477 11007 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:35.782413  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 107726us
I20260709 13:02:35.782629 11024 consensus_queue.cc:260] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:35.783794  8297 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:35.784080 11024 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:35.785992 11003 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:35.786790 11003 consensus_queue.cc:1059] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:35.773317  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:35.782648  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.785298  8297 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:35.795176  8253 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:35.712900  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:35.760792 11032 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:35.747653  8250 raft_consensus.cc:2410] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:35.666134 11070 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.749922  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:35.750411  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:35.750927  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:35.751088  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:35.803661  8254 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 2.
I20260709 13:02:35.804046  8251 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:35.749922  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:35.804656  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:35.675324 11008 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:35.754742 11023 consensus_queue.cc:237] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:35.755229  8249 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:35.653560 11040 consensus_queue.cc:260] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:35.756286 10835 consensus_queue.cc:237] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:35.670032 11083 consensus_queue.cc:1059] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.088s
I20260709 13:02:35.757656 10965 consensus_queue.cc:237] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:35.816414  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:35.816932  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:35.749465 11014 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 7. Current term is 8.
I20260709 13:02:35.762121  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:35.764472 10970 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:35.799571  8297 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:35.787561 11003 raft_consensus.cc:3055] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:35.801143  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:37920 (request call id 20) took 1179 ms. Trace:
I20260709 13:02:35.801990 11007 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:35.807123  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:35.809695  8256 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:35.712786 11067 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.087s
I20260709 13:02:35.705096 11072 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:35.753705 11074 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:35.811977  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 103125us
I20260709 13:02:35.814060  8397 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:02:35.815270  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:35.816282  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:35.819638  8405 raft_consensus.cc:1240] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:35.819942  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:35.779237 11062 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:35.825224  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 112372us
I20260709 13:02:35.800316 11065 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:35.826586  8323 rpcz_store.cc:276] 0709 13:02:34.608817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.609515 (+   698us) service_pool.cc:224] Handling call
0709 13:02:35.788480 (+1178965us) spinlock_profiling.cc:243] Waited 572 ms on lock 0x7b540020ec54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287b8853 00007f882d78660a 00007f8826d0c7d4 00007f8826d0c762 00007f8826d0c691 00007f8826d0c60c 00007f8826d0b8c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:35.788594 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1314643840}
I20260709 13:02:35.801268  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:35.857259  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:35.842669 11067 consensus_queue.cc:1243] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer ff7a6fe533e94c1bae555b4cae427728 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:02:35.829375  8297 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:35.748056 11079 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:35.831055  8324 raft_consensus.cc:1240] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:35.831274  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:35.862552  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:35.862682  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:35.863093  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:35.863335  8332 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:35.864647  8332 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:35.832079 11046 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:35.832962  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:35.834436  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:35.834897  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:35.867920  8330 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:35.836213  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:35.836613  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:35.807898 10926 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:35.842929  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:35.868012  8220 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:35.845141  8255 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:35.870957  8297 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:35.819175 11041 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:35.849575  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:35.827566 10970 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:35.828150 11003 raft_consensus.cc:740] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:35.855551  8257 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 3.
W20260709 13:02:35.708072  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 70156.7us
I20260709 13:02:35.841012 11040 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:35.828780  8250 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:35.856432  8369 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:35.859750  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:35.844161 11074 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.869820 11046 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.879096  8254 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:35.877082  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:35.869858 11012 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:35.877137  8972 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:35.889319  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:35.851356 11014 consensus_queue.cc:1059] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:02:35.813337 11070 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:35.896196  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:35.898285  8972 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:35.898839  8323 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:35.905308  8369 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:35.873500 10926 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.907433  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:35.908099  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:35.919659  8326 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:35.874788 11048 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:35.876024 11007 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:35.893034 10870 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:35.924472  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:35.935508  8323 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:35.936444  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:35.897091 11046 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:35.925494  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:35.894068 11074 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:35.927332  8328 raft_consensus.cc:1275] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:35.913738 11065 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:35.932698  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:35.892729 11023 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:35.896078 11086 consensus_queue.cc:1048] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 13:02:35.938206  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:35.940172  8332 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:35.931136  8972 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:02:35.962366  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 94409.6us
I20260709 13:02:35.986927  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:36.119506  8323 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:36.119988  8400 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:36.016594  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:36.091308  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:36.013885  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:35.936632 11033 consensus_meta.cc:287] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.609s	user 0.010s	sys 0.002s
I20260709 13:02:36.122068 11007 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:36.134416 11033 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:35.942165 10965 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:36.134895 11007 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:35.941229 11081 consensus_queue.cc:1059] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.339s
I20260709 13:02:35.984601 11034 consensus_queue.cc:260] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:36.136786 11034 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:35.935797 11012 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:36.137717  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:36.001971 10921 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:36.138515  8403 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:36.139176  8327 raft_consensus.cc:2393] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 4: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:36.136001 11081 raft_consensus.cc:3055] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:36.139679 11081 raft_consensus.cc:740] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:36.140547 10921 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:36.122040  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.142843  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2012) took 1017 ms. Trace:
I20260709 13:02:36.095757  8330 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:36.141084  8220 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:36.143225  8405 rpcz_store.cc:276] 0709 13:02:35.125916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.126056 (+   140us) service_pool.cc:224] Handling call
0709 13:02:35.175251 (+ 49195us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.845701 (+670450us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.142641 (+296940us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b54000fc8d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:36.142790 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1481128704}
I20260709 13:02:36.144101 11089 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:36.144244  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:36.144529 11089 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:36.144912  8254 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:36.144946 11089 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:36.143338  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:36.146238  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.146384  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:36.147094  9005 raft_consensus.cc:2393] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 8: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:36.147511  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:36.148034  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:36.148938  9004 raft_consensus.cc:2410] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:02:36.149026  9007 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:02:36.149794  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:02:36.143808 11022 consensus_meta.cc:287] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.658s	user 0.011s	sys 0.000s
I20260709 13:02:36.150791  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:36.137687 11012 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:36.151580 11012 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:36.152576  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:36.153268  8253 raft_consensus.cc:2393] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 7: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:36.153575 10921 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.154022  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:36.154085 10921 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:36.154690  8253 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:36.156170 10921 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:36.150262  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.158224  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:36.150661 11022 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:36.107163  8251 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:36.160243  8324 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:02:35.979434  8328 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:02:36.159449  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:36.161233  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:36.161755  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:36.162285  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.162636  8254 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:36.155489  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:02:36.143278 11004 consensus_meta.cc:287] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.551s	user 0.007s	sys 0.003s
I20260709 13:02:36.167331  8326 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:02:36.084625  8250 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:35.928855 10926 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:36.168462  8401 raft_consensus.cc:2393] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 4: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:36.137176 11007 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:35.943920  8331 raft_consensus.cc:2393] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 7: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:02:35.942611  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 71723.8us
W20260709 13:02:36.176436  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 245309us
I20260709 13:02:36.177410  8972 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:02:35.986378 11090 log.cc:927] Time spent T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:02:36.179445  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:36.195315  9002 raft_consensus.cc:2393] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 6: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:36.187549  8297 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:36.193053  8257 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:36.193668  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.198333  8399 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:36.199812  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:35.997669 11072 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:35.986501 11087 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:36.019264 10930 raft_consensus.cc:3055] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:36.209458  8254 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:35.980849 11092 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:36.204873  8297 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:02:36.241037  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 335825us
I20260709 13:02:36.209914  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:36.223600  8326 raft_consensus.cc:1240] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:02:35.986639 11032 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:36.026839 11079 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:35.981287 11064 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.314s
W20260709 13:02:36.249317  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 44510.9us
I20260709 13:02:36.209870  8323 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:36.198002 10931 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:36.223537 11007 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:36.261696  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 421201us
I20260709 13:02:36.234490 11087 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.263098  8295 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:36.264473 11098 raft_consensus.cc:2764] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:02:36.264842  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:36.283466  8399 raft_consensus.cc:2393] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 6: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:36.267251  8400 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:36.239557 10930 raft_consensus.cc:740] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:36.268169 11032 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:36.270056  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:36.271838  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:36.282838 10921 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:36.287346  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:36.287981  8400 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
I20260709 13:02:36.288877  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:36.291052 11022 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:36.293851  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:36.294098 10902 consensus_meta.cc:287] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.552s	user 0.009s	sys 0.000s
I20260709 13:02:36.294535  8406 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:36.294687 10902 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:36.295387  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:36.295724  8329 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:36.296080  8406 raft_consensus.cc:2393] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 7: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:36.296902  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:36.297446  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:36.298507  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:36.299019  8329 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:36.299718  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:36.300154  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.300421  8329 raft_consensus.cc:2410] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 9 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:02:36.279023 11070 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:36.303213  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:36.304407 10902 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:36.306753  8326 raft_consensus.cc:1275] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:02:36.307255  8326 pending_rounds.cc:85] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 2
I20260709 13:02:36.307530  8326 pending_rounds.cc:107] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:02:36.307843  8326 raft_consensus.cc:2889] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:36.246030 11092 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.192324 11004 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:36.304538  8403 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:02:36.284662  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:36.308203  8329 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:36.267968  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:36.275601 11089 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:36.309912  8295 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:35.981130 11091 consensus_queue.cc:1048] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.320s
I20260709 13:02:36.018389 11003 consensus_queue.cc:260] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:36.320273  8406 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:36.319448  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:36.337656  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:36.338555  8329 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:36.339581  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:36.340282  8329 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:36.328004  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:36.262305 11007 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.346019  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:36.347113  8327 raft_consensus.cc:2410] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:36.335137  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:36.348168  8403 raft_consensus.cc:2393] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 6: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:36.341208  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:36.350078  8329 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:36.343406  8323 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:36.344179  8405 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:02:36.358981  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 49150.6us
I20260709 13:02:36.359910  8295 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:36.365012  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:36.365573  8295 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:36.367491  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:36.283169 11087 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:36.358940  8297 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:36.374182  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 106267us
I20260709 13:02:36.367058  8295 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:36.373543  8332 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
W20260709 13:02:36.381312  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 203973us
I20260709 13:02:36.381489  8253 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:36.261194 11012 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:36.374850  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:36.380228  8325 raft_consensus.cc:2393] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 7: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:36.378329  8295 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:36.380075 10835 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:36.326175 11092 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:36.386081  8328 raft_consensus.cc:606] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
W20260709 13:02:36.374137 11020 consensus_meta.cc:287] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.574s	user 0.009s	sys 0.000s
I20260709 13:02:36.406539  8328 raft_consensus.cc:614] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:36.366837 11007 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:36.386847  8295 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:36.373730 10902 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:36.385701  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:36.412781  8972 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:36.413718  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 167256us
I20260709 13:02:36.413615  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:36.395318  8253 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:36.379668 11022 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:36.403963 10747 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:02:36.388520 10970 consensus_meta.cc:287] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.513s	user 0.009s	sys 0.000s
I20260709 13:02:36.388160 11010 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:36.407223 11070 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:36.416836  8369 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:36.410233 10835 raft_consensus.cc:697] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 5 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:36.415253 10921 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:36.378448 11003 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:36.383644 11098 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:36.410516 11020 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:36.406937  8153 catalog_manager.cc:5697] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:02:36.434626  8329 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
W20260709 13:02:36.425908 11039 consensus_meta.cc:287] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.843s	user 0.006s	sys 0.006s
I20260709 13:02:36.412343 11064 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:02:36.420529  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 252853us
W20260709 13:02:36.421828 11057 consensus_meta.cc:287] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.647s	user 0.012s	sys 0.000s
I20260709 13:02:36.487984 11064 raft_consensus.cc:697] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:36.490643 11108 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:36.491487 11108 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:36.491715  8248 raft_consensus.cc:2410] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:02:36.492026 11108 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.492511 11108 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:36.420488 11048 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:36.445789 10835 consensus_queue.cc:237] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:36.493773 11048 raft_consensus.cc:697] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:36.493999  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:36.494688  8248 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:02:36.494728 11048 consensus_queue.cc:237] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:36.432934 11032 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:36.426918 11010 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.426479 10970 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:36.501605  9000 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:36.501816  9002 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:36.435250 11098 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.502466 11098 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:36.502676 10970 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:36.450345 11012 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:36.450511 11041 consensus_meta.cc:287] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.574s	user 0.008s	sys 0.000s
I20260709 13:02:36.506677  8327 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:02:36.507170  8327 pending_rounds.cc:85] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 3
I20260709 13:02:36.507380  8327 pending_rounds.cc:107] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:02:36.507606  8327 raft_consensus.cc:2889] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:36.488991 11064 consensus_queue.cc:237] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:36.450911  8151 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55754 (request call id 198) took 14767 ms (client timeout 15000 ms). Trace:
I20260709 13:02:36.439340 10921 raft_consensus.cc:697] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 5 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:36.513777 10921 consensus_queue.cc:237] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:36.514523  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:36.515700  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:36.516400  8253 raft_consensus.cc:2410] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:02:36.455487 11046 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:36.460134  8257 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:02:36.426532 11070 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.432737 11091 consensus_queue.cc:1059] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:02:36.464924 11074 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:36.519796 11070 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:36.469586  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:36.521003  9005 raft_consensus.cc:2393] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 4: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:36.470757 11089 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:36.405241 10930 consensus_queue.cc:260] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:36.479939 11040 consensus_meta.cc:287] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.576s	user 0.008s	sys 0.002s
I20260709 13:02:36.482285 11004 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:36.464823 10926 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:36.528708 10926 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:36.490514  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 109101us
I20260709 13:02:36.428412  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:36.529765  8297 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:02:36.529939  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 101577us
I20260709 13:02:36.531039  8369 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:36.531096  8297 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:36.531837 11032 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:36.532039  8251 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 8. (term mismatch)
I20260709 13:02:36.532212 11008 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:36.532529  8251 pending_rounds.cc:85] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 7
I20260709 13:02:36.532567 11032 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:36.532907  8251 pending_rounds.cc:107] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260709 13:02:36.533277  8251 raft_consensus.cc:2889] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:36.498179  8255 consensus_meta.cc:287] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.627s	user 0.009s	sys 0.000s
I20260709 13:02:36.438512  8153 catalog_manager.cc:5697] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 4 to 6, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 6 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:02:36.500655 11010 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:36.509127  8151 rpcz_store.cc:269] 0709 13:02:21.650212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.650502 (+   290us) service_pool.cc:224] Handling call
0709 13:02:21.885496 (+234994us) spinlock_profiling.cc:243] Waited 38.1 ms on lock 0x7b54003bf2a8. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e9b0331 00007f882e9ab3fc 00007f882e57d1b4 00007f882e67b37e 00007f882cb3c2a4 00007f882cb3c232 00007f882cb3c161 00007f882cb3c0dc 00007f882cb3b392 00007f8826af948c 00007f8826af88c6
0709 13:02:36.411726 (+14526230us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:29.934909 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:29.942808 (+  7899us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:29.942867 (+    59us) write_op.cc:435] Acquired schema lock
0709 13:02:29.942890 (+    23us) tablet.cc:662] Decoding operations
0709 13:02:29.959835 (+ 16945us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:29.959926 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:29.959952 (+    26us) tablet.cc:685] Acquiring locks for 14 operations
0709 13:02:29.961216 (+  1264us) tablet.cc:701] Row locks acquired
0709 13:02:29.961271 (+    55us) write_op.cc:260] PREPARE: finished
0709 13:02:29.961643 (+   372us) write_op.cc:270] Start()
0709 13:02:29.967156 (+  5513us) write_op.cc:276] Timestamp: P: 1783602149961371 usec, L: 0
0709 13:02:29.967175 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:02:30.011548 (+ 44373us) log.cc:844] Serialized 5432 byte log entry
0709 13:02:32.684333 (+2672785us) op_driver.cc:464] REPLICATION: finished
0709 13:02:32.886301 (+201968us) spinlock_profiling.cc:243] Waited 39.6 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f88256b40ac 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882c1f4ef0
0709 13:02:32.996197 (+109896us) write_op.cc:301] APPLY: starting
0709 13:02:33.003799 (+  7602us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:33.073706 (+ 69907us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:33.073757 (+    51us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:33.080644 (+  6887us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:33.081066 (+   422us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=14
0709 13:02:33.081113 (+    47us) write_op.cc:312] APPLY: finished
0709 13:02:33.110054 (+ 28941us) log.cc:844] Serialized 101 byte log entry
0709 13:02:33.129693 (+ 19639us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:33.205472 (+ 75779us) write_op.cc:454] Released schema lock
0709 13:02:33.205561 (+    89us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:33.206184 (+   623us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:33.206224 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":294218,"reported_tablets":22,"spinlock_wait_cycles":87591296,"child_traces":[["op",{"apply.queue_time_us":279628,"apply.run_cpu_time_us":5632,"apply.run_wall_time_us":224041,"num_ops":14,"prepare.queue_time_us":1350956,"prepare.run_cpu_time_us":46572,"prepare.run_wall_time_us":2034717,"raft.queue_time_us":444801,"raft.run_cpu_time_us":22970,"raft.run_wall_time_us":332647,"replication_time_us":2668771,"spinlock_wait_cycles":91062656,"thread_start_us":3026950,"threads_started":4,"wal-append.queue_time_us":2120924,"wal-append.run_cpu_time_us":34485,"wal-append.run_wall_time_us":2308753}]]}
I20260709 13:02:36.533394  8297 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:36.534049  8252 raft_consensus.cc:2393] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 4: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:36.535043  8153 catalog_manager.cc:5697] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:36.536435  8249 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:36.538175  8222 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:36.534036  8255 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:36.533859  8369 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:36.554957  8255 raft_consensus.cc:1275] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:02:36.517405  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 75624.9us
I20260709 13:02:36.525141  8248 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:36.429883 10931 raft_consensus.cc:697] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:36.558374  8248 raft_consensus.cc:1275] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:36.558487 11101 consensus_queue.cc:1059] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:02:36.559054 10833 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:36.559800  8248 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:36.526512  8254 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:36.561960  8249 raft_consensus.cc:1275] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:36.562616 11012 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.563206 11012 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:36.559309 11101 consensus_queue.cc:1243] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 13:02:36.565788  8220 leader_election.cc:336] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.819s (SENT)
I20260709 13:02:36.579205  8220 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:36.550381  8297 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:36.554382  8153 catalog_manager.cc:5697] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 2 to 3, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 3 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:36.571451 11012 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:36.533735 11041 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:36.567257  8369 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:36.534365 11004 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:36.620550  8369 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:36.600835  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 188073us
I20260709 13:02:36.540511 11003 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.554095 10970 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:36.622442  8972 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:36.554463 11089 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:36.625061  8255 raft_consensus.cc:1240] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:36.625056  8972 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:36.628358  8153 catalog_manager.cc:5697] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 2 to 4, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 4 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:02:36.629923  8972 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:02:36.632296  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.529266 11092 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:36.599691 11036 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:36.600050 11041 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:36.662410  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:36.564919 11062 consensus_queue.cc:1059] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.072s
I20260709 13:02:36.665481  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:36.628126 11039 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:36.630491 11012 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:36.676785  8972 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:36.628247 11089 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.601624 11022 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:36.677807  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:36.678359  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:02:36.684202  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 127842us
I20260709 13:02:36.633420 11091 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:36.678411  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:36.678026  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:36.688771  9007 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:36.678051  9002 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:36.682276  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:36.659147 10926 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:36.618816 11070 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:36.695848  8972 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:36.665702 11036 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.674080 11041 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:36.671137 11106 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:36.683316 10747 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:36.677789 11107 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.714026 11114 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:36.717609 11046 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:36.718139 11108 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:36.704965 11036 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:36.718611 11070 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:36.691169 11089 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:36.718855 11087 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:36.699688 10931 consensus_queue.cc:237] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:36.719739 11032 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:36.691568 11039 raft_consensus.cc:697] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:36.691413  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:02:36.720486 11117 log.cc:927] Time spent T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:02:36.720908 11106 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:36.722069  8327 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:02:36.719758 11119 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:36.722909  8329 pending_rounds.cc:85] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 4
I20260709 13:02:36.795802  8329 pending_rounds.cc:107] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:02:36.796118  8329 raft_consensus.cc:2889] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:36.723793  8326 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:36.724423  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:36.806015  8330 raft_consensus.cc:2393] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 6: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:36.725667 11045 raft_consensus.cc:3055] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:36.751446  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:36.691581 11006 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:02:36.701229 11116 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:36.741892  9001 raft_consensus.cc:2393] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 9: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:36.682870 11064 consensus_queue.cc:1059] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 13:02:36.822197 11064 raft_consensus.cc:3055] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:36.762127 11066 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.823318 11064 raft_consensus.cc:740] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:36.824004 11066 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:36.824760 11064 consensus_queue.cc:260] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:36.826386 11066 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:36.826578 11064 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:36.827519 11024 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:36.827611  8298 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:36.828151 11024 raft_consensus.cc:697] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 9 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:36.828398 11072 raft_consensus.cc:3055] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:36.828773 11072 raft_consensus.cc:740] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:36.829182 10971 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:36.829566 11072 consensus_queue.cc:260] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:36.829952 10971 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:36.831691 11081 consensus_queue.cc:260] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:36.832365 11092 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:36.832723 11081 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:36.834019  9007 raft_consensus.cc:2410] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:02:36.834434  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.630587 11003 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:36.755618  8153 catalog_manager.cc:5697] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 4 to 5, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 5 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
W20260709 13:02:36.759660 11115 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:02:36.790464  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 74866.8us
I20260709 13:02:36.833746  8331 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:36.721412 11010 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:36.753135 11074 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:36.726557 11007 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:36.789359 11106 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:36.758901 11114 consensus_queue.cc:1059] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:02:36.808970 11045 raft_consensus.cc:740] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:36.817777 11087 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:36.761970 11063 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:36.754482 11018 consensus_queue.cc:1048] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 13:02:36.899860 11080 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:36.901732  8323 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:36.834975  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 78083.2us
W20260709 13:02:36.839922  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.840811  8298 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:36.792135  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:36.904559  8327 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:36.904695  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:36.851181  8404 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:02:36.792969 11032 raft_consensus.cc:697] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:02:36.860981  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.871034  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:36.872711  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:36.878304  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:36.883721  8153 catalog_manager.cc:5697] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 4 to 5, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 5 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:36.891253 11003 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:36.747087 11107 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:36.920424  8369 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:36.917922  8326 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:36.916306  8324 raft_consensus.cc:2410] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:02:36.922984  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:36.923251  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 82506.8us
W20260709 13:02:36.925999  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.923555  8327 raft_consensus.cc:2376] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8. Re-sending same reply.
I20260709 13:02:36.951349  9003 raft_consensus.cc:2410] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:02:36.921577 10690 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.937985 10902 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:36.967188  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:36.968253  8326 raft_consensus.cc:2368] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 7.
I20260709 13:02:36.935667 11063 raft_consensus.cc:697] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:36.920923 11057 consensus_queue.cc:1048] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:02:36.829176 11024 consensus_queue.cc:237] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:36.944870 11087 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.935504 11018 consensus_queue.cc:1059] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:02:36.971326  8220 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:36.937790 11066 consensus_queue.cc:1059] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:02:36.984345  8220 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:36.949134 10747 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:36.985152  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 137246us
I20260709 13:02:36.946044 11080 consensus_queue.cc:1048] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:02:36.944295 11046 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:02:36.989739 10926 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:36.923362 11039 consensus_queue.cc:237] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:37.020640 11121 log.cc:927] Time spent T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
W20260709 13:02:37.019763 11126 log.cc:927] Time spent T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:02:37.017997 10931 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:36.988515  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:37.015928 11003 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:36.985503 11125 log.cc:927] Time spent T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:02:36.960012 11040 consensus_queue.cc:1048] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:02:36.967504 10690 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:36.986358  8220 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:37.114706  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 128421us
I20260709 13:02:37.006383 11032 consensus_queue.cc:237] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:37.115760  8220 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:37.116629 11066 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:37.117389 10926 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:37.117287 11066 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:37.117841 10926 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.118232 10926 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:37.121241 11004 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:37.123210 10930 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:37.124521  8220 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:37.071313  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 132059us
I20260709 13:02:37.034469 10833 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:36.955142 11045 consensus_queue.cc:260] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:36.986593 11087 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:37.086825  8326 raft_consensus.cc:2393] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 4: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:02:37.087663  8325 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 289) took 1250 ms (client timeout 1590 ms). Trace:
I20260709 13:02:37.089524  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1689) took 1198 ms. Trace:
W20260709 13:02:37.140659  8325 rpcz_store.cc:269] 0709 13:02:35.836993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.862851 (+ 25858us) service_pool.cc:224] Handling call
0709 13:02:37.087516 (+1224665us) spinlock_profiling.cc:243] Waited 516 ms on lock 0x7b540011d5d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:37.087616 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1187185536}
I20260709 13:02:36.987056 10934 consensus_queue.cc:260] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:37.143375  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:37.144095  8325 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:37.144915  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:37.150067 11046 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:37.150707 11072 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:37.100466 11098 consensus_meta.cc:287] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.597s	user 0.004s	sys 0.004s
I20260709 13:02:37.101819 11010 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:37.090801 11007 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:37.106017  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:37920 (request call id 21) took 1138 ms. Trace:
W20260709 13:02:37.160784  8252 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1645) took 1461 ms (client timeout 1923 ms). Trace:
I20260709 13:02:37.160950  8328 rpcz_store.cc:276] 0709 13:02:35.967522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.969696 (+  2174us) service_pool.cc:224] Handling call
0709 13:02:37.105797 (+1136101us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b540014c654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287b8853 00007f882d78660a 00007f8826d0c7d4 00007f8826d0c762 00007f8826d0c691 00007f8826d0c60c 00007f8826d0b8c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:37.105944 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":517555456}
W20260709 13:02:37.161199  8252 rpcz_store.cc:269] 0709 13:02:35.699498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.699882 (+   384us) service_pool.cc:224] Handling call
0709 13:02:37.160592 (+1460710us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b5400406d54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:37.160728 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1678607232}
I20260709 13:02:37.107230  9008 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:02:37.126495  8220 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:37.141122  8257 rpcz_store.cc:276] 0709 13:02:35.890706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.016261 (+125555us) service_pool.cc:224] Handling call
0709 13:02:37.089330 (+1073069us) spinlock_profiling.cc:243] Waited 55.1 ms on lock 0x7b54001301d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:37.089466 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126698240}
I20260709 13:02:37.084895 11127 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:37.071729 11036 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:37.067101 11106 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:37.156208  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:37.157409  8325 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:37.144963 11136 consensus_queue.cc:1059] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:02:37.171955  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:37.133778 10833 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.172725  8326 raft_consensus.cc:2393] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 4: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:37.134711 11057 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:37.150760 11046 consensus_queue.cc:1243] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer ff7a6fe533e94c1bae555b4cae427728 log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:02:37.180759  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.163194 11045 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:37.173136  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 46708.2us
I20260709 13:02:37.185820 10855 raft_consensus.cc:3055] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:37.186533  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:37.176479 10833 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:37.128298 11003 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:37.230360  8220 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:37.176478 11119 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:37.191557  8403 raft_consensus.cc:1240] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:02:37.175832 11072 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:37.236967 10747 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:37.193176 11136 raft_consensus.cc:3055] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:37.242667  8398 raft_consensus.cc:1240] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:37.243000  8325 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:37.124043 11063 consensus_queue.cc:237] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:37.245736  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:37.251430  8326 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 5 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:37.251399  8325 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:02:37.171268 11098 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:37.256538  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:37.257050  8325 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:37.238133 11119 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:37.261581 11133 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:02:37.244959 11072 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.243355 11036 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:37.233246 10855 raft_consensus.cc:740] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:37.285481 10855 consensus_queue.cc:260] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:37.286516 10855 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:37.276671 10690 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:37.278807 11106 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:37.281443 11087 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:37.254235 11057 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:02:37.272732 11119 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:37.293882 11138 log.cc:927] Time spent T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:02:37.174255 11127 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:37.296144  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1679) took 1593 ms. Trace:
I20260709 13:02:37.297722 11127 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:37.300348  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.302017  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:37.302598  9007 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:37.248020 11136 raft_consensus.cc:740] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:37.304888  9007 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:37.305306 11136 consensus_queue.cc:260] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:37.306285  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:37.306593 11136 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:37.293814  9006 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:02:37.309176  8324 raft_consensus.cc:1275] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:37.309520  8298 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:37.311292  8297 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:37.253090 11066 raft_consensus.cc:2764] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:02:37.296530  8250 rpcz_store.cc:276] 0709 13:02:35.702357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.172181 (+469824us) service_pool.cc:224] Handling call
0709 13:02:36.172494 (+   313us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:36.173778 (+  1284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:36.173809 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:37.107783 (+933974us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b2c000014c8. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:02:37.108463 (+   680us) log.cc:844] Serialized 52 byte log entry
0709 13:02:37.140110 (+ 31647us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:37.179080 (+ 38970us) log.cc:844] Serialized 14 byte log entry
0709 13:02:37.179291 (+   211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.179361 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:37.295845 (+116484us) raft_consensus.cc:1685] finished
0709 13:02:37.295948 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:37.296087 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":872,"prepare.queue_time_us":6575,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":3243,"raft.queue_time_us":1029,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":56180,"spinlock_wait_cycles":41297024,"thread_start_us":31225,"threads_started":1,"wal-append.queue_time_us":61711}
I20260709 13:02:37.319355  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:37.319985  8404 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:37.322185  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:37.322676  8397 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:37.323348  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:37.323578  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:37.324273 11098 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:37.324559  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:37.325542 10847 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.326025 10847 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:37.329197 10847 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:37.329659 11106 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:37.329967 11023 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:37.330677 11106 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:37.332020  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:37.332762 11106 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:37.335570 11089 consensus_meta.cc:287] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.546s	user 0.009s	sys 0.000s
I20260709 13:02:37.336005 11089 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:37.336885  8405 raft_consensus.cc:1275] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:37.337169  8330 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:02:37.339373  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:37.316036  8298 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:37.335616  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:37.345554 10921 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:37.346256  9009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 635) took 1118 ms. Trace:
I20260709 13:02:37.242012 11041 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:37.348664  8974 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:37.358587  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 176043us
I20260709 13:02:37.375294  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:37.358806  9009 rpcz_store.cc:276] 0709 13:02:36.227825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.227983 (+   158us) service_pool.cc:224] Handling call
0709 13:02:37.102648 (+874665us) spinlock_profiling.cc:243] Waited 3.62 ms on lock 0x7b5800175898. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:02:37.102873 (+   225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:37.298412 (+195539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:37.298441 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:37.298483 (+    42us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:37.298511 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.346023 (+ 47512us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b54006e23d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:37.346073 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:37.346185 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":457484288}
W20260709 13:02:37.376294  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.359834  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:37.377198  9008 raft_consensus.cc:2410] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:02:37.377610  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:37.361842  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.379417  8403 raft_consensus.cc:2393] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 6: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:37.365780  8325 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:37.375986  8327 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:37.381085  8223 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:37.384761  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:37.308014 11072 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:37.391824  9007 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:37.392019  9003 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:37.392480  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:37.392781  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:37.396816  8332 raft_consensus.cc:1275] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:37.397099  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:37.397729  9005 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:37.318465 11087 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:37.404985  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:37.405689  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:37.412124  8330 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:02:37.417495  8223 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:37.418267  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.418548  8406 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:37.420533  8297 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:37.421300  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:37.382602 11014 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:37.420082  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:37.423310  8399 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:37.385610 11114 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:37.398706 11008 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:37.385200 11010 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.394522 11091 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:02:37.435659 11089 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:37.438107  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:37.438426 11107 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:37.439074 11136 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:37.439579 11136 raft_consensus.cc:697] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:37.448107  8402 raft_consensus.cc:2410] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:37.422749 10930 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.425192 11034 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:37.426837 11014 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.383921 11106 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:02:37.399049 11066 consensus_queue.cc:1059] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:02:37.393963 11023 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:02:37.427754 11114 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.408663 11079 consensus_queue.cc:1059] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:02:37.455996 11014 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:37.461056 11089 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:37.465284 11114 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:37.480831  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 250517us
I20260709 13:02:37.447190 11010 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:37.458812 10930 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:37.512037  8220 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:37.487708 11081 consensus_meta.cc:287] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.651s	user 0.011s	sys 0.000s
W20260709 13:02:37.524142  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 273408us
W20260709 13:02:37.491156 11048 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.015s	sys 0.002s
I20260709 13:02:37.471500 11087 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:02:37.527791 11091 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:37.480729 11066 raft_consensus.cc:3055] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:37.534061  8405 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 13:02:37.545390  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:37.529518 11048 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.016s	sys 0.002s
I20260709 13:02:37.546555  8220 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:37.526469 11107 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.487507 11136 consensus_queue.cc:237] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:37.537916 10926 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:37.561108 11056 raft_consensus.cc:3055] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:37.531509  8295 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:37.561508 10926 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:37.569051 11056 raft_consensus.cc:740] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:37.554889 11003 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:37.569747 11003 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:37.570228 11003 raft_consensus.cc:697] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:37.569969 11056 consensus_queue.cc:260] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:37.570966 11056 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:37.571374  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:37.572237  8332 raft_consensus.cc:2393] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 7: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:37.573589  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:37.574172  8332 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:37.565126 10833 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:37.565995 11140 log.cc:927] Time spent T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:02:37.577543  8972 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:37.578377 11012 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:37.578987  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:37.567783 11119 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:37.568614 11106 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.580689 11106 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:37.582321 11119 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:37.582681 11010 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:37.583786 11041 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:37.562029 11107 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:37.584945 10833 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:37.586941 11107 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:37.583879 11106 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:37.589416  8401 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:02:37.589859  8401 pending_rounds.cc:85] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 7
I20260709 13:02:37.590070  8401 pending_rounds.cc:107] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:02:37.590276  8401 raft_consensus.cc:2889] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:37.590515 10930 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:37.533540 11089 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:37.564615  8399 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:02:37.592893  8400 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:37.594105  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:37.595016  8399 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:37.571074 11003 consensus_queue.cc:237] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:37.605504  8972 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:37.596877  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 50432us
I20260709 13:02:37.606570  8220 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:37.606997  8372 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:37.535650 11066 raft_consensus.cc:740] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:37.609580  8220 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:37.614775 11048 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:37.616562  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:37.617199  8329 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:37.618798  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:37.615206  8220 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:37.622699  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 283305us
I20260709 13:02:37.616145  8372 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:37.621707  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 204264us
I20260709 13:02:37.608925 11089 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:37.618216 10833 raft_consensus.cc:2764] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:02:37.618348 11106 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:37.672340  8329 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
I20260709 13:02:37.619884 10926 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:37.553957 11142 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.069s
I20260709 13:02:37.689033  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:37.699023  8399 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
W20260709 13:02:37.690270  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 74199.9us
W20260709 13:02:37.624598 11018 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.000s	sys 0.019s
I20260709 13:02:37.694331  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:37.695789  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 82129.9us
I20260709 13:02:37.624056 11130 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:37.663090 11041 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.708520  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:37.732692  8399 raft_consensus.cc:2393] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 7: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:37.671079 11089 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.696674 10926 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.706887  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:37.692430 11091 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.738834  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:37.707195 11018 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.000s	sys 0.019s
I20260709 13:02:37.738482  8329 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:02:37.670221 11048 consensus_queue.cc:1048] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:02:37.775935  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2021) took 1453 ms. Trace:
I20260709 13:02:37.779461  8323 rpcz_store.cc:276] 0709 13:02:36.322351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.353266 (+ 30915us) service_pool.cc:224] Handling call
0709 13:02:36.403721 (+ 50455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:36.902161 (+498440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.775676 (+873515us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b5400363b54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:37.775872 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1098250624}
I20260709 13:02:37.776594  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:37.781629  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:37.692409 11024 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:02:37.792779  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 50370.7us
W20260709 13:02:37.709622  9001 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56940 (request call id 637) took 1563 ms (client timeout 1935 ms). Trace:
I20260709 13:02:37.793460  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:37.793848  8220 leader_election.cc:400] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 7.
I20260709 13:02:37.794095  8400 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:37.795063  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:37.704649 11143 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.101s
I20260709 13:02:37.798108 11143 consensus_queue.cc:1243] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:02:37.694912 11036 consensus_queue.cc:1059] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.106s
I20260709 13:02:37.778357  9004 raft_consensus.cc:1240] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:02:37.794283  8220 leader_election.cc:403] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:02:37.803316  8974 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:37.737808 11089 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:37.806525  8220 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:37.740871 10926 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:37.735191 11106 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:37.802788 11062 raft_consensus.cc:3055] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:37.739586 11041 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:37.802553  9004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 633) took 1611 ms. Trace:
W20260709 13:02:37.820991  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 190461us
W20260709 13:02:37.815814 10855 consensus_meta.cc:287] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.529s	user 0.007s	sys 0.003s
I20260709 13:02:37.734318 10833 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 13:02:37.826349  9004 rpcz_store.cc:276] 0709 13:02:36.191013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.191133 (+   120us) service_pool.cc:224] Handling call
0709 13:02:37.704607 (+1513474us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:37.801329 (+ 96722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.802331 (+  1002us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54006c2d50. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:37.802489 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3316077824}
I20260709 13:02:37.789515 11091 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:37.829142  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:37.831132  8250 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:02:37.833325  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:37.809377 11080 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 7.
I20260709 13:02:37.833923  8257 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:37.812641  8220 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:02:37.838346  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.813051 11143 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:37.814256 10931 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:37.844780  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.848287 11091 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:37.848634 10931 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.818979 11062 raft_consensus.cc:740] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:37.850625  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 296) took 1001 ms. Trace:
I20260709 13:02:37.823894 11106 raft_consensus.cc:697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 5 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:37.850930 11062 consensus_queue.cc:260] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:37.851053  8326 rpcz_store.cc:276] 0709 13:02:36.848886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.252416 (+403530us) service_pool.cc:224] Handling call
0709 13:02:37.273283 (+ 20867us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:37.273747 (+   464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:37.273775 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:37.274296 (+   521us) log.cc:844] Serialized 52 byte log entry
0709 13:02:37.533993 (+259697us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:37.534040 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.534108 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:37.850382 (+316274us) raft_consensus.cc:1685] finished
0709 13:02:37.850407 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:37.850567 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":91,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":4468,"thread_start_us":129234,"threads_started":1,"wal-append.queue_time_us":409509}
I20260709 13:02:37.851967 11106 consensus_queue.cc:237] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:37.852475 11062 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:37.849200 10931 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:37.804268  9001 rpcz_store.cc:269] 0709 13:02:36.146290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.149558 (+  3268us) service_pool.cc:224] Handling call
0709 13:02:37.693716 (+1544158us) spinlock_profiling.cc:243] Waited 533 ms on lock 0x7b54006c2d54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:37.705294 (+ 11578us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1226032384}
W20260709 13:02:37.862150 11144 log.cc:927] Time spent T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
I20260709 13:02:37.739296 11066 consensus_queue.cc:260] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:37.867081  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:37.809083  8399 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:37.873781 11042 raft_consensus.cc:3055] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:37.874193 11042 raft_consensus.cc:740] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:37.874971 11042 consensus_queue.cc:260] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:37.876116 11042 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:37.875952 11114 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:37.878829 11041 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:37.878532 11072 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:37.880921  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 174704us
W20260709 13:02:37.883273  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.855791 11057 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:37.888427  8403 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:37.894194  8250 raft_consensus.cc:1240] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:02:37.895886  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.896337  8257 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:37.899497  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:37.900290  8256 raft_consensus.cc:2393] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 4: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:37.901546  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:37.901803  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:37.902279  8253 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 7: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:02:37.902622  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:37.904094  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 168051us
W20260709 13:02:37.919064  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.920861  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:37.925100  8255 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:37.925551  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:37.920078  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:37.921717  8249 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
I20260709 13:02:37.925025  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:37.926414  8252 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:37.929385  8251 raft_consensus.cc:2393] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 9: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:02:37.931910  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.932857  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:37.933682  8253 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:37.893381 11057 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:37.906610 11048 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.935174  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:37.937958  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:37.938220  8256 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:37.938519  8257 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:37.908872 11056 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.936990  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:37.941239  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:37.942796  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:37.945787  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:37.946444  8254 raft_consensus.cc:2410] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:02:37.924659 11066 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:37.956140  8251 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:37.943471  8252 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:02:37.953836  8253 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:02:37.957314  8253 pending_rounds.cc:85] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 7
I20260709 13:02:37.957645  8253 pending_rounds.cc:107] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:02:37.957891  8253 raft_consensus.cc:2889] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:37.955643  8397 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 5. Current term is 6. Ops: []
W20260709 13:02:37.958155  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.889281 11014 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:37.965104  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:02:37.966101 11128 log.cc:927] Time spent T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.921s	user 0.001s	sys 0.005s
I20260709 13:02:37.966394  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:37.967087  8255 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:37.968302  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:37.950531 11056 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:37.971463 11056 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:37.972265 11108 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:37.972833 11108 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:37.974982 11108 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:37.975560  8256 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:37.977684 11014 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:37.978472  8325 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5. Ops: [2.5-4.6]
I20260709 13:02:37.988339 11072 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:37.933790 11127 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:37.992578 11114 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:37.941207 11057 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:37.943931 11048 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:38.004594 11048 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:38.006108  8257 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:02:38.028319  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:38.037945  8251 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:02:38.045848  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:38.046270  8251 raft_consensus.cc:1275] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:38.050462  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:38.051010  8251 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:37.989854 11127 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:38.053112  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:38.053915  8249 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
I20260709 13:02:38.053381 11056 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.084523 11127 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:38.034896 11008 consensus_queue.cc:1059] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:02:38.089982  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:38.092713  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:38.093477  8251 raft_consensus.cc:2410] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:02:38.094563  8255 raft_consensus.cc:1275] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:38.097002 10934 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:38.096556 10931 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:38.098604  8256 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:38.102188 11170 consensus_queue.cc:1059] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
W20260709 13:02:38.114930  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 272401us
I20260709 13:02:38.084560 11056 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:38.116539  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:38.117635  8254 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:38.122187 11041 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:02:38.123893  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:38.124207  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:38.125304 11056 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:38.126569  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:38.126819  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:38.127504  8325 raft_consensus.cc:2368] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6.
I20260709 13:02:38.127635  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:38.128504  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:38.128578  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:38.129302  8325 raft_consensus.cc:2368] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 8. Current term is 9.
I20260709 13:02:38.124643  8253 raft_consensus.cc:2393] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 7: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:38.131191  8972 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:38.133953  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:38.135553  9005 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 5 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:38.137044  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:38.125820 11034 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.137003  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:38.138505 11034 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:38.140183  8972 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:38.141007 11114 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:38.141271 11107 consensus_queue.cc:1059] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:38.142570 11107 raft_consensus.cc:3055] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:38.142977 11107 raft_consensus.cc:740] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:38.143584  8295 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:38.143916 11107 consensus_queue.cc:260] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:38.144616  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:38.120863 11098 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:38.128284  8327 raft_consensus.cc:2376] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5. Re-sending same reply.
I20260709 13:02:38.126356  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:38.145404  8295 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:38.140563 11034 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:38.145192 11107 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:38.148701  8295 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:38.145452  9005 raft_consensus.cc:2393] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 6: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:38.146826  8328 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:38.152827  8223 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:38.146391  8972 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:38.154621  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:38.156659  8253 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:38.157905  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:38.158608  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:38.159492  8326 raft_consensus.cc:2410] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:38.167095  8254 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:38.145599 11014 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:38.123258 11089 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:38.169764 11106 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:38.170187  8972 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:38.170614  8257 raft_consensus.cc:2393] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 6: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:38.170965 11108 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:38.170857 10926 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:38.171325  8972 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:38.172103 11108 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:38.172523 11108 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:38.172894  8972 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:38.172919 11108 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:38.174309  8248 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:02:38.174881  8252 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:02:38.174844  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:38.195994  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:38.196369  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:38.154551 11127 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:38.178970  9007 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 7: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:38.206314  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:38.215142  8326 raft_consensus.cc:2368] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 8. Current term is 9.
I20260709 13:02:38.195186  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:38.215991  8254 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:38.154433 11007 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:38.224968  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:38.225736  8253 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:02:38.145860 11003 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:02:38.178021 11091 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:38.222048 11007 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.176573 11008 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:02:38.228094  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.173276 10926 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:38.161936 11034 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:38.236394  8251 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:38.178493  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:38.237847  8328 raft_consensus.cc:1217] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Deduplicated request from leader. Original: 5.7->[7.8-7.8]   Dedup: 7.8->[]
I20260709 13:02:38.232327  8254 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:38.145721 11098 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:38.254129  8328 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:02:38.255687  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:38.250612  8974 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:38.263880  8326 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 4. Current term is 5. Ops: [2.5-4.6]
I20260709 13:02:38.257261  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:38.265967  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:38.266628  8253 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:38.270320  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:38.271247  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:38.271806  8406 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:38.238121 11007 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:38.270381  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:38.272807  8403 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:38.273005  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:38.273777  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:38.274394  8402 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:38.275265  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:38.275822  8397 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:38.280848 10934 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:38.281857 11183 log.cc:927] Time spent T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 13:02:38.282176  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 376956us
I20260709 13:02:38.282944 11098 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:38.222700 11062 consensus_queue.cc:1059] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:02:38.301259  9008 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:38.302691  9002 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:38.304545  8331 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:38.306509 11057 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:38.307428 11062 consensus_queue.cc:1243] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:02:38.308776 11057 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:38.308974 10690 raft_consensus.cc:3055] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:38.309402 11048 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.310019 11048 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:38.312816 11048 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:38.312943 11046 consensus_queue.cc:1059] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:38.240978 11089 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:38.240008 11064 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:38.266283  9004 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:38.291657  8402 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:38.287113 11018 consensus_queue.cc:1059] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:02:38.293990  8295 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:38.272171 11091 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.294523 10747 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:38.287312 11119 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.295452  8406 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:38.295908 11024 consensus_queue.cc:1059] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:02:38.296475 11181 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.190s	user 0.001s	sys 0.001s
W20260709 13:02:38.287361  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 49441.2us
I20260709 13:02:38.290695  9005 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:02:38.316464  8222 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:38.313410  8397 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:02:38.316769  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:38.318895  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 53835.5us
I20260709 13:02:38.325083  8295 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:38.315594  8223 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:38.315439  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 142576us
I20260709 13:02:38.337807  8295 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:38.318410 10934 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:02:38.309417 10690 raft_consensus.cc:740] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:38.341490 10690 consensus_queue.cc:260] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:38.342021 11014 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:38.342782 10690 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:38.343547 11098 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:38.343871 10965 consensus_queue.cc:1048] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:38.345551 11064 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.346206 11039 raft_consensus.cc:3055] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:38.346020 11064 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:38.346575 11039 raft_consensus.cc:740] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:38.347009 11119 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:38.313879 11046 raft_consensus.cc:3055] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:38.347893 11046 raft_consensus.cc:740] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:38.348157 11064 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:38.358706  8972 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:38.360333  8295 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:38.359867  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:38.352594  8330 raft_consensus.cc:1240] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:02:38.353073  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:38.362440  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 46021.9us
I20260709 13:02:38.364465  8327 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:02:38.367568  8326 raft_consensus.cc:2393] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 7: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:38.345252 10965 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:38.372679  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:38.378599  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:38.375468  8295 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:38.383586  8295 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:38.335801 11034 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:38.387967  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:38.390293  8972 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:38.365953 11003 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:38.366189 11170 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:38.391422  8369 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:02:38.393862  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:38.394014  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.376006 11130 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:38.373705 11098 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:38.348881 11046 consensus_queue.cc:260] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:38.313377 11114 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:38.403167  8330 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:38.403803 11114 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:38.404302 11114 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.404707 11114 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:38.405228  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:38.405963  8330 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:38.397805  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:38.407773  8326 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:38.388236 10921 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:38.408347  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:38.369864 11044 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:02:38.393677 11003 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.397449 11023 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:02:38.337028 11091 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:38.397799 11130 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.392176 11063 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:38.444128  8248 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:38.444402  8257 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:38.445498  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:38.446100  8251 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:38.376767 11081 consensus_queue.cc:1048] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:02:38.446597  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:38.396796 11170 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.366464 10934 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:02:38.447695 11170 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:38.448197 11161 raft_consensus.cc:3055] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:38.448623 11161 raft_consensus.cc:740] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:38.448717 10934 consensus_queue.cc:1059] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:38.388890 10930 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:38.449802 10930 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.449610 11161 consensus_queue.cc:260] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:38.450357 10930 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:38.450479  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:38.450838 11161 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:38.452972 11036 raft_consensus.cc:3055] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:38.399920 11098 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.443300  8250 raft_consensus.cc:1275] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:02:38.388667 10965 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:38.381968 10847 consensus_queue.cc:1059] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:02:38.449745  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:38.392004 11024 consensus_queue.cc:1048] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:02:38.410542 11045 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:38.447209  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:38.454591  8253 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:38.444037  8254 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:38.459642  8256 raft_consensus.cc:2376] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8. Re-sending same reply.
W20260709 13:02:38.460865  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 68158.8us
I20260709 13:02:38.461517  8297 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:38.461798  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:38.461910  8369 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:38.426085 11003 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:38.429629 10921 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:38.502036  8972 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:38.445611 11130 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:38.453667 11098 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:38.480708  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.501653  8330 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:38.466387  8252 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:38.503105  8324 raft_consensus.cc:2393] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 6: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:02:38.503211  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.453359 11036 raft_consensus.cc:740] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:38.454421 10931 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:38.506754  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 44895.4us
I20260709 13:02:38.430148 11018 consensus_queue.cc:1059] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 13:02:38.503455 11106 raft_consensus.cc:3055] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:38.466071 11045 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.454707 11192 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:38.505199 11127 raft_consensus.cc:697] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:38.453948 11107 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.054s
W20260709 13:02:38.521752  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 138227us
I20260709 13:02:38.508895 11157 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.514525 11063 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:38.530298  8295 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:38.506908 11034 raft_consensus.cc:697] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:38.527181 11045 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:38.523275 11106 raft_consensus.cc:740] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:38.553562 10847 consensus_queue.cc:1059] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:38.559438 11091 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:38.560679 11119 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:38.563757  9001 pending_rounds.cc:85] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: Aborting all ops after (but not including) 6
I20260709 13:02:38.569618 11036 consensus_queue.cc:260] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:38.570874  8328 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:38.574880 11170 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:38.524209 11149 consensus_queue.cc:1048] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 13:02:38.533567 11199 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:38.535552 11157 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:38.593766 10965 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:38.600200  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:38.529474 11192 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.622284 11063 raft_consensus.cc:697] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:38.626590  9001 pending_rounds.cc:107] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:02:38.625027 10747 consensus_queue.cc:1059] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:02:38.638672 11036 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:38.625082 11108 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:38.642108  9004 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:38.647766  9001 raft_consensus.cc:2889] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:38.645205 11199 raft_consensus.cc:697] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:02:38.690006  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 57743.6us
I20260709 13:02:38.648398 11192 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:38.692245  8248 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:02:38.693039  8257 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:38.688562  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:38.693919  8256 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 7: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:38.660108 11008 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.694733 11008 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:38.697052 11008 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:38.674012  9006 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:38.644783 11034 consensus_queue.cc:237] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:38.698237  8257 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:02:38.674641  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:38.699869  8250 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:38.701647  8972 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:38.701905  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:38.702490  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.703167 11091 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:38.703603 11091 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.703950  8250 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:38.703965 11091 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:38.702019 11192 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:38.704799  8257 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:02:38.705605  9003 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:02:38.706175  8248 raft_consensus.cc:1275] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:38.707886  8253 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:02:38.665925 10747 consensus_queue.cc:1243] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer ff7a6fe533e94c1bae555b4cae427728 log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:02:38.736159  8298 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:38.710722  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:38.647414 11039 consensus_queue.cc:260] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:38.649880 11127 consensus_queue.cc:237] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:38.653568 11106 consensus_queue.cc:260] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:38.678316 11007 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:38.679332 11114 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:38.763352  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.585733  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:02:38.776434 11204 log.cc:927] Time spent T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:02:38.784736 11098 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:38.764952 11032 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 13:02:38.825464  9005 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:02:38.789708 11207 consensus_queue.cc:1059] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
W20260709 13:02:38.821547  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.838260 10930 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:38.824527 11114 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:38.824664 11018 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.865185 11007 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:38.870935 10930 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:38.857003 10690 consensus_meta.cc:287] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.000s
I20260709 13:02:38.872844  8251 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:02:38.874302 10965 raft_consensus.cc:3055] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:38.874810 10965 raft_consensus.cc:740] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:38.875008 11157 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:38.855324 11199 consensus_queue.cc:237] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:38.875646 10965 consensus_queue.cc:260] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:38.873318  8251 pending_rounds.cc:85] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 5
I20260709 13:02:38.874380 11098 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:38.877985  8251 pending_rounds.cc:107] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:02:38.886154  8251 raft_consensus.cc:2889] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:38.878772  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:38.885234  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.877259 10965 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:38.761281 11063 consensus_queue.cc:237] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:38.908247  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 197847us
I20260709 13:02:38.871907 11091 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:38.868491 11003 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:38.909883  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:38.869109 11130 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:38.906112  9004 raft_consensus.cc:1240] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:38.921546 11081 consensus_queue.cc:1048] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:38.877789 11018 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:38.920081 11108 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:38.922593 10747 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:38.922140 11192 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:38.915053 11042 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:02:38.940354 11091 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:38.915972 11045 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:38.927450  8369 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:02:38.947525  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.932438 10934 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:38.966117 11003 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:38.966771 11218 consensus_queue.cc:1059] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:38.967080 11045 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:38.933171 11173 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:02:38.933341 11210 consensus_queue.cc:1059] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:02:38.938742 10747 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.971484 11048 consensus_queue.cc:1059] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:38.973369  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:38.997962  8405 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:38.970906 10934 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:39.000782  9007 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:38.955051 11081 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:39.004019 11045 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:38.999082 10747 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:39.008682 11149 raft_consensus.cc:3055] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:39.010358 11018 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:39.010234 11042 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:39.014914  8405 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:38.969575 11130 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:39.021013  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.066931  8222 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:39.010895 11063 raft_consensus.cc:3055] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:39.010809 11210 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
W20260709 13:02:39.079896  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.066088 11149 raft_consensus.cc:740] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:39.011250 11081 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:38.998716 11136 consensus_queue.cc:1059] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:02:39.069216 11063 raft_consensus.cc:740] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:39.074503 10934 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:39.108794  8974 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:39.098112 11081 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:39.113090  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:39.113736  8399 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:39.113970  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:39.114567  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:39.114830  8404 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 7: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:39.115190  8401 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:39.116559 11170 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:39.119426 11063 consensus_queue.cc:260] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:39.120716 11063 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:39.102178 11173 raft_consensus.cc:3055] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:39.122633 11173 raft_consensus.cc:740] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:39.123579 11173 consensus_queue.cc:260] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:39.124938 11173 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:39.111837 10965 raft_consensus.cc:3055] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:39.129208 11199 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:39.129506 10965 raft_consensus.cc:740] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:39.129686 11199 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:39.130110 11199 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:39.130405 10965 consensus_queue.cc:260] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:39.136656  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 126618us
I20260709 13:02:39.138934  8369 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:39.144631  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:39.144735  9001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 650) took 1013 ms. Trace:
I20260709 13:02:39.146108 11149 consensus_queue.cc:260] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:39.150532 11149 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:39.149655 11024 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:39.145251  9001 rpcz_store.cc:276] 0709 13:02:38.130915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.145975 (+ 15060us) service_pool.cc:224] Handling call
0709 13:02:38.563047 (+417072us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b5800175898. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:02:38.563410 (+   363us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:38.750556 (+187146us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:38.752860 (+  2304us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:38.841942 (+ 89082us) log.cc:844] Serialized 52 byte log entry
0709 13:02:38.956070 (+114128us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:39.072524 (+116454us) log.cc:844] Serialized 14 byte log entry
0709 13:02:39.103065 (+ 30541us) log.cc:844] Serialized 14 byte log entry
0709 13:02:39.136050 (+ 32985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.138829 (+  2779us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:39.144480 (+  5651us) raft_consensus.cc:1685] finished
0709 13:02:39.144510 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:39.144670 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10580,"prepare.queue_time_us":189477,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":2535,"raft.queue_time_us":22864,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":8978,"spinlock_wait_cycles":47389312,"thread_start_us":225734,"threads_started":2,"wal-append.queue_time_us":162887}
I20260709 13:02:39.154825  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:39.156817  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:39.157536  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.158296  8974 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:39.159134 11207 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:39.159488 11207 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.159793 11207 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:39.162025  8974 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:02:39.162642  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.164794  8974 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:39.141463 11045 consensus_queue.cc:1048] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:02:39.166142 11127 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:39.167169 11091 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:39.179183 11091 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.169534 11136 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.185247  8330 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:39.185643  8329 raft_consensus.cc:1240] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:02:39.186966  8329 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:02:39.192445 11208 log.cc:927] Time spent T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.515s	user 0.005s	sys 0.007s
I20260709 13:02:39.193382  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:39.193903  8325 raft_consensus.cc:1275] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:39.171530 11192 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.194340 11041 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:39.194902 11041 raft_consensus.cc:697] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:39.194732 11192 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:39.196113  8330 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:02:39.196538  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 666183us
I20260709 13:02:39.197062  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:39.198132  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:39.195729 11041 consensus_queue.cc:237] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:39.198719  8332 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
I20260709 13:02:39.199797  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:39.200675  8325 raft_consensus.cc:2393] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 7: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:39.201571  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:39.208118  8329 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:39.203269  8325 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:02:39.204116  8324 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:39.202797  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:39.211637  8332 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 7: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:39.214223 11200 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:39.214696 11200 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.215061 11200 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:39.216650 11044 raft_consensus.cc:3055] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:39.217110 11044 raft_consensus.cc:740] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:39.226015  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:39.182133 11091 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:39.227029  8325 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:39.231541 11106 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:39.231139 11081 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:39.201290 11136 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:39.233765  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 437238us
I20260709 13:02:39.235231 11136 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:39.237022 11062 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.237752  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:39.238818  8250 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:39.239022  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:39.239636  8255 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:39.239400 10747 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:39.241544 11192 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:39.241802  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:39.242367  8253 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:39.243019  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:39.243600  8404 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
W20260709 13:02:39.243914  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.244673  8974 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:39.244696  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:39.245622  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:39.245926  9005 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:39.246152  8251 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:39.247462 11044 consensus_queue.cc:260] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:39.248291 11119 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:39.248667 11119 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.248823 11044 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:39.249053 11119 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:39.239485  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:39.250524  8248 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:39.250969  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 514759us
I20260709 13:02:39.237597 11062 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:39.257663  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:39.255847  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:39.258596  8325 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:39.263782  8325 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:39.265815  8255 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
I20260709 13:02:39.269229  8251 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:02:39.249396 11108 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:39.243810 11081 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:39.275786  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 208896us
I20260709 13:02:39.282312 11003 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:02:39.282647  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.283054 11034 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.282825 11003 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:39.262485 10930 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:39.283551 11034 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:39.283852 10930 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:39.286298 10930 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:39.286794 11170 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:39.287335 11170 raft_consensus.cc:697] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:39.281381 11192 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:39.288188 11192 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:39.288142 11170 consensus_queue.cc:237] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:39.288779 11192 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:39.282152 11198 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:02:39.290120  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:39.291136  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.262455 11007 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:39.293358 11007 raft_consensus.cc:697] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 5 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:39.293987 11056 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.294227 11007 consensus_queue.cc:237] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:39.294617 11056 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:39.297685 11098 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.298187 11098 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:39.300180 11098 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:39.300969 10747 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:39.301271 10690 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.302233  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:39.303115 11034 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:39.297077 11056 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:39.302249 10690 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:39.310441  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 848777us
I20260709 13:02:39.312042  8297 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:39.314004  8297 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:02:39.316605  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 49927.8us
I20260709 13:02:39.317085  8331 raft_consensus.cc:1275] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:02:39.280738  8248 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:02:39.334314  8972 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:39.346112  8972 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:39.346578  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:39.347198  8325 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:39.371395  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:39.372231  8332 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:39.373149  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.259161 11172 consensus_queue.cc:1059] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.116s
I20260709 13:02:39.318236 10930 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:39.421604  8220 leader_election.cc:400] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6.
I20260709 13:02:39.422173  8220 leader_election.cc:403] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:02:39.416160 11035 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:39.423004  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:39.423096 11035 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:39.416239 10926 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.426190 10926 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:39.428393 10926 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:39.429008 10926 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6.
I20260709 13:02:39.430737  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:39.433295  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.437516  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:39.438881  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:39.439456  8256 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:39.440073  8972 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:39.441030  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:39.437713 11127 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.442778  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:39.446200  8250 raft_consensus.cc:1275] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:39.449705  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:39.453871  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:39.455119  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:39.455911  8406 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:39.457516  8401 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:39.433223 11035 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:39.383088 11108 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:39.458313  8325 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:02:39.458437  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:39.458899  8332 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
I20260709 13:02:39.459925  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:39.474622  8399 raft_consensus.cc:2410] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:02:39.359230 11034 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:39.345108 11003 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:39.390573 11044 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.382316 11198 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:39.482877  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:39.416576 11089 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.478232  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:39.488348  8253 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:39.382025 10921 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:39.478820  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:39.489482  8252 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:02:39.490628  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:39.491240  8331 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:39.419097 10930 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:39.496246  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:39.484839  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:39.497952  8404 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:39.477248  8256 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:39.419924 10690 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:39.465288 11219 log.cc:927] Time spent T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:02:39.478262  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:39.500129  8406 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:39.502017  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:39.531068  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2052) took 1145 ms. Trace:
I20260709 13:02:39.503654  8295 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:39.550437  8328 rpcz_store.cc:276] 0709 13:02:38.385943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.386112 (+   169us) service_pool.cc:224] Handling call
0709 13:02:38.386466 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.570608 (+184142us) spinlock_profiling.cc:243] Waited 8.05 ms on lock 0x7b5400339620. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f8828735dc0 00007f88287c01c0 00007f88287c1231 00007f88287bdd8f 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c
0709 13:02:38.628549 (+ 57941us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.530554 (+902005us) spinlock_profiling.cc:243] Waited 566 ms on lock 0x7b5400339620. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f8828735dc0 00007f88287c1a02 00007f88287bddd6 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:02:39.530968 (+   414us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1320108672}
I20260709 13:02:39.503239  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:39.553086  8256 raft_consensus.cc:2393] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 6: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:39.416692 11197 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.130s
I20260709 13:02:39.556614  8249 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 9: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:39.557644  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:39.563192  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.585651  8253 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
W20260709 13:02:39.592185  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.495784 11034 raft_consensus.cc:697] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:39.499837 11231 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.499343 10930 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:39.555195 10926 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:39.589709 11041 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:39.602912 11220 log.cc:927] Time spent T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:02:39.467029 11127 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:39.615338  9005 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:39.632323  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:39.635844  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:39.643642  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:39.644377  8249 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:39.500135 10921 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:39.650624  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:39.489706 11044 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:39.669067  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:39.654594  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:39.495923 11089 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:39.588809 11007 consensus_queue.cc:1048] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 13:02:39.624195 11057 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:39.690181  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:39.690924  8253 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:39.649042 11014 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:39.668663 11215 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.653347 11237 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:39.639829 11200 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:39.626536 11231 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:39.705092  8249 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:39.711081  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:39.650784 10926 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:39.697244 11014 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:39.713864  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:39.720067  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:39.700222  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 278703us
I20260709 13:02:39.650722 11041 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:39.693940 11062 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:39.726857  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.680512 11192 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:39.728320  8331 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:02:39.740448  8324 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:39.722328 11199 consensus_meta.cc:287] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.589s	user 0.011s	sys 0.000s
I20260709 13:02:39.717760 11237 raft_consensus.cc:697] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:39.721537  8249 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:39.735634  8325 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:39.699179 11057 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:39.743548  8223 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:39.748052  8249 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: [6.8-8.9]
W20260709 13:02:39.683943  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:39.754233  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:39.758476  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.706322 11215 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:39.755228 11218 consensus_queue.cc:1059] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:02:39.778853 11218 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:39.779392 11218 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:39.781615  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 278029us
I20260709 13:02:39.781951  9004 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:39.782706 11127 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:39.798055  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.797289  8252 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
W20260709 13:02:39.785398 11119 consensus_meta.cc:287] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.536s	user 0.006s	sys 0.001s
I20260709 13:02:39.770756 11161 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:39.720949 11034 consensus_queue.cc:237] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:39.830114 11119 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:39.831530  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:39.788931 11198 raft_consensus.cc:697] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:39.834028  8252 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
I20260709 13:02:39.836949  8295 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:39.781661 11089 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:39.792769 11218 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:39.800012 11172 raft_consensus.cc:3055] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:39.838521  8295 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:39.846604  8252 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:02:39.848661  8252 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6. Ops: [3.3-5.4]
W20260709 13:02:39.832233 11230 log.cc:927] Time spent T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 13:02:39.750324 11199 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:39.801930 11248 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:39.825335 11044 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:39.826849 11127 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:02:39.840400 11251 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.843885 11172 raft_consensus.cc:740] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:39.835469 11161 raft_consensus.cc:697] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:02:39.893055 11240 log.cc:927] Time spent T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
W20260709 13:02:39.909256 11241 log.cc:927] Time spent T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 13:02:39.906616  8252 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:02:39.868521 11098 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:02:39.899348 11235 log.cc:927] Time spent T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
W20260709 13:02:39.899290 11091 consensus_meta.cc:287] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.670s	user 0.011s	sys 0.014s
W20260709 13:02:39.902074 11247 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
W20260709 13:02:39.906615 11106 consensus_meta.cc:287] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.675s	user 0.010s	sys 0.000s
I20260709 13:02:39.896705 11248 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:02:39.931818 11239 log.cc:927] Time spent T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
W20260709 13:02:39.907058  8323 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.008s	sys 0.018s
W20260709 13:02:39.950248  8323 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.009s	sys 0.018s
I20260709 13:02:39.950464  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:39.944329  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:39.940910  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 218033us
I20260709 13:02:39.954543  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:39.955474  9005 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 4.
W20260709 13:02:39.957329  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 497230us
I20260709 13:02:39.958482  8372 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:39.966758  8252 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:02:39.934263 10747 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.969235  8972 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:02:39.979730  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.936879 11250 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:39.989892  8252 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:39.983681  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 202395us
I20260709 13:02:39.993076  8369 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:39.962548 11203 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.007s	sys 0.009s
I20260709 13:02:39.931035 11251 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:39.957333 11200 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:39.952647 11248 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:39.970770 11041 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:39.983685 11197 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.040776  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:40.039100  8975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.005s	sys 0.015s
I20260709 13:02:39.983045 11231 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:40.013007 11203 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.007s	sys 0.010s
W20260709 13:02:40.051409  8975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.005s	sys 0.015s
W20260709 13:02:40.020483 11246 log.cc:927] Time spent T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 13:02:39.953547 11091 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:40.010304 11192 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:39.917814 11237 consensus_queue.cc:237] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:40.044306 11249 log.cc:927] Time spent T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.276s	user 0.001s	sys 0.000s
I20260709 13:02:40.016340 11089 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.015650 10926 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:40.016610 11250 raft_consensus.cc:697] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:40.092204  8324 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
W20260709 13:02:40.053890 11247 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:02:39.948300 11198 consensus_queue.cc:237] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:39.983914 10747 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:40.087646 11215 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:40.095279 11114 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.992482 11172 consensus_queue.cc:260] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:40.089982 11057 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:40.130200 11172 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:40.126959  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:40.132105  9005 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 4.
I20260709 13:02:40.134740  8223 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:40.137373 11057 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:40.126214 11143 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.018s	sys 0.013s
W20260709 13:02:40.127463 11252 log.cc:927] Time spent T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.274s	user 0.001s	sys 0.000s
I20260709 13:02:40.148921  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:40.157181  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:40.164799  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.166294  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 173259us
I20260709 13:02:40.172874  8369 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:40.142076 11106 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:40.176779  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:40.094142 11197 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:40.138274 11057 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:40.151496 11143 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.019s	sys 0.013s
I20260709 13:02:40.177678  8254 raft_consensus.cc:2410] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:02:40.118129 11089 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:40.173955  8369 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:40.144485 11268 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:40.107848 11161 consensus_queue.cc:237] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:40.231560  8248 raft_consensus.cc:2393] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 6: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:40.135766 11119 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:40.150795 11199 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:40.173584 11256 log.cc:927] Time spent T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.249s	user 0.000s	sys 0.001s
W20260709 13:02:40.175730 11255 log.cc:927] Time spent T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
W20260709 13:02:40.256817  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.261642  8974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.022s	sys 0.010s
W20260709 13:02:40.262030  8974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.022s	sys 0.010s
I20260709 13:02:40.144814 11114 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:40.263831  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:40.277863  9005 raft_consensus.cc:2393] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 8: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:40.264632  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:40.226902 11057 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:40.279278  9003 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 9: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
W20260709 13:02:40.269275  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.236406 11254 log.cc:927] Time spent T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.314s	user 0.001s	sys 0.001s
I20260709 13:02:40.244057 11098 raft_consensus.cc:697] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:40.233878 11268 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.291082  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:40.236099 11106 raft_consensus.cc:697] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:02:40.286638 11257 log.cc:927] Time spent T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.321s	user 0.001s	sys 0.000s
I20260709 13:02:40.286952 11280 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:40.314765 11268 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:40.319095 11288 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:40.370309  8220 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:40.371353 11243 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:40.378297  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.382750 11271 log.cc:927] Time spent T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
W20260709 13:02:40.383745 11269 log.cc:927] Time spent T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
W20260709 13:02:40.387666  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.388741  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 214783us
I20260709 13:02:40.390002  8369 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:40.397709  8220 leader_election.cc:400] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 8. Current term is 9.
I20260709 13:02:40.246376 11250 consensus_queue.cc:237] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:40.375001 11280 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.423703  8220 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:40.373179 11251 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:40.400974 11199 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.387692 11091 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:40.446738  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1674) took 1011 ms. Trace:
W20260709 13:02:40.423208 11170 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.100s	user 0.005s	sys 0.016s
I20260709 13:02:40.450928  8255 rpcz_store.cc:276] 0709 13:02:39.435520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.440152 (+  4632us) service_pool.cc:224] Handling call
0709 13:02:39.440443 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:39.478516 (+ 38073us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:39.496357 (+ 17841us) log.cc:844] Serialized 14 byte log entry
0709 13:02:39.741311 (+244954us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:39.741772 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 13:02:39.741888 (+   116us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:39.741919 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.741987 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:40.446530 (+704543us) raft_consensus.cc:1685] finished
0709 13:02:40.446551 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.446685 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7077,"prepare.queue_time_us":6788,"prepare.run_wall_time_us":6739,"raft.queue_time_us":24046,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":16373,"spinlock_wait_cycles":108236032,"thread_start_us":149370,"threads_started":1,"wal-append.queue_time_us":361720}
I20260709 13:02:40.453279  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:40.455109  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:40.455619  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.444824 11243 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:40.432487 10747 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:40.466768  8220 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:02:40.479460  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.469029  8255 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:02:40.481225  8255 pending_rounds.cc:85] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 5
I20260709 13:02:40.481518  8255 pending_rounds.cc:107] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:02:40.483153  8255 raft_consensus.cc:2889] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:40.485124  8220 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:40.486501  8220 leader_election.cc:400] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 8. Current term is 9.
I20260709 13:02:40.487322  8220 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:40.492714 11284 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:40.493197 11284 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:40.493781 11284 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:40.494467 10930 consensus_meta.cc:287] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.854s	user 0.011s	sys 0.000s
I20260709 13:02:40.495066 11089 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:40.494880 10930 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:40.494887 11289 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:40.495888 11289 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.496258 11289 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:40.497072 10930 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:40.503821 11266 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.382s	user 0.000s	sys 0.002s
I20260709 13:02:40.386807 11098 consensus_queue.cc:237] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:40.493228 11173 raft_consensus.cc:3055] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:02:40.459242 11149 consensus_meta.cc:287] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 1.306s	user 0.013s	sys 0.000s
W20260709 13:02:40.458235 11170 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.141s	user 0.005s	sys 0.017s
I20260709 13:02:40.516423  8220 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:40.517836 11279 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:40.519474 11018 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:02:40.519652 11264 log.cc:927] Time spent T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 13:02:40.535817  8398 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:40.530320  8220 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:40.489528 11215 raft_consensus.cc:2764] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 13:02:40.373629 11288 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:40.428635 11280 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:40.506345 10747 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:40.542819  9001 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:02:40.457091 11199 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:40.541613 11215 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:40.521996 11173 raft_consensus.cc:740] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:40.535334 11018 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.579051  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:40.444314 11106 consensus_queue.cc:237] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:40.538978 11197 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:40.600081 11207 consensus_meta.cc:287] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 1.440s	user 0.013s	sys 0.000s
I20260709 13:02:40.600528 11288 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:40.601321  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 211324us
I20260709 13:02:40.568351 11114 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:40.606573 11280 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:40.607337 11057 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:40.610859  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:40.611578  9001 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:40.579942 11018 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:40.579520 10747 raft_consensus.cc:697] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:40.545097 11170 consensus_queue.cc:1059] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:02:40.618031  8372 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:40.624527 11300 log.cc:927] Time spent T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
W20260709 13:02:40.626250 11229 log.cc:927] Time spent T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 1.141s	user 0.009s	sys 0.009s
I20260709 13:02:40.576576 11035 consensus_queue.cc:1059] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:02:40.626812 11035 consensus_queue.cc:1243] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:02:40.627136 11291 consensus_queue.cc:1059] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:40.628554 11210 consensus_queue.cc:1059] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:40.628648 10747 consensus_queue.cc:237] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:40.629667 11210 consensus_queue.cc:1059] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:40.629953 11291 raft_consensus.cc:3055] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:40.630403 11210 raft_consensus.cc:3055] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:40.630568 11291 raft_consensus.cc:740] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:40.630864 11210 raft_consensus.cc:740] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:40.631765 11291 consensus_queue.cc:260] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:40.633401 11291 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:40.633710  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 382) took 1481 ms. Trace:
I20260709 13:02:40.634132  8401 rpcz_store.cc:276] 0709 13:02:39.152481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.154600 (+  2119us) service_pool.cc:224] Handling call
0709 13:02:40.633551 (+1478951us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b54002689d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:40.633653 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":634579200}
I20260709 13:02:40.637724 11280 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:40.637775 11114 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:40.641417 11280 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.641849 11280 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:40.641698 11114 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:40.643919 11114 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:40.600570 11207 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:40.627816 11149 consensus_queue.cc:1059] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:02:40.647256 11035 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:02:40.646921 11042 consensus_queue.cc:1059] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:02:40.631791 11210 consensus_queue.cc:260] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:40.698413 11299 log.cc:927] Time spent T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:02:40.694680 11268 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:40.702793 11289 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:40.706547 11284 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:40.708416 11284 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:40.708500 11289 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:40.708537 11268 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:40.704478 11280 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:40.738732  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.741899  8249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1731) took 1108 ms. Trace:
I20260709 13:02:40.742497  8249 rpcz_store.cc:276] 0709 13:02:39.633282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.771061 (+137779us) service_pool.cc:224] Handling call
0709 13:02:39.771453 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:39.771775 (+   322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:39.771803 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:39.772269 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 13:02:39.870858 (+ 98589us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:39.870923 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.871054 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:40.741058 (+870004us) raft_consensus.cc:1685] finished
0709 13:02:40.741152 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.741627 (+   475us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1018,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":18292,"thread_start_us":97411,"threads_started":1,"wal-append.queue_time_us":544065}
W20260709 13:02:40.747272  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 216859us
I20260709 13:02:40.749133  8220 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:40.754906  8220 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:40.763031  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 104811us
I20260709 13:02:40.772593  8369 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:40.775081  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.781911 10934 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:40.783267  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:40.784866  9001 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:40.785774  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.783306 10934 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:40.792716 11018 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:40.795912 10934 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:40.796824 11035 consensus_queue.cc:1048] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:40.799208 11042 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:40.661314 11044 consensus_queue.cc:1048] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:02:40.688759 11057 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:40.813598 11268 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.804517 11170 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:40.815727 11289 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:40.822209  9001 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:40.825613 11018 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:40.822840  8220 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:40.829823 11207 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:40.830381 11215 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:40.832041 11199 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:40.833024  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:40.833873 11280 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:40.850776 11237 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:02:40.851244  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 78911.7us
I20260709 13:02:40.851766 11237 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:40.852346  8369 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:40.853009  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 95787.5us
I20260709 13:02:40.854166  8220 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:40.835291 11250 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:40.855391 11035 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:40.855592 11250 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:40.836009 11275 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:40.856906 11268 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:40.816650 11274 consensus_queue.cc:1059] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:02:40.838617 11210 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:40.859552 11268 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:40.799916  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.832098  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:40.844009  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:40.844172  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:40.834496  8400 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:02:40.853991 11081 consensus_queue.cc:1175] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ff7a6fe533e94c1bae555b4cae427728" responder_term: 6 status { last_received { term: 3 index: 3 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:40.865780 11288 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:40.866204  8405 raft_consensus.cc:2393] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 5: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:40.866462 11288 raft_consensus.cc:697] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:40.867396  8401 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:40.850226  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:40.869130  8406 raft_consensus.cc:2393] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 8: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:40.869696  9005 raft_consensus.cc:2393] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 5: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:40.854187 11057 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:40.874019 11289 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:40.874534 11251 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:40.876691 11289 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:40.866024 11292 raft_consensus.cc:3055] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:40.880240 11207 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:40.881229 11207 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:40.883881 11196 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:40.884150 11207 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:40.884704 11196 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:40.885113  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:40.872875  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:40.878933 11292 raft_consensus.cc:740] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:40.890852  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1826) took 1261 ms. Trace:
I20260709 13:02:40.901377 11277 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:40.902300 11277 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:40.902711 11281 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:40.903652 11281 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:40.904829 11277 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:40.906973 11237 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:40.891817 11197 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:40.868433 11288 consensus_queue.cc:237] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:40.898468  9001 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:40.897729  8331 rpcz_store.cc:276] 0709 13:02:39.629768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.646656 (+ 16888us) service_pool.cc:224] Handling call
0709 13:02:39.675753 (+ 29097us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:39.740010 (+ 64257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.890239 (+1150229us) spinlock_profiling.cc:243] Waited 51.8 ms on lock 0x7b540011e9d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:40.890645 (+   406us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":119193344}
I20260709 13:02:40.922190  8220 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:40.922837 11197 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:40.886802  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 1832) took 1237 ms. Trace:
I20260709 13:02:40.889247  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:40.927835  8325 rpcz_store.cc:276] 0709 13:02:39.649092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.650308 (+  1216us) service_pool.cc:224] Handling call
0709 13:02:40.886352 (+1236044us) spinlock_profiling.cc:243] Waited 75.2 ms on lock 0x7b54003cb254. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:40.886546 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172952832}
I20260709 13:02:40.891462 11286 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.940620  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 1834) took 1290 ms. Trace:
W20260709 13:02:40.941857  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 89721.8us
I20260709 13:02:40.942418 10965 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:40.944588  8372 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:40.934026 11018 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:40.945078  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:40.945779  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:40.946493 11292 consensus_queue.cc:260] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:40.947347 11281 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:40.947849 11286 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:40.950165 11196 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:40.950665  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:40.951679  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 73597.4us
I20260709 13:02:40.952287 11170 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:40.954857  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:40.955583 11089 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.958518  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:40.960027  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:40.960934 11279 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:02:40.962554 11309 log.cc:927] Time spent T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:02:40.963358  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:40.964349  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:40.965963  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:40.970520  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1726) took 1338 ms. Trace:
I20260709 13:02:40.971536 11250 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:40.973780  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:40.975616 11237 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:40.976233 11277 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.976926 10930 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:40.977612  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:40.979800 11218 raft_consensus.cc:3055] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:41.025835  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:41.026762  8406 raft_consensus.cc:2393] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 5: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:41.027819  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:40.980585 11251 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.029874 11251 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:41.030393  8329 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:41.030939 11248 consensus_meta.cc:287] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.981s	user 0.004s	sys 0.016s
I20260709 13:02:41.031446 11248 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.032321 11251 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:40.981566  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:41.033406  8251 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 7: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:41.033893  8327 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:41.034667  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:41.035452  8330 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:41.036127  9003 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:40.984601  8326 rpcz_store.cc:276] 0709 13:02:39.649881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.672861 (+ 22980us) service_pool.cc:224] Handling call
0709 13:02:40.888818 (+1215957us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b58000e3498. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:02:40.940560 (+ 51742us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4867456}
I20260709 13:02:41.002272  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:41.037546  8325 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:41.037923 11286 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:41.038965 11310 log.cc:927] Time spent T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:02:41.039567 11197 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:41.040347 11292 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:41.040565  8397 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:02:41.041893 11289 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:41.019966 10930 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:41.042464 11289 raft_consensus.cc:697] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:41.044095 10930 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:41.047377 11039 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:41.048060 11277 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:41.049636 11280 consensus_queue.cc:1059] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:41.050742 11280 raft_consensus.cc:3055] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:41.051092 11280 raft_consensus.cc:740] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:41.025885 11218 raft_consensus.cc:740] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:41.052197 11280 consensus_queue.cc:260] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:41.053579 11218 consensus_queue.cc:260] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:41.055101 11218 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:41.041676  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 97250.8us
I20260709 13:02:41.056836 11237 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.057502 10747 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.058091 11072 raft_consensus.cc:3055] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:41.058213 10747 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.060591 10747 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:41.012451  8256 rpcz_store.cc:276] 0709 13:02:39.632132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.635445 (+  3313us) service_pool.cc:224] Handling call
0709 13:02:39.641085 (+  5640us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.796098 (+155013us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:39.824113 (+ 28015us) log.cc:844] Serialized 14 byte log entry
0709 13:02:40.149135 (+325022us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:40.149291 (+   156us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:40.149517 (+   226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.970061 (+820544us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b5400178754. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:40.970119 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.970336 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":29402,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":64,"raft.queue_time_us":3773,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":16564,"spinlock_wait_cycles":355819392,"thread_start_us":324091,"threads_started":1,"wal-append.queue_time_us":756701}
I20260709 13:02:41.019896 11279 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:41.009299 11245 log.cc:927] Time spent T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.913s	user 0.000s	sys 0.009s
I20260709 13:02:41.020126 11089 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.055018  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:41.058597 11072 raft_consensus.cc:740] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:40.981676  8222 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:41.043692 11289 consensus_queue.cc:237] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.035563  8251 raft_consensus.cc:2393] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 9: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:41.005496  9001 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:41.063690  8331 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:41.079906  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:41.080708 11057 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:41.080931  9005 raft_consensus.cc:2393] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 8: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:41.081211 11057 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.081612 11274 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:41.082324 11274 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:41.079928  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 98643.2us
I20260709 13:02:41.084277  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:41.085122  8254 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:02:41.085697  8297 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:41.086397 11275 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:41.086934  8222 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:41.088701  8222 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:41.089092 11275 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:41.075564  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:41.090129  8372 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:41.088727  8297 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:41.085060 11274 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:41.081645 11057 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:41.103421  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:41.096931 11286 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.017127 11018 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:41.085717 11284 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.097703 11248 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:41.101320 11280 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:41.101468 11197 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:41.104142 11072 consensus_queue.cc:260] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:41.117549  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:41.118136  8220 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:41.118496  8404 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:41.119218  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 397) took 1784 ms. Trace:
I20260709 13:02:41.119577 11007 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:41.120409  8220 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:41.119626  8257 rpcz_store.cc:276] 0709 13:02:39.334724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.338111 (+  3387us) service_pool.cc:224] Handling call
0709 13:02:39.338494 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:39.338892 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:39.348866 (+  9974us) log.cc:844] Serialized 14 byte log entry
0709 13:02:39.627238 (+278372us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:39.628673 (+  1435us) log.cc:844] Serialized 26 byte log entry
0709 13:02:39.628820 (+   147us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:39.628865 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.628960 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:41.008391 (+1379431us) spinlock_profiling.cc:243] Waited 85.7 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:41.118731 (+110340us) raft_consensus.cc:1685] finished
0709 13:02:41.118844 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.119043 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9712,"prepare.queue_time_us":10236,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":57,"raft.queue_time_us":987,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":15806,"spinlock_wait_cycles":197179264,"thread_start_us":277979,"threads_started":1,"wal-append.queue_time_us":662435}
I20260709 13:02:41.121619 11275 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:41.122027 11275 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:41.122411 11275 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:41.125562 11210 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:41.126127 11210 raft_consensus.cc:697] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:41.126394 11072 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:41.131587 11286 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:41.132789 11279 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:41.134423  8329 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:41.136348  8222 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:41.112334 11277 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:41.139747  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:41.140394 11089 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:41.140509  8331 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:41.146018  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:41.146735  8326 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
W20260709 13:02:41.147058 11258 log.cc:927] Time spent T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.874s	user 0.004s	sys 0.000s
I20260709 13:02:41.147513  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:41.148213  8325 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:41.149395  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:41.149698  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1828) took 1519 ms. Trace:
I20260709 13:02:41.150090  8332 rpcz_store.cc:276] 0709 13:02:39.630349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.630477 (+   128us) service_pool.cc:224] Handling call
0709 13:02:39.630787 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.688036 (+ 57249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:39.688064 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:39.688112 (+    48us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:39.688166 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.149282 (+1461116us) spinlock_profiling.cc:243] Waited 57 ms on lock 0x7b540011cbd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:41.149342 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.149543 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":131170048}
I20260709 13:02:41.151113  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:41.151718  8330 raft_consensus.cc:2376] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7. Re-sending same reply.
I20260709 13:02:41.150782  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:41.132809 11018 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:41.131606 11284 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.155928 11284 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:41.156895 11314 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:41.157363 11314 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:41.157791 11314 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:41.158416  8324 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 1830) took 1527 ms (client timeout 1856 ms). Trace:
W20260709 13:02:41.158840  8324 rpcz_store.cc:269] 0709 13:02:39.630870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.632982 (+  2112us) service_pool.cc:224] Handling call
0709 13:02:41.158099 (+1525117us) spinlock_profiling.cc:243] Waited 69.2 ms on lock 0x7b54003c9954. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:41.158260 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":159235840}
I20260709 13:02:41.165795 11197 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.166523 11197 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:41.168776 11197 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:41.160634 11279 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:41.169824 11279 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:41.170265 11279 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:41.171038  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:41.169754  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:41.178050  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:41.178697  8397 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:41.180193 10921 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:41.181806 11294 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.182331 11294 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.183172  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 384) took 1942 ms. Trace:
I20260709 13:02:41.183775  8400 raft_consensus.cc:2410] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:02:41.184259  8248 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:41.185302 11286 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:41.186421  8974 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.3:37367 timed out after 2.240s (SENT) [suppressed 19 similar messages]
I20260709 13:02:41.186470  8220 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:41.173775 11170 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:41.183578  8398 rpcz_store.cc:276] 0709 13:02:39.240618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.240774 (+   156us) service_pool.cc:224] Handling call
0709 13:02:39.241091 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.536976 (+1295885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.182677 (+645701us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400276d54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:41.182969 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2968299904}
I20260709 13:02:41.193776  8222 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:41.127350 11210 consensus_queue.cc:237] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.214571  9005 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:41.210623  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:41.223416  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:41.226467  9008 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:41.226558 11046 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:41.227882  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:41.228722  8330 raft_consensus.cc:2393] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 6: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:41.221073 11089 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:41.229535 11089 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.229895 11089 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:41.230165  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:41.230942  9007 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 4.
W20260709 13:02:41.233865  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 47532us
I20260709 13:02:41.234483  9006 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:41.234709  8220 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:41.235340 11161 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:41.236274 11161 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:41.238461  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:41.239209  8323 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
W20260709 13:02:41.264322  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.221721 11284 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:41.265610 11284 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:41.267973 11284 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:41.268843 11311 log.cc:927] Time spent T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
W20260709 13:02:41.273556 11312 log.cc:927] Time spent T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.284s	user 0.000s	sys 0.001s
I20260709 13:02:41.275556  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1738) took 1594 ms. Trace:
I20260709 13:02:41.275964  8252 rpcz_store.cc:276] 0709 13:02:39.680946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.990952 (+310006us) service_pool.cc:224] Handling call
0709 13:02:40.020235 (+ 29283us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:40.038495 (+ 18260us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:02:40.038548 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:40.859474 (+820926us) spinlock_profiling.cc:243] Waited 1.17 ms on lock 0x7b2c000014c8. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:02:40.860447 (+   973us) log.cc:844] Serialized 52 byte log entry
0709 13:02:40.871301 (+ 10854us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:40.871347 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.871460 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:41.275057 (+403597us) raft_consensus.cc:1685] finished
0709 13:02:41.275084 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.275355 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":93197,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":76,"spinlock_wait_cycles":2699520,"thread_start_us":10251,"threads_started":1,"wal-append.queue_time_us":10787}
I20260709 13:02:41.344111  9001 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: [8.9-8.9]
W20260709 13:02:41.325133 11317 log.cc:927] Time spent T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:02:41.362473  8332 raft_consensus.cc:1240] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:02:41.380626  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:41.381567  9006 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:02:41.382723  8403 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:02:41.388552  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:41.408583 11108 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.720s	user 0.003s	sys 0.039s
W20260709 13:02:41.409148 11108 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.912s	user 0.004s	sys 0.039s
I20260709 13:02:41.409488 11108 raft_consensus.cc:697] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 5 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:41.410854 11108 consensus_queue.cc:237] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.388692  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:41.413683 11280 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.414465 11280 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.417306  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:41.417852  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:41.422658  8327 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:41.422928  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:41.423887  9006 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:41.425985  8372 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:41.431779  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.427155 11280 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:41.433084  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 48382.2us
I20260709 13:02:41.433974  8298 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:41.433101 11289 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:41.435035 11231 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:41.435155 11289 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.435366  8298 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:41.435736 11289 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:41.443471  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:41.444368  9007 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:02:41.444954  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:41.445869  8332 raft_consensus.cc:2393] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 7: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:41.446597 10965 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.447149 10965 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:41.448613 11018 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.449297 11018 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.416311 11275 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:41.471312  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.478132  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:41.493988  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:41.457988 11314 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:41.480245 10921 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:41.503856  8298 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:41.521252  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:41.522149  8328 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:41.525921 11034 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:41.527189  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 51801.9us
I20260709 13:02:41.528209 11279 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.529238 11034 consensus_queue.cc:1059] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:41.530478 11286 raft_consensus.cc:3055] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:41.531068 11286 raft_consensus.cc:740] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:41.532815 11277 consensus_queue.cc:1059] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:41.532418 11286 consensus_queue.cc:260] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.533174 11057 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:41.534039 11286 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:41.534341 11203 raft_consensus.cc:3055] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:41.534792 11203 raft_consensus.cc:740] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:41.535912 11057 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:41.535714 11203 consensus_queue.cc:260] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.530580 11279 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:41.536772  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:41.537336  8404 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:41.537470 11203 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:41.539266 11248 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.539898 11248 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.543619  8324 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:41.547353  8404 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:41.547031 11089 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.547127 11248 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:41.550812 11089 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:41.552299 11260 log.cc:927] Time spent T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 1.498s	user 0.000s	sys 0.012s
I20260709 13:02:41.494385 10965 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:41.557391  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.559386  8327 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:41.576426  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:41.577065  8397 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:41.577553  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 44977.9us
I20260709 13:02:41.578449  8298 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:41.556815 11218 consensus_meta.cc:287] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.501s	user 0.008s	sys 0.006s
I20260709 13:02:41.585538  8298 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:41.552428 11314 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:41.594813  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.585016 11277 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:41.617803  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 267483us
I20260709 13:02:41.622191  8298 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:41.622308  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:41.615579  8398 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:41.624094  8324 raft_consensus.cc:1275] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:41.624383  8298 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:41.623715  8222 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:41.594358 10921 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:41.594712 11018 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:41.631675  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:41.632441  8327 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:41.594264 11292 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.637809  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:41.638546  8402 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:41.641651 11201 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:41.642294 11201 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:41.642771 11201 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:41.651019 10930 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:41.651767 10930 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:41.652347 10930 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:41.624372 11274 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:41.654080 11274 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.658656  9004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 544) took 2404 ms. Trace:
I20260709 13:02:41.659097  9004 rpcz_store.cc:276] 0709 13:02:39.254082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.256220 (+  2138us) service_pool.cc:224] Handling call
0709 13:02:39.304676 (+ 48456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.807821 (+503145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.658224 (+1850403us) spinlock_profiling.cc:243] Waited 444 ms on lock 0x7b54005cf8d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:41.658484 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1022167424}
I20260709 13:02:41.662482 11289 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.667403  8397 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:02:41.669992  8327 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:41.619144 11279 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.679042 11279 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:41.623927 11277 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:41.686573 11277 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:41.693588 11292 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:41.631203 11314 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:41.696632 11314 raft_consensus.cc:697] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 8 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:41.700024  9003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 662) took 1177 ms. Trace:
I20260709 13:02:41.700416  9003 rpcz_store.cc:276] 0709 13:02:40.522989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.524037 (+  1048us) service_pool.cc:224] Handling call
0709 13:02:40.524364 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.036559 (+512195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.699727 (+663168us) spinlock_profiling.cc:243] Waited 511 ms on lock 0x7b540057f8d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:41.699941 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1175851904}
I20260709 13:02:41.701725  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:41.709511 11248 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:41.710228 11248 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.712491 11248 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:41.714519  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.630046 11315 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:02:41.715248  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 80911.1us
I20260709 13:02:41.721156 11106 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:41.657123 11274 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:41.731640  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 283640us
I20260709 13:02:41.730297  8222 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:41.748178 10965 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.748546  8399 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:02:41.749583 11046 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.718465 11315 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:41.748857 10965 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.729592 11289 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:41.738466 11288 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:41.759544  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:41.764534  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:41.768049  8405 raft_consensus.cc:2410] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:02:41.752628 11039 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.752413 10747 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.768981  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:41.772132  8399 raft_consensus.cc:2410] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:41.756248 11315 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:41.750140 11046 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.756923 11210 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:41.757185 11279 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:41.777738  8406 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 5. Current term is 6. Ops: []
W20260709 13:02:41.769835 11072 consensus_meta.cc:287] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.643s	user 0.013s	sys 0.004s
I20260709 13:02:41.780848  8369 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:41.785665  8402 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:41.768893 11288 raft_consensus.cc:697] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:41.795336  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:41.740188 11314 consensus_queue.cc:237] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.802947  8222 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:41.805850 11003 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.774338 11081 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.807010 11289 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:41.808631 11231 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:41.808151 11288 consensus_queue.cc:237] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.809940 11034 consensus_queue.cc:1059] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:41.810810 11203 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.811800  9003 raft_consensus.cc:1240] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:41.774850 11039 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:41.812424  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:41.820780 11072 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:41.839521  8402 raft_consensus.cc:2393] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 8: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:41.839702 11203 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.838352 11081 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.842244  8406 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:02:41.842695 11203 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:41.822113 11289 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:41.848309 11014 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:41.882153  8406 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:41.873448  8326 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:41.819969 10747 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:41.848757 11072 raft_consensus.cc:697] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:41.871189 11289 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:41.837256 11210 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:41.948030  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 262165us
I20260709 13:02:41.868637 11218 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.870069 11203 consensus_queue.cc:1059] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 13:02:41.962566  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:41.951005  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:41.967139 11046 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:41.967502 11081 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:41.960453 11039 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:41.968633  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:41.969630 11218 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:41.986568  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 213960us
I20260709 13:02:41.988662  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1684) took 2204 ms. Trace:
I20260709 13:02:41.989065  8248 rpcz_store.cc:276] 0709 13:02:39.784331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.260441 (+476110us) service_pool.cc:224] Handling call
0709 13:02:40.260746 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.184757 (+924011us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.988394 (+803637us) spinlock_profiling.cc:243] Waited 913 ms on lock 0x7b54001306d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:41.988596 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2099400960}
I20260709 13:02:41.989761 11218 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:41.990312  8255 raft_consensus.cc:1275] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:02:41.990486  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:41.991258  8256 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:41.991760  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:41.993772  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:41.994647  8251 raft_consensus.cc:2393] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 5: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:41.970223 11263 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:41.995750 11263 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:41.870298 11003 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.000080 11003 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:42.000499  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:41.976692  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.001807  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:42.002660  9003 raft_consensus.cc:2410] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 8 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:02:41.981607 11210 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:41.982193  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:41.982815  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:41.983461  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:41.990629  9004 raft_consensus.cc:2393] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 7: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
W20260709 13:02:41.970974  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.006363 11277 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.006601  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:42.011198  8403 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 1730) took 2532 ms (client timeout 1972 ms). Trace:
I20260709 13:02:42.011590  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.006954  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.012528  8402 raft_consensus.cc:2393] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 7: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:42.012779  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.002466  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.006810  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:41.981863 11203 raft_consensus.cc:3055] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:42.007829  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.007382  8220 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:42.012548  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.023736  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.026002  8398 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:02:42.008880  8254 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:42.004142  8369 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:42.014022  8399 raft_consensus.cc:2410] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:02:42.011615  8403 rpcz_store.cc:269] 0709 13:02:39.478640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.480108 (+  1468us) service_pool.cc:224] Handling call
0709 13:02:42.010960 (+2530852us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b54001cc854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:42.011092 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4212970240}
I20260709 13:02:42.013335  8401 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:42.017009  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.029624  8397 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:02:42.023362  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.031378  8399 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:02:42.031740  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 51112.2us
I20260709 13:02:42.031757  8256 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:02:42.008468 11279 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:42.033689 11130 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.034103 11279 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:42.034287 11130 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.034420 11341 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:42.035668 11341 raft_consensus.cc:697] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:42.036223 11279 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:42.023882 11170 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:42.036841 11130 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:42.037046 11170 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:41.977602 11173 consensus_queue.cc:260] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.015005 11218 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:42.015661 11275 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:42.052938  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 137825us
I20260709 13:02:42.048731 11039 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:02:42.024811 11203 raft_consensus.cc:740] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:02:42.052570  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 45265.6us
I20260709 13:02:42.051167  8401 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:02:42.054106  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:42.058096  8403 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:42.045615 11072 consensus_queue.cc:237] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:42.060637  8398 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:42.060911  8220 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:42.055552  8401 pending_rounds.cc:85] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 3
I20260709 13:02:42.063050  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:42.037309 11341 consensus_queue.cc:237] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:42.065856  8397 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:42.053331 11218 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:02:42.069947  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.070122  8400 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:02:42.072618  8400 pending_rounds.cc:85] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 4
I20260709 13:02:42.066695  8401 pending_rounds.cc:107] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:02:42.074610  8401 raft_consensus.cc:2889] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:42.073026  8400 pending_rounds.cc:107] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:02:42.075214  8400 raft_consensus.cc:2889] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:42.032582 11201 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.064594 10965 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:42.063840 11263 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:42.080168  8220 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:42.067750 11279 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:42.070824 11277 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:42.075246 11218 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:42.083031  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:42.079756  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:42.065171 10930 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.083927 11279 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.092216 11294 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:42.093216  9004 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:42.092924 11203 consensus_queue.cc:260] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.097074 11203 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:42.098925  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:42.099591  8323 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
W20260709 13:02:42.093487  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.102159  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.103041  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.103904  8398 raft_consensus.cc:2410] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:02:42.103811  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:42.107560 11056 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.108156 11056 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:42.094390  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:42.103248  8403 raft_consensus.cc:2393] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 6: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:42.082175 11173 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:42.114056  8332 raft_consensus.cc:1275] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:02:42.093261  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:42.112772  8220 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:42.114754  8400 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:42.116295  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:42.117663  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:42.118243  8323 raft_consensus.cc:2376] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6. Re-sending same reply.
I20260709 13:02:42.118940 10930 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:42.087731 11275 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:42.121969 11039 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.080880 11315 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.111716 11279 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:42.127306  8222 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:42.125259  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:42.124814  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:42.133255  8252 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:42.134804  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:42.135076 11056 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:42.135419  9007 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:42.137593 10747 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:42.137769  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:42.138358  8323 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:42.138298  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:42.139103  8254 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:42.139703  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:42.140558  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:42.129791 11130 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:42.145767  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:42.146531  8220 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:42.147279  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:42.152627  9008 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:02:42.150291  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:42.153311  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:42.154569  8401 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:42.158162  8298 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:42.124246 11275 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:42.162173  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:42.157786 11130 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:42.164079  9005 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:42.164235 11130 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:42.164790 11130 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:42.162753  8330 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:42.163167 11275 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:42.119942 11294 consensus_queue.cc:1059] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:02:42.139726 11315 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:42.149686 11288 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:42.168361  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:42.162261 11003 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:42.171840  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:42.152428 11289 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.173770  9006 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:42.175038  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.176990  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1727) took 2544 ms. Trace:
I20260709 13:02:42.178323  8220 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:42.179340 11243 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:42.179562  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:42.278872  9003 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:42.279529  9001 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:02:42.280377  9003 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:42.281073  8298 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:42.261420  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.274714  8328 raft_consensus.cc:1240] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:42.282836  8400 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:42.222676  8254 raft_consensus.cc:1240] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:42.253324  8332 raft_consensus.cc:2410] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:02:42.285275  9005 raft_consensus.cc:2393] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 7: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:42.266067  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.285900  8402 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
W20260709 13:02:42.289276  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.290882  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:42.243633  8325 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:42.291054  8372 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:42.281864  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:42.294695  9004 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:42.296334  9002 raft_consensus.cc:1240] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:42.297317  8298 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:42.297870 11341 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:42.298022  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:42.298285 11203 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:42.298422 11341 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.298717  8256 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:02:42.298905 11203 raft_consensus.cc:697] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 7 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:42.300163 11203 consensus_queue.cc:237] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.301613 11277 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:42.301973 11042 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:42.303117  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:42.298992 11341 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:42.302259 11277 raft_consensus.cc:697] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 5 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:42.309759  8403 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:42.310384  8256 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:42.303858  8250 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:42.311318  9008 raft_consensus.cc:2393] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 7: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:42.300177  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:42.313758  9007 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:42.329365  8401 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:42.315462  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:42.252534 11347 log.cc:927] Time spent T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
W20260709 13:02:42.190037  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.343393  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:42.353309  8248 raft_consensus.cc:2410] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:42.346184  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:42.354414  8405 raft_consensus.cc:2393] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 7: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:42.357583 11201 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:42.357743  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:42.273288 11130 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:42.358246  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:42.158516 11039 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.196918 11288 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:02:42.358601  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 46652.3us
I20260709 13:02:42.359431 11149 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:42.312273 11003 raft_consensus.cc:697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:42.371138  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:42.332284  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:42.231505  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:42.375655  8401 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:42.353246  8403 raft_consensus.cc:2368] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 6. Current term is 7.
I20260709 13:02:42.403644  9009 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:02:42.285699  8253 rpcz_store.cc:276] 0709 13:02:39.632384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.691689 (+ 59305us) service_pool.cc:224] Handling call
0709 13:02:39.692062 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:39.692510 (+   448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:39.692535 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:39.692957 (+   422us) log.cc:844] Serialized 26 byte log entry
0709 13:02:39.888201 (+195244us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:39.888241 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.888308 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:41.146455 (+1258147us) spinlock_profiling.cc:243] Waited 4.99 ms on lock 0x7b2400168908. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255579c4 00007f8825555acb 00007f8825710dc9 00007f8825710a55 00007f882571009d 00007f882834210e 00007f882835988c 00007f8828358bb6 00007f88283587d8 00007f8828362221
0709 13:02:42.176041 (+1029586us) spinlock_profiling.cc:243] Waited 29.2 ms on lock 0x7b2400168998. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255579c4 00007f8825555acb 00007f8825710dc9 00007f8825710a55 00007f882570ffb0 00007f8828386488 00007f8828386be8 00007f882835e398 00007f882835976a 00007f8828358bb6
0709 13:02:42.176580 (+   539us) raft_consensus.cc:1685] finished
0709 13:02:42.176600 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.176799 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":97,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":7561,"spinlock_wait_cycles":78719104,"thread_start_us":184794,"threads_started":1,"wal-append.queue_time_us":445680}
I20260709 13:02:42.380916  8251 raft_consensus.cc:1275] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:42.405014  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:42.383251  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:42.230675  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:42.386858  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.408730  8402 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:42.385915  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:42.390547  8257 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:42.391242  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.385039 11056 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.393371 11039 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:42.238330 11046 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.146s
I20260709 13:02:42.394511 11130 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:42.395612 11275 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:42.396432  8248 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:42.397248  8255 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:42.405903  8250 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:02:42.406518  8398 raft_consensus.cc:2376] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8. Re-sending same reply.
I20260709 13:02:42.411096  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:42.412184  8406 raft_consensus.cc:2410] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:02:42.397562 11201 consensus_queue.cc:1059] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:02:42.397863 11263 consensus_queue.cc:1059] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:02:42.385392 11243 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:42.412463  8297 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:42.417500  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:42.418475  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 240174us
I20260709 13:02:42.398700 11288 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:42.417348  8329 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:42.421293  9005 raft_consensus.cc:2410] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 8 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:02:42.421108  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:42.414036  9000 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:42.426398  8254 raft_consensus.cc:2410] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:02:42.428468  8403 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:02:42.433713  8220 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:42.428975  8403 pending_rounds.cc:85] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 6
I20260709 13:02:42.438197  8403 pending_rounds.cc:107] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:02:42.438512  8403 raft_consensus.cc:2889] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:42.440320  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 49971us
I20260709 13:02:42.441880  8297 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:42.443032  8297 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:42.442688 11276 consensus_queue.cc:1059] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:42.442065 11197 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:42.444307  8401 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:42.426879 11243 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:42.445276 11243 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:42.446341 11289 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:42.420889 11277 consensus_queue.cc:237] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.451534 11248 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:42.451699  8972 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:42.452327  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:42.452939  8406 raft_consensus.cc:2376] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6. Re-sending same reply.
I20260709 13:02:42.453068 11321 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:42.453550 11321 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.453889 11321 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:42.456363 11345 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.297s	user 0.003s	sys 0.000s
I20260709 13:02:42.457621 11062 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:42.458187 11062 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:42.461293 11062 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:42.452083 11248 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.466650 11284 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:42.471282 10690 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:42.471424 11062 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.471944 11284 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:42.472141 10690 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:42.472294 11062 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:42.461870 11003 consensus_queue.cc:237] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.504165  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:42.414889 11294 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:42.414335 11056 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.508245 11056 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:42.464500  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:42.449986  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:42.463460  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 170833us
I20260709 13:02:42.509295 11127 raft_consensus.cc:3055] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:42.519281 11127 raft_consensus.cc:740] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:42.520373 11127 consensus_queue.cc:260] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.521735 11127 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:42.486326 11062 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:42.488763 11248 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:42.490278 10747 raft_consensus.cc:3055] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:42.491317  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.494705 11275 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:42.398890 11265 consensus_queue.cc:1059] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:02:42.473776  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:42.512111 11276 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:42.522323  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:42.522590  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:42.480489 11130 consensus_queue.cc:1059] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 13:02:42.524109  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:42.525275  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:42.485726 10690 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:42.509935  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.532166  8402 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:42.526703 11218 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:42.538998  8404 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:02:42.567711  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:42.581231  8406 raft_consensus.cc:2393] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 5: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:02:42.583001  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 140032us
I20260709 13:02:42.583290  8369 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:42.568265  8974 leader_election.cc:400] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 6. Current term is 7.
I20260709 13:02:42.586973  8974 leader_election.cc:403] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:02:42.481508 11072 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:42.512764 11197 raft_consensus.cc:3055] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:42.511083 11170 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:42.587224  8398 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:42.580870  8372 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:42.577695 11149 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.538552 10747 raft_consensus.cc:740] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:42.600030  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.607381  8402 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:02:42.565397 11248 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:42.610486  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:42.606766 11149 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:42.611203 11286 raft_consensus.cc:3055] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:42.579988 11108 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.617048  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.612473  9002 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:02:42.617750  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.618842  8403 raft_consensus.cc:2410] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:02:42.579629 11161 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.616813 11089 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.614148 11265 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:02:42.626328 11089 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:42.626928 11218 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:42.627157 11294 consensus_queue.cc:1059] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:42.616995 11286 raft_consensus.cc:740] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:42.628430 11089 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:42.628630 11292 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:42.629055 11108 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.629801 11284 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:42.630293 11042 raft_consensus.cc:3055] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:02:42.630597  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 193142us
I20260709 13:02:42.630760 11042 raft_consensus.cc:740] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:42.629818  9002 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:42.631395  8974 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:42.632104 11108 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:42.631716 11042 consensus_queue.cc:260] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.632624 11108 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 6. Current term is 7.
I20260709 13:02:42.633373 11275 raft_consensus.cc:3055] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:42.633752 11275 raft_consensus.cc:740] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:42.634503 11275 consensus_queue.cc:260] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:42.634829  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:42.634755 11288 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:42.635591  8324 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:02:42.578058 11341 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:42.637614 11341 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:42.637679 11288 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:42.608350 11289 raft_consensus.cc:2764] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:02:42.638320 10747 consensus_queue.cc:260] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.639580 10747 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:42.640148 11170 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:42.602458 11197 raft_consensus.cc:740] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:42.629102 11286 consensus_queue.cc:260] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.617758  8404 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:42.602427 11039 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:42.572422 10965 consensus_queue.cc:1048] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.048s
W20260709 13:02:42.658900  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 43854.8us
I20260709 13:02:42.645489  9007 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:42.660845  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:42.666091  8403 raft_consensus.cc:1240] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:42.619897 11046 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:42.675760  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:42.677203  8222 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:42.645177 11108 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:42.685809  8332 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:42.686833  8297 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:42.659989 11039 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.742192  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:42.742946  8325 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:42.662324 11288 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:42.723484  8324 raft_consensus.cc:2393] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 7: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:42.667488 11284 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.756248 11284 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:42.758828 11284 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:42.620224 11248 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.761155 11248 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:42.673043 11281 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:42.687390 11173 consensus_meta.cc:287] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.562s	user 0.010s	sys 0.001s
W20260709 13:02:42.738471  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 61319.6us
I20260709 13:02:42.658968 11321 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.765964 11321 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:42.726815 11286 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:42.688910 11046 raft_consensus.cc:697] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:42.770397 11046 consensus_queue.cc:237] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.751950  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:42.746064 11039 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:42.745951 11130 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:42.777796 11130 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.778270 11130 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:42.755652  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.766314 11081 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:42.784332  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 152966us
I20260709 13:02:42.787710  8974 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:42.794724  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 213874us
I20260709 13:02:42.738621 11201 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.676437 11161 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.739091 11108 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:02:42.796279 11279 consensus_meta.cc:287] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.637s	user 0.010s	sys 0.000s
I20260709 13:02:42.784762 10930 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.823122  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:42.828176  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:42.842942  8403 raft_consensus.cc:2376] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7. Re-sending same reply.
I20260709 13:02:42.830673  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.828117  8222 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:42.821611  8372 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:42.857508  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:42.828581  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:42.745653 11243 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:42.804225 11062 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.807226 10965 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.862418  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:42.817554 11108 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:42.864135  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:42.864249  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:42.876655  8257 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:42.864354  8372 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:42.863380 11072 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:42.880069  8253 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:02:42.876816  8254 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:02:42.862773  8248 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:42.865113  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:42.881608  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.728401 11197 consensus_queue.cc:260] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:42.818415 11321 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:42.906435  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:42.828307 11248 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.817281 11003 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:42.877803 11062 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.882826  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:42.917668  8403 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:42.784261 11081 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.892576  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:42.897218  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.900852  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:42.901242  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:42.904764  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:42.907275  8252 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:02:42.923203  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:42.907900  8255 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:42.923815  8254 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:42.804904 11288 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.870864 11218 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:02:42.912035  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 302837us
I20260709 13:02:42.927284  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:42.923614  9006 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:42.928002  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:42.929108  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:42.929922  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:42.864955 11343 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.809141 11289 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.918599  8250 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:42.923375  8327 raft_consensus.cc:2410] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:02:42.908978  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:42.910610  8326 raft_consensus.cc:2393] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 6: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:42.922646  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:42.933642  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.828080 11279 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:42.932703  8248 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:02:42.906229 11237 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:42.858147 10930 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.939849  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:42.911780 11341 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.941056  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.910324 11072 raft_consensus.cc:697] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:42.944556  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:42.942914  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:42.948273  8324 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:42.948745  8326 raft_consensus.cc:2393] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 7: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:42.949111  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:42.944831  8398 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:42.950423  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:42.944833  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:42.951479  8402 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:42.879783 10965 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:42.947207  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:42.953301  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:42.955976  8249 raft_consensus.cc:2393] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 8: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:42.956283  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:42.956874  8252 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:42.933575 11218 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.958060  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:42.959257  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 287977us
I20260709 13:02:42.963828  8399 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:02:42.966388  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:42.967244  8331 raft_consensus.cc:2393] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 7: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:42.968187  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:42.968454  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:42.969156  8253 raft_consensus.cc:2376] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9. Re-sending same reply.
I20260709 13:02:42.969416  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:42.970260  8331 raft_consensus.cc:2393] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 6: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:42.970461  8253 raft_consensus.cc:1240] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 8 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:42.971032  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:42.972013  8331 raft_consensus.cc:2410] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:02:42.972786  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:42.974956 10965 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:42.973361  8331 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:42.981323 11161 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:42.933763 11321 raft_consensus.cc:697] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:42.936988 11203 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.964282  8399 pending_rounds.cc:85] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 8
I20260709 13:02:42.882640 11201 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } }
I20260709 13:02:42.920907 11248 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:42.959945  8403 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:02:42.990325  8399 pending_rounds.cc:107] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:02:42.990139  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:42.985100  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:42.997241  8399 raft_consensus.cc:2889] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:42.964078  8297 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:43.004309  8326 raft_consensus.cc:1275] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:42.997464  8252 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 4.
I20260709 13:02:43.011287 11286 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:02:43.011684  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 47689.3us
I20260709 13:02:43.012567  8297 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:43.013595 11286 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:43.014664  8297 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:43.015663 11277 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:42.968204 11218 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:42.962842 11341 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:42.939972 11343 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:43.011332  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:42.938558 11289 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:43.018373  8972 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:43.019423  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:43.017411  8297 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:43.060079  8324 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:02:43.018626  8402 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:02:42.994469 10965 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:02:43.065361  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 48033.4us
I20260709 13:02:43.066702  8297 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:02:43.071113  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 52805.8us
I20260709 13:02:43.073750  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:43.050056 11353 log.cc:927] Time spent T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.240s	user 0.001s	sys 0.000s
I20260709 13:02:43.041708 11072 consensus_queue.cc:237] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:43.078796 11289 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:43.054574 11039 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:43.082489  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 1746) took 1104 ms. Trace:
I20260709 13:02:43.082904  8405 rpcz_store.cc:276] 0709 13:02:41.978497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.978737 (+   240us) service_pool.cc:224] Handling call
0709 13:02:43.082217 (+1103480us) spinlock_profiling.cc:243] Waited 339 ms on lock 0x7b54000fc3d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:43.082405 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":778861312}
I20260709 13:02:43.084340  8406 raft_consensus.cc:1240] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:43.041558 11321 consensus_queue.cc:237] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:43.085089  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:43.085711  9005 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:02:43.085516 10965 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:43.086498  8397 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:43.087277  8406 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:43.087754 10965 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:43.088119  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 389) took 1015 ms. Trace:
W20260709 13:02:43.088474 11354 log.cc:927] Time spent T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 13:02:43.089860 11315 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:43.090308 11315 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:43.090682 11315 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:43.072021  8972 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:43.092444  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:43.093019  8402 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:42.995339 11364 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:43.084481  8405 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:02:43.088506  8397 rpcz_store.cc:276] 0709 13:02:42.072628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.073659 (+  1031us) service_pool.cc:224] Handling call
0709 13:02:43.086348 (+1012689us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:43.086892 (+   544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.087899 (+  1007us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54000fc3d0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:43.088031 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2320753536}
I20260709 13:02:43.048575 11243 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:43.047734 11062 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:43.055684 11277 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:43.081238 11039 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:43.116873  8974 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:43.109638  8249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1750) took 1079 ms. Trace:
I20260709 13:02:43.117702  8249 rpcz_store.cc:276] 0709 13:02:42.032328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.042318 (+  9990us) service_pool.cc:224] Handling call
0709 13:02:43.105329 (+1063011us) spinlock_profiling.cc:243] Waited 892 ms on lock 0x7b540034bcd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:43.109554 (+  4225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2051580800}
I20260709 13:02:43.118793  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:43.119590  8331 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:43.109722 11358 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:43.110262 11366 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:43.126271  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:43.120357  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:43.136375  8403 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:43.137897  8398 raft_consensus.cc:2393] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 6: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:43.043720 11248 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:43.005182 11203 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:43.126901  8323 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:02:43.099745 11364 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:43.144289  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:43.063638 11130 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:43.145561  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:43.150187  8256 raft_consensus.cc:2393] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 7: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:02:43.148427  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 220254us
I20260709 13:02:43.094286 11108 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:43.145231  8328 raft_consensus.cc:2376] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8. Re-sending same reply.
I20260709 13:02:43.152693  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:43.155243  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:43.155390  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:43.156075  8253 raft_consensus.cc:2410] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:02:43.156252 11023 raft_consensus.cc:3055] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:43.156581 11023 raft_consensus.cc:740] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:43.156828  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:43.156893  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:43.155273  8254 raft_consensus.cc:2410] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:02:43.135496 11366 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:43.202138  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:43.202951  8256 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:43.178261  8331 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:02:43.162403  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:43.192693 11042 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:43.162499 11003 consensus_queue.cc:1059] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:02:43.194149  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:43.084488 11149 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:43.195315 11130 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:43.195277  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:43.167659 11127 consensus_meta.cc:287] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.635s	user 0.009s	sys 0.001s
I20260709 13:02:43.149636 11364 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:43.165081 10930 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:43.226794  9001 raft_consensus.cc:2410] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260709 13:02:43.196925  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.157361 11023 consensus_queue.cc:260] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:43.199090  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:43.233719  8248 raft_consensus.cc:2410] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:02:43.140321 11253 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:43.208782  8249 raft_consensus.cc:2393] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 7: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:02:43.163002 11365 log.cc:927] Time spent T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:02:43.137414 11277 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:43.214846  8252 raft_consensus.cc:2393] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 6: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:43.243938  8404 raft_consensus.cc:2368] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 7. Current term is 8.
I20260709 13:02:43.150595 11248 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:02:43.220620  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:43.166082 11201 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:43.207381 11366 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:43.249867 11341 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:43.250072 11368 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.230047  8222 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:43.240999  8398 raft_consensus.cc:1275] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:43.250972  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:43.241689  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 107195us
I20260709 13:02:43.243317  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:43.243920  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:43.245163  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 122697us
I20260709 13:02:43.235359  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:43.150414 11072 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.079s
I20260709 13:02:43.245967  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:43.258844  8399 raft_consensus.cc:2410] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 10 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:02:43.246927  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:43.248831  9000 raft_consensus.cc:3055] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:43.249022  8401 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:43.249605 11108 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:43.251506  8402 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:43.214613 11003 consensus_queue.cc:1243] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 13:02:43.210356  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 58819us
W20260709 13:02:43.263410  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 146224us
I20260709 13:02:43.263914  8369 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:43.264079  8222 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:43.257032  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:43.264947  8369 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:43.265327  8972 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:43.268108  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:43.265946  8974 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:43.247485 11279 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:43.271744  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:43.274606  8369 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:43.274741  8325 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:43.277171  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:43.278568  8253 raft_consensus.cc:2393] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 9: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:43.279008  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:43.256112 11253 raft_consensus.cc:697] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:43.260133 11243 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:43.280457  8222 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:43.282704  8325 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:43.283589  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:43.284209  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:43.260288  9000 raft_consensus.cc:740] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:43.288622  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:43.288070  8324 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:43.289286  8323 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:43.289959  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:43.290540  8324 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:43.292284  9003 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:02:43.292670  9003 pending_rounds.cc:85] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: Aborting all ops after (but not including) 5
I20260709 13:02:43.292906  9003 pending_rounds.cc:107] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:02:43.293138  9003 raft_consensus.cc:2889] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:43.288012 10926 consensus_queue.cc:1059] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:02:43.294114 11314 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:43.294545  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:43.295257  8329 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:43.295432 11008 raft_consensus.cc:3055] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:43.281561 11369 raft_consensus.cc:2764] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:02:43.295979 11008 raft_consensus.cc:740] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:43.297421  8297 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:43.275189 11294 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:43.299880  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:43.269256 11341 raft_consensus.cc:2764] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:02:43.309158  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:43.275672 11023 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:43.273557 11130 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:43.315279  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:43.304179  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:43.276692 11173 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:43.316130  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:02:43.322710  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 60012.1us
I20260709 13:02:43.323273  8252 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:43.307138  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:43.307878  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:43.308734  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:43.324224  8256 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:43.324467  8332 raft_consensus.cc:2393] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 6: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:43.301594  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:43.313971 11343 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:43.270073 10930 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:02:43.294560 11314 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.338198 11314 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:43.342410 11173 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:43.293658 11243 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.263887 11127 consensus_queue.cc:1059] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:02:43.343930 11108 consensus_queue.cc:1059] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:43.317616 11381 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:43.345254 11355 raft_consensus.cc:3055] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:43.345661 11355 raft_consensus.cc:740] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:43.346542 11355 consensus_queue.cc:260] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:43.343978 11243 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:43.282608 11057 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:02:43.349190  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:43.273689 11003 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:02:43.285925 11376 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:43.357023  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:43.276424 11281 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:02:43.327346  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:43.363615  8254 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:43.327683  8328 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:43.364481  9009 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:43.250562 11368 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:43.333791 11203 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:43.334714  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:43.335752 11253 consensus_queue.cc:237] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:43.286382 11357 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:02:43.318089  8329 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:43.353155  8402 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:43.306645 11294 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:43.358641  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.358758  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:43.344918 11218 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:43.359629  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 94320.1us
I20260709 13:02:43.354569 11248 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:43.309212 10926 consensus_queue.cc:1048] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:02:43.356671 11130 raft_consensus.cc:697] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:43.357941 11380 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:43.401682  8295 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:43.402159  8972 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:43.388754 11294 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:43.377116  8256 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
W20260709 13:02:43.411667  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 149970us
I20260709 13:02:43.413218  8295 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:43.362596 11358 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
W20260709 13:02:43.404940 10747 consensus_meta.cc:287] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.765s	user 0.010s	sys 0.002s
I20260709 13:02:43.418421  8295 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:43.422864  8295 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:02:43.424063  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 55212.5us
I20260709 13:02:43.424468  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:43.431434  8256 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:43.433495  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:43.434254  9008 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:43.396015 11366 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:43.365707 11008 consensus_queue.cc:260] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:43.417862 11149 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:43.412793 11380 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:43.413059 11248 raft_consensus.cc:697] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:43.422101 11003 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:43.413272 11343 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.460968  9007 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:02:43.439833 11382 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:43.469024  8297 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:02:43.472195  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 68994.5us
I20260709 13:02:43.435356 11383 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:43.470259  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:43.470261  9001 raft_consensus.cc:1240] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:43.436218 11081 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:43.473656  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:43.448742 11203 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:43.469020  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:43.446072  9009 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:43.448366 11277 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:43.479779  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:43.481973  8297 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:43.484220  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:43.484800  8403 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:02:43.490314  8397 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:02:43.491114  8399 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:02:43.492086  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:43.486128 10930 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:43.497921  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 75139.3us
I20260709 13:02:43.498641  8295 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:43.499338  9009 raft_consensus.cc:1275] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:43.499702  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:43.472486 11382 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.500701  8323 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:02:43.502280  8295 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:43.472040 11321 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:02:43.494009 11008 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:43.502899  9008 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:43.509153  8295 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:43.512073  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:43.529898  8252 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:43.519418  8972 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:43.537930  8222 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:43.467680 11315 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:43.540235  8295 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:43.509323 11286 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:43.493909 11380 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:43.548952  8295 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:43.549566  8254 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:43.550772  9007 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:43.506006 11383 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:02:43.516703 11263 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:43.507926 11003 raft_consensus.cc:2764] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:02:43.507970 11203 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:02:43.543016 11237 raft_consensus.cc:3055] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:43.503301 11248 consensus_queue.cc:237] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:43.545764  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:43.553339  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:43.481276 11130 consensus_queue.cc:237] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:43.555333 11277 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:43.597324 11275 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:43.555781 11057 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:43.556339  8249 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:43.601677 11057 raft_consensus.cc:697] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:43.474254 11343 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:43.602636 11057 consensus_queue.cc:237] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:43.531301 11382 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:43.603792  8223 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:43.561268  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:43.604480  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 74106.5us
I20260709 13:02:43.557411 11380 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:43.605067 11380 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:43.610095  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.574005  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:43.570194  8372 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:43.610893  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 49701.1us
I20260709 13:02:43.576051  8327 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:43.577155  8326 raft_consensus.cc:2410] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:02:43.524997 11334 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:02:43.513341  9000 consensus_queue.cc:260] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:43.569301 11237 raft_consensus.cc:740] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:43.548224 11321 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:43.519779 11243 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:43.523327 11294 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:43.557727 11369 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:43.595381 11288 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:43.605144  8223 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:43.605355 11387 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:43.560828  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:43.606624  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.559799 11265 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:43.612951 11391 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:43.626294 11341 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:43.626677 10965 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:43.560070 11263 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.633556 11321 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.615931  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:43.655413  8323 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:43.649878  9000 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:43.626173  8252 raft_consensus.cc:1275] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:43.639932  8223 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:43.662985  8223 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:43.664317  8223 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:43.642815  8324 raft_consensus.cc:1240] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:43.643560  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:43.643476  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:43.616170 11357 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:02:43.647625  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.648540  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:43.649209  8251 raft_consensus.cc:2368] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7.
I20260709 13:02:43.651012  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:43.644902 11286 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:43.546314 11108 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:02:43.656854  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:43.628731  8327 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:43.641366 11089 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:43.656555 11343 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:43.641461 11201 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.659441  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:43.636196 11368 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:43.669691  8972 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:43.707839  8972 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:43.673882  8223 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:43.709182  8972 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:43.640920 11387 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:43.686582  8253 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:43.690825  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:43.653678 11263 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:43.684605  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:43.713093  8324 raft_consensus.cc:2410] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:02:43.658028 11321 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:43.668900 11253 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:43.650061 11341 raft_consensus.cc:697] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:43.645138 11023 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.646003 11391 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.711272  8329 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:43.645610 11334 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:02:43.716325  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:43.702854  8252 raft_consensus.cc:1240] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:43.681463 11161 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:43.689872 11357 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:43.722664  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 48883.6us
I20260709 13:02:43.689554 11288 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:43.689777 11284 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:43.724306  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 73339.7us
I20260709 13:02:43.715791 11358 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:43.733935  8329 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:43.659183 11149 consensus_queue.cc:1048] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 13:02:43.740226  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:43.744541  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.659430 11314 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:43.714035 11387 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:43.715557 11389 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:43.720649 11343 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:43.722460 11201 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:43.753319  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:43.764098  8369 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:43.745342  8223 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:43.755344  8252 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:43.721728 11366 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:43.705003 11286 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:43.733021 11391 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:43.737989 11161 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.772262  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:43.790923  8329 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:43.791656  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:43.792096  8329 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:43.775061  8248 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 9: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:43.746064 10930 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:43.747567 11357 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:43.753381 11388 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:43.798856  8328 raft_consensus.cc:2393] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 7: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:43.805490  8329 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:02:43.732882 11023 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:43.811028  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:43.811913  8329 raft_consensus.cc:2393] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 8: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:43.754220 11381 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:43.756803 11389 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:43.815717  8252 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:43.816655  8332 raft_consensus.cc:2376] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7. Re-sending same reply.
I20260709 13:02:43.786543 11376 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:43.772413 11089 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:43.825698 11314 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:43.797793 11161 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:43.787573 11288 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:43.829998  8249 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:43.831383  8369 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:43.832290  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:43.798549 11315 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:43.835271 11149 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:43.835539  8369 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:43.841579  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:43.842238  8329 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:43.836126  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:43.786875 11279 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:02:43.838754  8331 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 5. (term mismatch)
I20260709 13:02:43.848695  8331 pending_rounds.cc:85] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 4
I20260709 13:02:43.848991  8331 pending_rounds.cc:107] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 4.5
I20260709 13:02:43.849068 11286 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:43.850172  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:43.849253  8331 raft_consensus.cc:2889] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:43.844535  8252 raft_consensus.cc:2368] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 7. Current term is 8.
I20260709 13:02:43.846658  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:43.854948  8295 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:43.797645 11072 consensus_queue.cc:1059] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:02:43.860718  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:43.838651 11364 consensus_meta.cc:287] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.594s	user 0.005s	sys 0.005s
I20260709 13:02:43.849514 10965 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:43.819060 11389 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:43.861720  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:43.863065  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:43.869777  8220 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:43.810747 11284 consensus_queue.cc:1059] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:02:43.871217  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:43.887642  8329 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:43.886548 11364 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:43.888746  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:43.813056 11203 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:02:43.889719  8400 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:43.890133  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:43.891170  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:43.891614  8329 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:43.897258 11364 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:43.850159 11294 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:43.860285 11286 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:43.818245 11341 consensus_queue.cc:237] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:43.954041  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:43.956351  8252 raft_consensus.cc:2410] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 11 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:02:43.943440 11414 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:43.937108  8369 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:43.902325  8329 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:43.901867  8220 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:43.953051 11334 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:43.887110 11243 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:43.886483 11343 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:43.866384 11042 consensus_queue.cc:1059] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.084s
I20260709 13:02:43.849789 11288 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:43.953410 11398 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
W20260709 13:02:43.955364  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 246204us
W20260709 13:02:43.972900 11406 log.cc:927] Time spent T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:02:43.975463 11321 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:43.977052 11334 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:43.978787  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:43.979498 11161 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:43.983350  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:43.849171 11237 consensus_queue.cc:260] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:43.987594 11263 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:43.989557  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 52556.4us
W20260709 13:02:43.953569 11400 log.cc:927] Time spent T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 13:02:43.930413 11253 raft_consensus.cc:697] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:02:43.969115  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 67341.1us
I20260709 13:02:44.029938  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:44.030707  8220 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:44.032481  8220 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:43.950508 11003 consensus_queue.cc:1059] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:02:44.011101  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:44.039619  8332 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:02:43.931411 11149 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.058s
I20260709 13:02:44.017169  8252 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:43.965075 10930 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:44.046700  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:44.047097 11161 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:44.046945 10930 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:43.995527 11294 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.049307 11294 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:44.052234 11294 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:44.053797  9000 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:02:44.055069 11127 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:02:44.056686  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:44.057601  8398 raft_consensus.cc:2393] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 10: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:44.062690 11288 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:44.067415 11277 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:44.076887 11081 raft_consensus.cc:3055] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:44.077555 11081 raft_consensus.cc:740] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:02:44.024947 11397 log.cc:927] Time spent T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.278s	user 0.001s	sys 0.000s
I20260709 13:02:43.994833  8972 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:44.096007  8332 raft_consensus.cc:1275] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:43.954773 11276 consensus_queue.cc:1059] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:02:44.036608  8369 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:02:44.034081 11416 log.cc:927] Time spent T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:02:44.045980 11396 log.cc:927] Time spent T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.285s	user 0.001s	sys 0.000s
I20260709 13:02:44.046140  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:44.118481  8406 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:02:44.032306  8255 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
W20260709 13:02:44.085685  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 53302.5us
I20260709 13:02:44.121275  8220 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:44.115588  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:44.086599  8329 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
W20260709 13:02:44.115898  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 79373.8us
I20260709 13:02:44.125005  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:44.081341 11314 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:44.126458  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:44.127954  8220 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:44.099186 11243 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:44.078629 11081 consensus_queue.cc:260] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:44.117223 11406 log.cc:927] Time spent T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 13:02:44.117797 11380 consensus_meta.cc:287] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.510s	user 0.005s	sys 0.004s
I20260709 13:02:44.131644  8220 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:44.116719 11355 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:44.024276 11382 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:44.119271 11237 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:44.135380  8369 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:44.180555  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 45249.1us
W20260709 13:02:44.129134 11394 log.cc:927] Time spent T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.364s	user 0.001s	sys 0.000s
I20260709 13:02:44.123123 11294 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:44.119318 11201 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:44.148980 11314 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:44.145828 11130 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:44.131309 11062 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:44.131059 11414 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:44.132350 11161 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:44.131496 11315 raft_consensus.cc:3055] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:44.175228  8295 leader_election.cc:400] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 7. Current term is 8.
I20260709 13:02:44.097232 11357 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:44.149950  8220 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:44.147207 11362 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:44.187511  9007 raft_consensus.cc:1240] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:02:44.145411 11243 raft_consensus.cc:697] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 8 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:44.190034  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:44.199146  8295 leader_election.cc:403] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:02:44.171936 10965 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:44.200843  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:44.282842  8331 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:02:44.283656  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:44.203104  8332 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:44.198877 11315 raft_consensus.cc:740] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:44.187747 11149 consensus_queue.cc:1059] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.090s
I20260709 13:02:44.207686 11161 raft_consensus.cc:697] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:44.185413 11294 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.189129 11062 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:44.276063  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:44.290464  8252 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
W20260709 13:02:44.276762  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.277344  8254 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:44.276707  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:44.151456 11380 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
W20260709 13:02:44.281999  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 106934us
I20260709 13:02:44.302137  8295 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:44.302937 11380 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:44.303563 11380 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:44.304141 11380 consensus_queue.cc:1059] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:44.304363 11201 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:44.304929 11380 consensus_queue.cc:1059] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:44.305617 11276 raft_consensus.cc:3055] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:44.305505 11417 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.111s
I20260709 13:02:44.306103 11276 raft_consensus.cc:740] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:44.306639 11357 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:44.307017 11276 consensus_queue.cc:260] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:44.176795 11081 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:44.292588  8328 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:44.208945 11314 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 13:02:44.293788 11263 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:44.292160 11062 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:44.290876 11294 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:44.289096 11321 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:44.297860 11243 consensus_queue.cc:237] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:44.298511 11149 raft_consensus.cc:3055] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:44.299438 11161 consensus_queue.cc:237] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:44.304253 10930 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:44.284924  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:44.327714  8369 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:44.308046 11422 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.281312 11414 raft_consensus.cc:697] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:44.308455  8331 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:44.288247 11364 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:44.290521 11277 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 7. Current term is 8.
I20260709 13:02:44.198365 11391 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:44.193181 11253 consensus_queue.cc:237] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:44.286789 11362 raft_consensus.cc:697] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:44.307387 11415 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.115s
I20260709 13:02:44.328696  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:44.312462  8406 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:44.329138  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:44.317102 11380 consensus_queue.cc:1243] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer 8c42259d7c4b463a9f485f42c320c81b log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:02:44.284652 11387 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:44.325461 11275 consensus_meta.cc:287] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.727s	user 0.010s	sys 0.001s
I20260709 13:02:44.362761  8331 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:44.323884 11149 raft_consensus.cc:740] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:44.330593 11276 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:44.344311 11414 consensus_queue.cc:237] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:44.331483 11042 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:44.332398 11417 consensus_queue.cc:1059] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:02:44.333671 11422 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:44.335602 11315 consensus_queue.cc:260] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:44.384764 11197 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:44.362768  8332 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:44.347261  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1776) took 1037 ms. Trace:
I20260709 13:02:44.402312  8251 rpcz_store.cc:276] 0709 13:02:43.309694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.348919 (+ 39225us) service_pool.cc:224] Handling call
0709 13:02:44.347053 (+998134us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b54000767d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:44.347186 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":687741312}
W20260709 13:02:44.387368  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 58291.9us
I20260709 13:02:44.393141  8330 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:44.406898  8328 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:44.409729  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1712) took 1262 ms. Trace:
I20260709 13:02:44.410247  8254 rpcz_store.cc:276] 0709 13:02:43.147766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.558791 (+411025us) service_pool.cc:224] Handling call
0709 13:02:43.559492 (+   701us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.292404 (+732912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.409398 (+116994us) spinlock_profiling.cc:243] Waited 706 ms on lock 0x7b54000762d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:44.409651 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1623281920}
I20260709 13:02:44.412102  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2100) took 1565 ms. Trace:
I20260709 13:02:44.412762 11089 raft_consensus.cc:3055] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:44.413697  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1715) took 1264 ms. Trace:
I20260709 13:02:44.414036  8250 rpcz_store.cc:276] 0709 13:02:43.149273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.588883 (+439610us) service_pool.cc:224] Handling call
0709 13:02:43.589181 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:43.589506 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:43.589533 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:44.063636 (+474103us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b2c000014c8. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:02:44.064196 (+   560us) log.cc:844] Serialized 26 byte log entry
0709 13:02:44.136451 (+ 72255us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:44.153000 (+ 16549us) log.cc:844] Serialized 14 byte log entry
0709 13:02:44.413286 (+260286us) spinlock_profiling.cc:243] Waited 1.28 ms on lock 0x7b180034e3a0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88283664bc 00007f88256b448b 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256ce455 00007f88287b6060 00007f88287c8716 00007f88287d9287
0709 13:02:44.413390 (+   104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.413476 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:44.413513 (+    37us) raft_consensus.cc:1685] finished
0709 13:02:44.413526 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.413646 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":101169,"prepare.queue_time_us":33206,"prepare.run_wall_time_us":6716,"raft.queue_time_us":47488,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":6822,"spinlock_wait_cycles":59179776,"thread_start_us":93856,"threads_started":2,"wal-append.queue_time_us":212816}
I20260709 13:02:44.415313 11315 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:44.417721  8332 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:44.388633 11415 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:44.413203 11089 raft_consensus.cc:740] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:44.412563  8400 rpcz_store.cc:276] 0709 13:02:42.846354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.076039 (+229685us) service_pool.cc:224] Handling call
0709 13:02:43.076326 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:43.890112 (+813786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.411808 (+521696us) spinlock_profiling.cc:243] Waited 813 ms on lock 0x7b540020fb54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:44.412034 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1869899648}
I20260709 13:02:44.422381 11089 consensus_queue.cc:260] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:44.423583 11089 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:44.426515  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.427071  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.404596 11420 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 13:02:44.429811  8328 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:44.431529  8298 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:44.415946 11382 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:44.435833  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:44.442062  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:44.431291 11279 raft_consensus.cc:3055] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:44.454658 11422 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:44.433626 11201 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.462169  8369 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:44.435382 11380 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:44.458824 11376 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:44.456110 11279 raft_consensus.cc:740] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:44.438333 11237 consensus_queue.cc:1048] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:02:44.466399 11417 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:44.434041 11149 consensus_queue.cc:260] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:44.459785 11062 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:44.491158 11081 consensus_queue.cc:1059] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:44.491840 11173 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:44.476912 11391 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:44.492060 11429 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:02:44.495479 11173 raft_consensus.cc:3055] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:44.466509 11422 consensus_queue.cc:1059] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:02:44.486172 11417 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.475373 11201 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:44.510476 11389 consensus_meta.cc:287] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.610s	user 0.009s	sys 0.002s
I20260709 13:02:44.439325 11362 consensus_queue.cc:237] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:44.515679 11429 consensus_queue.cc:1243] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer ff7a6fe533e94c1bae555b4cae427728 log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:02:44.490167 11341 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:02:44.514986 11173 raft_consensus.cc:740] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:44.523808 11417 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:44.525575 11149 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:44.538652  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 71800.5us
I20260709 13:02:44.532639 11376 raft_consensus.cc:697] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 9 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:44.540124 11173 consensus_queue.cc:260] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:44.541189 11173 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:44.546800 11201 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:44.550073 11294 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:44.552363 11294 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:44.553798 11062 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:44.556314  8974 leader_election.cc:400] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 7. Current term is 8.
I20260709 13:02:44.556653  8974 leader_election.cc:403] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:02:44.556880 11376 consensus_queue.cc:237] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:44.559978  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:44.560709  9007 raft_consensus.cc:2393] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 10: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
W20260709 13:02:44.561448  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 566600us
I20260709 13:02:44.564802  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:44.566018  8326 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:44.535611 11387 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:44.566983  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:44.569048  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:44.569768  9007 raft_consensus.cc:2393] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 6: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:44.570257  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:44.566720  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:44.574275  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:44.574936  8332 raft_consensus.cc:2393] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 7: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:44.575780  9003 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:02:44.574887  9006 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:02:44.577867  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:44.580137  8400 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:44.580204 11438 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:44.580790 11438 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:44.581573  8325 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:44.583184  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:44.583997 11421 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 7. Current term is 8.
I20260709 13:02:44.584396  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:44.597076  8327 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:44.597550  8406 raft_consensus.cc:2393] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 10: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:44.585992 11387 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:44.586222 11355 consensus_queue.cc:1048] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:44.588095  8405 raft_consensus.cc:1275] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:44.525758 11389 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:44.595005  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:44.597788  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:44.601599  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:44.602023  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.602170  8326 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:44.602514  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.596067 11286 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.612600  8972 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:44.617223  8332 pending_rounds.cc:85] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 4
I20260709 13:02:44.617700  8332 pending_rounds.cc:107] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:02:44.618000  8332 raft_consensus.cc:2889] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:44.618377  8332 pending_rounds.cc:107] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:02:44.618657  8332 raft_consensus.cc:2889] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:44.600621 11294 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.625317  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:44.609776 11277 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.625967  8405 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:02:44.631314  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:44.631704  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:44.632081  9001 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 8: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:44.600445 11276 consensus_queue.cc:1059] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:02:44.633345  9009 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:44.603433 11197 consensus_queue.cc:1059] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:02:44.635285  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:44.632586  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:44.636559  8406 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:02:44.632267  9007 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:44.620983 11387 raft_consensus.cc:697] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:44.642210  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:44.613703 11429 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:02:44.644454  9003 raft_consensus.cc:2393] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 10: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:44.645742  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:44.628032 11411 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:44.648193  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:44.649325  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:44.650019  8402 raft_consensus.cc:2368] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7.
I20260709 13:02:44.648814  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:44.650821  8328 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:02:44.647475  8372 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:44.653021 11003 raft_consensus.cc:3055] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:44.653337 11358 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:44.633527 11218 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:44.655833 11437 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:02:44.656898  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:44.633623 11277 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:44.628226 11294 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:44.662050  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:44.662470 11277 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:44.662889  8400 raft_consensus.cc:2410] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:02:44.663147 11294 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:44.633260 11263 consensus_queue.cc:1048] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:02:44.663687 11294 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.664111 11294 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:44.665347  8325 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 1881) took 1366 ms (client timeout 1693 ms). Trace:
I20260709 13:02:44.651216  8328 pending_rounds.cc:85] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 6
W20260709 13:02:44.665804  8325 rpcz_store.cc:269] 0709 13:02:43.299189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.299640 (+   451us) service_pool.cc:224] Handling call
0709 13:02:44.665202 (+1365562us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400064854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:44.665295 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2929406720}
I20260709 13:02:44.653416 11003 raft_consensus.cc:740] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:44.699208  8326 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:44.616412 11286 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:44.673943 11422 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:44.646075 11046 consensus_queue.cc:1059] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:02:44.641101 11391 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:44.663707 11042 raft_consensus.cc:3055] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:44.645684 11321 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:44.629263 11127 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:02:44.690794  8295 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:44.691293  8328 pending_rounds.cc:107] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:02:44.720293  8328 raft_consensus.cc:2889] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:44.691749  8406 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:44.692402  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:44.693529  8369 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:44.721511  8331 raft_consensus.cc:2410] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
W20260709 13:02:44.708015  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 57955.5us
I20260709 13:02:44.709004 11294 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:44.710449 11443 log.cc:927] Time spent T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:02:44.713649  8400 raft_consensus.cc:1275] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:44.716205  8325 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:44.723145  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2087) took 1078 ms. Trace:
I20260709 13:02:44.722913 11391 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:44.729604  8323 rpcz_store.cc:276] 0709 13:02:43.645885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.646001 (+   116us) service_pool.cc:224] Handling call
0709 13:02:43.646325 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:43.655890 (+  9565us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.722936 (+1067046us) spinlock_profiling.cc:243] Waited 8.69 ms on lock 0x7b54002c9a54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:44.723094 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19991424}
I20260709 13:02:44.719054 11042 raft_consensus.cc:740] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:44.723562 11358 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:44.721639  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:44.727263  8297 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:44.713980 11003 consensus_queue.cc:260] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:44.725235 11286 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:44.745443 11286 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:44.745812 11286 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.746148 11286 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:44.744501  8154 catalog_manager.cc:5697] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: config changed from index 5 to 2, term changed from 3 to 5, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1), f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from VOTER to NON_VOTER, now has a pending config: VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1), VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2), VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3), VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 5 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } health_report { overall_health: HEALTHY } } } pending_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:44.741539  8369 leader_election.cc:400] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7.
I20260709 13:02:44.728219 11173 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:44.744656 11003 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:44.748601  8369 leader_election.cc:403] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:44.727952 11281 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.731561 11042 consensus_queue.cc:260] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:44.750850  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:44.751231  8369 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:44.752708  8323 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:44.753916  8369 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:44.757225 11294 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:44.757479 11149 consensus_queue.cc:1059] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:02:44.760128 11418 log.cc:927] Time spent T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.493s	user 0.000s	sys 0.002s
I20260709 13:02:44.766886 11438 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:44.758224 10965 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:44.736299 11315 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:44.810006 11315 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:44.777994 11281 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:44.810815 11315 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:44.812429 11281 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:44.764755 11387 consensus_queue.cc:237] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:44.814291 11417 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:44.815461  8403 raft_consensus.cc:2368] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7.
I20260709 13:02:44.748834 11321 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:02:44.774925  8972 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:44.771276  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:44.795863  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.749203 11376 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:44.756866 11414 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7.
I20260709 13:02:44.809903 11042 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:44.778131 11108 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:02:44.743847 11445 consensus_queue.cc:1048] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.092s
I20260709 13:02:44.816658 11417 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:44.829648 11358 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:02:44.839691  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 76000.1us
I20260709 13:02:44.821856 11127 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:02:44.840261 11046 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:44.840801 11289 raft_consensus.cc:3055] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:44.847635 11389 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:44.851493  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2096) took 1193 ms. Trace:
I20260709 13:02:44.870411  8330 rpcz_store.cc:276] 0709 13:02:43.658196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.685476 (+ 27280us) service_pool.cc:224] Handling call
0709 13:02:44.851333 (+1165857us) spinlock_profiling.cc:243] Waited 660 ms on lock 0x7b5400364554. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:44.851429 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1518262656}
I20260709 13:02:44.871948  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:44.853868  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2103) took 1166 ms. Trace:
I20260709 13:02:44.873709  8329 rpcz_store.cc:276] 0709 13:02:43.687573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.978501 (+290928us) service_pool.cc:224] Handling call
0709 13:02:44.853694 (+875193us) spinlock_profiling.cc:243] Waited 63.1 ms on lock 0x7b54003c9454. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:44.853798 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":145029504}
I20260709 13:02:44.854087  8324 raft_consensus.cc:1240] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:44.802593 11294 raft_consensus.cc:697] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:44.819314 10965 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.875145 10965 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:44.875573 11294 consensus_queue.cc:237] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:44.819238 11173 raft_consensus.cc:697] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:44.863992  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:44.879091  8323 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:02:44.822743 11376 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:44.894618  8323 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:44.897244 11376 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:44.867887 11046 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:44.902544  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 175301us
I20260709 13:02:44.906217  8297 leader_election.cc:400] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7.
I20260709 13:02:44.906543  8297 leader_election.cc:403] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:02:44.870118 11289 raft_consensus.cc:740] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:44.877842 11023 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:44.905709 11046 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:44.913664  8297 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:44.917068  8297 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:44.931396  8297 leader_election.cc:400] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7.
I20260709 13:02:44.931851  8297 leader_election.cc:403] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:44.936110  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:44.936614  8406 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:02:44.937255  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:44.938069  8405 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:02:44.946098  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:44.916867 11023 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:44.950387 11450 log.cc:927] Time spent T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
W20260709 13:02:44.953527 11415 consensus_meta.cc:287] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.528s	user 0.008s	sys 0.004s
I20260709 13:02:44.953956 11415 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:44.954512  8329 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:44.955533 11286 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:44.956036 11358 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:44.958210  8326 raft_consensus.cc:1240] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:02:44.912791 11380 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7.
I20260709 13:02:44.962353  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:44.905805 11173 consensus_queue.cc:237] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
W20260709 13:02:44.974771  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.938685 11003 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:44.942898 11444 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7.
I20260709 13:02:44.980592  8297 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:45.000934 11391 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:45.000830 11453 consensus_queue.cc:1059] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:02:44.995383 11286 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:44.914996 11289 consensus_queue.cc:260] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:44.969177 11358 consensus_queue.cc:1059] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:02:44.994042 11197 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:45.016215 11197 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.016685 11197 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:45.020761 11289 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:44.973333 11376 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:45.032788 11203 raft_consensus.cc:3055] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:45.037806  8328 raft_consensus.cc:1275] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:45.042011 11358 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:45.043738 11315 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:45.038653 11203 raft_consensus.cc:740] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:02:45.046576  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.047176 11203 consensus_queue.cc:260] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:45.047959  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:45.048391 11203 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:45.050104  8328 raft_consensus.cc:1275] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:02:45.051193 11376 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:45.051543 11315 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:45.054207  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:45.054873  8326 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:45.055034  9005 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:45.057176  8372 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:45.060333  8369 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:45.062129  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:45.063544  8974 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:45.057595  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:45.064594  9001 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:45.077840  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:45.080874  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:45.083985 11457 log.cc:927] Time spent T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:02:45.064847 11417 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:45.075953 11438 consensus_queue.cc:1048] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.016s
W20260709 13:02:45.087411 11455 log.cc:927] Time spent T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.171s	user 0.002s	sys 0.000s
I20260709 13:02:45.071501 11391 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:45.095204 11288 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.095485 10965 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:45.095749 11288 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:45.096062 11334 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:45.080092  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:45.099328  9006 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:45.104410 11081 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:02:45.066535 11376 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:45.105796 11376 raft_consensus.cc:697] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:45.109912  8397 raft_consensus.cc:2393] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 7: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:02:45.110698  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.126696  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:45.127571  8256 raft_consensus.cc:2393] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 10: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:45.128691  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:45.129772  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:45.129864 10965 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:45.134469  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:45.134704  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:45.139326  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:45.146106  8251 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:45.146359  8254 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:45.147696  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.148947  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:45.149989  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:45.155185  9006 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:45.162330  8257 raft_consensus.cc:2368] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 7. Current term is 8.
I20260709 13:02:45.163525  8405 raft_consensus.cc:2393] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 6: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:02:45.164788  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 1.01466e+06us
I20260709 13:02:45.166900  8220 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:45.169094  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 61390.8us
I20260709 13:02:45.171205  8256 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:02:45.172214  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 44025.2us
I20260709 13:02:45.175767  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:45.187731  8249 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:45.187103 11343 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:45.189576  8223 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:45.193924  8254 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:45.193902  8220 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:45.195158  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:45.191888  8332 raft_consensus.cc:1275] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:45.196138  8330 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:45.173079 11288 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:45.190634 10926 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:45.225610  8220 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:45.243031  8220 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:45.250712  8405 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:45.224682 11243 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:45.252288 11464 log.cc:927] Time spent T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:02:45.252647 11243 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:45.261829 11363 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:45.262394 11363 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:45.264335 11363 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:45.266170  9005 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:45.266825 11046 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:45.268273  8406 raft_consensus.cc:1240] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:45.199615 11089 consensus_queue.cc:1059] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 13:02:45.271024 11197 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:45.289561  8251 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:02:45.292996  8249 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
I20260709 13:02:45.295040 11237 raft_consensus.cc:3055] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:45.295488 11237 raft_consensus.cc:740] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:45.300406  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1891) took 1092 ms. Trace:
I20260709 13:02:45.300863  8324 rpcz_store.cc:276] 0709 13:02:44.207915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.240420 (+ 32505us) service_pool.cc:224] Handling call
0709 13:02:44.853909 (+613489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.874935 (+ 21026us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.300194 (+425259us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b54003c9450. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:45.300343 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1409919360}
I20260709 13:02:45.318560  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:45.320735  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 77819.2us
I20260709 13:02:45.249624  8332 raft_consensus.cc:2955] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Committing config change with OpId 6.6: config changed from index 3 to 6, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:45.324775  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:45.260007 11275 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:45.199268 11376 consensus_queue.cc:237] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:45.318508 11130 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:45.336148  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:45.318266 11288 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:45.340256  8254 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:45.342338  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:45.341871  8251 raft_consensus.cc:2393] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 10: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:45.330395 10926 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:02:45.354081  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:45.355109 11466 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.271s	user 0.000s	sys 0.007s
I20260709 13:02:45.355792  8220 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:45.357831 11275 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:45.358486 11452 log.cc:927] Time spent T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.441s	user 0.002s	sys 0.000s
W20260709 13:02:45.358788  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.358560 11275 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:45.344520 11197 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:45.358883  8220 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:45.296571 11237 consensus_queue.cc:260] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:45.367027 11237 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:45.357653 11334 consensus_queue.cc:1048] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:45.360037 11275 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:45.360169  8220 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:45.360409 11130 raft_consensus.cc:697] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:02:45.357946 11469 log.cc:927] Time spent T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 13:02:45.379437  8255 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:45.341364 11057 raft_consensus.cc:3055] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:45.381635  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:45.386361  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:45.386598  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:45.386961  8324 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:45.404284  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:45.405079  8400 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:02:45.405845  8329 raft_consensus.cc:1275] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:02:45.358644 11465 log.cc:927] Time spent T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 13:02:45.408507  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:45.413110  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:45.416543 11130 consensus_queue.cc:237] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:45.418869  8255 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 8 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:45.419404  8332 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:02:45.419901  8332 pending_rounds.cc:85] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 3
I20260709 13:02:45.420192  8332 pending_rounds.cc:107] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:02:45.420498  8332 raft_consensus.cc:2889] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:45.422478  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:45.424947  8257 raft_consensus.cc:1240] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:45.426357  8220 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:45.427757  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:45.428328  8325 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
W20260709 13:02:45.428344  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 56383.4us
I20260709 13:02:45.374580 11275 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:45.384104 11057 raft_consensus.cc:740] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:45.357657 11357 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 13:02:45.441607  8324 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
W20260709 13:02:45.435952  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.449393  8254 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:45.459250  8400 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:02:45.429280 11243 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:45.434861 11161 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:45.436733 11275 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:45.474352  8328 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
W20260709 13:02:45.475301  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 69437us
I20260709 13:02:45.474578 11281 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.442305 11062 consensus_queue.cc:1048] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:02:45.455212 11434 consensus_queue.cc:1048] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.248s
I20260709 13:02:45.476434 11243 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.494262  8330 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:45.500062  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1727) took 1310 ms. Trace:
I20260709 13:02:45.500458  8253 rpcz_store.cc:276] 0709 13:02:44.189301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.189465 (+   164us) service_pool.cc:224] Handling call
0709 13:02:44.241950 (+ 52485us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:45.090263 (+848313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.090295 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:45.090839 (+   544us) log.cc:844] Serialized 52 byte log entry
0709 13:02:45.173501 (+ 82662us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:45.186763 (+ 13262us) log.cc:844] Serialized 14 byte log entry
0709 13:02:45.191647 (+  4884us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.191714 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:45.499871 (+308157us) raft_consensus.cc:1685] finished
0709 13:02:45.499890 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.499998 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":886338,"prepare.queue_time_us":13242,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":862768,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":22199,"thread_start_us":82294,"threads_started":1,"wal-append.queue_time_us":162818}
I20260709 13:02:45.505959 11428 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.506024 11471 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.506065 11419 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:45.480370 11476 consensus_queue.cc:1059] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 13:02:45.553267  8328 raft_consensus.cc:1240] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 7. Current term is 8. Ops: [7.7-7.7]
W20260709 13:02:45.554550 11479 log.cc:927] Time spent T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:02:45.514281 11486 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.556373 11486 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:45.559484 11486 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:45.560161  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:45.560312 11486 consensus_queue.cc:1059] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:45.560811  8328 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:45.487581 11281 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:45.561918  8972 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:45.563016 11486 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:45.563287 11281 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:45.563377 11486 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.563872 11486 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:45.564582 11203 consensus_meta.cc:287] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.516s	user 0.008s	sys 0.004s
I20260709 13:02:45.568807  8972 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:02:45.569974 11477 log.cc:927] Time spent T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:02:45.570382 11203 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:45.571290 11203 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:45.572209  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:45.573609  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:45.574793  8972 leader_election.cc:400] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 7. Current term is 8.
I20260709 13:02:45.575026  8329 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:02:45.575202  8972 leader_election.cc:403] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:02:45.517709 11411 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:45.514783 11243 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:45.548380 11419 raft_consensus.cc:697] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:45.527302 11428 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:45.600705 11486 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:45.602331 11428 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:45.602665 11243 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:45.607739 11417 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:45.584259 11203 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:45.585626 11411 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:45.544191 11471 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:45.609594  8252 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:45.497087 11057 consensus_queue.cc:260] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:45.622190  8223 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:45.611505 11473 log.cc:927] Time spent T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
W20260709 13:02:45.626258 11475 log.cc:927] Time spent T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:02:45.628364 11471 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:45.628923  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:45.629667  8248 raft_consensus.cc:2368] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7.
I20260709 13:02:45.630363 11411 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:45.630409 11072 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.631062 11411 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:45.631153 11072 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:45.608443 11417 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:45.632750 11387 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 7. Current term is 8.
I20260709 13:02:45.633630 11417 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:45.591835 11275 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:45.636438 11472 raft_consensus.cc:3055] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:45.636838 11472 raft_consensus.cc:740] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:45.636910 11275 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:45.638085 11483 log.cc:927] Time spent T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.062s	user 0.002s	sys 0.000s
I20260709 13:02:45.626202 11428 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:45.639346 11428 raft_consensus.cc:697] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 8 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:45.640244 11428 consensus_queue.cc:237] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:45.642498 11057 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:45.646554  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:45.647229  8328 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:45.648164  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:45.648686  8328 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:45.649963 11419 consensus_queue.cc:237] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:45.651433  8328 raft_consensus.cc:1275] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:45.650364  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:45.652781  8325 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:45.630505  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:45.653813  8252 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:45.656730  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:45.657577  8248 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:45.662544  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.622795 11243 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.602676 11486 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:45.663935  8369 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:45.658730 11358 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.014s	sys 0.029s
I20260709 13:02:45.668363  9006 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:45.655069 11072 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:45.658720 11434 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.672341 11363 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:45.662627 11210 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.667690 11472 consensus_queue.cc:260] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:45.697131  8295 leader_election.cc:400] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7.
W20260709 13:02:45.704917  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 128692us
I20260709 13:02:45.670992  8369 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:45.684790  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.685276  8220 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:45.737872  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:45.687927  8325 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
W20260709 13:02:45.751410  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 80476.4us
I20260709 13:02:45.700390  8295 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:45.705904  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:45.633374 11387 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
W20260709 13:02:45.754518  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 69303.4us
I20260709 13:02:45.783800 11471 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:45.784523  8220 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:45.785549 11471 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:45.735638  8297 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:45.670130 11358 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.014s	sys 0.029s
I20260709 13:02:45.786139 11471 raft_consensus.cc:697] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:45.799727  8328 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
W20260709 13:02:45.801558  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 66025.8us
W20260709 13:02:45.801774  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 101468us
I20260709 13:02:45.808058  8330 raft_consensus.cc:1240] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 7. Current term is 8. Ops: [6.4-7.5]
W20260709 13:02:45.808781  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:45.811722  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 105900us
I20260709 13:02:45.812619  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:45.815021  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:45.679684 11288 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:45.819029  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:45.675483 11434 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:45.681054 11210 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:45.809906  8220 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:45.820021  8328 raft_consensus.cc:2393] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 7: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:45.825253  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:45.691195 11237 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.827415  8972 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:45.778482 11364 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.755084 11363 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:45.830542  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:45.778831 11417 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.801175 11203 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.798727 11275 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:45.667204 11243 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:45.869474  8223 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:45.835778  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:45.960480  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 122049us
I20260709 13:02:45.807971 11281 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:45.889732  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.813944 11276 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:45.825852 11387 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:02:45.870962  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 43634.8us
W20260709 13:02:45.961299  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.023864  8972 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:45.970080  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.832770 11368 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.962841 11288 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:45.962016 11495 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:45.971663 11419 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:45.959874 11275 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.833024 11237 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:45.816442 11279 consensus_queue.cc:260] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:45.837127 11364 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:46.029948 11363 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:45.894825 11417 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:46.067291 11497 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:02:46.082732 11451 log.cc:927] Time spent T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 1.163s	user 0.000s	sys 0.005s
W20260709 13:02:46.082482 11492 log.cc:927] Time spent T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:02:46.063939 11376 consensus_queue.cc:1059] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:02:46.092062 11275 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:46.057421 11368 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:46.114321 11470 consensus_queue.cc:1059] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:02:46.112545 11495 raft_consensus.cc:697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:46.143971 11364 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:45.969040 11203 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:46.164681 11062 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.139429 11237 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:46.069259 11471 consensus_queue.cc:237] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:46.156056  8248 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
I20260709 13:02:46.196362 11288 raft_consensus.cc:697] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:02:46.206259  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 422181us
I20260709 13:02:46.230108  8369 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:46.208160  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 398232us
I20260709 13:02:46.222590  8295 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:46.116928 11363 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:46.233291  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:46.242595  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:46.244306  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:46.248759 11288 consensus_queue.cc:237] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:46.250319 11363 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:46.225384 11062 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:46.252231 11062 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:46.252763 11062 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:46.254137 11502 log.cc:927] Time spent T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:02:46.255033  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:46.255775  8251 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:02:46.256786  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:46.259514 11363 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.260082 11363 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:44.811092  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:46.248126  8248 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:46.187863 11364 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:02:46.261315  8153 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45456 (request call id 282) took 23737 ms (client timeout 15000 ms). Trace:
I20260709 13:02:46.269821  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:46.270555  8251 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:46.243445 11281 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.242604 11237 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.272755 11281 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:46.272924 11237 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:46.273070  8154 catalog_manager.cc:5697] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 2 to 5, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 5 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:46.275218 11237 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:46.274761  8154 catalog_manager.cc:5697] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 5 to 6, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 6 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:46.275807 11237 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:46.276321 11237 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:46.275463 11281 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:46.277668 11281 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:46.276579  8154 catalog_manager.cc:5697] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:46.278107 11281 raft_consensus.cc:697] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:46.286844 11517 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:46.287458 11517 raft_consensus.cc:697] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:46.287519 11281 consensus_queue.cc:237] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:46.287868 11504 log.cc:927] Time spent T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:02:46.287868 11521 consensus_queue.cc:1059] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:46.248011 11428 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:46.243005 11516 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:46.286558  8154 catalog_manager.cc:5697] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:46.253217 11062 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:46.309789  8154 catalog_manager.cc:5697] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 4 to 5, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 5 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:46.282209 11210 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:46.269460 11364 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:46.314056 11276 raft_consensus.cc:697] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:46.298334 11243 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:46.320412  8154 catalog_manager.cc:5697] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:02:46.288638 11517 consensus_queue.cc:237] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:46.299924 11428 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:46.309688 11516 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:46.293088 11495 consensus_queue.cc:237] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:46.344139  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2133) took 1252 ms. Trace:
I20260709 13:02:46.344784  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1793) took 1282 ms. Trace:
W20260709 13:02:46.420100  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.422991  8404 rpcz_store.cc:276] 0709 13:02:45.062128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.063136 (+  1008us) service_pool.cc:224] Handling call
0709 13:02:45.075740 (+ 12604us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.094759 (+ 19019us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.094796 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:45.094840 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:45.094876 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.344525 (+1249649us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b5400277254. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:46.344585 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.344714 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43196672}
I20260709 13:02:46.421715  8406 rpcz_store.cc:276] 0709 13:02:45.091719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.095415 (+  3696us) service_pool.cc:224] Handling call
0709 13:02:45.095721 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.268640 (+172919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.343802 (+1075162us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54003a1954. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:46.344056 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":396366592}
I20260709 13:02:46.321795 11387 raft_consensus.cc:697] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:46.439091  8154 catalog_manager.cc:5697] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 2 to 3, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 3 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:46.415450 11516 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:46.358949 11197 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.443439 11276 consensus_queue.cc:237] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:46.412899 11417 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:46.446898 11279 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:46.362094 11494 log.cc:927] Time spent T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.731s	user 0.000s	sys 0.002s
I20260709 13:02:46.447320 11062 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:46.408854 11503 log.cc:927] Time spent T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:02:46.465936 11428 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:46.265584  8153 rpcz_store.cc:269] 0709 13:02:22.523489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.580198 (+ 56709us) service_pool.cc:224] Handling call
0709 13:02:46.261095 (+23680897us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:37.345607 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:37.346207 (+   600us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:37.346259 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:02:37.346273 (+    14us) tablet.cc:662] Decoding operations
0709 13:02:37.463565 (+117292us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:37.472709 (+  9144us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:37.474923 (+  2214us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:02:37.592657 (+117734us) tablet.cc:701] Row locks acquired
0709 13:02:37.592713 (+    56us) write_op.cc:260] PREPARE: finished
0709 13:02:37.592905 (+   192us) write_op.cc:270] Start()
0709 13:02:37.593077 (+   172us) write_op.cc:276] Timestamp: P: 1783602157592839 usec, L: 0
0709 13:02:37.593104 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:02:37.686395 (+ 93291us) log.cc:844] Serialized 2818 byte log entry
0709 13:02:38.230222 (+543827us) op_driver.cc:464] REPLICATION: finished
0709 13:02:38.416155 (+185933us) write_op.cc:301] APPLY: starting
0709 13:02:38.416259 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:38.416535 (+   276us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:38.416686 (+   151us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:38.417821 (+  1135us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:38.418156 (+   335us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 13:02:38.418222 (+    66us) write_op.cc:312] APPLY: finished
0709 13:02:38.418797 (+   575us) log.cc:844] Serialized 59 byte log entry
0709 13:02:38.418969 (+   172us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:38.419078 (+   109us) write_op.cc:454] Released schema lock
0709 13:02:38.419118 (+    40us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:38.421567 (+  2449us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:38.421605 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3199003,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":185669,"apply.run_cpu_time_us":3115,"apply.run_wall_time_us":5343,"num_ops":7,"prepare.queue_time_us":156152,"prepare.run_cpu_time_us":27996,"prepare.run_wall_time_us":591748,"raft.queue_time_us":144445,"raft.run_cpu_time_us":33702,"raft.run_wall_time_us":164910,"replication_time_us":636961,"thread_start_us":468924,"threads_started":4,"wal-append.queue_time_us":364498,"wal-append.run_cpu_time_us":30513,"wal-append.run_wall_time_us":1383986}]]}
I20260709 13:02:46.466002  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1788) took 1027 ms. Trace:
W20260709 13:02:46.468089 11480 log.cc:927] Time spent T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.919s	user 0.003s	sys 0.002s
W20260709 13:02:46.352468 11488 log.cc:927] Time spent T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.723s	user 0.002s	sys 0.002s
I20260709 13:02:46.324774 11210 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:46.452905 11368 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:46.327498 11046 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:46.403014 11490 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:02:46.451447 11057 consensus_meta.cc:287] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.808s	user 0.011s	sys 0.001s
I20260709 13:02:46.428707 11127 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:46.455683 11203 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:46.441089  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.461725  8154 catalog_manager.cc:5697] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 6 to 7, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 7 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:46.467628  8257 rpcz_store.cc:276] 0709 13:02:45.438266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.438388 (+   122us) service_pool.cc:224] Handling call
0709 13:02:45.438693 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:45.459406 (+ 20713us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:45.480777 (+ 21371us) log.cc:844] Serialized 14 byte log entry
0709 13:02:45.544985 (+ 64208us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:45.545544 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 13:02:45.546123 (+   579us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:45.547541 (+  1418us) log.cc:844] Serialized 14 byte log entry
0709 13:02:45.549220 (+  1679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.549331 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:46.361310 (+811979us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:46.465792 (+104482us) raft_consensus.cc:1685] finished
0709 13:02:46.465815 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.465944 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1285,"prepare.queue_time_us":30903,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":1688,"raft.queue_time_us":8723,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":9239,"spinlock_wait_cycles":170502656,"thread_start_us":27670,"threads_started":1,"wal-append.queue_time_us":136542}
I20260709 13:02:46.529718  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:46.530988  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:46.324188  8251 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
W20260709 13:02:46.532902  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.532994  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:46.451102 11434 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:46.534868  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:46.538906  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:46.540627  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:46.541388  8251 raft_consensus.cc:2393] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 8: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:46.542569  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:46.543587  8257 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
I20260709 13:02:46.544698 11434 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:46.545177 11434 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:46.545727  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1784) took 1207 ms. Trace:
I20260709 13:02:46.330235 11391 raft_consensus.cc:2955] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Committing config change with OpId 6.6: config changed from index 3 to 6, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:46.546098  8256 rpcz_store.cc:276] 0709 13:02:45.338654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.339039 (+   385us) service_pool.cc:224] Handling call
0709 13:02:45.339360 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:45.384393 (+ 45033us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.384437 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:45.409834 (+ 25397us) log.cc:844] Serialized 26 byte log entry
0709 13:02:45.493047 (+ 83213us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:45.493089 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.493152 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:46.351666 (+858514us) spinlock_profiling.cc:243] Waited 22.9 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:46.537508 (+185842us) raft_consensus.cc:1685] finished
0709 13:02:46.537538 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.545666 (+  8128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45073,"raft.queue_time_us":64672,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":17705,"spinlock_wait_cycles":52595968,"thread_start_us":40262,"threads_started":1,"wal-append.queue_time_us":154228}
I20260709 13:02:46.544339  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:46.547338 11434 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:46.548105  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:46.548676  8257 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:46.548655  8256 raft_consensus.cc:2376] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9. Re-sending same reply.
I20260709 13:02:46.549703  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:46.539104 11127 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:46.550750  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:46.551517  8256 raft_consensus.cc:2410] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:02:46.551944 11127 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:46.552528  8248 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:46.557031  8248 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:46.557730  9006 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:46.560704  8248 raft_consensus.cc:1275] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:02:46.560808  8257 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:46.459993 11197 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:46.537796 11490 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:46.470639 11516 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:46.582754 11275 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:46.570600 11203 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:46.590118  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:46.591225  9009 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:02:46.593982  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.596648  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:46.591390  8154 catalog_manager.cc:5697] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:46.599195 11130 raft_consensus.cc:3055] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:46.586076 11387 consensus_queue.cc:237] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:46.606405  8249 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
W20260709 13:02:46.607631  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.611670  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.764052 11508 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.764848 11508 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:46.766197 11519 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:46.766903 11519 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:46.769970 11368 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.770620 11368 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:46.773118 11368 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:46.644073  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1803) took 1076 ms. Trace:
I20260709 13:02:46.775449  8248 raft_consensus.cc:1240] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:46.722348  8251 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:02:46.776750  8154 catalog_manager.cc:5697] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:46.774775  8257 rpcz_store.cc:276] 0709 13:02:45.567650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.544147 (+976497us) service_pool.cc:224] Handling call
0709 13:02:46.643991 (+ 99844us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:46.782107  8154 catalog_manager.cc:5697] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 1 to 3, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 3 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:46.784260  8154 catalog_manager.cc:5697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 6 to 7, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 7 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:46.773988 11519 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:46.755752 11130 raft_consensus.cc:740] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:46.803145  8251 raft_consensus.cc:1275] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:46.774995 11470 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:46.803960  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1801) took 1237 ms. Trace:
I20260709 13:02:46.804059  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1912) took 1815 ms. Trace:
I20260709 13:02:46.806046  8326 rpcz_store.cc:276] 0709 13:02:44.988734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.989182 (+   448us) service_pool.cc:224] Handling call
0709 13:02:44.989606 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.055442 (+ 65836us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.803832 (+1748390us) spinlock_profiling.cc:243] Waited 65 ms on lock 0x7b5400365954. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:46.804003 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":149513600}
I20260709 13:02:46.782052 11057 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:46.780251 11279 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:46.807509  8251 rpcz_store.cc:276] 0709 13:02:45.566984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.788288 (+1221304us) service_pool.cc:224] Handling call
0709 13:02:46.788914 (+   626us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:46.803665 (+ 14751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.803903 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:46.753517 11516 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:46.755968 11203 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:46.811990  8154 catalog_manager.cc:5697] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:46.814702  8251 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:46.815642  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1730) took 1232 ms. Trace:
I20260709 13:02:46.816171  8251 rpcz_store.cc:276] 0709 13:02:45.583685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.809397 (+1225712us) service_pool.cc:224] Handling call
0709 13:02:46.814495 (+  5098us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:46.815315 (+   820us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.815576 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:46.817442  8154 catalog_manager.cc:5697] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 3 to 5, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 5 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:46.837723  8974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.016s	sys 0.063s
W20260709 13:02:46.838625  8974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.016s	sys 0.063s
I20260709 13:02:46.843253  8251 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10. Ops: [8.8-9.9]
I20260709 13:02:46.844000  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1732) took 1259 ms. Trace:
I20260709 13:02:46.844339  8251 rpcz_store.cc:276] 0709 13:02:45.584914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.817150 (+1232236us) service_pool.cc:224] Handling call
0709 13:02:46.842936 (+ 25786us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:46.843721 (+   785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.843946 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:46.845014  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:46.853397  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:46.810076 11279 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:46.808173 11470 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:46.858294  8154 catalog_manager.cc:5697] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:46.810714 11057 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:46.862972  8298 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.017s	sys 0.044s
W20260709 13:02:46.867012  8298 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.017s	sys 0.044s
I20260709 13:02:46.868294  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:46.870751  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.873039  8251 raft_consensus.cc:1217] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Deduplicated request from leader. Original: 7.5->[9.6-9.6]   Dedup: 9.6->[]
I20260709 13:02:46.910156  8251 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 13:02:46.873907  8369 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.036s	sys 0.037s
W20260709 13:02:46.910833  8369 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.036s	sys 0.037s
I20260709 13:02:46.910979  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1737) took 1324 ms. Trace:
I20260709 13:02:46.911468  8251 rpcz_store.cc:276] 0709 13:02:45.586319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.845257 (+1258938us) service_pool.cc:224] Handling call
0709 13:02:46.852489 (+  7232us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:46.910573 (+ 58084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.910897 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:46.912339  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.916319  8255 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.019s	sys 0.041s
I20260709 13:02:46.915715  8154 catalog_manager.cc:5697] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 2 to 4, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 4 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:02:46.916643  8255 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.019s	sys 0.041s
I20260709 13:02:46.917287  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1790) took 1439 ms. Trace:
I20260709 13:02:46.917704  8255 rpcz_store.cc:276] 0709 13:02:45.477613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.477756 (+   143us) service_pool.cc:224] Handling call
0709 13:02:45.478142 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.478392 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:45.819959 (+341567us) log.cc:844] Serialized 14 byte log entry
0709 13:02:46.311532 (+491573us) log.cc:844] Serialized 14 byte log entry
0709 13:02:46.316023 (+  4491us) log.cc:844] Serialized 14 byte log entry
0709 13:02:46.916932 (+600909us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:46.916985 (+    53us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:46.917027 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.917088 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.917222 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18187,"prepare.queue_time_us":110265,"prepare.run_cpu_time_us":140,"prepare.run_wall_time_us":3801,"raft.queue_time_us":163559,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":197,"thread_start_us":1413605,"threads_started":3}
I20260709 13:02:46.932979  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 438) took 1311 ms. Trace:
I20260709 13:02:46.933382  8255 rpcz_store.cc:276] 0709 13:02:45.621825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.932159 (+1310334us) service_pool.cc:224] Handling call
0709 13:02:46.932486 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:46.932645 (+   159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:46.932682 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:46.932725 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:46.932758 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.932809 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.932921 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:46.837865 11130 consensus_queue.cc:260] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:46.911532  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 681348us
I20260709 13:02:46.913221  8251 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:46.956954  8972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.890s	user 0.026s	sys 0.049s
W20260709 13:02:46.957414  8972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.026s	sys 0.050s
W20260709 13:02:46.939750 11284 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.000s	sys 0.041s
I20260709 13:02:46.950757 11490 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:46.958060  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 934085us
W20260709 13:02:46.951864 11314 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.003s	sys 0.014s
I20260709 13:02:46.958791  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1738) took 1372 ms. Trace:
W20260709 13:02:46.952467  9000 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.001s	sys 0.017s
I20260709 13:02:46.940263 11197 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:46.958411 11334 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:46.897780 11497 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.008s	sys 0.042s
I20260709 13:02:46.904326  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:46.944700 11275 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:46.991657  8369 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:46.996071  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.041054  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 83068.7us
I20260709 13:02:46.953569 11130 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:46.996071  8323 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.000s	sys 0.019s
I20260709 13:02:46.984591 11428 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:47.017634  8251 rpcz_store.cc:276] 0709 13:02:45.586576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.912582 (+1326006us) service_pool.cc:224] Handling call
0709 13:02:46.912996 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:46.957355 (+ 44359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.958719 (+  1364us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:47.018385  9000 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.002s	sys 0.018s
I20260709 13:02:47.041870  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:47.046584  8323 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.801s	user 0.000s	sys 0.019s
I20260709 13:02:47.047863  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2123) took 1314 ms. Trace:
I20260709 13:02:47.048322  8323 rpcz_store.cc:276] 0709 13:02:45.733826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.740698 (+  6872us) service_pool.cc:224] Handling call
0709 13:02:45.753313 (+ 12615us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:45.753764 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.753801 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:45.781019 (+ 27218us) log.cc:844] Serialized 26 byte log entry
0709 13:02:46.207903 (+426884us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:47.047134 (+839231us) log.cc:844] Serialized 14 byte log entry
0709 13:02:47.047458 (+   324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.047552 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:47.047604 (+    52us) raft_consensus.cc:1685] finished
0709 13:02:47.047622 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.047799 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8935,"prepare.queue_time_us":132142,"raft.queue_time_us":26860,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":10231,"spinlock_wait_cycles":63799040,"thread_start_us":1049151,"threads_started":2,"wal-append.queue_time_us":487281}
I20260709 13:02:47.051359  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:47.001690 11284 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.002s	sys 0.042s
W20260709 13:02:47.018683 11314 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.005s	sys 0.016s
W20260709 13:02:47.021371 11281 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.006s	sys 0.026s
W20260709 13:02:47.040622 11497 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.854s	user 0.009s	sys 0.043s
I20260709 13:02:47.054881  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:47.057096  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:47.055234  8256 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.000s	sys 0.022s
W20260709 13:02:47.060864  8256 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.000s	sys 0.022s
W20260709 13:02:47.055154 11495 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.007s	sys 0.016s
I20260709 13:02:47.041916 11203 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:47.053624 11536 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:47.041239 11445 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.946s	user 0.001s	sys 0.020s
W20260709 13:02:47.058108 11281 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.006s	sys 0.026s
I20260709 13:02:47.053967 11470 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:47.056962 11279 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:47.063531 11485 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.053742 11534 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:02:47.063755 11288 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.014s	sys 0.027s
I20260709 13:02:47.077922  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:47.064466 11495 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.008s	sys 0.016s
I20260709 13:02:47.046672 11334 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:47.070947 11276 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.012s	sys 0.025s
I20260709 13:02:47.052088 11428 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:47.057411 11497 raft_consensus.cc:697] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 5 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:02:47.073329 11445 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.987s	user 0.001s	sys 0.020s
I20260709 13:02:47.083338 11534 raft_consensus.cc:697] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:47.072984 11203 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:47.078444 11434 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:02:47.088131 11459 log.cc:927] Time spent T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 2.036s	user 0.005s	sys 0.011s
I20260709 13:02:47.089041 11237 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:47.083777 11288 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.014s	sys 0.027s
I20260709 13:02:47.075564 11532 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:47.076205 11525 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:47.086297 11276 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.013s	sys 0.025s
I20260709 13:02:47.080314  9000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 708) took 1412 ms. Trace:
W20260709 13:02:47.100286  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.083503 11485 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:47.063155 11057 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:47.111792  9007 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.008s	sys 0.019s
I20260709 13:02:47.074308 11536 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:47.132416  9007 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.008s	sys 0.019s
W20260709 13:02:47.111485 11391 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.013s	sys 0.013s
W20260709 13:02:47.129283 11526 log.cc:927] Time spent T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 13:02:47.118026  9004 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 13:02:47.157964 11517 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.013s	sys 0.037s
I20260709 13:02:47.104355 11434 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:02:47.158645  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 89287.7us
W20260709 13:02:47.212082  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.166854 11391 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.013s	sys 0.014s
W20260709 13:02:47.157820 11471 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.026s	sys 0.025s
I20260709 13:02:47.112285 11203 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:47.144526 11525 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:47.149267 11497 consensus_queue.cc:237] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:47.144476 11532 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:47.506079  8974 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.3:37367 timed out after 1.792s (SENT) [suppressed 26 similar messages]
W20260709 13:02:47.433230  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 391356us
W20260709 13:02:47.549834  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.563977  8252 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1793) took 2015 ms (client timeout 1859 ms). Trace:
W20260709 13:02:47.564538  8252 rpcz_store.cc:269] 0709 13:02:45.528383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.573313 (+ 44930us) service_pool.cc:224] Handling call
0709 13:02:47.543710 (+1970397us) spinlock_profiling.cc:243] Waited 35.2 ms on lock 0x7b540026c854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:47.543835 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":81037184}
I20260709 13:02:47.565882  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:47.569955  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1743) took 1944 ms. Trace:
W20260709 13:02:47.571605  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.541039 11471 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.086s	user 0.026s	sys 0.037s
W20260709 13:02:47.296912  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.293658 11526 log.cc:927] Time spent T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:02:47.581722  9001 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
W20260709 13:02:47.231802 11459 log.cc:927] Time spent T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:02:47.231329 11434 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:47.230275 11517 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.014s	sys 0.038s
W20260709 13:02:47.538911  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.570453  8255 rpcz_store.cc:276] 0709 13:02:45.625348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.934478 (+1309130us) service_pool.cc:224] Handling call
0709 13:02:46.934825 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:46.935063 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:47.187356 (+252293us) log.cc:844] Serialized 14 byte log entry
0709 13:02:47.569532 (+382176us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:47.569597 (+    65us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:47.569657 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.569741 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.569889 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7329,"thread_start_us":606296,"threads_started":2}
W20260709 13:02:47.386384 11542 log.cc:927] Time spent T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
W20260709 13:02:47.300912 11537 log.cc:927] Time spent T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.217s	user 0.000s	sys 0.000s
W20260709 13:02:47.386590 11541 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 13:02:47.594369 11203 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:47.230790 11276 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.250300 11057 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.488847 11490 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:47.229106 11445 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:47.553741 11288 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.231292 11552 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:47.543291 11491 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:02:47.563807 11543 log.cc:927] Time spent T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 13:02:47.553149 11279 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:47.563093 11470 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:47.597415 11387 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:47.611410 11491 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:47.637275 11491 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:47.637387 11275 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:47.637926 11491 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:47.609226 11057 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:47.638437 11491 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:47.638845  8249 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1785) took 2270 ms (client timeout 2243 ms). Trace:
W20260709 13:02:47.639240  8249 rpcz_store.cc:269] 0709 13:02:45.367996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.368355 (+   359us) service_pool.cc:224] Handling call
0709 13:02:47.638649 (+2270294us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540009ecd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:47.638755 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2721006336}
I20260709 13:02:47.640507 11057 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:47.640789 11491 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:47.615991 11279 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:47.652132 11429 raft_consensus.cc:3055] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:47.652611 11429 raft_consensus.cc:740] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:47.653527 11429 consensus_queue.cc:260] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:47.654428 11279 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:47.654749 11429 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:47.617072 11470 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:47.658450 11470 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:47.654476 11532 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:47.610831 11490 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:47.670217 11490 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:47.631418  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1905) took 3015 ms. Trace:
I20260709 13:02:47.130113  9000 rpcz_store.cc:276] 0709 13:02:45.665464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.666067 (+   603us) service_pool.cc:224] Handling call
0709 13:02:45.680566 (+ 14499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.680795 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:46.224904 (+544109us) log.cc:844] Serialized 14 byte log entry
0709 13:02:47.059951 (+835047us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:47.062655 (+  2704us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:47.065689 (+  3034us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.070280 (+  4591us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.077981 (+  7701us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77684,"prepare.queue_time_us":188053,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":45,"raft.queue_time_us":8450,"spinlock_wait_cycles":5335296,"thread_start_us":1160238,"threads_started":2,"wal-append.queue_time_us":867646}
I20260709 13:02:47.629787 11485 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:47.632838 11489 log.cc:927] Time spent T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 1.999s	user 0.005s	sys 0.010s
I20260709 13:02:47.612202 11288 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:47.672748  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:47.609123 11445 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:47.673492  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2120) took 2260 ms. Trace:
I20260709 13:02:47.675040 11445 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:47.614596 11552 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:47.674031  8331 rpcz_store.cc:276] 0709 13:02:45.413051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.414146 (+  1095us) service_pool.cc:224] Handling call
0709 13:02:45.414498 (+   352us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:45.414937 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.414975 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:45.434172 (+ 19197us) log.cc:844] Serialized 52 byte log entry
0709 13:02:45.489776 (+ 55604us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:45.489853 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.489945 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:47.632253 (+2142308us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:47.673189 (+ 40936us) raft_consensus.cc:1685] finished
0709 13:02:47.673210 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.673414 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":113,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":6264,"spinlock_wait_cycles":41243008,"thread_start_us":23946,"threads_started":1,"wal-append.queue_time_us":169180}
I20260709 13:02:47.600903 11428 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:47.642860  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1800) took 2076 ms. Trace:
I20260709 13:02:47.679432  8248 rpcz_store.cc:276] 0709 13:02:45.566638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.561705 (+995067us) service_pool.cc:224] Handling call
0709 13:02:46.562017 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:46.775927 (+213910us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.642560 (+866633us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54002ee354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:47.642785 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":490192768}
W20260709 13:02:47.617298 11544 log.cc:927] Time spent T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.231s	user 0.000s	sys 0.000s
W20260709 13:02:47.645454 11520 log.cc:927] Time spent T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.695s	user 0.000s	sys 0.001s
I20260709 13:02:47.649150  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1792) took 2150 ms. Trace:
W20260709 13:02:47.651908  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.616614 11550 log.cc:927] Time spent T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.329s	user 0.000s	sys 0.000s
W20260709 13:02:47.662598  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 2.43565e+06us
I20260709 13:02:47.623294 11525 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:47.667255 11387 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:47.671195  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:47.641769 11472 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:47.673532  8251 raft_consensus.cc:2393] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 8: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:47.670758  8327 rpcz_store.cc:276] 0709 13:02:44.616340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.619500 (+  3160us) service_pool.cc:224] Handling call
0709 13:02:44.619911 (+   411us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:44.690243 (+ 70332us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:44.777094 (+ 86851us) log.cc:844] Serialized 14 byte log entry
0709 13:02:44.979531 (+202437us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:44.989097 (+  9566us) log.cc:844] Serialized 52 byte log entry
0709 13:02:44.989203 (+   106us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:44.989422 (+   219us) log.cc:844] Serialized 14 byte log entry
0709 13:02:44.994458 (+  5036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.084376 (+ 89918us) spinlock_profiling.cc:243] Waited 54.4 ms on lock 0x7b54003cb254. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:45.084426 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:47.062514 (+1978088us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:47.631140 (+568626us) raft_consensus.cc:1685] finished
0709 13:02:47.631162 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.631317 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4724,"prepare.queue_time_us":27792,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":1456,"raft.queue_time_us":20527,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":17157,"spinlock_wait_cycles":154500736,"thread_start_us":181275,"threads_started":2,"wal-append.queue_time_us":194152}
I20260709 13:02:47.678640  9005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 710) took 2012 ms. Trace:
W20260709 13:02:47.671691  8295 connection.cc:256] RPC call timeout handler was delayed by -1.2022775980000233s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1897s for a response
I20260709 13:02:47.604835 11276 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:47.371573 11534 consensus_queue.cc:237] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:47.599553 11536 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:47.921362  8295 connection.cc:256] RPC call timeout handler was delayed by -1.1293775980000191s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1978s for a response
I20260709 13:02:47.910918  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:47.913765  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 407665us
I20260709 13:02:47.863919  9007 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 5 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:47.921378  8371 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.010s	sys 0.015s
I20260709 13:02:47.924517  8298 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:47.926869  8371 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.010s	sys 0.015s
W20260709 13:02:47.928308  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.680982  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:47.922049  8256 rpcz_store.cc:276] 0709 13:02:45.499164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.552033 (+1052869us) service_pool.cc:224] Handling call
0709 13:02:46.552318 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:46.552666 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:46.552692 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:46.553127 (+   435us) log.cc:844] Serialized 26 byte log entry
0709 13:02:47.061364 (+508237us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:47.061418 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.061525 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:47.648867 (+587342us) raft_consensus.cc:1685] finished
0709 13:02:47.648896 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.649084 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5683,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":5809,"thread_start_us":507853,"threads_started":1,"wal-append.queue_time_us":1093422}
I20260709 13:02:47.923997  9005 rpcz_store.cc:276] 0709 13:02:45.666619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.666719 (+   100us) service_pool.cc:224] Handling call
0709 13:02:47.677812 (+2011093us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b5800175898. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:02:47.678057 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:47.678232 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:47.678264 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:47.678322 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:47.678368 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.678442 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.678573 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2550528}
I20260709 13:02:47.940984  8256 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:02:47.943317  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:47.943434  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:47.943912  8255 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
W20260709 13:02:47.780802 11130 consensus_meta.cc:287] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.730s	user 0.010s	sys 0.000s
W20260709 13:02:47.682644 11481 log.cc:927] Time spent T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 2.133s	user 0.008s	sys 0.000s
W20260709 13:02:47.946869  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 178230us
W20260709 13:02:47.948856  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.952817  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:47.681041 11364 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:47.958514  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.953871  8251 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:47.928855 11376 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:47.962574  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:47.964356  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:47.972658  8402 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:02:47.940848 11203 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:47.974267  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:47.977062  8249 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:47.978606  8402 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:02:47.946264 11237 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.946136 11497 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.981889  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:47.951884 11552 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:47.955299 11288 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:48.028048 11276 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:48.028699 11552 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:48.029104 11552 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.033620  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2129) took 2225 ms. Trace:
I20260709 13:02:48.034081  8330 rpcz_store.cc:276] 0709 13:02:45.807698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.983368 (+175670us) service_pool.cc:224] Handling call
0709 13:02:45.983785 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:45.984221 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.984258 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:46.033851 (+ 49593us) log.cc:844] Serialized 26 byte log entry
0709 13:02:46.423525 (+389674us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:46.423579 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.423668 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:47.644693 (+1221025us) spinlock_profiling.cc:243] Waited 2.59 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:48.008066 (+363373us) raft_consensus.cc:1685] finished
0709 13:02:48.008101 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:48.033535 (+ 25434us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":454,"raft.queue_time_us":653,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":39127,"spinlock_wait_cycles":6209664,"thread_start_us":354705,"threads_started":1,"wal-append.queue_time_us":824229}
W20260709 13:02:48.037950 11557 log.cc:927] Time spent T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 13:02:48.039801  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 494924us
I20260709 13:02:48.019200  8404 raft_consensus.cc:2393] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 9: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:48.020599  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:48.026415 11481 log.cc:927] Time spent T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:02:48.046815  9006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56948 (request call id 711) took 1511 ms. Trace:
I20260709 13:02:48.051587  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:48.048895  9006 rpcz_store.cc:276] 0709 13:02:46.530563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.530710 (+   147us) service_pool.cc:224] Handling call
0709 13:02:48.041668 (+1510958us) spinlock_profiling.cc:243] Waited 25.9 ms on lock 0x7b54004c2f54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:48.041853 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59575168}
I20260709 13:02:48.058189  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:47.976346 11434 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:48.029419 11552 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:48.040408 11276 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.983371 11203 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:47.977573 11376 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:48.017637 11237 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:48.104470 11429 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:48.109743  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:48.109601  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:48.114715  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:48.115927  8397 raft_consensus.cc:2393] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 5: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:48.117935  8249 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:48.114722  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:48.058164 11445 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.026721 11497 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:48.071398 11558 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:48.093912 11364 consensus_queue.cc:1048] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:02:48.134565  8251 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:02:48.200174  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 275694us
I20260709 13:02:48.207793  8405 raft_consensus.cc:1275] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:02:48.114136 11411 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.188512  8402 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:02:48.172610  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:48.217415  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:48.218469  8400 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:48.219079  8404 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:48.220347  8397 raft_consensus.cc:1275] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:48.238495  8404 raft_consensus.cc:1240] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 7. Current term is 8. Ops: []
W20260709 13:02:48.125463 11357 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.021s	user 0.017s	sys 0.058s
I20260709 13:02:48.260668  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:48.126461 11429 raft_consensus.cc:697] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:48.110651 11276 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:48.128643 11391 raft_consensus.cc:3055] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:48.265782  8401 raft_consensus.cc:1275] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:02:48.209959 11558 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:02:48.269726 11357 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.169s	user 0.025s	sys 0.077s
I20260709 13:02:48.266702 11387 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:48.244035 11411 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:48.143745 11445 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:48.482499  8296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.097s	sys 0.061s
I20260709 13:02:48.357767  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:48.832141  8296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.883s	user 0.098s	sys 0.132s
W20260709 13:02:48.350361 11549 log.cc:927] Time spent T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.706s	user 0.003s	sys 0.000s
I20260709 13:02:48.393905  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:48.834328 11411 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:48.834348  9009 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:48.593538 11421 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:48.349807 11578 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:48.838397 11237 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:48.839161 11237 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.839731 11237 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:48.718242 11334 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:48.487145 11429 consensus_queue.cc:237] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:48.305495 11567 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:02:48.851163  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 440) took 1806 ms. Trace:
I20260709 13:02:48.851621  8252 rpcz_store.cc:276] 0709 13:02:47.044704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.567155 (+522451us) service_pool.cc:224] Handling call
0709 13:02:47.567528 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:47.567948 (+   420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:47.567989 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:47.574891 (+  6902us) log.cc:844] Serialized 26 byte log entry
0709 13:02:47.902173 (+327282us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:47.945070 (+ 42897us) log.cc:844] Serialized 14 byte log entry
0709 13:02:48.321360 (+376290us) spinlock_profiling.cc:243] Waited 5.22 ms on lock 0x7b4800002dd8. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882d6e7aa1 00007f882d6deb37 00007f882d6de9ca 00007f882d7d626f 00007f882d7d5f37 00007f882d7ea55c 00007f882d7ea508 00007f882d7ea488 00007f882d7ea443 00007f882d7e96ec 00007f882c713a96
0709 13:02:48.348342 (+ 26982us) spinlock_profiling.cc:243] Waited 4.46 ms on lock 0x7b180004bb80. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882564bc0a 00007f882ed8b219 00007f88256d3a40 00007f88256ce4e4 00007f88256ce455 00007f88287b6060 00007f88287c8716 00007f88287d9287 00007f88287d9208
0709 13:02:48.348561 (+   219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.348653 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:48.850909 (+502256us) raft_consensus.cc:1685] finished
0709 13:02:48.850951 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:48.851096 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":131184,"prepare.queue_time_us":44059,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":404044,"raft.run_cpu_time_us":1567,"raft.run_wall_time_us":748021,"spinlock_wait_cycles":22265856,"thread_start_us":304989,"threads_started":1,"wal-append.queue_time_us":514720}
I20260709 13:02:48.304752 11391 raft_consensus.cc:740] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:48.855655 11391 consensus_queue.cc:260] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:48.857218 11391 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:48.392777 11558 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:48.447743 11482 log.cc:927] Time spent T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 2.926s	user 0.022s	sys 0.040s
W20260709 13:02:48.549799  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:48.862800  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.864493  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:48.864733  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:48.871404 11445 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:48.872910 11445 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.826767 11130 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.926362  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:48.929059  8404 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
W20260709 13:02:48.831756  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.876514  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 426) took 3755 ms. Trace:
I20260709 13:02:48.692324  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:48.891862  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:48.938047  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:48.446673  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1791) took 2959 ms. Trace:
I20260709 13:02:48.932886  8250 rpcz_store.cc:276] 0709 13:02:45.120638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.140044 (+ 19406us) service_pool.cc:224] Handling call
0709 13:02:45.140451 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:45.149923 (+  9472us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:45.149949 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:45.165474 (+ 15525us) log.cc:844] Serialized 26 byte log entry
0709 13:02:45.417812 (+252338us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:45.417859 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.417917 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:48.425845 (+3007928us) spinlock_profiling.cc:243] Waited 59.8 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:48.876151 (+450306us) raft_consensus.cc:1685] finished
0709 13:02:48.876180 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:48.876442 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6505,"raft.queue_time_us":8697,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":9507,"spinlock_wait_cycles":137629440,"thread_start_us":250253,"threads_started":1,"wal-append.queue_time_us":307455}
W20260709 13:02:48.343786  8325 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2125) took 2609 ms (client timeout 2162 ms). Trace:
W20260709 13:02:48.953421  8325 rpcz_store.cc:269] 0709 13:02:45.734488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.742589 (+  8101us) service_pool.cc:224] Handling call
0709 13:02:48.343513 (+2600924us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b5400364a54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:48.343682 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3014619008}
W20260709 13:02:48.414117  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 357348us
I20260709 13:02:48.334378  9008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 707) took 1756 ms. Trace:
W20260709 13:02:48.414492  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 206328us
I20260709 13:02:48.956926  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:48.942799  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:48.959971  8256 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.003s	sys 0.011s
W20260709 13:02:48.961350  8256 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.003s	sys 0.011s
W20260709 13:02:48.351614  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.962249  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 444) took 1255 ms. Trace:
I20260709 13:02:48.962806  8256 rpcz_store.cc:276] 0709 13:02:47.706754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.942133 (+235379us) service_pool.cc:224] Handling call
0709 13:02:47.942472 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:47.942718 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:48.414943 (+472225us) log.cc:844] Serialized 14 byte log entry
0709 13:02:48.961914 (+546971us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:48.961981 (+    67us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:48.962023 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.962086 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:48.962201 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36182,"prepare.queue_time_us":74486,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"thread_start_us":981824,"threads_started":2}
I20260709 13:02:48.967839  8401 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:48.955852  9008 rpcz_store.cc:276] 0709 13:02:46.578097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.578236 (+   139us) service_pool.cc:224] Handling call
0709 13:02:48.041977 (+1463741us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:48.042217 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:48.042255 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:48.042310 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:48.042354 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.042428 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:48.334172 (+291744us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54004c2f50. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:48.334303 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3364644608}
I20260709 13:02:48.976924  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:48.951189  8253 rpcz_store.cc:276] 0709 13:02:45.487395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.518493 (+ 31098us) service_pool.cc:224] Handling call
0709 13:02:45.545267 (+ 26774us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:45.628710 (+ 83443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.628739 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:45.629201 (+   462us) log.cc:844] Serialized 26 byte log entry
0709 13:02:45.832120 (+202919us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:45.832202 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.446221 (+2614019us) spinlock_profiling.cc:243] Waited 83.1 ms on lock 0x7b5400178254. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:48.446278 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:48.446324 (+    46us) raft_consensus.cc:1685] finished
0709 13:02:48.446336 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:48.446601 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":210,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":81,"spinlock_wait_cycles":191126784,"thread_start_us":193788,"threads_started":1,"wal-append.queue_time_us":610607,"wal-append.run_cpu_time_us":2697,"wal-append.run_wall_time_us":1246595}
I20260709 13:02:48.982801  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:48.983666  8401 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
W20260709 13:02:48.342695 11551 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.748s	user 0.001s	sys 0.001s
W20260709 13:02:48.993284  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.972891 11130 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:49.050808  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:49.051101  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:49.051822  9006 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
W20260709 13:02:48.341020 11545 log.cc:927] Time spent T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.778s	user 0.001s	sys 0.001s
I20260709 13:02:48.911108 11203 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:48.830348 11434 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:48.335619 11516 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:48.984817 11321 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.979799 11314 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.837826 11421 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:49.010367 11580 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:48.838670 11578 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:48.972123 11334 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:49.014197 11376 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:49.032078 11472 consensus_meta.cc:287] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 1.093s	user 0.008s	sys 0.000s
I20260709 13:02:49.019773 11276 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:48.980135 11445 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:49.022127 11579 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:49.033569  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.085683  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:49.086524  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.088222  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:49.065104 11363 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:48.320645 11508 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:49.059614  8295 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.739s	user 0.044s	sys 0.105s
I20260709 13:02:49.061059  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:49.101554  8295 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.045s	sys 0.105s
W20260709 13:02:49.064182  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:49.102415  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.068998  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.103253  8400 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:49.104789  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:49.105620  8330 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:49.107303  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:49.108891  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:49.109764  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:49.110423  9006 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:02:49.111208  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:49.111737  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:49.112505  8326 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
I20260709 13:02:49.111275  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:49.113787  8328 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:49.115885  8328 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:49.076668 11434 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.084149 11580 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:49.117604  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:49.119141 11580 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:49.109867 11276 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:49.120091 11276 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:49.122237 11276 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:49.089684 11429 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.123924 11578 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:49.127468  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:49.123709 11429 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:49.109784 11516 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.131695 11516 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:49.132231  8329 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260709 13:02:49.132654  8329 pending_rounds.cc:85] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 8
I20260709 13:02:49.132918  8329 pending_rounds.cc:107] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 13:02:49.133153  8329 raft_consensus.cc:2889] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:49.134128  8295 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:49.135160  8295 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:49.055258 11576 log.cc:927] Time spent T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.625s	user 0.001s	sys 0.001s
I20260709 13:02:49.077383  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:49.131060 11429 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:49.082196  9002 raft_consensus.cc:2410] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:02:49.135269  8298 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:49.070927  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.148161 11517 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.148844 11588 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:49.139703  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:49.084213 11357 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.160931  8295 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:49.074548  8406 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.003s	sys 0.012s
I20260709 13:02:49.146783  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.148659 11517 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:49.139778  8328 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:49.189364 11508 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.140019  9008 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:49.080564 11237 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:49.243376  8404 raft_consensus.cc:1275] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:49.150625  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:49.243642 11237 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:49.244177 11237 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:49.245536  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.246248  8400 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:49.145516 11434 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:49.269665  9004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 583) took 2426 ms. Trace:
I20260709 13:02:49.270298  9004 rpcz_store.cc:276] 0709 13:02:46.843690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.847605 (+  3915us) service_pool.cc:224] Handling call
0709 13:02:46.914319 (+ 66714us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:47.217198 (+302879us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.159489 (+1942291us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b54006de2d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:49.269602 (+110113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":418742656}
W20260709 13:02:49.271205 11500 log.cc:927] Time spent T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 3.034s	user 0.005s	sys 0.023s
I20260709 13:02:49.271880 11516 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:49.272473 11516 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.272686  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1789) took 3811 ms. Trace:
I20260709 13:02:49.273082  8254 rpcz_store.cc:276] 0709 13:02:45.461363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.475341 (+ 13978us) service_pool.cc:224] Handling call
0709 13:02:45.475749 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:45.479210 (+  3461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.479250 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:45.479740 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 13:02:45.628119 (+148379us) spinlock_profiling.cc:243] Waited 961 us on lock 0x7b1800385420. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e0d1 00007f8828779a0f
0709 13:02:45.832534 (+204415us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:46.041971 (+209437us) log.cc:844] Serialized 14 byte log entry
0709 13:02:46.045207 (+  3236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.045284 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:49.264489 (+3219205us) spinlock_profiling.cc:243] Waited 3.27 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:49.272502 (+  8013us) raft_consensus.cc:1685] finished
0709 13:02:49.272522 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:49.272635 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84892,"prepare.queue_time_us":22788,"prepare.run_cpu_time_us":6334,"prepare.run_wall_time_us":14474,"raft.queue_time_us":74311,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":12152,"spinlock_wait_cycles":9736832,"thread_start_us":204223,"threads_started":1,"wal-append.queue_time_us":642794}
W20260709 13:02:49.280728 11552 consensus_meta.cc:287] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.165s	user 0.009s	sys 0.000s
I20260709 13:02:49.283063 11130 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:49.283722 11130 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.284286 11130 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:49.286643 11445 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:49.287389 11445 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:49.090039 11579 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:49.289688 11579 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:49.290340 11566 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:02:49.292563 11576 log.cc:927] Time spent T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:02:49.133862 11203 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:49.293403 11203 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:49.269142 11434 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:49.295769 11203 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:49.158298  8330 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:49.068650 11321 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:49.168324  8406 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.004s	sys 0.012s
I20260709 13:02:49.084981 11497 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:49.149336 11588 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.218946 11578 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:49.219213 11421 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:49.083901 11334 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:49.073177 11314 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:49.219971  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 59148.1us
I20260709 13:02:49.221400  8401 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:02:49.225750 11357 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:49.226743  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:49.150815 11551 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:02:49.076324  8325 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:02:49.308522  8325 raft_consensus.cc:606] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:02:49.308928  8325 raft_consensus.cc:614] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:49.152026  8298 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:49.242666 11508 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:49.300297  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2135) took 1343 ms. Trace:
I20260709 13:02:49.314191  8406 rpcz_store.cc:276] 0709 13:02:47.956906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.957056 (+   150us) service_pool.cc:224] Handling call
0709 13:02:47.957412 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:47.957699 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:48.460926 (+503227us) log.cc:844] Serialized 14 byte log entry
0709 13:02:49.299896 (+838970us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:49.299961 (+    65us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:49.300017 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.300102 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:49.300234 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":248374,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":162,"raft.queue_time_us":928,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":1038,"thread_start_us":1329582,"threads_started":2}
I20260709 13:02:49.147111 11580 raft_consensus.cc:2764] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:02:49.272925 11516 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
W20260709 13:02:49.303359 10930 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.964s	user 0.008s	sys 0.040s
I20260709 13:02:49.303376  8295 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:02:49.322180  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 170185us
I20260709 13:02:49.341288  9003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 712) took 2800 ms. Trace:
I20260709 13:02:49.147296 11276 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:49.325459 11321 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:49.327684 11357 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:49.330897 11334 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:49.302745 11588 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:49.281230 11552 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:49.345335  9003 rpcz_store.cc:276] 0709 13:02:46.540967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.541146 (+   179us) service_pool.cc:224] Handling call
0709 13:02:46.541448 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:46.564645 (+ 23197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:46.564681 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:46.575806 (+ 11125us) log.cc:844] Serialized 26 byte log entry
0709 13:02:47.019062 (+443256us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:47.054217 (+ 35155us) log.cc:844] Serialized 14 byte log entry
0709 13:02:47.054508 (+   291us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.054596 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:48.342071 (+1287475us) spinlock_profiling.cc:243] Waited 19.2 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:49.340902 (+998831us) raft_consensus.cc:1685] finished
0709 13:02:49.341028 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:49.341213 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88001,"prepare.queue_time_us":33468,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":5456,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":8373,"spinlock_wait_cycles":44117888,"thread_start_us":432585,"threads_started":1,"wal-append.queue_time_us":1008321}
W20260709 13:02:49.348875  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 45623.8us
W20260709 13:02:49.344316 10930 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.006s	user 0.008s	sys 0.040s
W20260709 13:02:49.352092  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.356499  8295 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:49.338766 11314 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:49.337391 11593 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:49.353497 11357 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.367552  8324 raft_consensus.cc:2393] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 7: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:49.352605 11203 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:49.364763  8295 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:49.368145 11558 consensus_meta.cc:287] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.000s
I20260709 13:02:49.367017 10930 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:49.376014  8325 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:02:49.384301  8325 raft_consensus.cc:606] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:49.384915  8325 raft_consensus.cc:614] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:49.352998 11276 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:49.387729 11276 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:49.388399  8972 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:49.391096  8295 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:49.369614 11203 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.369876  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:49.389611  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:02:49.372898 11580 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:49.396745  8295 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:49.395151  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:49.404970  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.392697 11421 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:49.407030  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.368971 11357 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:49.405789  8406 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:02:49.404956  8295 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:49.410236  8401 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:49.413913  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.374109 11578 consensus_queue.cc:1059] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:02:49.393747 11203 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:49.417454  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.401827 11593 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:49.406162 11364 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:49.416755 11421 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.392813 11497 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:02:49.461112  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.390966 11558 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:49.422509 11516 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:49.424297 11508 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:49.396044 11382 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:02:49.397817 11580 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.402482 11376 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.445075  9001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 586) took 2588 ms. Trace:
I20260709 13:02:49.445914  9007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 585) took 2601 ms. Trace:
I20260709 13:02:49.478976  9007 rpcz_store.cc:276] 0709 13:02:46.844346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.228654 (+384308us) service_pool.cc:224] Handling call
0709 13:02:47.229763 (+  1109us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:47.923615 (+693852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.445667 (+1522052us) spinlock_profiling.cc:243] Waited 634 ms on lock 0x7b54006c07d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:49.445849 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1457438080}
I20260709 13:02:49.477188  9001 rpcz_store.cc:276] 0709 13:02:46.856240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.659871 (+803631us) service_pool.cc:224] Handling call
0709 13:02:47.660175 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:47.676829 (+ 16654us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:47.950951 (+274122us) log.cc:844] Serialized 14 byte log entry
0709 13:02:48.427346 (+476395us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:48.427423 (+    77us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:48.427474 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.444801 (+1017327us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b5400527bd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:49.444870 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:49.445008 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5707,"prepare.queue_time_us":364396,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":9470,"raft.queue_time_us":2300,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":12637,"spinlock_wait_cycles":37805312,"thread_start_us":691253,"threads_started":2,"wal-append.queue_time_us":1059933}
W20260709 13:02:49.480453 11583 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.428s	user 0.010s	sys 0.008s
I20260709 13:02:49.436199  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.457101  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.490000  8406 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
I20260709 13:02:49.490175  8404 raft_consensus.cc:2410] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:02:49.420245 11552 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:49.432921  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:49.480340 11580 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:49.420850 11237 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:49.405942 11363 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:49.392935 11276 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:02:49.465019  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:49.525377  9002 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56954 (request call id 584) took 2675 ms (client timeout 1906 ms). Trace:
I20260709 13:02:49.525789  8405 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
W20260709 13:02:49.526046  9002 rpcz_store.cc:269] 0709 13:02:46.844087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.844747 (+   660us) service_pool.cc:224] Handling call
0709 13:02:49.518005 (+2673258us) spinlock_profiling.cc:243] Waited 2.23 s on lock 0x7b54005695d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:49.519085 (+  1080us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5133256576}
I20260709 13:02:49.526784  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.514381 11516 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:49.527503 11516 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:49.527937  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.528605  8405 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
I20260709 13:02:49.467410  8324 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:02:49.529573  8324 pending_rounds.cc:85] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 8
I20260709 13:02:49.521366 11533 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:49.530068  8324 pending_rounds.cc:107] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:02:49.530328 11533 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:49.530422  8324 raft_consensus.cc:2889] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:49.530743 11533 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:49.468160  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.532307  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.406615 11362 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:49.533406  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.533677 11362 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:49.534351  8403 raft_consensus.cc:2393] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 8: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:49.535638 11362 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:49.449893 11546 log.cc:927] Time spent T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 1.855s	user 0.010s	sys 0.017s
I20260709 13:02:49.470700  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:49.471124  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:49.472779  8326 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:02:49.536567  8330 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:49.425825 11130 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:49.438779 11445 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:49.428263 11593 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:49.460695 11421 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:49.476112  8325 raft_consensus.cc:3055] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:49.491598  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.539556  8325 raft_consensus.cc:740] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:49.539783  8406 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:49.540194  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.540747  8404 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
I20260709 13:02:49.540948  8406 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:02:49.462028  8402 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:02:49.519456  8398 raft_consensus.cc:3055] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:49.463910 11588 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:49.521356  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.529520 11516 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:49.406297 11434 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:49.542865  8399 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:02:49.529366  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:49.514814  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 81987.9us
I20260709 13:02:49.544664  8972 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:49.543941  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 190692us
I20260709 13:02:49.542018  8398 raft_consensus.cc:740] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:49.483942 11376 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:49.543915  8325 consensus_queue.cc:260] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:49.547731  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.548484  8404 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
W20260709 13:02:49.549723  8369 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.014s	sys 0.023s
W20260709 13:02:49.550122  8369 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.014s	sys 0.023s
W20260709 13:02:49.550524  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 595101us
I20260709 13:02:49.551921  8328 raft_consensus.cc:2393] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 8: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:49.539124 11445 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:49.464177 11364 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:49.553722  8972 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:49.558990  8325 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:49.560204  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:49.560755  9002 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:49.560629  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:49.561702  9001 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 8: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:49.542426 11593 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:49.563256  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:49.565049  8401 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:02:49.566248  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:49.566984  8405 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:49.551384  8398 consensus_queue.cc:260] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:49.549451 11497 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:49.537150 11363 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:49.557641 11445 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:49.563710 11513 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:49.568554 11530 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
W20260709 13:02:49.553999 11237 consensus_queue.cc:1175] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f865c4d761f5446eb12b3a029234772e" responder_term: 5 status { last_received { term: 5 index: 8 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 4. Current term is 5. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:49.546185 11434 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:49.592576  9002 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:02:49.561365 11490 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:02:49.576921 11497 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:49.594308  8298 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:49.594641  8372 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.015s	sys 0.006s
W20260709 13:02:49.598393  8372 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.015s	sys 0.006s
W20260709 13:02:49.598944  8297 connection.cc:256] RPC call timeout handler was delayed by -2.3203344830000674s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1897s for a response
W20260709 13:02:49.599993  8297 connection.cc:256] RPC call timeout handler was delayed by -1.563468553000007s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1906s for a response
I20260709 13:02:49.558106 11382 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:02:49.602066  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:49.602113  8330 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:02:49.602762  8326 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:49.604395  8326 raft_consensus.cc:1275] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:02:49.606859  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:49.607424  8329 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:49.607772  8325 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:02:49.568311 11486 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:02:49.610082  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:49.611310  8326 raft_consensus.cc:1275] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:02:49.616478  8329 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:49.624040  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.628649  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:49.624907  8404 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:02:49.629456  8330 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:02:49.602608 11263 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:49.594331 11513 raft_consensus.cc:697] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:49.603291 11276 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.603466 11497 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:49.603631 11357 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:49.608384 11552 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:02:49.639293  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 484895us
I20260709 13:02:49.635032  8372 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:49.650730  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:49.651518  8329 raft_consensus.cc:2410] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:02:49.651906  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:49.652593  8325 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:49.613668 11589 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:02:49.640915 11585 log.cc:927] Time spent T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.730s	user 0.009s	sys 0.004s
I20260709 13:02:49.636268 11558 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:49.654927  8325 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:02:49.659873  8222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.003s	sys 0.018s
I20260709 13:02:49.625365 11391 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:02:49.662793  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:49.608642 11595 consensus_queue.cc:1059] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
W20260709 13:02:49.666183  8222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.003s	sys 0.018s
I20260709 13:02:49.666806  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:49.672610  8325 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:49.672689  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:49.673560  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:49.674152  8325 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:49.664861  8223 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:49.674783  8404 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:49.669782  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:49.676252  8327 raft_consensus.cc:2410] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:02:49.677596  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:49.652663 11552 raft_consensus.cc:697] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 7 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:49.679328  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:49.678707  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:49.681316 11600 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:49.685679  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:49.677062 11334 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.688184  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:49.688756  8325 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:49.689517  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:49.683076  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:49.684639  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.643288 11276 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:49.686245  8327 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:49.686471  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.687253  8254 raft_consensus.cc:1275] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:49.687144  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.689963  8402 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:02:49.702078  8974 leader_election.cc:336] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Timed out: RequestConsensusVote RPC to 127.6.255.3:37367 timed out after 1.557s (SENT)
I20260709 13:02:49.702713  8974 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:49.675662 11314 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.705132  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:49.705696  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:49.707402  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.708352  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.686858  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:49.706280  8256 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:49.718196  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:49.729467  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.722812 11263 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:02:49.724071 11579 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:02:49.723888  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:49.740706  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:49.725052  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:49.741695  9007 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:49.725423  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:49.742645  8255 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:49.696661 11513 consensus_queue.cc:237] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:49.684015 11357 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:49.745754  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.745985 11434 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:49.746141 11357 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.746417  8406 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:49.746631 11357 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:49.747373  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.748018 11552 consensus_queue.cc:237] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:49.722445  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:49.727537 11314 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:49.752259 11314 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:49.730038  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:49.686043 11600 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:49.710453 11530 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:49.740504  8974 leader_election.cc:336] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Timed out: RequestConsensusVote RPC to 127.6.255.3:37367 timed out after 1.625s (SENT)
I20260709 13:02:49.683290 11536 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:49.664052 11589 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:49.745272  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:49.757822  8329 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:49.759177  8327 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:02:49.705582 11334 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:49.732594  8250 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
W20260709 13:02:49.695221  8369 connection.cc:256] RPC call timeout handler was delayed by -3.818970089000004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1764s for a response
I20260709 13:02:50.385289  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:50.225365  8405 consensus_meta.cc:287] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.647s	user 0.009s	sys 0.000s
W20260709 13:02:50.602422  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 861786us
I20260709 13:02:50.602669  8404 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:50.602905  8401 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:02:50.603018  8405 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:49.898701  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:50.431824  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:50.456735  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:50.352802  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:50.604796  8328 raft_consensus.cc:2393] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 8: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
W20260709 13:02:49.889693  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 156884us
I20260709 13:02:50.604974  8403 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:50.611749  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:50.456851  8220 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.783s	user 0.013s	sys 0.030s
W20260709 13:02:50.612433  8220 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.938s	user 0.015s	sys 0.031s
W20260709 13:02:50.612883  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 943380us
I20260709 13:02:49.845686  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:50.614367  8249 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
W20260709 13:02:49.886777  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 251765us
I20260709 13:02:49.776700  9007 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
W20260709 13:02:50.221712  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.054520  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:50.617882  8332 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:50.456772  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:50.619536  8330 raft_consensus.cc:2410] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:02:49.960736  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:50.620077  8331 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:50.612077  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:49.734378 11588 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:50.600713  8256 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.007s	sys 0.007s
I20260709 13:02:50.590330  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:50.385655  8398 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:49.936463  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:50.518049  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:50.603816  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 430) took 1038 ms. Trace:
I20260709 13:02:49.732456 11263 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:50.573083  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:50.604067  8974 leader_election.cc:336] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Timed out: RequestConsensusVote RPC to 127.6.255.3:37367 timed out after 1.792s (SENT)
I20260709 13:02:49.740218 11376 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:50.621275  8328 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
W20260709 13:02:50.134845 11533 consensus_meta.cc:287] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.604s	user 0.010s	sys 0.000s
I20260709 13:02:50.622087  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:50.665915  8327 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 5.
I20260709 13:02:50.666455  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2144) took 1007 ms. Trace:
I20260709 13:02:50.666610  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:50.667050  8325 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:50.667304  8249 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
I20260709 13:02:50.666826  8327 rpcz_store.cc:276] 0709 13:02:49.659119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.621791 (+962672us) service_pool.cc:224] Handling call
0709 13:02:50.666402 (+ 44611us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:49.765478  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:50.623827  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:50.615911  8372 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:50.624887  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:50.404667 11608 log.cc:927] Time spent T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc: Append to log took a long time: real 0.253s	user 0.000s	sys 0.002s
W20260709 13:02:50.659382  8256 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.008s	sys 0.012s
I20260709 13:02:50.659950  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:50.689561  8328 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:02:50.690347  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2152) took 1009 ms. Trace:
I20260709 13:02:49.838749  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:50.660167  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:50.660452  8326 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:50.656651  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:50.692731  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:50.693481  8403 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:50.694350  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:50.695043 11611 log.cc:927] Time spent T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.545s	user 0.003s	sys 0.011s
I20260709 13:02:50.695385  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:50.696470  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:50.697238  8403 raft_consensus.cc:2410] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 11 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:02:50.697743  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:50.698145 11616 log.cc:927] Time spent T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.186s	user 0.000s	sys 0.000s
I20260709 13:02:50.698360  8400 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:49.960146 10930 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:50.623265 11357 consensus_meta.cc:287] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.876s	user 0.015s	sys 0.003s
I20260709 13:02:50.547842 11599 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:50.701175 11357 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:50.701916 11599 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:50.702986 11334 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:50.703409 11357 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:50.704883 11599 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:50.621680 11391 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:50.661356  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 57314.2us
I20260709 13:02:50.661034  8323 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:02:50.656596 11487 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:02:50.715015  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 99153.4us
I20260709 13:02:50.660642  8405 rpcz_store.cc:276] 0709 13:02:49.565711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.565973 (+   262us) service_pool.cc:224] Handling call
0709 13:02:50.603759 (+1037786us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:50.626808  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:50.466691 11580 consensus_meta.cc:287] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.935s	user 0.011s	sys 0.000s
I20260709 13:02:49.974123 11593 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:50.112414 11421 consensus_meta.cc:287] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.566s	user 0.009s	sys 0.000s
W20260709 13:02:50.099997 11605 log.cc:927] Time spent T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:02:50.052713 11602 log.cc:927] Time spent T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
W20260709 13:02:50.540125 11203 consensus_meta.cc:287] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.077s	user 0.011s	sys 0.001s
I20260709 13:02:50.744673 11421 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:50.745034 11203 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:50.622928  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:50.460376 11276 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:50.668663  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:50.750028  8327 raft_consensus.cc:2393] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 9: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:50.674628  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:50.750617  8399 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:50.680001  8331 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:50.751029  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2138) took 1093 ms. Trace:
I20260709 13:02:50.751564  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 1939) took 1054 ms. Trace:
I20260709 13:02:50.752013  8331 rpcz_store.cc:276] 0709 13:02:49.697235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.960438 (+263203us) service_pool.cc:224] Handling call
0709 13:02:50.751506 (+791068us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:50.751623  8323 rpcz_store.cc:276] 0709 13:02:49.657405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.714758 (+1057353us) service_pool.cc:224] Handling call
0709 13:02:50.750101 (+ 35343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:50.750507 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:50.750551 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:50.750609 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:50.750674 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.750759 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.750948 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:50.148757 11363 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:49.764961 11600 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:50.399906 11607 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.001s	sys 0.016s
I20260709 13:02:50.686091  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:50.767513  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:50.768113  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2141) took 1109 ms. Trace:
I20260709 13:02:50.628261 11517 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:50.012189 11364 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:50.016120 11508 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:50.705837 11589 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:50.678304  8972 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:50.780741  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 102532us
W20260709 13:02:50.706442  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 100713us
I20260709 13:02:50.690784  8328 rpcz_store.cc:276] 0709 13:02:49.680786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.659680 (+978894us) service_pool.cc:224] Handling call
0709 13:02:50.690269 (+ 30589us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:50.710219  8369 connection.cc:256] RPC call timeout handler was delayed by -3.3878700889999891s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2243s for a response
W20260709 13:02:50.783560  8369 connection.cc:256] RPC call timeout handler was delayed by -3.3318433819999882s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1859s for a response
I20260709 13:02:50.718065  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:50.784236  8328 raft_consensus.cc:1275] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:02:50.784818  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:50.717453  9001 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:50.718098  8298 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:50.720678  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2142) took 1062 ms. Trace:
I20260709 13:02:50.721791  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:50.711246 11497 consensus_meta.cc:287] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 1.060s	user 0.012s	sys 0.000s
I20260709 13:02:50.722839  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:50.787765  8251 raft_consensus.cc:2393] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 8: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:50.723318  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:50.788386  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1815) took 1022 ms. Trace:
I20260709 13:02:50.726768  8332 raft_consensus.cc:1275] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:50.784832  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2150) took 1108 ms. Trace:
I20260709 13:02:50.789167  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2148) took 1125 ms. Trace:
I20260709 13:02:50.789113  8328 rpcz_store.cc:276] 0709 13:02:49.676331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.783641 (+1107310us) service_pool.cc:224] Handling call
0709 13:02:50.783927 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:50.784602 (+   675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.784781 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:50.789599  8332 rpcz_store.cc:276] 0709 13:02:49.664077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.726114 (+1062037us) service_pool.cc:224] Handling call
0709 13:02:50.726392 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:50.788933 (+ 62541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.789116 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:50.790535  8328 raft_consensus.cc:1275] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:50.790683  8332 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:50.791090  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2151) took 1113 ms. Trace:
I20260709 13:02:50.791229  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1937) took 1110 ms. Trace:
I20260709 13:02:50.791585  8332 rpcz_store.cc:276] 0709 13:02:49.680410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.790285 (+1109875us) service_pool.cc:224] Handling call
0709 13:02:50.790511 (+   226us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:50.791022 (+   511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.791186 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:50.791419  8328 rpcz_store.cc:276] 0709 13:02:49.677577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.789982 (+1112405us) service_pool.cc:224] Handling call
0709 13:02:50.790268 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:50.790878 (+   610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.791035 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:50.622531  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:50.661705  8397 raft_consensus.cc:2376] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8. Re-sending same reply.
I20260709 13:02:50.730095  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:50.793080  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 448) took 1048 ms. Trace:
I20260709 13:02:50.793354  8253 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:50.793537  8397 rpcz_store.cc:276] 0709 13:02:49.744498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.517695 (+773197us) service_pool.cc:224] Handling call
0709 13:02:50.793027 (+275332us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:50.712895 11294 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:50.731858 11445 consensus_meta.cc:287] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.144s	user 0.010s	sys 0.002s
I20260709 13:02:50.739092  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:50.798594  8250 raft_consensus.cc:2410] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:02:50.740644  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:50.708050 11533 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:50.621618 11536 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:50.805876 11625 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:50.807742 11536 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:50.755936  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:50.761822  8326 raft_consensus.cc:1240] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 13:02:50.763456  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:50.809644 11362 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.810418  8249 raft_consensus.cc:2393] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 10: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:50.810212 11362 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:50.763871  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:50.764122  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:50.811437  8257 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:02:50.765170  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2156) took 1036 ms. Trace:
I20260709 13:02:50.812464  8327 rpcz_store.cc:276] 0709 13:02:49.729056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.668430 (+939374us) service_pool.cc:224] Handling call
0709 13:02:50.765091 (+ 96661us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:50.812783  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:50.762007  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2147) took 1102 ms. Trace:
I20260709 13:02:50.768507  8330 rpcz_store.cc:276] 0709 13:02:49.658286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.625342 (+967056us) service_pool.cc:224] Handling call
0709 13:02:50.768040 (+142698us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:50.814527  8329 rpcz_store.cc:276] 0709 13:02:49.659954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.622654 (+962700us) service_pool.cc:224] Handling call
0709 13:02:50.761941 (+139287us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:50.815703  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1942) took 1050 ms. Trace:
I20260709 13:02:50.816212  8330 rpcz_store.cc:276] 0709 13:02:49.765334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.814794 (+1049460us) service_pool.cc:224] Handling call
0709 13:02:50.815104 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:50.815294 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:50.815323 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:50.815366 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:50.815404 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.815480 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.815615 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:50.726264 11614 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:50.768225  9003 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:50.773623  8295 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:50.777976  8323 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:02:50.826658  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2137) took 1169 ms. Trace:
I20260709 13:02:50.831192  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 392) took 1159 ms. Trace:
I20260709 13:02:50.827236  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:50.831638  8323 rpcz_store.cc:276] 0709 13:02:49.671903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.762361 (+1090458us) service_pool.cc:224] Handling call
0709 13:02:50.777650 (+ 15289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:50.830952 (+ 53302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.831142 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:50.832108  8257 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:50.831274 11614 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:50.786391  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 68351.8us
I20260709 13:02:50.786322  8325 rpcz_store.cc:276] 0709 13:02:49.658582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.689292 (+ 30710us) service_pool.cc:224] Handling call
0709 13:02:50.720516 (+1031224us) spinlock_profiling.cc:243] Waited 977 ms on lock 0x7b540037cb54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:50.720619 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2246560640}
I20260709 13:02:50.794950  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:50.840270  9001 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 9: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:50.788719  8254 rpcz_store.cc:276] 0709 13:02:49.766135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.766280 (+   145us) service_pool.cc:224] Handling call
0709 13:02:50.723182 (+956902us) spinlock_profiling.cc:243] Waited 8.61 ms on lock 0x7b5800002a98. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:02:50.788331 (+ 65149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19791616}
I20260709 13:02:50.785616  8248 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:50.800284  8222 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:50.800370  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:50.722546 11487 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:50.718640 11391 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:50.728116 11580 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:50.803758  8369 connection.cc:256] RPC call timeout handler was delayed by -3.1901852689999828s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18500000000000003s for a response
I20260709 13:02:50.799150  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 446) took 1055 ms. Trace:
I20260709 13:02:50.798112 11357 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:02:50.776556 11203 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:50.819685  8327 raft_consensus.cc:1275] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:02:50.833417  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 59893us
I20260709 13:02:50.846304  8257 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:02:50.857831  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:50.858911  8254 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 9: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:50.859455  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1940) took 1151 ms. Trace:
I20260709 13:02:50.859573  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:50.859872  8327 rpcz_store.cc:276] 0709 13:02:49.708326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.813634 (+1105308us) service_pool.cc:224] Handling call
0709 13:02:50.814011 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:50.859131 (+ 45120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.859377 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:50.844022  8222 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:50.833630  9008 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:50.830677  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:50.747215 11421 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:50.797741 10930 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:50.825966  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1943) took 1040 ms. Trace:
I20260709 13:02:50.882200  8329 rpcz_store.cc:276] 0709 13:02:49.785864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.825192 (+1039328us) service_pool.cc:224] Handling call
0709 13:02:50.825504 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:50.825674 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:50.825698 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:50.825735 (+    37us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:50.825767 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.825824 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.825918 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:50.857221  8369 connection.cc:256] RPC call timeout handler was delayed by -2.7068163659999982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20510000000000003s for a response
W20260709 13:02:50.883737  8369 connection.cc:256] RPC call timeout handler was delayed by -2.6580412700000124s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.252s for a response
W20260709 13:02:50.844734 11613 log.cc:927] Time spent T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.323s	user 0.001s	sys 0.000s
I20260709 13:02:50.837236 11382 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:50.886827  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:50.887421  8255 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:50.831382  8326 rpcz_store.cc:276] 0709 13:02:49.657101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.714030 (+1056929us) service_pool.cc:224] Handling call
0709 13:02:50.755445 (+ 41415us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:50.809624 (+ 54179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.826434 (+ 16810us) spinlock_profiling.cc:243] Waited 41 ms on lock 0x7b5400364050. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:50.826601 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":94256896}
I20260709 13:02:50.838577  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:50.843202  8372 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:50.890232  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 47091.6us
I20260709 13:02:50.857070  8401 rpcz_store.cc:276] 0709 13:02:49.744071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.622274 (+878203us) service_pool.cc:224] Handling call
0709 13:02:50.799073 (+176799us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:50.862409  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:50.890771  8327 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:50.864754  8295 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:50.868083  8222 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:50.870410  9000 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:50.872815  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:50.891880  8369 connection.cc:256] RPC call timeout handler was delayed by -2.5423974519999888s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15780000000000002s for a response
I20260709 13:02:50.873749  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:50.892294  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:50.893013  8254 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:02:50.874342  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:50.892310  8369 connection.cc:256] RPC call timeout handler was delayed by -2.3866974520000213s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
I20260709 13:02:50.894275  8251 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
W20260709 13:02:50.894743  8369 connection.cc:256] RPC call timeout handler was delayed by -2.3272974520000389s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1817s for a response
W20260709 13:02:50.895110  8369 connection.cc:256] RPC call timeout handler was delayed by -2.000333535999971s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2781s for a response
W20260709 13:02:50.895658  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 176812us
I20260709 13:02:50.863013  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:50.897019  8369 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:50.899271  8249 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
W20260709 13:02:50.899730  8220 connection.cc:256] RPC call timeout handler was delayed by -1.1010543749999897s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23660000000000003s for a response
I20260709 13:02:50.803813 11497 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:50.803174 11508 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:50.814338 11445 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:50.814898 11294 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:50.855530 11614 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:50.909087  8295 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:50.909356  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:50.910553  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:50.910880  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:50.926908  8401 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
I20260709 13:02:50.911491  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:50.927698  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:50.912413  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:50.928373  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:50.928946  8399 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:02:50.928937  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:50.929921  8404 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:02:50.930799  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:50.912603  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:50.913679  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:50.932348  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.932552  8403 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 5.
I20260709 13:02:50.876740 11625 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:50.917950  8974 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:50.921363  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:50.934209  8400 raft_consensus.cc:2393] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 8: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:50.934157  8253 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:50.922711  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1814) took 1192 ms. Trace:
I20260709 13:02:50.911652  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:50.937093 11625 consensus_queue.cc:1059] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:50.936357  8256 rpcz_store.cc:276] 0709 13:02:49.730198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.730343 (+   145us) service_pool.cc:224] Handling call
0709 13:02:49.730687 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:49.740286 (+  9599us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:49.776240 (+ 35954us) log.cc:844] Serialized 14 byte log entry
0709 13:02:50.059492 (+283252us) spinlock_profiling.cc:243] Waited 6.77 ms on lock 0x7b18002d8020. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e23a 00007f88287c8a2a
0709 13:02:50.922385 (+862893us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:50.922431 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:50.922472 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.922543 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.922656 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9205,"prepare.queue_time_us":94705,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":27106,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":247,"spinlock_wait_cycles":15575936,"thread_start_us":862608,"threads_started":2,"wal-append.queue_time_us":882931}
I20260709 13:02:50.941699  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1808) took 1450 ms. Trace:
I20260709 13:02:50.941377 11625 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:50.943043  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 391) took 1271 ms. Trace:
I20260709 13:02:50.943423 11625 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:50.944064 11629 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:50.942505  8398 rpcz_store.cc:276] 0709 13:02:49.491774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.492100 (+   326us) service_pool.cc:224] Handling call
0709 13:02:49.519320 (+ 27220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:50.126079 (+606759us) spinlock_profiling.cc:243] Waited 25.4 ms on lock 0x7b58007b38d0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f882871a0aa 00007f882879c447 00007f88287b4fe7 00007f88287b67de 00007f88287c0b49 00007f88287c1416 00007f88287bdd8f 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4
0709 13:02:50.941347 (+815268us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:50.941377 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:50.941443 (+    66us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:50.941474 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.941534 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.941651 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14667,"raft.queue_time_us":54235,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":2634,"spinlock_wait_cycles":58416384,"thread_start_us":207594,"threads_started":1}
I20260709 13:02:50.898904 11363 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:50.941043  8256 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:02:50.930066  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:50.943588  8331 rpcz_store.cc:276] 0709 13:02:49.671581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.753275 (+1081694us) service_pool.cc:224] Handling call
0709 13:02:50.753616 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:50.753982 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:50.754016 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:50.754475 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:02:50.914504 (+160029us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:50.914542 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.914607 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:50.942828 (+ 28221us) raft_consensus.cc:1685] finished
0709 13:02:50.942848 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.942978 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9158,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":5395,"thread_start_us":159747,"threads_started":1,"wal-append.queue_time_us":186921}
I20260709 13:02:50.945297 11629 raft_consensus.cc:3055] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:50.907632 11554 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:50.893393 10930 raft_consensus.cc:697] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:50.949921  8295 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:50.923941 11589 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:50.883076 11517 consensus_queue.cc:1059] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:02:50.960472  8324 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7. Ops: []
W20260709 13:02:50.949640  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 321457us
W20260709 13:02:50.961199  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 48669.8us
I20260709 13:02:50.920287 11579 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:50.976290  8295 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:50.959601  8974 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:50.961248 11588 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:50.923290 11536 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:50.996378 11487 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:50.999729 11294 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:51.002183  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1938) took 1310 ms. Trace:
I20260709 13:02:51.003152 11508 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:51.003768 11593 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:51.013876  8332 rpcz_store.cc:276] 0709 13:02:49.691920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.796494 (+1104574us) service_pool.cc:224] Handling call
0709 13:02:50.796873 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:50.836639 (+ 39766us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:50.836670 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:50.856491 (+ 19821us) log.cc:844] Serialized 26 byte log entry
0709 13:02:50.974733 (+118242us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:50.974791 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.974889 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:51.000714 (+ 25825us) spinlock_profiling.cc:243] Waited 5.5 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:51.001860 (+  1146us) raft_consensus.cc:1685] finished
0709 13:02:51.001886 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:51.002121 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35338,"raft.queue_time_us":35497,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"spinlock_wait_cycles":12640000,"thread_start_us":54251,"threads_started":1,"wal-append.queue_time_us":81019}
I20260709 13:02:51.015640  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:51.016274  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:51.004053 10930 consensus_queue.cc:237] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:50.946018 11630 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 13:02:50.986127 11579 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:51.007934 11580 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:51.008021 11497 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:50.950968 11363 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:50.962735 11513 raft_consensus.cc:2764] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:02:50.962543 11534 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:50.959257 11629 raft_consensus.cc:740] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:50.983520 11203 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:02:51.011087 11624 log.cc:927] Time spent T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 13:02:51.036175 11629 consensus_queue.cc:260] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:51.037206 11615 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.302s	user 0.002s	sys 0.001s
I20260709 13:02:51.034044 11203 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:51.037658 11391 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:51.038040 11203 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:51.038980 11533 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:51.040292 11625 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:51.040834 11382 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.041290 11382 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:51.041863 11362 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:51.042124 11629 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:51.046957 11622 log.cc:927] Time spent T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.338s	user 0.000s	sys 0.003s
I20260709 13:02:51.022161 11593 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:51.048141 11445 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:51.047664 11263 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.011351 11588 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:51.048902 11588 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:51.026503 11363 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:51.013641 11631 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:51.053617 11589 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:51.053874 11631 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:51.054172 11589 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:51.054512 11631 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:51.023808 11579 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:51.048331 11593 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:51.057364 11593 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:51.058257 11620 log.cc:927] Time spent T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.324s	user 0.001s	sys 0.001s
W20260709 13:02:51.059966 11618 log.cc:927] Time spent T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.368s	user 0.001s	sys 0.001s
I20260709 13:02:51.064658 11382 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.071080 11382 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:51.006600 11614 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.094802 11589 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.135417  8220 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:51.138374  8220 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:02:51.143501  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 275257us
I20260709 13:02:51.146221  8222 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:51.147673  8222 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:50.976769  8251 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:50.978178  8256 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:02:51.151266  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:51.166296 11656 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:51.011832  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2153) took 1322 ms. Trace:
I20260709 13:02:51.175395  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 393) took 1032 ms. Trace:
I20260709 13:02:51.176327  8401 raft_consensus.cc:2393] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 7: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:51.176229  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:51.018626  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:51.175410  8328 rpcz_store.cc:276] 0709 13:02:49.688701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.792693 (+1103992us) service_pool.cc:224] Handling call
0709 13:02:50.792970 (+   277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:50.907170 (+114200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:50.907205 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:50.907851 (+   646us) log.cc:844] Serialized 26 byte log entry
0709 13:02:50.973057 (+ 65206us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:50.973112 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.973181 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:51.010764 (+ 37583us) raft_consensus.cc:1685] finished
0709 13:02:51.010780 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:51.010903 (+   123us) inbound_call.cc:177] Queueing success response
0709 13:02:51.060911 (+ 50008us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b48001701d8. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882d6e7aa1 00007f882d6deb37 00007f882d6de9ca 00007f882d7d626f 00007f882d7d5f37 00007f882d7ea55c 00007f882d7ea508 00007f882d7ea488 00007f882d7ea443 00007f882d7e96ec 00007f882c713a96
Metrics: {"mutex_wait_us":113667,"raft.queue_time_us":113851,"raft.run_cpu_time_us":2912,"raft.run_wall_time_us":153212,"spinlock_wait_cycles":36136320,"thread_start_us":64878,"threads_started":1,"wal-append.queue_time_us":65207}
I20260709 13:02:51.019217  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:51.180634  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:51.178042  8332 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:51.176012  8330 rpcz_store.cc:276] 0709 13:02:50.142700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.831487 (+688787us) service_pool.cc:224] Handling call
0709 13:02:50.831781 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:50.832105 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:50.832133 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:50.857696 (+ 25563us) log.cc:844] Serialized 26 byte log entry
0709 13:02:51.009131 (+151435us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:51.009170 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:51.009261 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:51.174261 (+165000us) raft_consensus.cc:1685] finished
0709 13:02:51.174300 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:51.175138 (+   838us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":440,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"thread_start_us":103113,"threads_started":1,"wal-append.queue_time_us":142423}
I20260709 13:02:51.182200  8328 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:51.183674  8297 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:51.180531  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:51.186113  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 162792us
I20260709 13:02:51.186939  8331 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 10: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:51.181178  8405 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:02:51.023753  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:51.018958  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:51.189241  9003 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:51.189378  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 229682us
I20260709 13:02:51.179517  8397 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:51.190784  8974 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:51.194437  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 394) took 1051 ms. Trace:
I20260709 13:02:51.194893  8325 rpcz_store.cc:276] 0709 13:02:50.142984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.839192 (+696208us) service_pool.cc:224] Handling call
0709 13:02:50.839507 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:50.859664 (+ 20157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:50.859715 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:50.872303 (+ 12588us) log.cc:844] Serialized 26 byte log entry
0709 13:02:50.995410 (+123107us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:50.995462 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.995545 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:51.193872 (+198327us) raft_consensus.cc:1685] finished
0709 13:02:51.193928 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:51.194232 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6188,"raft.queue_time_us":2074,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":18538,"thread_start_us":43985,"threads_started":1,"wal-append.queue_time_us":120283}
I20260709 13:02:51.184314  8220 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:51.204310  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:51.203887 11508 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:02:51.205529 11334 consensus_queue.cc:1048] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:51.208504 11263 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:51.209247 11614 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:51.209858 11294 consensus_queue.cc:1048] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:51.210541 11614 consensus_queue.cc:1059] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:51.212603  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:51.213680  8327 raft_consensus.cc:2376] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10. Re-sending same reply.
I20260709 13:02:51.213850 11589 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:51.215317 11599 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:51.215447 11237 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:51.216969 11237 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.218904 11607 consensus_queue.cc:1048] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:51.220846  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:51.222836  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:51.223369  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:51.224069  9000 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:02:51.224676 11625 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:51.205271 11600 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.203336 11655 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:02:51.225286 11625 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.104126 11631 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.225418  8253 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9. Ops: [6.8-8.9]
I20260709 13:02:51.245673  8369 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:51.247720  8972 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:51.226019 11626 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:51.249482 11652 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:51.265707 11652 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:51.266132 11652 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.266476 11652 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:51.266932 11654 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:51.267400 11554 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:51.267534 11654 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:51.268163 11654 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:51.268298 11554 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:51.269615  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 69310.6us
I20260709 13:02:51.251818 11656 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:51.271615 11554 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:51.272194 11645 consensus_queue.cc:1059] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:51.272368  8222 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:02:51.272966  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 88962.7us
I20260709 13:02:51.273531 11645 consensus_queue.cc:1048] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:51.274048 11649 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:51.274478 11643 raft_consensus.cc:3055] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:51.274796 11649 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.275091 11643 raft_consensus.cc:740] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:51.275491 11649 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:51.276050 11656 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:51.248685 11631 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:51.248838  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 58183.1us
I20260709 13:02:51.232191 11445 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.277993  8220 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:51.231912 11362 consensus_queue.cc:1048] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:02:51.216038 11599 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.278506 11445 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.279232 11599 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:51.254837  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:51.274274  8399 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:51.233568 11276 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:51.282832  8974 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:51.282514 11276 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.210556 11536 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:51.285805 11536 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.286079 11276 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:51.276604 11643 consensus_queue.cc:260] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:51.286264 11536 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:51.273583 11357 consensus_queue.cc:1048] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:02:51.288275 11643 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:51.232105  8406 raft_consensus.cc:2393] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 6: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:02:51.291262  8406 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 450) took 1546 ms (client timeout 1518 ms). Trace:
W20260709 13:02:51.291785  8406 rpcz_store.cc:269] 0709 13:02:49.744969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.747148 (+  2179us) service_pool.cc:224] Handling call
0709 13:02:51.290957 (+1543809us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54003e6054. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:51.291149 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3411683712}
W20260709 13:02:51.292604  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 109389us
I20260709 13:02:51.225507 11363 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.293608  8297 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:51.294370 11421 raft_consensus.cc:3055] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:51.294909 11421 raft_consensus.cc:740] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:51.295410  8297 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:51.296290  8402 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:51.297621 11276 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:51.257860  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:51.298543 11276 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:51.299044 11276 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:51.263015  9005 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:51.240803  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1930) took 1634 ms. Trace:
I20260709 13:02:51.299952  8324 rpcz_store.cc:276] 0709 13:02:49.606189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.606298 (+   109us) service_pool.cc:224] Handling call
0709 13:02:49.606545 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:50.982867 (+1376322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:51.240155 (+257288us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54003c9954. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:51.240545 (+   390us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3112792960}
I20260709 13:02:51.237986 11625 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:51.303408  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 108905us
I20260709 13:02:51.303508 11558 raft_consensus.cc:3055] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:51.304023 11558 raft_consensus.cc:740] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:51.264168 11626 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.305811 11362 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:51.306537 11362 raft_consensus.cc:697] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:51.305941 11626 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:51.295570 11363 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:51.299176  9000 raft_consensus.cc:2393] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 7: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:51.281625 11445 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:51.239123  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:51.241746 11237 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:51.299494 11276 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:51.305307 11558 consensus_queue.cc:260] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.308034 11362 consensus_queue.cc:237] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:51.318420  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 46339.2us
I20260709 13:02:51.319125 11508 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:51.319770 11508 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:51.319906  8222 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:51.320314 11508 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:51.321384 11653 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:51.321844 11653 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:51.312417 11607 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:02:51.322316 11653 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:51.322866 11357 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:51.323347 11357 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:51.323779 11357 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:51.313331 11364 consensus_queue.cc:1059] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:02:51.315133  8251 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:02:51.321910 11558 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:51.326241 11631 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:51.234390 11600 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:51.296715 11421 consensus_queue.cc:260] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.330324 11421 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:51.330219 11588 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.330518 11363 consensus_queue.cc:1048] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:51.335822 11588 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:51.337699  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:51.338541  9003 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:02:51.340176 11364 raft_consensus.cc:3055] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:51.340677 11364 raft_consensus.cc:740] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:51.342460 11364 consensus_queue.cc:260] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:51.344179  8222 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:51.345170 11607 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:51.345837 11607 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:51.345852 11631 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:51.346621 11607 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:51.348758  8369 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:02:51.248008  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 271549us
W20260709 13:02:51.350307  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 83879us
W20260709 13:02:51.364910  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 87079.2us
W20260709 13:02:51.370126  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 87546.3us
I20260709 13:02:51.371093  8974 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:51.372064 11631 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:51.372507 11631 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:51.372946 11631 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:51.373976  8295 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:51.374991 11579 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:51.375770  8295 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:51.376178 11593 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:51.376797 11237 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:51.377625 11445 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:51.377709 11579 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:51.378068 11445 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:51.378561 11445 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:51.381604  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:51.381809 11334 consensus_queue.cc:1059] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:51.382848 11334 raft_consensus.cc:3055] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:51.381597 11237 raft_consensus.cc:697] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:51.383301 11334 raft_consensus.cc:740] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:51.384370 11237 consensus_queue.cc:237] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.384636 11334 consensus_queue.cc:260] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.385866 11364 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:51.359565 11363 consensus_queue.cc:1059] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
W20260709 13:02:51.400164  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 103013us
I20260709 13:02:51.400997  8297 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:51.410140  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:51.411386  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:51.412565  8222 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:51.412331 11654 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:51.422442  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 73883.7us
W20260709 13:02:51.428239  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 52559.7us
I20260709 13:02:51.429327  8295 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:51.430079 11363 raft_consensus.cc:3055] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:51.430588 11363 raft_consensus.cc:740] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:51.431694 11363 consensus_queue.cc:260] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.432092 11626 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:51.435135 11536 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:51.436863 11655 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:51.437358 11655 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.437783 11655 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:51.395027 11599 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.438925 11649 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.439862 11599 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:51.437798  9003 raft_consensus.cc:2368] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9.
W20260709 13:02:51.429386  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 58430.9us
I20260709 13:02:51.412526 11276 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:51.433144 11363 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:51.455358  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 54504.6us
I20260709 13:02:51.456122  8972 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:51.441360 11263 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:02:51.458252 11631 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.459328 11263 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:51.460747 11651 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:51.461210 11651 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:51.461658 11651 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:51.463544 11445 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.465622  9005 raft_consensus.cc:1240] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:51.466778 11536 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:51.467350 11588 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:51.441679 11579 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:51.468164 11579 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:51.468323  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:51.468590 11579 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:51.467775 11588 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:51.472047 11588 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:51.427130 11334 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:51.453835  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:51.476506  8220 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:51.441267 11649 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:51.482472 11593 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:51.483436 11357 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.483903 11653 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.485180 11445 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:51.485898  8372 leader_election.cc:400] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9.
I20260709 13:02:51.521662  8372 leader_election.cc:403] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:02:51.485586 11203 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:51.534368 11364 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:51.534684 11203 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:51.490809  8369 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:02:51.536999  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 46283.8us
W20260709 13:02:51.541613  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 66673.4us
I20260709 13:02:51.493496 11276 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:51.509398 11508 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.542223  8369 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:51.510681 11651 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.543349 11558 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:51.545012 11651 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:51.493616 11536 consensus_queue.cc:1059] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:02:51.514194  8297 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:51.545262 11643 consensus_queue.cc:1048] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:51.514463 11626 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:51.514215  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 68870.4us
I20260709 13:02:51.546067 11643 consensus_queue.cc:1243] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:02:51.546370 11536 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:51.526942 11579 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:51.531045  8974 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:51.531487 11357 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:51.532371  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 46813.5us
I20260709 13:02:51.474282 11362 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:02:51.532730 11657 log.cc:927] Time spent T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:02:51.538458  8329 raft_consensus.cc:2393] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 8: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:51.539562  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:51.460757 11631 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:51.548354  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 52768us
I20260709 13:02:51.556219  8220 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:51.537726 11294 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:51.563593  9005 raft_consensus.cc:1240] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:51.558717 11362 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.570081 11362 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:51.558699 11599 consensus_queue.cc:1048] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 13:02:51.567171 11645 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:51.573722 11276 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:51.573797 11263 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:51.575572 11627 consensus_queue.cc:1048] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:02:51.563580 11294 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:51.561290 11536 raft_consensus.cc:3055] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:51.559204 11625 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:51.571105 11357 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:02:51.594565  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 100626us
W20260709 13:02:51.595980  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 65072.7us
I20260709 13:02:51.580667 11645 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:51.598706 11629 consensus_meta.cc:287] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.536s	user 0.012s	sys 0.007s
I20260709 13:02:51.592324 11294 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:51.585112 11364 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9.
I20260709 13:02:51.592017 11536 raft_consensus.cc:740] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:51.602747  8222 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:51.604238  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:51.605863 11652 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:51.608641  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.614876  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:51.615605  8329 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:51.597440 11665 consensus_queue.cc:1048] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
W20260709 13:02:51.617733  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.619566  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:51.620316 11508 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:51.620487  8256 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:02:51.621790  8324 raft_consensus.cc:1275] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:51.624270  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:51.624375  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:51.625164  8324 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 10: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:51.625276  8256 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:51.629491 11421 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:51.630576  8974 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:51.631470 11421 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:51.631309 11655 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.634430 11655 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:51.634536  8251 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 13:02:51.635788  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:51.636161  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:51.636926  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:51.637732 11654 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:51.638262  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:51.657174  8257 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:51.635156  8151 catalog_manager.cc:5697] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: config changed from index 2 to 5, term changed from 5 to 6, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to <none>, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1), VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2), VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3), VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 6 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:51.604848 11607 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.640082  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:51.640648  8399 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:51.637759  8253 raft_consensus.cc:1240] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:02:51.639442 11652 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:51.641469 11558 consensus_queue.cc:1059] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:51.642202  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:51.643128  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:51.643584  8297 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:51.644271  8222 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:51.644588 11579 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:51.640960 11421 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:51.649803  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:51.650072  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:51.651577  8327 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:51.652168  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:51.653920  8256 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
I20260709 13:02:51.654120 11536 consensus_queue.cc:260] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.605247 11653 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:51.599488 11645 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:51.656443  8252 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:51.656471  8249 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:51.659430  8329 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
W20260709 13:02:51.662014  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 105937us
I20260709 13:02:51.663766  8248 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:51.678663  9005 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:51.679088 11536 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:51.680666  8326 raft_consensus.cc:2393] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 8: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:51.685544 11364 raft_consensus.cc:3055] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:51.686125 11364 raft_consensus.cc:740] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:51.685979  8220 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:51.687377 11364 consensus_queue.cc:260] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.688804 11364 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:51.691969  8154 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45434 (request call id 281) took 18755 ms (client timeout 15000 ms). Trace:
I20260709 13:02:51.694545  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:51.694653  8151 catalog_manager.cc:5697] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 4 to 5, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 5 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:02:51.671885  8250 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:51.696803  8151 catalog_manager.cc:5697] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 1 to 4, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 4 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:51.728016  8249 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:02:51.730346  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:51.731668  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:51.732853  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:02:51.733731  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 89538.7us
I20260709 13:02:51.735102  8222 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:51.736337  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.737948  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:51.738925 11554 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.739681 11558 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:51.739544 11554 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:51.740216 11558 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.740490  8369 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:51.740718 11558 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:51.741946  8331 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:51.742388 11376 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:51.742823 11376 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.743225 11376 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:51.743927  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:51.744585  8331 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:51.745886  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:51.746542  8328 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:02:51.748528  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:51.749130  8328 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:51.750437  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 50042us
I20260709 13:02:51.756182  8257 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:02:51.749689  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:51.735111 11294 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.769420  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:51.772588  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:51.773478  8257 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:02:51.774583  8331 raft_consensus.cc:1240] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:51.775606  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:51.776784 11294 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:51.777006  8330 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:02:51.774582  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:51.777973  8329 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:51.779225  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:51.785611  8329 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:02:51.796002  8328 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:51.725168 11626 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:51.798924  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:51.807679  8254 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:02:51.728848 11665 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:51.751278 11655 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:51.809791  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.813537  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:51.804893  8332 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:02:51.814098  8326 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:51.812700  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:51.810506  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:51.817759  8248 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: [5.3-7.4]
I20260709 13:02:51.816525  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:51.820617  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:51.822355  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 87345.1us
I20260709 13:02:51.824903  8253 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:51.822558  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:51.826108  8328 raft_consensus.cc:1275] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:51.826581  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:51.827399  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:51.828223  9000 raft_consensus.cc:2410] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
W20260709 13:02:51.832444  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 92032.3us
I20260709 13:02:51.833675  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:51.833751 11664 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:51.834007  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:51.834756  8400 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:51.835834  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:51.836851 11625 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:51.838215  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:51.838974  8404 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:51.840699 11607 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:51.842381  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 156545us
I20260709 13:02:51.844758  8324 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:51.845256  8251 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:51.840759 11362 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:51.853682 11588 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:51.855005 11362 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:51.858982 11558 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:51.829490 11554 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:51.862187  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:51.863417  8222 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:51.865063  9005 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:02:51.862920  9003 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:02:51.867944 11376 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:51.827858 11579 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
W20260709 13:02:51.692498  8154 rpcz_store.cc:269] 0709 13:02:32.936350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.936531 (+   181us) service_pool.cc:224] Handling call
0709 13:02:51.691625 (+18755094us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:48.195902 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:48.350421 (+154519us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:48.350480 (+    59us) write_op.cc:435] Acquired schema lock
0709 13:02:48.350504 (+    24us) tablet.cc:662] Decoding operations
0709 13:02:48.893751 (+543247us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:48.949975 (+ 56224us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:48.957596 (+  7621us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:02:49.297120 (+339524us) tablet.cc:701] Row locks acquired
0709 13:02:49.297166 (+    46us) write_op.cc:260] PREPARE: finished
0709 13:02:49.297371 (+   205us) write_op.cc:270] Start()
0709 13:02:49.297560 (+   189us) write_op.cc:276] Timestamp: P: 1783602169297291 usec, L: 0
0709 13:02:49.297591 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:02:49.298452 (+   861us) log.cc:844] Serialized 6917 byte log entry
0709 13:02:50.960109 (+1661657us) op_driver.cc:464] REPLICATION: finished
0709 13:02:51.100176 (+140067us) write_op.cc:301] APPLY: starting
0709 13:02:51.100253 (+    77us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:51.100474 (+   221us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:51.100590 (+   116us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:51.102518 (+  1928us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:51.102833 (+   315us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0709 13:02:51.102882 (+    49us) write_op.cc:312] APPLY: finished
0709 13:02:51.103292 (+   410us) log.cc:844] Serialized 119 byte log entry
0709 13:02:51.103467 (+   175us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:51.103623 (+   156us) write_op.cc:454] Released schema lock
0709 13:02:51.103690 (+    67us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:51.104055 (+   365us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:51.104093 (+    38us) write_op.cc:454] Released schema lock
0709 13:02:51.215452 (+111359us) spinlock_profiling.cc:243] Waited 3.73 ms on lock 0x7b54003bf454. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bb32a 00007f8828740f14 00007f8828740ec3 00007f8828740e38 00007f8828740df3 00007f88287400dc 00007f8828769e16 00007f8828751cbb 00007f882873c39d 00007f882873fd75
Metrics: {"mutex_wait_us":1578443,"reported_tablets":37,"child_traces":[["op",{"apply.queue_time_us":120362,"apply.run_cpu_time_us":3735,"apply.run_wall_time_us":3868,"num_ops":17,"prepare.queue_time_us":417112,"prepare.run_cpu_time_us":45802,"prepare.run_wall_time_us":1547556,"raft.queue_time_us":70844,"raft.run_cpu_time_us":16161,"raft.run_wall_time_us":339626,"replication_time_us":1601478,"spinlock_wait_cycles":8583936,"thread_start_us":828880,"threads_started":4,"wal-append.queue_time_us":731716}]]}
W20260709 13:02:52.037719  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 50440us
I20260709 13:02:51.875147  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:52.042960  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 214936us
I20260709 13:02:52.045323  8404 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:51.824790 11654 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:51.898360  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:51.872789  8406 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:02:51.934080  8151 catalog_manager.cc:5697] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:02:51.898820  8972 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:51.861244 11558 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:52.053329  8250 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:52.081634  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:02:51.958786 11363 consensus_meta.cc:287] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.504s	user 0.011s	sys 0.002s
W20260709 13:02:51.901585  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 301397us
I20260709 13:02:51.874042  8295 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:51.934929  8248 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:02:51.993405  8397 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:02:51.898838  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:51.869145  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:52.083354  8257 raft_consensus.cc:1240] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:52.081986  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:52.089021  8405 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:52.090332  8220 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:52.090485  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:52.091216 11362 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:52.092759  8151 catalog_manager.cc:5697] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 4 to 5, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 5 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } }
W20260709 13:02:52.085845  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 187573us
I20260709 13:02:52.093595 11626 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:51.873660 11607 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:52.094267  8369 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:52.094601  8151 catalog_manager.cc:5697] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 2 to 4, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 4 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:52.096150  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:52.096259  8369 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:52.082450  8974 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:52.097546 11652 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:52.097823 11362 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:52.098555 11362 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:52.083535  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 184807us
I20260709 13:02:51.873502  8401 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:02:51.987253 11588 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:51.897837 11554 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.101065  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:52.101881  8400 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
W20260709 13:02:52.114264  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 240293us
I20260709 13:02:52.052753 11654 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.119117  8253 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:02:52.102734  8972 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:52.096269  8151 catalog_manager.cc:5697] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 3 to 4, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 4 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:52.124078  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 84637us
I20260709 13:02:52.127642  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:52.100872 11651 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:52.128774 11651 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:52.129202 11651 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:52.129887  9000 raft_consensus.cc:2393] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 6: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:52.134002  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:52.129772  8295 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:52.137977  8404 raft_consensus.cc:2410] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:02:52.137321  8369 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:52.140669  8297 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:52.142120  9003 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:52.148854  8405 raft_consensus.cc:2410] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 11 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:02:52.149267  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:52.151192  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.119683 11654 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:52.156985  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:52.152530  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:52.139685  8151 catalog_manager.cc:5697] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:52.164846 11558 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:52.165009  8974 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:52.165360 11558 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:52.165891 11558 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:52.170406 11666 log.cc:927] Time spent T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:02:52.170666 11363 consensus_queue.cc:1059] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:52.171772 11614 raft_consensus.cc:3055] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:52.172219 11614 raft_consensus.cc:740] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:52.172796  8398 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:52.173131 11614 consensus_queue.cc:260] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:52.154848 11276 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:52.174431 11614 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:52.175321 11588 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:02:52.172309 11669 log.cc:927] Time spent T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:02:52.175899 11588 raft_consensus.cc:697] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:02:52.193125  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 71680.7us
W20260709 13:02:52.206847  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.148968 11652 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:02:52.212375  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 47450.9us
I20260709 13:02:52.210258 11203 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:02:52.219210  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 99157.8us
I20260709 13:02:52.214287  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:52.223814  8151 catalog_manager.cc:5697] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 3 to 4, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 4 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:02:52.218569  8399 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:52.231736 11645 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:52.233601  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:52.235529 11579 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:52.192217 11648 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:52.214754 11652 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.237342 11652 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:52.237726 11376 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:52.241636 11321 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:52.244277  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:52.233687  8297 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:52.153545 11554 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:52.155341 11263 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 13:02:52.255954  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:52.256790  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:52.261315  8399 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:52.245990  8974 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:52.257594  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.257203  8151 catalog_manager.cc:5697] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 5 to 6, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 6 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:52.268034  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:52.269500  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:52.269824  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:52.270594 11645 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:52.271063  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:52.272146  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:52.272429  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:52.273460  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:52.273756  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:52.274189  8250 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:52.274730  8248 raft_consensus.cc:2393] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 6: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:52.269510  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:52.277158  8326 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:02:52.218335 11203 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:52.284565  8974 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:52.290990  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:52.291316 11517 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:52.291841  8249 raft_consensus.cc:2393] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 8: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:52.291875 11517 raft_consensus.cc:697] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 6 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:52.277201  8253 raft_consensus.cc:2368] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9.
I20260709 13:02:52.293125  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:52.298604  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:52.299364  8248 raft_consensus.cc:2368] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9.
W20260709 13:02:52.300297  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.297664  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:52.288167 11203 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:52.309893  8399 raft_consensus.cc:1240] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: []
W20260709 13:02:52.314013  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 80422.5us
I20260709 13:02:52.320295  8252 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:52.301061 11362 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:52.271050 11654 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:52.286784 11376 consensus_queue.cc:1059] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:02:52.328171  8297 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:52.345301  8151 catalog_manager.cc:5697] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 2 to 5, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 5 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:52.321998 11421 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:52.304472 11588 consensus_queue.cc:237] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:52.359659 11654 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:52.362949  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.361732  8151 catalog_manager.cc:5697] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 3 to 6, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 6 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:52.360363 11654 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:52.380934  8250 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:02:52.400115  8297 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:52.401751  8401 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:02:52.406702  8402 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:52.448356  9003 raft_consensus.cc:1240] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:52.340472 11517 consensus_queue.cc:237] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:52.447938  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:52.444162 11558 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:52.453574 11630 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:52.454164 11630 raft_consensus.cc:697] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:52.454495 11558 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:52.455475 11558 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:52.455334 11630 consensus_queue.cc:237] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:52.401502 10930 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:52.452666  8255 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
W20260709 13:02:52.477600  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 313582us
I20260709 13:02:52.470280  8151 catalog_manager.cc:5697] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 3 to 4, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 4 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:02:52.393841 11654 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:52.498668  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:52.496402 11554 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:52.487300  8151 catalog_manager.cc:5697] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:52.443053 11363 raft_consensus.cc:3055] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:52.469761 10930 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.456203 11558 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:52.560652  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:52.556092 10930 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:52.535425 11363 raft_consensus.cc:740] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:02:52.585888  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 315009us
W20260709 13:02:52.589588  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 375004us
I20260709 13:02:52.594187  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:52.595542  8406 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:52.609023  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:52.608894  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 302857us
W20260709 13:02:52.623857  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.606897 11276 raft_consensus.cc:697] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:52.607540 11321 raft_consensus.cc:697] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:52.627034 11362 raft_consensus.cc:697] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:52.649662  8972 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:52.651708  8972 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:52.640025 11294 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:52.667186  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:52.667924  8405 raft_consensus.cc:2393] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 8: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:52.668092  8402 pending_rounds.cc:85] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 2
I20260709 13:02:52.626129 11263 consensus_queue.cc:1059] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:02:52.672852  9003 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:02:52.673990  9003 pending_rounds.cc:85] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: Aborting all ops after (but not including) 8
W20260709 13:02:52.675668  8220 leader_election.cc:336] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.772s (SENT)
I20260709 13:02:52.668471  8402 pending_rounds.cc:107] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:02:52.680351  8402 raft_consensus.cc:2889] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:52.646040 11678 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:52.674281  9003 pending_rounds.cc:107] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:02:52.637396 11681 consensus_queue.cc:1059] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:02:52.703529  9003 raft_consensus.cc:2889] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:52.676174  8220 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:52.648435 11631 consensus_queue.cc:1059] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
W20260709 13:02:52.673674 11670 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.311s	user 0.001s	sys 0.000s
I20260709 13:02:52.662870 11614 consensus_queue.cc:1048] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 13:02:52.680521 11680 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:52.721380  8974 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:52.664708 11382 consensus_queue.cc:1059] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.209s
W20260709 13:02:52.722590  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 321561us
I20260709 13:02:52.708630 11678 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:52.667802 11363 consensus_queue.cc:260] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:52.732965  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 254233us
W20260709 13:02:52.749164  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.754374  8295 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:52.756315 11679 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:52.722234 11607 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:52.716009 11588 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
W20260709 13:02:52.757617  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 87105.3us
I20260709 13:02:52.759263  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:52.728531 11421 raft_consensus.cc:697] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:52.742705 11558 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:52.722607 11683 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:02:52.767969  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 49231us
W20260709 13:02:52.775976  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:52.748627 11651 consensus_meta.cc:287] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.586s	user 0.012s	sys 0.003s
W20260709 13:02:52.785923  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 123613us
W20260709 13:02:52.787860  8220 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.869s (SENT) [suppressed 19 similar messages]
I20260709 13:02:52.749768 11678 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:52.798250  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 76968.6us
I20260709 13:02:52.760977 11679 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:52.811595  8405 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:02:52.787705 11680 raft_consensus.cc:697] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:52.795306 11685 raft_consensus.cc:3055] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:02:52.821947 11676 log.cc:927] Time spent T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 13:02:52.805509 11630 raft_consensus.cc:3055] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:52.822363 11685 raft_consensus.cc:740] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:52.822546 11630 raft_consensus.cc:740] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:52.823443 11685 consensus_queue.cc:260] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:52.823585 11630 consensus_queue.cc:260] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:52.824103 11679 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:52.824703 11685 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:52.824937 11630 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:52.761646 11362 consensus_queue.cc:237] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:52.826375 11679 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:52.826721 11652 consensus_meta.cc:287] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.001s
I20260709 13:02:52.827005 11679 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:52.827276 11588 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:02:52.811342 11674 log.cc:927] Time spent T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 13:02:52.821736 11705 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:02:52.822021 11675 log.cc:927] Time spent T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:02:52.827569 11679 raft_consensus.cc:697] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 7 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:52.820987  8297 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:02:52.829308  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.829257  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1827) took 1141 ms. Trace:
W20260709 13:02:52.819722 11677 log.cc:927] Time spent T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.151s	user 0.001s	sys 0.001s
I20260709 13:02:52.818913 11558 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:52.806845 11363 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:52.833060  8222 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:52.834280  8406 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:02:52.838616 11706 consensus_queue.cc:1059] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:02:52.841420  8256 rpcz_store.cc:276] 0709 13:02:51.687740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.815728 (+127988us) service_pool.cc:224] Handling call
0709 13:02:51.818257 (+  2529us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:52.041711 (+223454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:52.041740 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:52.052587 (+ 10847us) log.cc:844] Serialized 52 byte log entry
0709 13:02:52.298436 (+245849us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:52.318138 (+ 19702us) log.cc:844] Serialized 14 byte log entry
0709 13:02:52.459220 (+141082us) spinlock_profiling.cc:243] Waited 1.05 ms on lock 0x7b1800421900. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882564bc0a 00007f882ed8b219 00007f88256d3a40 00007f88256ce4e4 00007f88256ce455 00007f882c1b63c7 00007f88287c89a7 00007f88287d80bf 00007f88287d8038
0709 13:02:52.459450 (+   230us) log.cc:844] Serialized 14 byte log entry
0709 13:02:52.459643 (+   193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:52.471271 (+ 11628us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b54003fd754. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:52.471347 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:52.829051 (+357704us) raft_consensus.cc:1685] finished
0709 13:02:52.829069 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:52.829197 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":122709,"prepare.queue_time_us":188513,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":101,"raft.queue_time_us":17763,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":5580,"spinlock_wait_cycles":70591488,"thread_start_us":211606,"threads_started":1,"wal-append.queue_time_us":493553}
I20260709 13:02:52.838491 11680 consensus_queue.cc:237] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:52.839196 11684 consensus_queue.cc:1059] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:52.747962 11321 consensus_queue.cc:237] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:52.753780 11276 consensus_queue.cc:237] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:52.824865  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:52.836058 11364 consensus_meta.cc:287] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 1.146s	user 0.011s	sys 0.008s
I20260709 13:02:52.794817 11651 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:52.866083  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:52.866320  9004 raft_consensus.cc:1240] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:52.868261  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:52.868348 11679 consensus_queue.cc:237] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:52.869036  8406 raft_consensus.cc:2393] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 9: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:52.854835 11651 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:52.827198 11652 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:52.847872 11558 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:52.873795  8222 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:52.881094  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:52.882675 11705 raft_consensus.cc:3055] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:52.883330 11705 raft_consensus.cc:740] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:52.884301 11705 consensus_queue.cc:260] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:52.886327  8398 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:52.882757 11697 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:02:52.896677  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 142462us
I20260709 13:02:52.889834  8405 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
W20260709 13:02:52.895680  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 74789.7us
I20260709 13:02:52.871800 11614 raft_consensus.cc:3055] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:52.877168 11607 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:52.920775  8295 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:52.938231  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 138876us
I20260709 13:02:52.880472 11421 consensus_queue.cc:237] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:52.896963 11697 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.920353 11614 raft_consensus.cc:740] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:02:52.970966  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 50297.8us
I20260709 13:02:52.972265  8295 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:52.975018  8295 leader_election.cc:400] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9.
I20260709 13:02:52.937649 11691 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.975476  8295 leader_election.cc:403] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:02:52.937166 11626 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:52.977875  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 104220us
I20260709 13:02:52.978452  8295 leader_election.cc:400] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9.
I20260709 13:02:52.978828  8295 leader_election.cc:403] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:02:52.980705  8295 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:52.982853  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 191254us
W20260709 13:02:52.985898  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 198038us
I20260709 13:02:52.974809 11579 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:52.974679 11697 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:52.975864 11694 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:52.977664 11648 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.013s	sys 0.003s
W20260709 13:02:52.996373  8220 leader_election.cc:336] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.933s (SENT)
I20260709 13:02:52.981055 11654 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:52.999260  8220 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:52.966709 11678 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:52.984615 11693 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:52.981710 11699 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:53.001201  8295 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:52.992588 11607 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:53.001952 11648 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.014s	sys 0.003s
I20260709 13:02:53.006861  8295 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:52.981304 11691 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:52.998912 11626 consensus_queue.cc:1059] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:02:53.006335 11654 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:53.013844  8295 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:53.026889 11686 log.cc:927] Time spent T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:02:53.027446 11558 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:53.043524 11691 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:53.025111 11704 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.048595  8297 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:53.015372 11694 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:53.026036 11712 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:53.009177 11648 raft_consensus.cc:697] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:53.062613  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:02:53.065352  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 51583.7us
I20260709 13:02:53.009830 11679 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:02:53.003564 11614 consensus_queue.cc:260] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:53.008757 11607 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:53.072046 11203 consensus_meta.cc:287] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.730s	user 0.010s	sys 0.003s
I20260709 13:02:53.082046  8297 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:53.084363  9005 raft_consensus.cc:1240] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7. Ops: []
W20260709 13:02:53.083755  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 88370.4us
I20260709 13:02:53.076361 11694 raft_consensus.cc:697] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 7 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:53.088865  8222 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:53.077852 11712 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:53.093214  8369 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:53.082190 11691 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:53.103231 11558 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:53.103585 11678 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:53.091435 11699 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9.
I20260709 13:02:53.104794 11579 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.094698 11614 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:53.106130 11691 raft_consensus.cc:697] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:53.092440 11645 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:02:53.108021  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.114526  8403 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:53.073269 11704 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:53.083603 11203 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:53.121481  8222 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:53.117811 11276 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:53.125362 11294 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:53.118271 11626 raft_consensus.cc:3055] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:53.126344 11626 raft_consensus.cc:740] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:53.126396 11645 consensus_queue.cc:1059] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:53.127511 11645 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:53.128208 11645 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:53.128785 11645 raft_consensus.cc:3055] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:53.129179 11645 raft_consensus.cc:740] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:53.129873 11705 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:53.129962 11645 consensus_queue.cc:260] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:53.131335 11645 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:53.127519 11626 consensus_queue.cc:260] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:53.134250 10930 consensus_meta.cc:287] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.519s	user 0.008s	sys 0.002s
I20260709 13:02:53.134732 11626 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:53.134768 11607 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:53.135270 11687 raft_consensus.cc:3055] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:53.135700 11687 raft_consensus.cc:740] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:53.118201 11579 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:53.119122 11652 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:53.116032 11678 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:53.141593 11678 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:53.142268  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.134678 10930 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:53.143797 11678 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:53.147878  8252 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:50420
I20260709 13:02:53.140133 11579 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:53.144940  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:53.150018  8256 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:02:53.138273  9004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 738) took 1037 ms. Trace:
I20260709 13:02:53.145318  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:53.144634  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:53.083693 11723 consensus_queue.cc:1059] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:02:53.148707  8252 raft_consensus.cc:606] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:02:53.209100  8252 raft_consensus.cc:614] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:53.193591  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 72185.9us
I20260709 13:02:53.145895  8257 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:02:53.209733  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 320511us
I20260709 13:02:53.193540  8297 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:53.144033 10930 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:53.192901 11704 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:53.194696 11710 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:53.125885 11276 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:53.195891  8256 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:53.196488 11607 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.149139 11693 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9.
W20260709 13:02:53.193374  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.151449  9004 rpcz_store.cc:276] 0709 13:02:52.100662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.102628 (+  1966us) service_pool.cc:224] Handling call
0709 13:02:52.176912 (+ 74284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.871838 (+694926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.138013 (+266175us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b54004c2554. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:53.138188 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1535022976}
W20260709 13:02:53.212275  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 92533.3us
I20260709 13:02:53.136662 11687 consensus_queue.cc:260] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:53.125960 11294 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:53.147413 11263 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:53.210433  8255 raft_consensus.cc:2393] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 9: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:02:53.241390  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.147552 11683 raft_consensus.cc:697] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:53.230142  8254 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:53.263433  8249 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:53.230310  8253 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:53.264287  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 54284.1us
I20260709 13:02:53.267752  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:53.269213  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 48962.9us
I20260709 13:02:53.230818 11607 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:53.215199 11276 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:53.296219  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.270061 11704 consensus_queue.cc:1059] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
W20260709 13:02:53.296548  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 82644.8us
I20260709 13:02:53.320662  8222 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:02:53.321889  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 54225.1us
W20260709 13:02:53.324512  8403 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 470) took 1453 ms (client timeout 1773 ms). Trace:
I20260709 13:02:53.297851 11697 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:53.299424 11687 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:53.302300 11294 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:53.303104  8297 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:53.215616 11710 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:53.307068 11729 consensus_queue.cc:1059] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:53.308285 11203 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:53.308877  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.335507  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.309574 11685 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:53.309633  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 98961.2us
I20260709 13:02:53.336570  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:53.337193  8250 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:02:53.310106 11630 consensus_queue.cc:1059] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:02:53.311239  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.311672 11263 raft_consensus.cc:697] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:02:53.343531 11725 log.cc:927] Time spent T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:02:53.213596 11579 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:53.119378 11648 consensus_queue.cc:237] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:53.145104 11691 consensus_queue.cc:237] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:53.276262 11709 raft_consensus.cc:3055] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:53.275367 11694 consensus_queue.cc:237] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:53.317240  8220 leader_election.cc:336] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.207s (SENT)
I20260709 13:02:53.329008  8222 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:53.324908  8403 rpcz_store.cc:269] 0709 13:02:51.870847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.871684 (+   837us) service_pool.cc:224] Handling call
0709 13:02:53.324355 (+1452671us) spinlock_profiling.cc:243] Waited 781 ms on lock 0x7b54002689d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:53.324450 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1796917632}
I20260709 13:02:53.331025  8328 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:53.333580  8332 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:53.339349  8329 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:53.354969  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:53.355193  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:53.334765 11723 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.344866  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:53.363343  8325 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:53.348824  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:53.349472  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:53.335019 11645 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:02:53.366763  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 98416.3us
I20260709 13:02:53.359537  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:53.367626  8297 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:53.342868  8369 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:53.359649  9004 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:02:53.344120 11263 consensus_queue.cc:237] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:53.335163 11710 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:53.363130  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 45916.5us
I20260709 13:02:53.330476 11554 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:53.364140  9008 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:53.360327  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:53.368623  8369 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:53.346860 11579 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:02:53.370848  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 57795.2us
I20260709 13:02:53.349395 11709 raft_consensus.cc:740] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:02:53.375787  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.382499  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:53.382622  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:53.357688 11607 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:53.385563  8250 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
I20260709 13:02:53.362432 11203 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:53.349923 11363 consensus_queue.cc:1048] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:02:53.389858  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:53.398708  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:53.384207 11579 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:53.390012 11699 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:53.345793 11685 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:53.411296  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:53.413004  8295 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:53.397851  8298 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:53.372455 11704 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:53.415995  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:53.416049  8332 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:53.417225  8297 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:53.417908 11740 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:53.377142 11554 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.416379  9004 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:53.420882 11554 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:53.420820  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 91883us
I20260709 13:02:53.421862  8372 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:53.428347  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.430603  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:53.430990  8249 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:53.431324  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:53.431757  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:53.431941  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:53.434190  8248 raft_consensus.cc:2393] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 9: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:53.434558  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:53.436174  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:53.430778  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:53.436810  8257 raft_consensus.cc:2410] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:02:53.387085 11645 raft_consensus.cc:697] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:53.438890  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:53.446826  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:53.447190  8255 raft_consensus.cc:2393] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 8: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:53.439459  8252 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:53.447510  8332 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:02:53.448917  9001 raft_consensus.cc:2410] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 10 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:02:53.389951 11203 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.439654  8330 raft_consensus.cc:2393] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 9: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:53.439760  8256 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:53.346163 11683 consensus_queue.cc:237] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:53.402282 11697 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:53.452586  8329 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:02:53.457007  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:53.468218  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 55318.3us
I20260709 13:02:53.468894  8326 raft_consensus.cc:2393] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 9: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:53.368739 11723 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:53.445971 11376 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:53.377678 10930 consensus_queue.cc:1059] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:02:53.482631 11684 raft_consensus.cc:3055] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:53.463632 11558 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:53.416087 11705 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:53.466727  8332 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:53.492853  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:53.493227  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:53.493844  8257 raft_consensus.cc:2410] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:02:53.494293  8250 raft_consensus.cc:2393] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 6: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:02:53.467458  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 69682.8us
I20260709 13:02:53.386683 11680 consensus_queue.cc:1059] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:02:53.419574 11614 consensus_queue.cc:1059] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:02:53.436794 11362 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.419916 11704 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:02:53.478924 11728 log.cc:927] Time spent T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:02:53.479542 11651 consensus_queue.cc:1059] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:53.479951 11654 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:53.482136 11294 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:53.482759 10930 consensus_queue.cc:1243] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:02:53.464036  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:53.465816 11203 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:53.518301  8326 raft_consensus.cc:3055] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:53.518685  8326 raft_consensus.cc:740] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:53.495384  8298 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:53.495679  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:53.418684 11382 consensus_queue.cc:1059] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
W20260709 13:02:53.481273 11733 log.cc:927] Time spent T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:02:53.531090  8325 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
W20260709 13:02:53.497354  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 74126.5us
I20260709 13:02:53.473568  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:53.501821  8253 raft_consensus.cc:1275] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:02:53.502972  8332 raft_consensus.cc:1240] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:53.483167 11684 raft_consensus.cc:740] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:53.472779 11588 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:53.483527 11376 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:53.508662  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 105444us
I20260709 13:02:53.512408  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:53.487094 11558 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.541664 11680 consensus_queue.cc:1059] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:53.541858 11558 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:53.504920 11704 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:53.523893  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:53.545356  8256 raft_consensus.cc:2393] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 8: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:53.524947 11685 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:53.525341 11723 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:53.504709 11362 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:53.526801  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.527374  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 63402.4us
I20260709 13:02:53.501996 11517 consensus_queue.cc:1059] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:02:53.552748  8369 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:53.553861  8369 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:53.529897 11614 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:53.532914  8222 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:53.535220  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:53.532780  8253 pending_rounds.cc:85] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 4
I20260709 13:02:53.538065  8326 consensus_queue.cc:260] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:53.539227 11376 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:53.539518 11645 consensus_queue.cc:237] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:53.541925 11688 raft_consensus.cc:3055] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:53.538992  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:53.548888 11703 raft_consensus.cc:3055] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:53.504597 11630 consensus_queue.cc:1059] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:02:53.557950  8253 pending_rounds.cc:107] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:02:53.585350  8253 raft_consensus.cc:2889] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:53.556857  8332 raft_consensus.cc:2376] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9. Re-sending same reply.
I20260709 13:02:53.534206 11679 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:53.564357  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:53.611930  8255 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:02:53.534070 11588 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:53.514703  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:53.568293  8297 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:53.612897 11588 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:53.569624  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:53.565203 11723 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:53.597288  8326 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:53.560122 11614 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:53.655220  8332 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 7 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:53.572664 11703 raft_consensus.cc:740] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:53.596134  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:53.556450 11689 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:53.556636 11685 raft_consensus.cc:3055] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:53.567967 11688 raft_consensus.cc:740] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:53.561825 11654 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:53.670534  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.624871 11203 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:53.676199  8325 raft_consensus.cc:1240] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:53.599694 11746 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:53.631079 11684 consensus_queue.cc:260] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:53.773377  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 215166us
W20260709 13:02:53.830178  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.788496  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 293126us
I20260709 13:02:53.701615 11652 raft_consensus.cc:3055] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:54.003434 11652 raft_consensus.cc:740] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:54.003849 11684 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:53.939177  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 427790us
I20260709 13:02:54.004556 11652 consensus_queue.cc:260] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.006404 11652 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:53.701982 11294 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.674949 11263 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.676034 11614 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:53.990182 11754 log.cc:927] Time spent T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.237s	user 0.000s	sys 0.000s
I20260709 13:02:54.067483  8329 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260709 13:02:53.987172 11741 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.387s	user 0.001s	sys 0.000s
I20260709 13:02:54.021818  8332 raft_consensus.cc:1275] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:53.982406  8326 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:54.002821 11685 raft_consensus.cc:740] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:02:54.002548  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 434251us
I20260709 13:02:54.063773  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:53.986017 11711 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:02:53.985663 11744 log.cc:927] Time spent T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 13:02:53.799376 11681 consensus_queue.cc:1059] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.263s
W20260709 13:02:54.068476 11579 consensus_meta.cc:287] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.638s	user 0.010s	sys 0.000s
I20260709 13:02:53.992908 11704 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:54.101029  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.013602 11734 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.009209 11363 consensus_queue.cc:1059] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 13:02:54.128837  8297 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:54.101943 11362 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:54.067533 11294 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:53.955242 11703 consensus_queue.cc:260] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.094867 11517 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:54.163416  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 162693us
W20260709 13:02:54.139557 11710 consensus_meta.cc:287] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.733s	user 0.010s	sys 0.000s
I20260709 13:02:54.016943 11746 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:54.183027  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 186923us
W20260709 13:02:54.183781  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:54.144011 11687 consensus_meta.cc:287] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.807s	user 0.009s	sys 0.002s
W20260709 13:02:54.185410  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 181995us
W20260709 13:02:54.187153  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.076237 11263 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.106153 11579 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:54.165239 11711 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:54.203270  8250 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:54.212524  9005 raft_consensus.cc:1240] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:54.214994  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.195087 11691 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:54.133960 11734 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:54.220867  8324 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.004s	sys 0.016s
W20260709 13:02:54.222060  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 101255us
I20260709 13:02:54.227388  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:54.229244  8403 raft_consensus.cc:1240] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:54.187155 11517 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.212283 11678 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:54.245750  8250 raft_consensus.cc:2410] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
W20260709 13:02:54.238803  8324 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.004s	sys 0.017s
I20260709 13:02:54.255167 11699 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:54.255716  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:54.256460  8324 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
W20260709 13:02:54.237519  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.217655 11703 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:54.145231 11688 consensus_queue.cc:260] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.237411  8972 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:54.186504 11685 consensus_queue.cc:260] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.260277  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:54.264045  8329 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 10: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:54.221520 11203 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:54.189972 11710 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.238441 11607 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:54.238245 11626 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:54.258210 11732 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.214273 11711 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.299901  8323 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:54.262348 11712 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:54.253118 11691 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:54.308506  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 45620us
W20260709 13:02:54.322320  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:54.323307  8220 leader_election.cc:336] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.890s (SENT)
I20260709 13:02:54.279793 11776 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:54.333659  8403 raft_consensus.cc:1240] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:54.318419 11704 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:54.339560  8297 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:54.339802 11782 consensus_queue.cc:1059] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:02:54.340691 11749 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.340932 11710 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:54.341226 11749 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.341540  8297 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:54.342218 11779 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:54.341666 11558 consensus_meta.cc:287] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.799s	user 0.008s	sys 0.002s
I20260709 13:02:54.326843 11688 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:54.306007 11626 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:54.355818  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 53031.4us
I20260709 13:02:54.301223 11607 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:54.370353  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:54.370872  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:54.371812  9005 raft_consensus.cc:2368] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12.
I20260709 13:02:54.331032 11645 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:02:54.384271 11723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.016s	sys 0.006s
I20260709 13:02:54.371057  9003 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:02:54.341634 11710 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:02:54.387636  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 46184.1us
I20260709 13:02:54.337276 11776 raft_consensus.cc:697] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:54.307847 11732 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.351648 11294 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:54.395912 11723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.016s	sys 0.007s
I20260709 13:02:54.398913 11778 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.488751 11778 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.402487 11746 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:54.405752 11734 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:54.355291 11558 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:54.370337 11704 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:54.418668  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 95359.6us
W20260709 13:02:54.448285 11765 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 13:02:54.449615 11758 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:54.451108 11771 log.cc:927] Time spent T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:02:54.451912 11767 log.cc:927] Time spent T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 13:02:54.452420 11705 raft_consensus.cc:3055] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:54.452322 11756 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:02:54.517970 11776 consensus_queue.cc:237] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.519604 11558 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:54.520133 11631 raft_consensus.cc:3055] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:54.520560 11631 raft_consensus.cc:740] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:54.454308 11654 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:54.455331 11511 raft_consensus.cc:3055] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:02:54.440037 11768 log.cc:927] Time spent T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
W20260709 13:02:54.458329 11276 consensus_meta.cc:287] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.123s	user 0.011s	sys 0.000s
I20260709 13:02:54.521662 11654 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:54.521770 11511 raft_consensus.cc:740] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:54.522040 11276 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.523380 11511 consensus_queue.cc:260] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.511201 11758 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:54.524621 11758 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:54.524823 11511 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:54.458921 11554 consensus_meta.cc:287] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 1.037s	user 0.006s	sys 0.005s
I20260709 13:02:54.395223 11579 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:54.464972 11770 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.464704 11614 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:54.525629 11554 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.525872 11770 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:54.527465 11614 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:54.528359 11770 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:54.528450  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:54.415382 11263 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:54.468120 11588 consensus_meta.cc:287] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.855s	user 0.004s	sys 0.005s
I20260709 13:02:54.468892 11517 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:54.362659 11779 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:54.469559 11777 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:02:54.470677 11757 log.cc:927] Time spent T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 13:02:54.476061  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:54.476468  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:54.514551 11760 consensus_queue.cc:1059] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:02:54.504740 10930 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:54.511289 11704 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.477732  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:54.478602  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:54.479722 11678 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:54.480136 11691 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:54.480417  8405 raft_consensus.cc:1240] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:54.483120 11781 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:54.484592  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:54.486307 11607 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:54.530799 11778 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:54.532271  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1800) took 1105 ms. Trace:
W20260709 13:02:54.486536 11759 log.cc:927] Time spent T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.284s	user 0.002s	sys 0.000s
W20260709 13:02:54.549947 11697 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:02:54.487608 11699 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:54.550551 11699 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:54.532272 11777 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:54.552451 11777 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:54.401321 11626 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:54.489172 11711 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:54.482105 11645 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:54.490585  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:54.532629 11749 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:54.532945 11796 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.554551 11796 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:54.492462  8369 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:54.533047  8399 raft_consensus.cc:2393] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 7: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:54.488162  8323 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
W20260709 13:02:54.499497  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.533166  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:54.413710 11710 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:02:54.507899  8220 leader_election.cc:336] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.899s (SENT)
I20260709 13:02:54.488183 11732 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:54.493566 11723 raft_consensus.cc:697] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:54.514639 11705 raft_consensus.cc:740] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:54.572963 11723 consensus_queue.cc:237] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.529399  9005 raft_consensus.cc:2368] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9.
W20260709 13:02:54.555665  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 63278.1us
I20260709 13:02:54.556322  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:54.538061  8401 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 10: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:54.545890  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:54.546808  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:54.530802  8400 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 12: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:54.575156  8369 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:54.556846  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.578003  8369 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:54.578346  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.578938  9004 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 12: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:54.579151  8369 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:54.562062  8403 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
I20260709 13:02:54.562129  8404 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
W20260709 13:02:54.494899  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 173806us
I20260709 13:02:54.571732  8220 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:54.580878  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:54.572924 11705 consensus_queue.cc:260] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.548944  8256 rpcz_store.cc:276] 0709 13:02:53.426844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.545993 (+119149us) service_pool.cc:224] Handling call
0709 13:02:53.546336 (+   343us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:53.546692 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:53.546722 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:53.547247 (+   525us) log.cc:844] Serialized 52 byte log entry
0709 13:02:53.829712 (+282465us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:53.829769 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.829866 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:54.532019 (+702153us) raft_consensus.cc:1685] finished
0709 13:02:54.532043 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:54.532212 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17090,"raft.queue_time_us":2241,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":14195,"thread_start_us":282012,"threads_started":1,"wal-append.queue_time_us":662537}
W20260709 13:02:54.579546  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.579691  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:54.586086  8398 raft_consensus.cc:2410] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:02:54.592003  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:02:54.560501  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 197333us
I20260709 13:02:54.587380  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:54.593531  8326 raft_consensus.cc:2410] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:02:54.594741  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:54.596100  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:54.555131 11645 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:54.535734 11376 consensus_meta.cc:287] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.963s	user 0.011s	sys 0.000s
I20260709 13:02:54.602983  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:54.575122  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:54.606920  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:54.607209  8255 raft_consensus.cc:1275] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:54.607436  8257 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:02:54.608400  8249 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:54.610806  8331 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:54.615914  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:54.571489 11710 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:54.544531 11704 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:54.624742  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:54.616716  8256 raft_consensus.cc:2368] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12.
I20260709 13:02:54.575661 11711 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:54.618515  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:54.619704  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:54.620239  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:54.620579  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:54.597318 11691 raft_consensus.cc:2764] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:02:54.622242  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:54.528398 11554 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:54.616081  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:54.556989 11276 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:54.531972 11779 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:54.624782  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:54.536023 10930 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:54.640489  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 132596us
I20260709 13:02:54.621253 11645 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:54.626628 11705 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:54.607672 11770 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:54.606386 11777 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:54.659965  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1969) took 1166 ms. Trace:
I20260709 13:02:54.634564  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:54.632138  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:54.630028  8248 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:02:54.644119 11740 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.631455 11678 consensus_queue.cc:1048] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:02:54.675506 10930 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:54.654661  8398 raft_consensus.cc:2368] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9.
I20260709 13:02:54.655014  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:54.676417  8326 raft_consensus.cc:2393] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 6: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:54.677227  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:54.678505  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:54.679006  8326 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:54.616582 11376 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.681687 11777 raft_consensus.cc:697] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 9 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:54.682745 11777 consensus_queue.cc:237] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:54.683739 11376 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:54.685613  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:54.565474 11631 consensus_queue.cc:260] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.547138 11781 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.616773 11749 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:02:54.687464  8403 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:54.672767  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:54.672793  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:54.674328 11740 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:54.529937 11588 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.669538  8330 rpcz_store.cc:276] 0709 13:02:53.493116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.493246 (+   130us) service_pool.cc:224] Handling call
0709 13:02:53.507130 (+ 13884us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:53.531519 (+ 24389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:53.531562 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:54.545444 (+1013882us) spinlock_profiling.cc:243] Waited 2.09 ms on lock 0x7b2c000d4468. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:02:54.545836 (+   392us) log.cc:844] Serialized 26 byte log entry
0709 13:02:54.609717 (+ 63881us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:54.609775 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.609894 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:54.653554 (+ 43660us) raft_consensus.cc:1685] finished
0709 13:02:54.653599 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:54.659900 (+  6301us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13910,"raft.queue_time_us":31459,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":5872,"spinlock_wait_cycles":4809984,"thread_start_us":63483,"threads_started":1,"wal-append.queue_time_us":98259}
I20260709 13:02:54.689213  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:02:54.689707  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.705871  9003 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:54.658306  8295 leader_election.cc:400] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12.
I20260709 13:02:54.671958  8331 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:54.706552  8295 leader_election.cc:403] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:02:54.697161  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:54.653729 11784 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:02:54.706162  8220 leader_election.cc:336] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.160s (SENT)
I20260709 13:02:54.708555  8220 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:54.708673  8369 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:54.700101  8405 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
I20260709 13:02:54.700918  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:54.701561  8326 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
W20260709 13:02:54.707018  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 48906.4us
I20260709 13:02:54.712272  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:54.711429  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:54.713680  8369 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:54.714432  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:54.660445 11779 consensus_queue.cc:1048] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:02:54.695030 11645 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.696074 11770 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.718515  8255 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:02:54.693953 11376 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:54.723595  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:54.717473  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:54.720367  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:54.707868 11631 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:54.725890  8369 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:54.727826  8331 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 9: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:54.715482 11558 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:54.716254 11680 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:54.728654  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:54.728929 11588 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:54.723587 11645 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.729789  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:54.730332  8331 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:54.731074 11781 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:54.731383 11645 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:54.731137  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:54.731961 11704 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:54.701948 11711 consensus_queue.cc:1059] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:02:54.732605  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:54.733023 11711 raft_consensus.cc:3055] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:54.733836  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:54.713253 10930 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12.
I20260709 13:02:54.729141 11740 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:54.735291  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:54.733371 11711 raft_consensus.cc:740] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:54.737370 11796 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:54.737653 11799 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:54.738154 11789 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:54.738544 11801 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:02:54.740028 11766 raft_consensus.cc:3055] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:54.766050  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:54.718393 11678 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.765839  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:54.766857 11678 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.768173 11709 consensus_queue.cc:260] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.769065 11678 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:54.740759  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:54.749680  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 43534.1us
I20260709 13:02:54.747584  8402 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:54.770778  8220 leader_election.cc:336] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.215s (SENT)
I20260709 13:02:54.753660  8399 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:02:54.716107 11784 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.756683 11732 consensus_queue.cc:1059] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:54.754467 11691 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:54.757464 11781 consensus_queue.cc:1048] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:54.759732  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:54.766220 11766 raft_consensus.cc:740] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:54.781395 11799 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:54.790031 11766 consensus_queue.cc:260] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.791455 11766 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:54.791873 11799 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:54.792047  9000 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:02:54.796482 11758 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:54.771173  8220 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:54.744053  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:54.808799  8255 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 12: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:54.751278  8369 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:54.772037  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:54.781035  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:54.782137  8372 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:54.782557  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:54.782845  8298 leader_election.cc:400] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12.
I20260709 13:02:54.787189  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:54.788964 11795 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:54.796566 11710 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.798564 11769 raft_consensus.cc:3055] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:54.725667 11770 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.755483 11704 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:54.767333  8326 raft_consensus.cc:2393] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 6: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:02:54.802919  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.805121  9003 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:02:54.799228 11699 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.806308  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:54.806397  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:54.841238  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 70464.7us
I20260709 13:02:54.841835  9004 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:54.784373 11709 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:54.807229 11784 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:54.811467  8399 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:02:54.812413  8405 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 9: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:54.812449  8298 leader_election.cc:400] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9.
I20260709 13:02:54.741977 11376 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:54.817584  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 66371.9us
I20260709 13:02:54.784934 11645 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:54.833853  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:54.820969 11769 raft_consensus.cc:740] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:54.839598  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:54.854321  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:54.835891  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:54.855362  8255 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:54.842401  8402 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
I20260709 13:02:54.844635  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:54.856820  8223 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:54.845124  8298 leader_election.cc:403] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:02:54.845832  8222 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:54.850606  8254 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:54.835105  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.850987  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:54.858004  8399 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
W20260709 13:02:54.858166  8323 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 420) took 1492 ms (client timeout 1952 ms). Trace:
I20260709 13:02:54.837420 11779 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:54.821130 11796 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:54.856972  9002 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
W20260709 13:02:54.858529  8323 rpcz_store.cc:269] 0709 13:02:53.355158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.355290 (+   132us) service_pool.cc:224] Handling call
0709 13:02:54.847350 (+1492060us) spinlock_profiling.cc:243] Waited 917 ms on lock 0x7b54003c9e54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:54.847441 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2107995520}
I20260709 13:02:54.856647  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:54.821288 11691 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:54.863147  8369 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:54.863745 11745 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:54.895218  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:54.895736 11745 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:54.896162 11745 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:54.896536 11699 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:54.845122 11704 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.898933 11704 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:54.817855 11781 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.867726  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:54.868378  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:54.868966  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:54.908294  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 45227.8us
I20260709 13:02:54.869688 11711 consensus_queue.cc:260] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:54.871141 11770 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:54.871090  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:54.871827  8254 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:54.873927 11363 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:54.875259 11790 consensus_queue.cc:1059] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:54.876219 11362 raft_consensus.cc:3055] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:54.879626 11685 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:54.882470 11787 log.cc:927] Time spent T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.335s	user 0.003s	sys 0.000s
I20260709 13:02:54.884241  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:54.885390  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:54.886826 11758 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:54.909174 11691 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:54.888584 11710 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:54.890333  9006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56948 (request call id 744) took 1500 ms. Trace:
I20260709 13:02:54.910518 11777 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:54.893411 11687 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:54.893627 10930 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:54.894052 11554 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:54.894248  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:54.894438 11558 consensus_queue.cc:1059] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:54.843731 11654 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.895087  8223 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:54.896831  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:54.900386  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:54.960865  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:54.960698  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 65671.5us
W20260709 13:02:54.902560  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 56825.3us
I20260709 13:02:54.866060 11769 consensus_queue.cc:260] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:54.906270 11376 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:54.906579 11730 raft_consensus.cc:3055] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:02:54.906957  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 85246.8us
I20260709 13:02:54.906772  8372 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:54.869516 11748 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:54.908685  8249 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:54.910334  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:54.914157  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:54.914114  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:54.913041  8972 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:54.916059  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:54.968973  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 62305.8us
W20260709 13:02:54.971381  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 58437.2us
I20260709 13:02:54.860634 11779 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:54.976058  8330 raft_consensus.cc:2393] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 11: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:54.918188  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:54.920946  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:54.922838  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:54.978284  8402 raft_consensus.cc:2393] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 9: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:54.909559 11796 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:02:54.888360  8400 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 480) took 1548 ms (client timeout 1952 ms). Trace:
W20260709 13:02:54.890931  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 78600.7us
I20260709 13:02:54.929528  8220 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:54.930794  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:54.980233  8405 raft_consensus.cc:2368] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12.
I20260709 13:02:54.932296  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:54.933084  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:54.964403 11732 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:54.918702 11362 raft_consensus.cc:740] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:55.026095 11362 consensus_queue.cc:260] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:55.027285 11362 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:54.926529  9006 rpcz_store.cc:276] 0709 13:02:53.357093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.365748 (+  8655us) service_pool.cc:224] Handling call
0709 13:02:54.799989 (+1434241us) spinlock_profiling.cc:243] Waited 1.37 ms on lock 0x7b5800175898. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:02:54.857646 (+ 57657us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3154304}
I20260709 13:02:54.958493  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:55.029836 11758 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.030145  8255 raft_consensus.cc:2410] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:02:55.030435 11758 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:55.031142  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:55.032294  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:55.033002  8255 raft_consensus.cc:2393] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 10: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:02:54.963868  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.964730  8331 raft_consensus.cc:2368] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12.
I20260709 13:02:55.034932 11654 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:55.035553  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:54.968679 11730 raft_consensus.cc:740] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:54.927654 11777 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.036971 11730 consensus_queue.cc:260] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:55.037715 11777 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:55.038376 11730 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:54.965924  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:54.915732 11781 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.939363 11554 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.040019 11554 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:54.939248 11711 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:54.979236  8328 raft_consensus.cc:2393] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 7: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:54.981691  8256 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:54.981870  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:55.042007  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:55.042737  8328 raft_consensus.cc:2393] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 8: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:55.042996  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:54.859689 11645 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:55.043566  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:54.979204  8400 rpcz_store.cc:269] 0709 13:02:53.340264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.340429 (+   165us) service_pool.cc:224] Handling call
0709 13:02:54.888178 (+1547749us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54001cbe54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:54.888302 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2650764544}
I20260709 13:02:55.044374  8328 raft_consensus.cc:2393] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 8: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:02:54.983201  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 53806.3us
I20260709 13:02:54.977939 11779 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:54.978081 11766 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:55.053498  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:54.981508 11796 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:54.984908 11740 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:54.985239  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:54.985929  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:54.986753  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:54.984917  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:54.999708 11710 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.004879  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:02:55.015935  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:55.089419  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.927803 11691 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:55.050081  8220 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:55.053421  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:55.095283  8400 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:55.037176  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:55.096002  8331 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:02:55.096154  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:55.097150  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:02:55.063712  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 85384.7us
I20260709 13:02:54.976147 11770 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:55.099484  9001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56940 (request call id 748) took 1714 ms. Trace:
I20260709 13:02:55.046593  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:54.934333 11687 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:54.976567 11769 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:55.101634  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 51661.7us
I20260709 13:02:55.094249  8323 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
W20260709 13:02:55.094760  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.098325  8297 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:54.972172 11784 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:55.102433  8220 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:55.106999  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:55.100845  8328 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:55.099988  9001 rpcz_store.cc:276] 0709 13:02:53.384663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.384846 (+   183us) service_pool.cc:224] Handling call
0709 13:02:55.093485 (+1708639us) spinlock_profiling.cc:243] Waited 35.8 ms on lock 0x7b540057d5d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:55.099407 (+  5922us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":82227968}
I20260709 13:02:55.103259  8297 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:55.108141  8256 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:02:55.108867  8255 raft_consensus.cc:2393] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 9: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:02:55.086547  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 120725us
I20260709 13:02:55.109614  8327 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 7 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:55.109344  8297 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:55.114945  8324 raft_consensus.cc:2368] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12.
I20260709 13:02:55.115339  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:55.024420 11732 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:55.114428  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:55.117905  8297 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:55.120970  8220 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:02:55.122215  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:55.122828  8402 raft_consensus.cc:2368] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9.
I20260709 13:02:55.096925 11376 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:55.118610  8298 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:55.070255 11796 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:55.126760  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:55.128803  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:55.127954  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:55.129806  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:55.129880  8331 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:55.131189  8220 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:55.129051  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:55.132586  8323 raft_consensus.cc:2393] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 9: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:55.133903  8328 raft_consensus.cc:2393] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 8: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:55.134454  8330 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:55.127960  8295 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:55.139065 11630 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.140857  9003 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:55.100313 11766 raft_consensus.cc:697] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:55.107666 11748 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:55.111160 11588 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:55.113464 11795 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.111559 11276 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:55.105096 11745 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:55.119704 11770 raft_consensus.cc:697] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 9 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:55.099777 11710 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:55.113049 11784 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:55.125720 11732 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:55.319465  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:02:55.379271  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.320297  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:55.215363  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:55.298652  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:55.201386  8400 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
I20260709 13:02:55.365761  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:55.175575  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.314071  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:55.164124  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.321127  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:55.203261  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:55.191735  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:55.316637  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:55.126701 11740 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:55.156227  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:55.318163  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:55.431035  8397 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:55.320775  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:55.176390  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:55.432446  8331 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:02:55.195478  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:55.148438  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:55.435154  8220 leader_election.cc:336] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.523s (SENT)
I20260709 13:02:55.420253  8406 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:55.119401 11697 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:55.420457  8248 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:02:55.420656  8252 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 5.
I20260709 13:02:55.174777  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:55.423777  8324 raft_consensus.cc:2393] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 9: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:55.421943  9004 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:02:55.440697  8254 raft_consensus.cc:2393] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 9: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:55.118568 11694 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.424279  8255 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:55.325965 11687 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:02:55.239804 11751 log.cc:927] Time spent T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 1.353s	user 0.000s	sys 0.009s
I20260709 13:02:55.437968  8405 raft_consensus.cc:2393] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 9: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:55.442919  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:55.449026  9004 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:55.141237 11789 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:55.146317 11799 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:02:55.444475  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 316533us
I20260709 13:02:55.139874 11729 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:55.145943 11630 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:55.428817 11763 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:55.320608 10930 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9.
I20260709 13:02:55.208626 11758 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:55.210711 11723 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.213243 11645 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:55.455466  8323 raft_consensus.cc:1240] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:02:55.306066 11781 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:55.457862  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.459558  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:55.457144 11789 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.460044  8329 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:02:55.460201  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:55.539311  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:55.429215 11276 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:55.539714  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:55.419207 11795 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:55.462034  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:55.547513  8397 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:55.466919  8253 raft_consensus.cc:2393] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 6: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:55.469383  8325 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:02:55.474813  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:55.549036  8399 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 10: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:55.477614  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:55.478971  8256 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "ff7a6fe533e94c1bae555b4cae427728"
 from {username='slave'} at 127.0.0.1:50420
I20260709 13:02:55.484485 11745 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:55.550415  8403 raft_consensus.cc:2393] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 7: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:55.550765  8256 raft_consensus.cc:606] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Received request to transfer leadership to TS ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:55.553548  8256 raft_consensus.cc:614] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:55.549881  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 435453us
I20260709 13:02:55.540604  8405 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:55.488222  8326 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:02:55.490512 11554 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:55.494906  8295 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:55.495498  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.499588 11770 consensus_queue.cc:237] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:55.502063 11630 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:55.566815  8251 raft_consensus.cc:1275] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:55.239885 11766 consensus_queue.cc:237] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:55.509902  8250 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1841) took 2104 ms (client timeout 1912 ms). Trace:
W20260709 13:02:55.574707  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:55.512048 11685 consensus_meta.cc:287] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.587s	user 0.009s	sys 0.001s
I20260709 13:02:55.513175  8249 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:55.518812  8972 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:55.547086 11781 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.523255  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:55.580396  8248 raft_consensus.cc:2410] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:02:55.456080 11697 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:55.528638  8400 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:02:55.459739  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:55.446321 11694 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:55.461229 11729 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.530762  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:55.461906  8324 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:02:55.587471  9004 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:55.587687  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:55.536026 11777 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:55.549961  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:55.588799  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1814) took 1003 ms. Trace:
I20260709 13:02:55.589042  8399 raft_consensus.cc:2393] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 8: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:02:55.562785  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.565743  8372 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:02:55.566398  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 71584us
I20260709 13:02:55.581511  9005 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:02:55.502166 11784 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:55.593487  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2177) took 1020 ms. Trace:
I20260709 13:02:55.589186  8248 rpcz_store.cc:276] 0709 13:02:54.585288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.462916 (+877628us) service_pool.cc:224] Handling call
0709 13:02:55.588739 (+125823us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:55.593966  8324 rpcz_store.cc:276] 0709 13:02:54.573381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.443901 (+870520us) service_pool.cc:224] Handling call
0709 13:02:55.461566 (+ 17665us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:55.593105 (+131539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.593414 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:55.591754  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1809) took 1008 ms. Trace:
I20260709 13:02:55.595347  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:55.569540  8323 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:02:55.565948 11376 raft_consensus.cc:697] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:55.573827  8223 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:55.577176  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 58489.5us
I20260709 13:02:55.522490  9002 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:02:55.600385  8972 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:55.574687  8250 rpcz_store.cc:269] 0709 13:02:53.404954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.502132 (+ 97178us) service_pool.cc:224] Handling call
0709 13:02:55.509635 (+2007503us) spinlock_profiling.cc:243] Waited 590 ms on lock 0x7b5400408654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:55.509796 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1357117696}
I20260709 13:02:55.598357  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:55.603137  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:55.543736 11789 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:55.590140  8222 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:55.617225  8222 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:55.529628 11723 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:55.561111  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 443240us
I20260709 13:02:55.633752  8297 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:55.595568  8251 rpcz_store.cc:276] 0709 13:02:54.583751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.590746 (+1006995us) service_pool.cc:224] Handling call
0709 13:02:55.591144 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.591364 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:55.591404 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:55.591462 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:55.591510 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.591585 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:55.591692 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:55.587046 11630 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:55.593559 11729 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:55.551539 11779 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:55.613308  8369 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:55.568459 11740 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.624758 11784 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:55.599195 11705 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:02:55.627804  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.632678  9004 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:02:55.634902  8250 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:02:55.666848  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 486) took 1031 ms. Trace:
I20260709 13:02:55.667353  8223 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:55.625317 11704 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:02:55.635937  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 517313us
I20260709 13:02:55.597107 11799 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.576283 11276 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:55.639842  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:55.643445  8400 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:55.761828  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2206) took 1018 ms. Trace:
I20260709 13:02:55.762244  8400 rpcz_store.cc:276] 0709 13:02:54.743214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.600128 (+856914us) service_pool.cc:224] Handling call
0709 13:02:55.643200 (+ 43072us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.761559 (+118359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.761761 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:55.644546  8324 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:02:55.763520  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2184) took 1113 ms. Trace:
I20260709 13:02:55.763999  8324 rpcz_store.cc:276] 0709 13:02:54.650098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.599682 (+949584us) service_pool.cc:224] Handling call
0709 13:02:55.636340 (+ 36658us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:55.763193 (+126853us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.763451 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:55.645946  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:55.765502  8405 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:55.647430  8297 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:55.597661 11697 raft_consensus.cc:697] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:55.667212  8250 rpcz_store.cc:276] 0709 13:02:54.635264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.601866 (+966602us) service_pool.cc:224] Handling call
0709 13:02:55.602145 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.666507 (+ 64362us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.666773 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:55.600919 11711 consensus_meta.cc:287] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.557s	user 0.009s	sys 0.004s
I20260709 13:02:55.625936 11685 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:55.588517 11781 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:55.756170  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1975) took 2226 ms. Trace:
W20260709 13:02:55.757967  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 144685us
I20260709 13:02:55.637995 11689 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:55.637612 11630 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.774719  8255 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:55.788666  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1852) took 1118 ms. Trace:
I20260709 13:02:55.777920  8327 rpcz_store.cc:276] 0709 13:02:53.529869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.754160 (+224291us) service_pool.cc:224] Handling call
0709 13:02:54.007272 (+253112us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.110155 (+1102883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.755819 (+645664us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540037cb54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:55.756069 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2534271872}
I20260709 13:02:55.663707 11784 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:55.788784  8249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49220 (request call id 488) took 1134 ms. Trace:
I20260709 13:02:55.637209 11758 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.136s
I20260709 13:02:55.755108 11699 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:02:55.761485 11654 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.004s	sys 0.049s
I20260709 13:02:55.791221  8298 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:55.750113 11704 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:02:55.792196  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 144841us
I20260709 13:02:55.805109  8324 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:02:55.818725  8297 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:55.819037  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2185) took 1168 ms. Trace:
I20260709 13:02:55.790423  8249 rpcz_store.cc:276] 0709 13:02:54.653998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.042732 (+388734us) service_pool.cc:224] Handling call
0709 13:02:55.788596 (+745864us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b5400308754. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:55.788724 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1079626496}
I20260709 13:02:55.819377  8324 rpcz_store.cc:276] 0709 13:02:54.650481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.775902 (+1125421us) service_pool.cc:224] Handling call
0709 13:02:55.804788 (+ 28886us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:55.818770 (+ 13982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.818987 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:55.821264  8405 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:02:55.774576 11685 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:55.755522 11705 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:55.770442 11276 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.789247  8255 rpcz_store.cc:276] 0709 13:02:54.670354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.774047 (+1103693us) service_pool.cc:224] Handling call
0709 13:02:55.774385 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.788316 (+ 13931us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.788593 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:55.805991  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1853) took 1119 ms. Trace:
I20260709 13:02:55.809826  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1983) took 1153 ms. Trace:
I20260709 13:02:55.794586  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:55.882253  8255 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:55.881762  8250 rpcz_store.cc:276] 0709 13:02:54.686792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.783148 (+1096356us) service_pool.cc:224] Handling call
0709 13:02:55.805472 (+ 22324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.805635 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:55.805661 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:55.805699 (+    38us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:55.805730 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.805781 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:55.805921 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:55.789271 11679 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.007s	sys 0.037s
W20260709 13:02:55.867584  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 48931.8us
I20260709 13:02:55.890165  8297 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:55.792829 11630 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:55.796690 11691 consensus_meta.cc:287] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.685s	user 0.009s	sys 0.002s
W20260709 13:02:55.808427 11654 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.005s	sys 0.050s
I20260709 13:02:55.751083 11740 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:55.879992  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 88828.7us
I20260709 13:02:55.880630  8324 raft_consensus.cc:1240] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 7. Current term is 8. Ops: [5.3-7.4]
I20260709 13:02:55.883074  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 490) took 1138 ms. Trace:
I20260709 13:02:55.908354  8255 rpcz_store.cc:276] 0709 13:02:54.744843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.881825 (+1136982us) service_pool.cc:224] Handling call
0709 13:02:55.882089 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.882820 (+   731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.883017 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:55.881851  8327 rpcz_store.cc:276] 0709 13:02:54.656720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.790102 (+1133382us) service_pool.cc:224] Handling call
0709 13:02:55.809140 (+ 19038us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.809365 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:55.809408 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:55.809482 (+    74us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:55.809530 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.809606 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:55.809752 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:55.884802  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 90302.6us
I20260709 13:02:55.910409  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:55.793659 11678 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.769241 11799 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:55.795900  8403 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:55.794071 11784 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:55.809409 10930 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:55.810628 11819 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.824994 11704 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:55.806241 11824 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:55.819388 11711 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.901417  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1860) took 1112 ms. Trace:
I20260709 13:02:55.962308  8400 rpcz_store.cc:276] 0709 13:02:54.788502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.808937 (+1020435us) service_pool.cc:224] Handling call
0709 13:02:55.890333 (+ 81396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.900968 (+ 10635us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:55.901011 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:55.901076 (+    65us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:55.901122 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.901205 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:55.901351 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:55.912724  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1858) took 1125 ms. Trace:
I20260709 13:02:55.936892 11648 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:55.937234 11651 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.937461  8327 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:55.937898 11821 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:55.938442  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2187) took 1263 ms. Trace:
I20260709 13:02:55.969802  8324 rpcz_store.cc:276] 0709 13:02:54.674796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.820242 (+1145446us) service_pool.cc:224] Handling call
0709 13:02:55.880369 (+ 60127us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:55.938117 (+ 57748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.938382 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:55.938596 11822 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:55.937467  8250 raft_consensus.cc:1275] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:02:55.893452 11362 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:55.939625 11678 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:55.940043 11681 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:55.943349 11740 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:56.023733  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2175) took 1451 ms. Trace:
I20260709 13:02:55.875036 11685 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:55.963958  8403 rpcz_store.cc:276] 0709 13:02:54.787896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.795332 (+1007436us) service_pool.cc:224] Handling call
0709 13:02:55.795681 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.912463 (+116782us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.912666 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:56.019488  8251 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:02:55.891080 11276 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:55.972846  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:55.980873  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.014180  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2189) took 1338 ms. Trace:
I20260709 13:02:56.036383  8327 rpcz_store.cc:276] 0709 13:02:54.675349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.937007 (+1261658us) service_pool.cc:224] Handling call
0709 13:02:55.937289 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.969000 (+ 31711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.014105 (+ 45105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:56.015210  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 492) took 1240 ms. Trace:
I20260709 13:02:55.939237 11699 raft_consensus.cc:697] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:02:55.952108 11791 log.cc:927] Time spent T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 1.550s	user 0.004s	sys 0.002s
I20260709 13:02:55.802697 11376 consensus_queue.cc:237] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:55.944610  8256 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:56.024155  8325 rpcz_store.cc:276] 0709 13:02:54.572817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.435311 (+862494us) service_pool.cc:224] Handling call
0709 13:02:55.448245 (+ 12934us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:55.548725 (+100480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.023492 (+474767us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b54003c8f54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:56.023678 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48004352}
I20260709 13:02:55.900723 11795 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:55.968261  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 78169.8us
I20260709 13:02:56.046101  8297 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:56.037585  8250 rpcz_store.cc:276] 0709 13:02:54.775348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.883182 (+1107834us) service_pool.cc:224] Handling call
0709 13:02:55.937125 (+ 53943us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.014886 (+ 77761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.015148 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:56.036389  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.042418  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 428) took 1408 ms. Trace:
W20260709 13:02:56.042546  8220 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.010s	sys 0.010s
W20260709 13:02:56.048579  8220 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.010s	sys 0.010s
I20260709 13:02:56.019119 11763 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:56.052639  8297 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:02:56.023876  8974 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.051s	sys 0.043s
I20260709 13:02:56.014693 10930 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:56.053830  8974 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.052s	sys 0.044s
W20260709 13:02:56.054140  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 742974us
I20260709 13:02:56.055078  8298 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:56.088999  8327 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:02:56.041296  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:56.089716  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 432) took 1271 ms. Trace:
W20260709 13:02:56.050853  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 383531us
I20260709 13:02:56.090056  8327 rpcz_store.cc:276] 0709 13:02:54.818499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.037338 (+1218839us) service_pool.cc:224] Handling call
0709 13:02:56.051030 (+ 13692us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.089453 (+ 38423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.089652 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:56.047284  8329 rpcz_store.cc:276] 0709 13:02:54.634061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.538732 (+904671us) service_pool.cc:224] Handling call
0709 13:02:55.539040 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.539243 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:55.539590 (+   347us) log.cc:844] Serialized 14 byte log entry
0709 13:02:56.041994 (+502404us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:56.042053 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:56.042117 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.042206 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.042357 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7236,"prepare.queue_time_us":1152,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"thread_start_us":494618,"threads_started":1}
I20260709 13:02:56.091468  8325 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:56.093597  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:56.094234  9002 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:02:56.041167 11740 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:56.027221 11821 raft_consensus.cc:697] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:55.955441 11691 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:56.048990  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 598199us
W20260709 13:02:56.098052  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.015098 11711 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:55.961843 11819 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:56.045760 11554 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:56.101043  8400 raft_consensus.cc:3055] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:56.112835  8400 raft_consensus.cc:740] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:56.107597  8223 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:56.105646  8327 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:02:56.107400  8220 leader_election.cc:336] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.869s (SENT)
I20260709 13:02:56.021574 11651 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:55.965611 11648 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:56.086447 11777 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:56.115257  8220 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:56.119115 11694 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:56.122071  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1823) took 1017 ms. Trace:
I20260709 13:02:56.114420  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1991) took 1245 ms. Trace:
W20260709 13:02:56.122705  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.037338 11681 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:56.103262 11758 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:56.125300  8400 consensus_queue.cc:260] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:56.135805  8327 rpcz_store.cc:276] 0709 13:02:54.868911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.090778 (+1221867us) service_pool.cc:224] Handling call
0709 13:02:56.105295 (+ 14517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.114179 (+  8884us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.114367 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:56.107033 11679 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:56.135740  8253 rpcz_store.cc:276] 0709 13:02:55.104209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.133476 (+ 29267us) service_pool.cc:224] Handling call
0709 13:02:56.121792 (+988316us) spinlock_profiling.cc:243] Waited 26.9 ms on lock 0x7b5400407754. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:56.121991 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":61984512}
I20260709 13:02:56.021937 11697 consensus_queue.cc:237] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:56.142175  8398 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 10: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:02:56.142307  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.157294 11781 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:56.170344 11821 consensus_queue.cc:237] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:56.144342 11723 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:56.123374 11779 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:56.114090 11820 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:56.167214 11699 consensus_queue.cc:237] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:56.095494 10930 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:56.316181 11740 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:56.320863 11758 raft_consensus.cc:697] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:02:56.311118  8254 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49220 (request call id 491) took 1548 ms (client timeout 1914 ms). Trace:
W20260709 13:02:56.317158  8222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.017s	sys 0.016s
I20260709 13:02:56.317989  8325 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:02:56.289737  8248 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.016s	sys 0.028s
W20260709 13:02:56.362532  8222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.018s	sys 0.016s
W20260709 13:02:56.342398 11829 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.199s	user 0.000s	sys 0.006s
W20260709 13:02:56.363240  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 746003us
W20260709 13:02:56.362761  8248 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.017s	sys 0.028s
I20260709 13:02:56.364733  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1811) took 1780 ms. Trace:
I20260709 13:02:56.315526 11777 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:56.324867 11796 consensus_meta.cc:287] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.184s	user 0.011s	sys 0.007s
W20260709 13:02:56.322984 11748 consensus_meta.cc:287] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.147s	user 0.014s	sys 0.004s
W20260709 13:02:56.314319  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:56.379024  8220 leader_election.cc:336] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.139s (SENT)
I20260709 13:02:56.379427  8220 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:56.362752 11799 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:56.363320  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 433) took 1539 ms. Trace:
I20260709 13:02:56.380456  8325 rpcz_store.cc:276] 0709 13:02:54.823655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.045384 (+1221729us) service_pool.cc:224] Handling call
0709 13:02:56.091347 (+ 45963us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.362962 (+271615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.363250 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:56.353746  8400 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:56.381783 11704 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:56.382848 11758 consensus_queue.cc:237] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:56.383853 11704 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:56.352380 11833 log.cc:927] Time spent T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.208s	user 0.001s	sys 0.006s
I20260709 13:02:56.368224  8253 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:50420
I20260709 13:02:56.365376  8248 rpcz_store.cc:276] 0709 13:02:54.584367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.601300 (+1016933us) service_pool.cc:224] Handling call
0709 13:02:55.633783 (+ 32483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.364311 (+730528us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:56.364355 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:56.364413 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:56.364459 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.364541 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.364672 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":493691,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":5145,"thread_start_us":730133,"threads_started":1}
W20260709 13:02:56.355316 11823 log.cc:927] Time spent T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.219s	user 0.003s	sys 0.003s
I20260709 13:02:56.388103  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 492) took 1550 ms. Trace:
W20260709 13:02:56.348042  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 437645us
I20260709 13:02:56.388468  8398 rpcz_store.cc:276] 0709 13:02:54.837273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.539044 (+701771us) service_pool.cc:224] Handling call
0709 13:02:56.387918 (+848874us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b540020fb54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:56.388049 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1363379072}
I20260709 13:02:56.389024  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:56.376267 11710 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:56.372962 11712 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.354735 11679 raft_consensus.cc:697] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:56.405822  8327 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:02:56.392475  8326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2186) took 1718 ms (client timeout 1819 ms). Trace:
I20260709 13:02:56.358644 11781 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.393652 11681 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:56.394912 11819 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:56.395172  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1863) took 1605 ms. Trace:
I20260709 13:02:56.400837  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:56.409797  9000 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
W20260709 13:02:56.361825  8254 rpcz_store.cc:269] 0709 13:02:54.762456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.911206 (+148750us) service_pool.cc:224] Handling call
0709 13:02:56.153043 (+1241837us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b5400407254. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:56.311049 (+158006us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1064122624}
I20260709 13:02:56.364678  8222 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:56.411176  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 46602.6us
I20260709 13:02:56.412300  8222 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:56.390164 11835 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:56.389595  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:56.393018 11363 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:02:56.418619  8328 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:02:56.416203  8398 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
W20260709 13:02:56.380896 11831 log.cc:927] Time spent T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:02:56.411834  8372 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:56.420357  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:56.406523  8326 rpcz_store.cc:269] 0709 13:02:54.674471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.853627 (+179156us) service_pool.cc:224] Handling call
0709 13:02:56.392301 (+1538674us) spinlock_profiling.cc:243] Waited 564 ms on lock 0x7b54002c9a54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:56.392411 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1297364224}
W20260709 13:02:56.419589  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.420967  8398 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:02:56.420588  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:56.405247 11558 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:56.423230  8399 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.023s	sys 0.014s
I20260709 13:02:56.423625  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:56.423745  8254 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:56.424749  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:56.425366  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 494) took 1456 ms. Trace:
I20260709 13:02:56.425716  8254 rpcz_store.cc:276] 0709 13:02:54.969239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.411111 (+1441872us) service_pool.cc:224] Handling call
0709 13:02:56.423535 (+ 12424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.425155 (+  1620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.425318 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:56.379427 11740 raft_consensus.cc:697] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:56.375845 11705 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:56.405233 11723 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:56.400579 11748 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:56.441200  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:56.422847  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1807) took 1839 ms. Trace:
W20260709 13:02:56.434809  8399 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.024s	sys 0.014s
I20260709 13:02:56.431818  8400 rpcz_store.cc:276] 0709 13:02:54.789386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.043054 (+1253668us) service_pool.cc:224] Handling call
0709 13:02:56.095922 (+ 52868us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.392721 (+296799us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:56.392757 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:56.392797 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:56.392833 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.394967 (+  2134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.395103 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":178,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90}
I20260709 13:02:56.443181  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:56.442260  8398 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:56.437943  8369 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:56.442312  8257 rpcz_store.cc:276] 0709 13:02:54.583147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.497065 (+913918us) service_pool.cc:224] Handling call
0709 13:02:55.497766 (+   701us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.498058 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:55.967081 (+469023us) log.cc:844] Serialized 14 byte log entry
0709 13:02:56.419924 (+452843us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:56.422499 (+  2575us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:56.422564 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.422650 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.422781 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1593,"prepare.queue_time_us":627787,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":10376,"thread_start_us":917329,"threads_started":2}
I20260709 13:02:56.418251 11710 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:56.418823 11730 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:56.410200 11799 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:56.454061  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:56.379599 11777 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:56.461251  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1865) took 1349 ms. Trace:
I20260709 13:02:56.422739 11681 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.379221 11796 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:56.413296 11712 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:56.461891  8254 rpcz_store.cc:276] 0709 13:02:55.112018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.437232 (+1325214us) service_pool.cc:224] Handling call
0709 13:02:56.460701 (+ 23469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.460908 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:56.460934 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:56.460976 (+    42us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:56.461007 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.461061 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.461181 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:56.471873  8254 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:56.436625 11723 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.468418  8257 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:02:56.427871 11704 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:56.444972 11691 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:56.477154  8221 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.006s	sys 0.026s
W20260709 13:02:56.501516  8221 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.011s	sys 0.027s
I20260709 13:02:56.505878  9003 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:02:56.507890  8372 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:56.494561  8332 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.016s	sys 0.013s
I20260709 13:02:56.447773 11651 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:56.493320  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 495) took 1353 ms. Trace:
I20260709 13:02:56.464529  8369 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:02:56.482888  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.428092 11825 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:56.418403 11781 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:56.472656  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 496) took 1219 ms. Trace:
I20260709 13:02:56.553627  8325 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:02:56.530786  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 478135us
I20260709 13:02:56.559341  8254 rpcz_store.cc:276] 0709 13:02:55.253830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.471060 (+1217230us) service_pool.cc:224] Handling call
0709 13:02:56.471456 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.472342 (+   886us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.472587 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:56.561316  8297 leader_election.cc:400] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12.
W20260709 13:02:56.555655  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 91242.1us
I20260709 13:02:56.563624  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:56.563858  8297 leader_election.cc:400] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9.
W20260709 13:02:56.557855  8220 leader_election.cc:336] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.641s (SENT)
I20260709 13:02:56.564944  8220 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:56.551002  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 458482us
I20260709 13:02:56.474241 11558 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:02:56.562067  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:56.580936  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.579368  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:56.556556  8257 rpcz_store.cc:276] 0709 13:02:55.140360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.446331 (+1305971us) service_pool.cc:224] Handling call
0709 13:02:56.467905 (+ 21574us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.492954 (+ 25049us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.493245 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:56.462368 11678 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:56.466416 11648 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:56.579909  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:56.494988 11704 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:56.566447  8972 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:56.529589 11730 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.527204 11588 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:56.550992 11819 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:56.586503  8220 leader_election.cc:336] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.819s (SENT)
I20260709 13:02:56.529779 11723 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:56.468050 11710 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:56.564971  8297 leader_election.cc:400] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9.
I20260709 13:02:56.590951  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:56.549913 11779 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.591459  9008 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:02:56.580559 11558 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.597506  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:56.603180  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 495566us
I20260709 13:02:56.554720 11711 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:56.606422  8257 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:02:56.604213  8220 leader_election.cc:336] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.913s (SENT)
I20260709 13:02:56.474615 11681 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:56.570889 11691 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:56.624187  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1867) took 1059 ms. Trace:
I20260709 13:02:56.626142  8257 rpcz_store.cc:276] 0709 13:02:55.564673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.582098 (+1017425us) service_pool.cc:224] Handling call
0709 13:02:56.598198 (+ 16100us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.623899 (+ 25701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.624124 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:56.610606  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 45757.3us
I20260709 13:02:56.582672  8369 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:56.579365 11705 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:02:56.631225  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 48659.5us
I20260709 13:02:56.624717  8220 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:56.618403  9009 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:56.630942 11705 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:56.632081  8369 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:56.630293  8297 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:02:56.588699 11835 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:56.634326  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:56.636739  8297 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:56.601912 11704 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:56.578917 11651 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:02:56.597682 11839 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:56.610334 11732 consensus_meta.cc:287] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.168s	user 0.011s	sys 0.006s
W20260709 13:02:56.619537 11789 consensus_meta.cc:287] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.974s	user 0.012s	sys 0.002s
I20260709 13:02:56.643958  9006 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:02:56.646663  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:56.648134  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 43953.8us
I20260709 13:02:56.651355  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1816) took 1874 ms. Trace:
I20260709 13:02:56.621187 11558 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:56.658336  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:56.659276 11784 consensus_meta.cc:287] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.682s	user 0.008s	sys 0.000s
I20260709 13:02:56.659731 11784 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:56.660708 11685 consensus_meta.cc:287] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.612s	user 0.007s	sys 0.000s
I20260709 13:02:56.655575  9001 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:02:56.633760 11705 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:56.652163 11836 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:56.654129  8297 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:02:56.662397 11844 log.cc:927] Time spent T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:02:56.664657  8220 leader_election.cc:336] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.008s (SENT)
I20260709 13:02:56.693583  8220 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:56.664753 11704 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:56.580106 11679 consensus_queue.cc:237] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:56.643456 11711 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:56.717906 11711 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:56.718461 11711 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:56.720309 11681 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:56.721607  8400 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
W20260709 13:02:56.722729 11630 consensus_meta.cc:287] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.767s	user 0.010s	sys 0.000s
I20260709 13:02:56.723259 11630 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:56.724376 11704 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:56.724627  9000 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 10: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:56.725015 11704 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.725576 11704 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:56.726177 11777 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:56.726541 11630 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:56.726822 11777 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:56.727541 11777 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:56.727916 11777 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.728210 11704 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:56.729228 11841 log.cc:927] Time spent T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:02:56.696471 11685 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:56.730556  9004 raft_consensus.cc:1240] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:56.731179 11710 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:56.731875 11710 consensus_queue.cc:1059] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:56.732426 11685 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:56.732815 11710 consensus_queue.cc:1243] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:02:56.736900 11276 consensus_meta.cc:287] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.688s	user 0.006s	sys 0.003s
W20260709 13:02:56.715855 11729 consensus_meta.cc:287] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.955s	user 0.012s	sys 0.000s
I20260709 13:02:56.665088 11558 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:56.655452  8255 rpcz_store.cc:276] 0709 13:02:54.777108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.909357 (+1132249us) service_pool.cc:224] Handling call
0709 13:02:55.911498 (+  2141us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.911755 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:56.325229 (+413474us) log.cc:844] Serialized 14 byte log entry
0709 13:02:56.650634 (+325405us) log.cc:844] Serialized 14 byte log entry
0709 13:02:56.650918 (+   284us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:56.650999 (+    81us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:56.651064 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.651150 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.651280 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36441,"prepare.queue_time_us":293526,"prepare.run_cpu_time_us":5580,"prepare.run_wall_time_us":142348,"raft.queue_time_us":35790,"thread_start_us":670855,"threads_started":2}
I20260709 13:02:56.608203 11588 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:56.610584 11730 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:56.708248 11828 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.332s	user 0.001s	sys 0.000s
I20260709 13:02:56.469038 11745 consensus_queue.cc:1059] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.210s
I20260709 13:02:56.690476 10930 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:56.620477 11779 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:56.611882 11740 consensus_queue.cc:237] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:56.746145  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:56.749543 10930 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:56.689822 11651 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.762367 10930 raft_consensus.cc:3055] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:56.767511 10930 raft_consensus.cc:740] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:02:56.768076  8330 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.005s	sys 0.046s
I20260709 13:02:56.754197  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:56.612403 11819 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:56.758723  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:56.746645 11712 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:56.754673  8974 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:56.737399 11276 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:56.768522 10930 consensus_queue.cc:260] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:56.763268  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 98625.2us
I20260709 13:02:56.770582  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:56.770848  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:56.711377 11705 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:02:56.778684  8330 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.005s	sys 0.046s
I20260709 13:02:56.783694  9008 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
W20260709 13:02:56.795631  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.728299 11777 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:56.744216 11763 consensus_meta.cc:287] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.657s	user 0.006s	sys 0.002s
I20260709 13:02:56.745982 11820 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:56.748739 11730 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:56.739720 11784 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:56.746014 11846 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.666955 11839 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:56.749271 11847 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:56.672655 11789 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:56.748111 11781 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:56.747162 11796 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:56.807785  8324 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:02:56.751282 11779 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:56.666725 11732 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:56.808712  8372 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:56.755038 11848 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:56.804348 10930 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:56.738133 11729 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:56.804841  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1977) took 2564 ms. Trace:
I20260709 13:02:56.793859 11626 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:56.774103 11748 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:56.800606 11362 consensus_meta.cc:287] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.772s	user 0.007s	sys 0.003s
I20260709 13:02:56.840667 11362 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:56.842048  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:56.842679  8403 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:02:56.843204  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:56.843628  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:56.844321  8403 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 12: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:56.844352  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:56.751498 11558 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:56.827509  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.846916 11732 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:56.848050  8403 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:02:56.848752  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 482) took 2034 ms. Trace:
I20260709 13:02:56.849088  8403 rpcz_store.cc:276] 0709 13:02:54.814131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.845093 (+2030962us) service_pool.cc:224] Handling call
0709 13:02:56.845369 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.848483 (+  3114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.848695 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:56.851188 11839 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:56.774741 11651 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:56.852169  8403 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:02:56.852836  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 484) took 2038 ms. Trace:
I20260709 13:02:56.852458 11711 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:56.853717 11651 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:56.804519 11763 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:56.856436  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.853188  8403 rpcz_store.cc:276] 0709 13:02:54.814755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.851615 (+2036860us) service_pool.cc:224] Handling call
0709 13:02:56.851946 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:56.852585 (+   639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.852783 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:56.859284  8403 raft_consensus.cc:1240] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:56.860065  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1992) took 1990 ms. Trace:
I20260709 13:02:56.860385  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:56.861032  9007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 624) took 3518 ms. Trace:
I20260709 13:02:56.861268  8255 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
I20260709 13:02:56.862660  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1817) took 2080 ms. Trace:
I20260709 13:02:56.861610  9007 rpcz_store.cc:276] 0709 13:02:53.342852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.365656 (+ 22804us) service_pool.cc:224] Handling call
0709 13:02:53.526819 (+161163us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b5800175898. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:02:55.096707 (+1569888us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:55.312370 (+215663us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:55.312415 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:55.312921 (+   506us) log.cc:844] Serialized 26 byte log entry
0709 13:02:55.789473 (+476552us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:55.789515 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.387477 (+597962us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b540057d5d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:56.387524 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:56.749052 (+361528us) raft_consensus.cc:1685] finished
0709 13:02:56.749072 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.860670 (+111598us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b540057d5d0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:56.860778 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8067,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":98,"spinlock_wait_cycles":4101519360,"thread_start_us":476114,"threads_started":1,"wal-append.queue_time_us":836499}
I20260709 13:02:56.863065  8250 rpcz_store.cc:276] 0709 13:02:54.782111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.053299 (+1271188us) service_pool.cc:224] Handling call
0709 13:02:56.098893 (+ 45594us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:02:56.108292 (+  9399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:56.108323 (+    31us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:02:56.115838 (+  7515us) log.cc:844] Serialized 78 byte log entry
0709 13:02:56.607805 (+491967us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:56.607855 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.607944 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:56.862433 (+254489us) raft_consensus.cc:1685] finished
0709 13:02:56.862455 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.862602 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":260576,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":9554,"thread_start_us":491593,"threads_started":1,"wal-append.queue_time_us":678896}
I20260709 13:02:56.846927 11825 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:56.864585  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 430) took 2108 ms. Trace:
I20260709 13:02:56.864984 11825 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:56.866034  8404 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:56.866258 11789 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:56.866212  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 434) took 1406 ms. Trace:
I20260709 13:02:56.867089  8405 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:56.867004  8327 rpcz_store.cc:276] 0709 13:02:55.459406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.353994 (+894588us) service_pool.cc:224] Handling call
0709 13:02:56.369860 (+ 15866us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.406181 (+ 36321us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.865950 (+459769us) spinlock_profiling.cc:243] Waited 31.2 ms on lock 0x7b540014ffd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:56.866147 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71861504}
I20260709 13:02:56.869105  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1825) took 1238 ms. Trace:
I20260709 13:02:56.865025  8324 rpcz_store.cc:276] 0709 13:02:54.756110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.970679 (+1214569us) service_pool.cc:224] Handling call
0709 13:02:56.027696 (+ 57017us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:56.818081 (+790385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.864334 (+ 46253us) spinlock_profiling.cc:243] Waited 780 ms on lock 0x7b540037d054. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:56.864528 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1793310080}
I20260709 13:02:56.883028  8254 rpcz_store.cc:276] 0709 13:02:55.630853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.583673 (+952820us) service_pool.cc:224] Handling call
0709 13:02:56.584039 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.584289 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:56.628792 (+ 44503us) log.cc:844] Serialized 14 byte log entry
0709 13:02:56.868543 (+239751us) log.cc:844] Serialized 14 byte log entry
0709 13:02:56.868765 (+   222us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:56.868817 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:56.868866 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.868933 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.869049 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16421,"prepare.queue_time_us":47080,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":76,"raft.queue_time_us":19887,"thread_start_us":190519,"threads_started":1}
I20260709 13:02:56.836273  8332 rpcz_store.cc:276] 0709 13:02:54.235684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.235828 (+   144us) service_pool.cc:224] Handling call
0709 13:02:54.254335 (+ 18507us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:54.302677 (+ 48342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:54.302734 (+    57us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:54.319283 (+ 16549us) log.cc:844] Serialized 52 byte log entry
0709 13:02:55.456564 (+1137281us) spinlock_profiling.cc:243] Waited 1.73 ms on lock 0x7b1800392800. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e0d1 00007f8828779a0f
0709 13:02:55.873752 (+417188us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:55.989479 (+115727us) log.cc:844] Serialized 14 byte log entry
0709 13:02:56.316951 (+327472us) spinlock_profiling.cc:243] Waited 5.49 ms on lock 0x7b1800392800. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:02:56.527000 (+210049us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.799711 (+272711us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b540029aed4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:56.799755 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:56.799796 (+    41us) raft_consensus.cc:1685] finished
0709 13:02:56.799809 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.799943 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29015,"prepare.queue_time_us":140331,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":53145,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":18858,"spinlock_wait_cycles":70263424,"thread_start_us":954077,"threads_started":2,"wal-append.queue_time_us":1795317}
I20260709 13:02:56.801206  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1859) took 2012 ms. Trace:
I20260709 13:02:56.867650  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1851) took 2164 ms. Trace:
I20260709 13:02:56.745923 11852 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:02:56.822012 11848 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.815402 11846 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:56.860325  9008 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:56.860033  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 485) took 2045 ms. Trace:
I20260709 13:02:56.801112 11685 raft_consensus.cc:2764] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:02:56.888417  8223 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:56.876338  8298 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:56.835455  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 181370us
I20260709 13:02:56.814260 11820 raft_consensus.cc:697] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:56.890552  8406 rpcz_store.cc:276] 0709 13:02:54.788218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.794461 (+1006243us) service_pool.cc:224] Handling call
0709 13:02:55.794905 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:55.805284 (+ 10379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:55.805322 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:55.805814 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 13:02:56.332481 (+526667us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:56.799999 (+467518us) log.cc:844] Serialized 14 byte log entry
0709 13:02:56.800188 (+   189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.800267 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:56.800310 (+    43us) raft_consensus.cc:1685] finished
0709 13:02:56.800323 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.800459 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":275955,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":19609,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":16004,"thread_start_us":870092,"threads_started":2,"wal-append.queue_time_us":730210}
I20260709 13:02:56.899113  8403 rpcz_store.cc:276] 0709 13:02:54.815136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.858795 (+2043659us) service_pool.cc:224] Handling call
0709 13:02:56.859107 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.859771 (+   664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.859976 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:56.890204  8295 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:56.860546  8329 rpcz_store.cc:276] 0709 13:02:54.869231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.091304 (+1222073us) service_pool.cc:224] Handling call
0709 13:02:56.108523 (+ 17219us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:02:56.110783 (+  2260us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:56.135713 (+ 24930us) log.cc:844] Serialized 14 byte log entry
0709 13:02:56.635731 (+500018us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:02:56.647757 (+ 12026us) log.cc:844] Serialized 78 byte log entry
0709 13:02:56.647891 (+   134us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:56.647921 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.648098 (+   177us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:56.859819 (+211721us) raft_consensus.cc:1685] finished
0709 13:02:56.859839 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.860000 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11013,"prepare.queue_time_us":31410,"prepare.run_cpu_time_us":5905,"prepare.run_wall_time_us":154228,"raft.queue_time_us":258668,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":45368,"thread_start_us":301799,"threads_started":1,"wal-append.queue_time_us":602142}
I20260709 13:02:56.902076  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:56.998298  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:56.998724  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2203) took 2159 ms. Trace:
I20260709 13:02:56.999351  8250 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
W20260709 13:02:56.972424  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 340380us
I20260709 13:02:56.886868  8401 rpcz_store.cc:276] 0709 13:02:54.703070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.704383 (+  1313us) service_pool.cc:224] Handling call
0709 13:02:54.720599 (+ 16216us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:55.045458 (+324859us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:55.045492 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:55.075963 (+ 30471us) log.cc:844] Serialized 26 byte log entry
0709 13:02:55.626046 (+550083us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:56.126803 (+500757us) log.cc:844] Serialized 14 byte log entry
0709 13:02:56.127054 (+   251us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.867221 (+740167us) spinlock_profiling.cc:243] Waited 324 ms on lock 0x7b54001dfe54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:56.867272 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:56.867322 (+    50us) raft_consensus.cc:1685] finished
0709 13:02:56.867430 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.867572 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":842171,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":2779,"raft.queue_time_us":85511,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":8563,"spinlock_wait_cycles":746080128,"thread_start_us":856247,"threads_started":2,"wal-append.queue_time_us":981817}
I20260709 13:02:56.998947  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:57.001662 11691 raft_consensus.cc:697] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:57.010896 11691 consensus_queue.cc:237] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:56.915182  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.001907  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:02:56.964267  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:57.020340  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 130232us
I20260709 13:02:57.000986  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:56.931707  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:02:57.023546  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 135195us
I20260709 13:02:57.025799  8297 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:56.902558  8298 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:56.931289  8253 raft_consensus.cc:606] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:02:57.027675  8253 raft_consensus.cc:614] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:57.028258  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:02:56.919232  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.018998  8255 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:02:56.917490 10930 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:57.031313 10930 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.031721 10930 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:57.032399  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.955060  8974 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:57.034561  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 79597.4us
I20260709 13:02:57.035247  8974 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:56.994985  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 395509us
I20260709 13:02:56.836719 11779 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:57.041951  8249 raft_consensus.cc:1240] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:56.897984 11848 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:57.066007 11777 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:57.067736  8400 raft_consensus.cc:1240] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:56.955386 11697 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.004s	sys 0.012s
I20260709 13:02:57.072448  8222 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:57.073760 10930 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:57.081390  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 178940us
I20260709 13:02:56.930401 11795 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:57.082657 10930 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:57.091023  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:02:57.098362  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 72644.6us
I20260709 13:02:57.100181  8297 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:57.044126 11745 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:57.106492 10930 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:57.050272 11779 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.109688  9007 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
W20260709 13:02:57.077034 11697 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.005s	sys 0.018s
I20260709 13:02:57.109408  9004 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:02:57.043709 11763 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:57.121852  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:57.122251  8251 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49220 (request call id 498) took 1514 ms (client timeout 1531 ms). Trace:
W20260709 13:02:57.122702  8251 rpcz_store.cc:269] 0709 13:02:55.607861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.639481 (+ 31620us) service_pool.cc:224] Handling call
0709 13:02:57.122074 (+1482593us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b5400408154. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:57.122196 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":593775488}
I20260709 13:02:57.112694  8297 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:57.068670 11651 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:57.124007  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:57.074584 11648 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:57.101336 11712 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:57.150511  8323 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:57.135351 10930 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:57.136904  8295 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:57.138707  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:57.136312  9001 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:02:57.121618 11763 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.096939 11796 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:57.119973 11779 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:57.083105 11766 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:02:57.069178 11363 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:02:57.161854  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:57.070766 11748 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:02:57.117980 11866 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:57.119658 11745 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:57.129479 11822 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.030s	user 0.000s	sys 0.019s
W20260709 13:02:57.172716  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:57.176002  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.185767  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:57.128022 11651 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.187443  8251 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:57.097545 11630 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:02:57.190205 10930 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:57.190863  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 54041.2us
I20260709 13:02:57.207991  9001 raft_consensus.cc:3055] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:57.163753 11796 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.163515 11685 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:02:57.203624 11723 consensus_meta.cc:287] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.585s	user 0.006s	sys 0.006s
W20260709 13:02:57.206486  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 93784.7us
I20260709 13:02:57.211223  9001 raft_consensus.cc:740] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:02:57.207269 11822 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.119s	user 0.001s	sys 0.019s
I20260709 13:02:57.143254  8255 raft_consensus.cc:2955] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Committing config change with OpId 6.6: config changed from index 3 to 6, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:57.160423 11712 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.163461 11763 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:57.209446 11824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.158s	user 0.015s	sys 0.032s
I20260709 13:02:57.228118  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:57.269889  8327 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:02:57.210317 11651 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:57.229897  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.230919  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:57.231832  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:57.232733  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:57.209201  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:57.160130 11648 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:57.277603  8251 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:57.208997 11694 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:57.238166 11796 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:57.201974 11745 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:57.336440  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1868) took 2545 ms. Trace:
I20260709 13:02:57.274605  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:57.278682  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:57.199594 11681 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:02:57.207840 11866 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:57.191372 11678 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:57.239444  9001 consensus_queue.cc:260] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:57.396178  8398 rpcz_store.cc:276] 0709 13:02:54.790679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.454028 (+1663349us) service_pool.cc:224] Handling call
0709 13:02:56.473277 (+ 19249us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:56.475061 (+  1784us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:56.475106 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:56.475701 (+   595us) log.cc:844] Serialized 26 byte log entry
0709 13:02:56.892772 (+417071us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:56.930563 (+ 37791us) log.cc:844] Serialized 14 byte log entry
0709 13:02:56.930904 (+   341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.931010 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:57.303105 (+372095us) spinlock_profiling.cc:243] Waited 3.16 ms on lock 0x7b1800309d00. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:02:57.336083 (+ 32978us) raft_consensus.cc:1685] finished
0709 13:02:57.336202 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:57.336366 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":37221,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":14393,"raft.queue_time_us":978,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":72875,"spinlock_wait_cycles":7269760,"thread_start_us":313614,"threads_started":1,"wal-append.queue_time_us":806379}
I20260709 13:02:57.323426  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 487) took 2627 ms. Trace:
W20260709 13:02:57.296366 11854 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 13:02:57.326933  8372 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:57.260555 11822 raft_consensus.cc:697] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:57.306604  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 431) took 2533 ms. Trace:
I20260709 13:02:57.299921  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 483) took 2485 ms. Trace:
I20260709 13:02:57.411981  8400 rpcz_store.cc:276] 0709 13:02:54.814463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.845392 (+2030929us) service_pool.cc:224] Handling call
0709 13:02:56.872076 (+ 26684us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:57.068206 (+196130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.299662 (+231456us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b54003e8354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:57.299856 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":449321728}
I20260709 13:02:57.330612  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 1996) took 1772 ms. Trace:
I20260709 13:02:57.300499  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:57.305733  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1862) took 2516 ms. Trace:
I20260709 13:02:57.411878  8330 rpcz_store.cc:276] 0709 13:02:54.772721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.022593 (+1249872us) service_pool.cc:224] Handling call
0709 13:02:56.044303 (+ 21710us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:56.051602 (+  7299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:56.051634 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:56.089479 (+ 37845us) log.cc:844] Serialized 26 byte log entry
0709 13:02:56.793405 (+703926us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:56.793474 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.306192 (+512718us) spinlock_profiling.cc:243] Waited 4.84 ms on lock 0x7b540014cb54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:57.306250 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:57.306298 (+    48us) raft_consensus.cc:1685] finished
0709 13:02:57.306312 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:57.306542 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2033,"raft.queue_time_us":2533,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":16255,"spinlock_wait_cycles":11136768,"thread_start_us":679859,"threads_started":1,"wal-append.queue_time_us":714358}
I20260709 13:02:57.330076  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2210) took 2585 ms. Trace:
I20260709 13:02:57.300717  8249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1855) took 2605 ms. Trace:
I20260709 13:02:57.309468  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:50420 (request call id 20) took 1216 ms. Trace:
I20260709 13:02:57.387499 11850 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.393908 11694 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.243691  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:57.298416 11819 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:57.399721  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:57.390509 11824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.342s	user 0.015s	sys 0.032s
I20260709 13:02:57.317283  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:57.429417  8403 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:57.406003 11882 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.403259  8256 rpcz_store.cc:276] 0709 13:02:54.640588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.604460 (+963872us) service_pool.cc:224] Handling call
0709 13:02:55.604852 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.044986 (+440134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.267325 (+1222339us) spinlock_profiling.cc:243] Waited 339 ms on lock 0x7b5400075dd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:57.267517 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":780642560}
I20260709 13:02:57.432631  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:57.281610 11870 consensus_queue.cc:1059] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.139s
I20260709 13:02:57.325964 11689 consensus_queue.cc:1059] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.098s
I20260709 13:02:57.266826 11712 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:57.437811 11870 raft_consensus.cc:3055] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:57.438243 11870 raft_consensus.cc:740] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:57.439090 11870 consensus_queue.cc:260] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:57.440299 11870 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:57.444649 11651 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:57.280580 11860 log.cc:927] Time spent T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:02:57.408736  9001 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:57.409135  8251 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
W20260709 13:02:57.409264  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 81641.9us
I20260709 13:02:57.327914  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:57.461849  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:57.415266  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:57.415419  8253 rpcz_store.cc:276] 0709 13:02:56.092690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.353067 (+260377us) service_pool.cc:224] Handling call
0709 13:02:57.309234 (+956167us) spinlock_profiling.cc:243] Waited 546 ms on lock 0x7b54003471d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287b8853 00007f882d78660a 00007f8826d0c7d4 00007f8826d0c762 00007f8826d0c691 00007f8826d0c60c 00007f8826d0b8c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:57.309352 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1256534272}
I20260709 13:02:57.301800 11276 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:57.288808  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:57.296764 11697 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.465965 11697 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:57.468364 11697 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:57.415284  8249 rpcz_store.cc:276] 0709 13:02:54.694897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.819673 (+1124776us) service_pool.cc:224] Handling call
0709 13:02:55.866535 (+ 46862us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:57.046082 (+1179547us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.300466 (+254384us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54003471d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:57.300636 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2702361856}
W20260709 13:02:57.469159 11858 log.cc:927] Time spent T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.308s	user 0.000s	sys 0.000s
I20260709 13:02:57.330143 11729 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:57.331151 11711 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:57.417495  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:57.470490  8398 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 10: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:57.471657  8398 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:57.472488  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 487) took 2652 ms. Trace:
W20260709 13:02:57.472635 11862 log.cc:927] Time spent T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.308s	user 0.000s	sys 0.000s
I20260709 13:02:57.304970  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:57.414106  8405 rpcz_store.cc:276] 0709 13:02:54.789133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.822508 (+1033375us) service_pool.cc:224] Handling call
0709 13:02:55.822869 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.867663 (+1044794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.305498 (+437835us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54000fdcd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:57.305675 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2400724352}
I20260709 13:02:57.474377 11836 raft_consensus.cc:3055] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:57.474771  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:57.413702  8325 rpcz_store.cc:276] 0709 13:02:55.558249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.388147 (+829898us) service_pool.cc:224] Handling call
0709 13:02:56.388418 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.559480 (+171062us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.330336 (+770856us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b540020ec54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:57.330529 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":378812160}
I20260709 13:02:57.477001  8254 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
W20260709 13:02:57.477984 11865 log.cc:927] Time spent T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.285s	user 0.003s	sys 0.000s
I20260709 13:02:57.415545 11678 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:57.300159 11777 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:57.325702 11363 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:57.332762 11748 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:57.298349 11626 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:57.416952  8404 rpcz_store.cc:276] 0709 13:02:54.744182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.647895 (+903713us) service_pool.cc:224] Handling call
0709 13:02:55.762417 (+114522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.866498 (+1104081us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.329760 (+463262us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54003160d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:57.329981 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2537357056}
I20260709 13:02:57.304473 11648 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:57.279549  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.496908  8223 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:57.425376  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 333816us
W20260709 13:02:57.426141  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.264181 11723 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:57.337782  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:57.333333 11872 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.414345 11790 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:57.431339  8400 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:57.565232  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 486) took 2745 ms. Trace:
I20260709 13:02:57.498091  8223 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:57.565683  8400 rpcz_store.cc:276] 0709 13:02:54.819715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.417157 (+2597442us) service_pool.cc:224] Handling call
0709 13:02:57.431167 (+ 14010us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:57.564965 (+133798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.565175 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:57.566385  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 68517.1us
I20260709 13:02:57.433851  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:57.286206  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:02:57.424134 11825 consensus_meta.cc:287] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.559s	user 0.010s	sys 0.000s
I20260709 13:02:57.190488 11820 consensus_queue.cc:237] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:57.570039  8400 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:57.569967 11825 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:57.570732  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 490) took 2735 ms. Trace:
I20260709 13:02:57.571045  8400 rpcz_store.cc:276] 0709 13:02:54.835400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.566512 (+2731112us) service_pool.cc:224] Handling call
0709 13:02:57.569878 (+  3366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:57.570479 (+   601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.570659 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:57.572059 11825 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:57.572196 11769 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:57.572706 11769 raft_consensus.cc:697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:57.572834 11883 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:57.573642 11769 consensus_queue.cc:237] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:57.574647 11790 raft_consensus.cc:697] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 9 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:57.575519 11790 consensus_queue.cc:237] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:57.577673 11872 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:57.564638 11748 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:57.579882 11872 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:57.580394 11748 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:57.581470 11876 raft_consensus.cc:3055] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:57.581859 11876 raft_consensus.cc:740] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:57.582605 11876 consensus_queue.cc:260] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:57.583689 11876 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:57.584525 11763 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:57.454479  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:57.458359  8403 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:02:57.479943  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 429) took 2836 ms. Trace:
I20260709 13:02:57.297117 11846 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:57.481282  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1863) took 2519 ms. Trace:
I20260709 13:02:57.589362  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:57.591679 11852 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.462783  8330 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:02:57.592263 11852 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:57.593206 11867 log.cc:927] Time spent T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.401s	user 0.001s	sys 0.000s
I20260709 13:02:57.594475 11852 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:57.595183  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:57.595759  9007 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 6.
I20260709 13:02:57.595799  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:57.596403  8254 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:57.327185 11868 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.119s
I20260709 13:02:57.264627 11864 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.209s
I20260709 13:02:57.429220 11824 raft_consensus.cc:697] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:02:57.486158  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 388383us
I20260709 13:02:57.474817 11836 raft_consensus.cc:740] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:57.472947  8398 rpcz_store.cc:276] 0709 13:02:54.820024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.471100 (+2651076us) service_pool.cc:224] Handling call
0709 13:02:57.471473 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:57.472135 (+   662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.472416 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:57.477043  8405 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:57.611099  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 488) took 2790 ms. Trace:
I20260709 13:02:57.611440  8405 rpcz_store.cc:276] 0709 13:02:54.820312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.476311 (+2655999us) service_pool.cc:224] Handling call
0709 13:02:57.476719 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:57.610849 (+134130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.611047 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:57.487659  9004 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:57.471652 11861 log.cc:927] Time spent T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.274s	user 0.000s	sys 0.000s
I20260709 13:02:57.491600  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:57.423383 11850 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:57.558543  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.424312 11694 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:57.629869  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.590000  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 493) took 2765 ms. Trace:
I20260709 13:02:57.590823  8404 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 7. Current term is 8. Ops: []
W20260709 13:02:57.609792 11818 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.510s	user 0.001s	sys 0.004s
I20260709 13:02:57.447614 11882 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:57.603318  8326 raft_consensus.cc:2393] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 9: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:57.494621 11626 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:57.497996 11729 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.605558  8298 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:57.609189  8254 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:02:57.612532  8256 raft_consensus.cc:2368] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11.
W20260709 13:02:57.754448  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 148944us
I20260709 13:02:57.564352 11835 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:57.469928 11712 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:57.614533  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:57.591539  8252 rpcz_store.cc:276] 0709 13:02:54.962650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.405842 (+1443192us) service_pool.cc:224] Handling call
0709 13:02:56.410713 (+  4871us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:56.473795 (+ 63082us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:56.473852 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:56.474403 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 13:02:56.820780 (+346377us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:56.820824 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.303980 (+483156us) spinlock_profiling.cc:243] Waited 53.8 ms on lock 0x7b540033e354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:57.304034 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:57.466752 (+162718us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:02:57.480996 (+ 14244us) raft_consensus.cc:1685] finished
0709 13:02:57.481027 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:57.481207 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19136,"raft.queue_time_us":7147,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":18025,"spinlock_wait_cycles":166311936,"thread_start_us":222882,"threads_started":1,"wal-append.queue_time_us":502375}
I20260709 13:02:57.490999 11363 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:57.589351  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.790940  8222 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:57.663945  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 489) took 2836 ms. Trace:
I20260709 13:02:57.589511  8323 rpcz_store.cc:276] 0709 13:02:54.643947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.598864 (+954917us) service_pool.cc:224] Handling call
0709 13:02:55.599473 (+   609us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:57.150979 (+1551506us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.479626 (+328647us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b540008ff54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:57.479866 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3566159232}
I20260709 13:02:57.791824  8404 rpcz_store.cc:276] 0709 13:02:54.827771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.490657 (+2662886us) service_pool.cc:224] Handling call
0709 13:02:57.590655 (+ 99998us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:57.663676 (+ 73021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.663886 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:57.792290  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:57.622188  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.496670 11732 consensus_queue.cc:1059] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.115s
I20260709 13:02:57.663496  8248 rpcz_store.cc:276] 0709 13:02:54.824423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.387371 (+1562948us) service_pool.cc:224] Handling call
0709 13:02:56.387711 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:56.407433 (+ 19722us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:56.407479 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:56.419454 (+ 11975us) log.cc:844] Serialized 26 byte log entry
0709 13:02:56.774182 (+354728us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:56.774221 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.774295 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:57.589795 (+815500us) raft_consensus.cc:1685] finished
0709 13:02:57.589821 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:57.589949 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19314,"raft.queue_time_us":22917,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":2638,"thread_start_us":312363,"threads_started":1,"wal-append.queue_time_us":564043}
I20260709 13:02:57.794639  8369 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:57.664263  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.675086  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:57.673620  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:57.613538  8325 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:02:57.757443 11651 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:57.758806 11763 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:57.611308 11795 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:57.495922 11822 consensus_queue.cc:237] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:57.761613 11819 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:57.762091  8405 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:57.846575  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 494) took 2526 ms. Trace:
I20260709 13:02:57.847026  8405 rpcz_store.cc:276] 0709 13:02:55.319878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.624649 (+2304771us) service_pool.cc:224] Handling call
0709 13:02:57.761777 (+137128us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:57.846263 (+ 84486us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.846507 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:57.763487 11779 consensus_meta.cc:287] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.565s	user 0.008s	sys 0.000s
I20260709 13:02:57.764083 11824 consensus_queue.cc:237] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:57.765074 11729 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:57.766021 11866 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:57.664923 11883 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.105s
I20260709 13:02:57.772755 11766 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.803695 11890 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:57.778395 11848 consensus_meta.cc:287] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.680s	user 0.008s	sys 0.000s
I20260709 13:02:57.858747 11779 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:57.879405 11848 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:57.881381 11779 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:57.884925  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:57.781072 11882 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:57.886875  8401 raft_consensus.cc:2393] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 10: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:02:57.782035  8255 consensus_meta.cc:287] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.513s	user 0.007s	sys 0.002s
I20260709 13:02:57.782698  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:57.783290 11745 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:57.784130 11723 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:57.787568 11796 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:57.752449 11818 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.657s	user 0.008s	sys 0.009s
I20260709 13:02:57.801308  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:57.802412  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:57.661948 11777 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.838160  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2216) took 2281 ms. Trace:
I20260709 13:02:57.842020  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:57.663612 11825 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:57.674016 11376 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:57.849550  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:57.850481  8403 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:57.604260 11846 consensus_queue.cc:1059] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.197s
I20260709 13:02:57.856128  8328 raft_consensus.cc:2393] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 7: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:57.610850 11852 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.234s
I20260709 13:02:57.859866  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:57.794014 11868 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:57.804030 11872 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:57.777832 11697 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.867396  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:57.883191 11678 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:57.875782 11890 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:57.845741 11711 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:02:57.848984 11819 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:58.035832 11819 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:58.036480  8403 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:02:57.878418 11766 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:58.038887 11766 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:57.885663 11626 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:58.040548 11745 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:58.040939 11626 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:58.041178 11745 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.889932  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:58.041741 11745 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:57.884373  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:58.042629  9001 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:58.042490 11697 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:58.043380  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:58.044099  8249 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:02:58.044960  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:57.890681  8253 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:58.046114 11363 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:58.047552 11694 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:57.866431 11835 raft_consensus.cc:697] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:58.050283 11835 consensus_queue.cc:237] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:58.050983  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:58.051631  8332 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:58.053366 11685 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:58.053835 11685 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:58.056568 11796 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:57.908452 11884 log.cc:927] Time spent T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:02:58.058686 11890 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:58.059089  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:58.059818  8329 raft_consensus.cc:2368] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12.
I20260709 13:02:58.060551  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:58.061038  8329 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:57.864085 11769 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.015158  8404 rpcz_store.cc:276] 0709 13:02:55.556422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.792689 (+2236267us) service_pool.cc:224] Handling call
0709 13:02:57.836881 (+ 44192us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:57.837119 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:57.837151 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:57.837212 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:57.837898 (+   686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.837962 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:57.838096 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:58.061699 11769 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:58.062475  8404 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:58.063035  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 496) took 2007 ms. Trace:
I20260709 13:02:58.063476  8254 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:58.063336  8404 rpcz_store.cc:276] 0709 13:02:56.055409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.061909 (+2006500us) service_pool.cc:224] Handling call
0709 13:02:58.062199 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.062813 (+   614us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.062984 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:57.966785  8252 raft_consensus.cc:2393] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 9: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:02:58.012109  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.064497  8249 raft_consensus.cc:2368] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 9. Current term is 10.
I20260709 13:02:57.856895 11836 consensus_queue.cc:260] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:58.009668  9000 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56948 (request call id 762) took 1567 ms (client timeout 1928 ms). Trace:
W20260709 13:02:58.019102  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 228198us
W20260709 13:02:58.073907  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.918947  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:58.074681  8323 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:02:58.017323  8325 raft_consensus.cc:2393] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 9: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:56.999460  8399 rpcz_store.cc:276] 0709 13:02:54.742464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.599274 (+856810us) service_pool.cc:224] Handling call
0709 13:02:55.634362 (+ 35088us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.634573 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:56.442579 (+808006us) log.cc:844] Serialized 14 byte log entry
0709 13:02:56.901042 (+458463us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:56.901100 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:56.901155 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.901228 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.901345 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":114339,"prepare.queue_time_us":931450,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":1837,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":28084,"thread_start_us":1148883,"threads_started":2}
I20260709 13:02:57.874761 11712 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:57.964437 11885 log.cc:927] Time spent T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:02:57.910760 11848 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:58.027661 11868 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:58.033401  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 230307us
I20260709 13:02:58.033981 11894 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:57.877861 11850 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:58.041929  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:58.155580  8248 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:02:58.156576  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:58.157291  8248 raft_consensus.cc:2368] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 9. Current term is 10.
I20260709 13:02:58.157665  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1878) took 1037 ms. Trace:
I20260709 13:02:57.902508  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:58.065737  9001 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:02:58.057337  8253 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:02:58.061098 11729 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:58.070919  9000 rpcz_store.cc:269] 0709 13:02:56.442707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.442856 (+   149us) service_pool.cc:224] Handling call
0709 13:02:58.009422 (+1566566us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b540067acd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:58.009600 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":645198592}
I20260709 13:02:58.086011  8254 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:02:57.959035 11891 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:58.092538 11796 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:58.091522 11740 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:58.140105  8369 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:57.919126 11777 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:58.144143  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:58.025457 11825 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:58.024715 11376 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:58.157998 11745 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:58.409677  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:58.469264  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 329192us
I20260709 13:02:58.470091  8369 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:58.472075  8369 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:58.157948  8248 rpcz_store.cc:276] 0709 13:02:57.120536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.156308 (+1035772us) service_pool.cc:224] Handling call
0709 13:02:58.157615 (+  1307us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:58.477051  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 500) took 1425 ms. Trace:
I20260709 13:02:58.477485  8250 rpcz_store.cc:276] 0709 13:02:57.052132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.052236 (+   104us) service_pool.cc:224] Handling call
0709 13:02:57.081906 (+ 29670us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:57.101154 (+ 19248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:57.101182 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:57.110329 (+  9147us) log.cc:844] Serialized 26 byte log entry
0709 13:02:57.488668 (+378339us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:57.488707 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.488772 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:58.476846 (+988074us) raft_consensus.cc:1685] finished
0709 13:02:58.476864 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:58.476995 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":699,"thread_start_us":323635,"threads_started":1,"wal-append.queue_time_us":472155}
I20260709 13:02:58.479348  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1872) took 1418 ms. Trace:
I20260709 13:02:58.483580  8253 rpcz_store.cc:276] 0709 13:02:57.061001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.884667 (+823666us) service_pool.cc:224] Handling call
0709 13:02:58.479296 (+594629us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:58.484834  8253 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:50420
I20260709 13:02:58.482393  8249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1833) took 1331 ms. Trace:
W20260709 13:02:58.343766  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.182066  8399 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:02:58.485519  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:58.310513  8974 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.3:37367 timed out after 1.506s (SENT) [suppressed 25 similar messages]
W20260709 13:02:58.456935  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.495714  8369 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:02:58.496654  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.498174  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2228) took 1450 ms. Trace:
I20260709 13:02:58.526165  8399 rpcz_store.cc:276] 0709 13:02:57.048349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.075763 (+1027414us) service_pool.cc:224] Handling call
0709 13:02:58.138968 (+ 63205us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.497839 (+358871us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.498108 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:58.498703  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.496300  8249 rpcz_store.cc:276] 0709 13:02:57.150847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.072129 (+921282us) service_pool.cc:224] Handling call
0709 13:02:58.482318 (+410189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:58.530133  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:58.530988  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1834) took 1286 ms. Trace:
I20260709 13:02:58.531347  8250 rpcz_store.cc:276] 0709 13:02:57.244226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.478749 (+1234523us) service_pool.cc:224] Handling call
0709 13:02:58.530914 (+ 52165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:58.469012 11363 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:58.532989  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:58.496031  8253 raft_consensus.cc:606] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:02:58.538482  8253 raft_consensus.cc:614] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:58.535951  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:58.509065 11891 raft_consensus.cc:697] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:58.529374 11876 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:58.543314 11876 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:58.468777 11796 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:58.548448  8329 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:02:58.380254 11769 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:58.488543 11745 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.489988 11694 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:58.543654 11876 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:58.536955 11363 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:58.547649  9004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 644) took 1329 ms. Trace:
I20260709 13:02:58.545864  8248 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 10: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:58.508400 11822 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.488148 11872 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:58.551076  9008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 646) took 1251 ms. Trace:
I20260709 13:02:58.298429 11894 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:58.570114  9008 rpcz_store.cc:276] 0709 13:02:57.299730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.300180 (+   450us) service_pool.cc:224] Handling call
0709 13:02:57.300475 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:57.303237 (+  2762us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:57.407666 (+104429us) log.cc:844] Serialized 14 byte log entry
0709 13:02:57.408174 (+   508us) log.cc:844] Serialized 14 byte log entry
0709 13:02:57.409041 (+   867us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:57.409092 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:57.409132 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.543442 (+1134310us) spinlock_profiling.cc:243] Waited 5.75 ms on lock 0x7b18002f17c0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882569f2a6 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419
0709 13:02:58.550799 (+  7357us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b540057d5d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:58.550870 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:58.551019 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":565,"prepare.queue_time_us":429341,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":75,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":6419,"spinlock_wait_cycles":19087488,"thread_start_us":103607,"threads_started":1}
I20260709 13:02:58.572674 11777 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:58.552466  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2223) took 1184 ms. Trace:
I20260709 13:02:58.552884  8398 raft_consensus.cc:1275] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:02:58.574537  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49220 (request call id 501) took 1522 ms. Trace:
I20260709 13:02:58.175604 11836 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:58.554565  8399 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:58.575132  8248 rpcz_store.cc:276] 0709 13:02:57.052448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.476015 (+1423567us) service_pool.cc:224] Handling call
0709 13:02:58.574473 (+ 98458us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:58.487994 11890 consensus_queue.cc:1059] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 13:02:58.561420  8249 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:58.298010 11850 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:58.183450 11711 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.370s
I20260709 13:02:58.566529  9004 rpcz_store.cc:276] 0709 13:02:57.206974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.207577 (+   603us) service_pool.cc:224] Handling call
0709 13:02:57.208738 (+  1161us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:57.612142 (+403404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.535622 (+923480us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b54005a8cd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:58.535838 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":640515328}
I20260709 13:02:58.510799 11740 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:58.303848 11852 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.576035  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2229) took 1388 ms. Trace:
I20260709 13:02:58.554986  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:58.585757  8248 raft_consensus.cc:1240] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:02:58.586082  9007 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:58.586647  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 276154us
I20260709 13:02:58.585490  8399 rpcz_store.cc:276] 0709 13:02:57.187357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.537948 (+1350591us) service_pool.cc:224] Handling call
0709 13:02:58.554305 (+ 16357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.575667 (+ 21362us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.575960 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:58.570466 11363 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:58.581669  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.591158  8298 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:58.573988 11872 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:58.579026 11777 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.578352 11822 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:58.577181 11893 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:58.605747 11777 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:58.606479 11893 raft_consensus.cc:697] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:58.607522 11822 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:58.607816 11893 consensus_queue.cc:237] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:58.609086 11777 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:58.573580 11894 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:58.543648 11891 consensus_queue.cc:237] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:58.659428  8972 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:02:58.661923 11864 consensus_queue.cc:1059] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:58.627576 11890 raft_consensus.cc:3055] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:58.667667 11890 raft_consensus.cc:740] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:58.668730  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2224) took 2221 ms. Trace:
I20260709 13:02:58.668588 11890 consensus_queue.cc:260] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:58.669971 11890 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:58.669253  8404 rpcz_store.cc:276] 0709 13:02:56.446929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.064258 (+1617329us) service_pool.cc:224] Handling call
0709 13:02:58.137774 (+ 73516us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.544741 (+406967us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:58.544784 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:58.544827 (+    43us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:58.544860 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.668406 (+123546us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b54003e6a54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:58.668468 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:58.668654 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":935359744}
I20260709 13:02:58.671778  8404 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:58.672492  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2231) took 1201 ms. Trace:
I20260709 13:02:58.627266 11846 consensus_queue.cc:1059] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.070s
W20260709 13:02:58.674116  8402 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 2207) took 3930 ms (client timeout 1869 ms). Trace:
I20260709 13:02:58.674427 11846 consensus_queue.cc:1243] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:02:58.675086 11846 raft_consensus.cc:3055] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:02:58.674719  8402 rpcz_store.cc:269] 0709 13:02:54.743481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.128532 (+385051us) service_pool.cc:224] Handling call
0709 13:02:58.673794 (+3545262us) spinlock_profiling.cc:243] Waited 1.97 s on lock 0x7b54000c8d54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:58.673985 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4533508352}
I20260709 13:02:58.678880  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:58.679718  8330 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:58.680907  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:58.682868  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:58.576943  8329 rpcz_store.cc:276] 0709 13:02:57.368193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.060321 (+692128us) service_pool.cc:224] Handling call
0709 13:02:58.552402 (+492081us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:58.625058 11796 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:58.619072 11825 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:58.629511  8399 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:58.684283  8397 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:58.684769  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2230) took 1412 ms. Trace:
I20260709 13:02:58.623307 11740 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:58.619165 11868 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:02:58.610249 11897 log.cc:927] Time spent T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:02:58.633713  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.626174 11878 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:58.628264 11872 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:58.638789 11588 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:58.641899  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:58.723667  8324 raft_consensus.cc:2368] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12.
I20260709 13:02:58.642366 11376 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:58.644454  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 2233) took 1055 ms. Trace:
I20260709 13:02:58.725006  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:58.725196  8403 rpcz_store.cc:276] 0709 13:02:57.588777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.589140 (+   363us) service_pool.cc:224] Handling call
0709 13:02:58.644228 (+1055088us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b54002684d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:58.644377 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":594762624}
I20260709 13:02:58.647959  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2218) took 3090 ms. Trace:
I20260709 13:02:58.726512  8405 rpcz_store.cc:276] 0709 13:02:55.557074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.025831 (+2468757us) service_pool.cc:224] Handling call
0709 13:02:58.060003 (+ 34172us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.556881 (+496878us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:58.556913 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:58.556957 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:58.556988 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.647693 (+ 90705us) spinlock_profiling.cc:243] Waited 497 ms on lock 0x7b54001cc354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:58.647755 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:58.647904 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1142076288}
W20260709 13:02:58.649349  8401 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 502) took 1753 ms (client timeout 2209 ms). Trace:
W20260709 13:02:58.728207  8401 rpcz_store.cc:269] 0709 13:02:56.896515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.304675 (+408160us) service_pool.cc:224] Handling call
0709 13:02:58.649099 (+1344424us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b54000fd7d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:58.649277 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":949021440}
I20260709 13:02:58.657624 11822 consensus_queue.cc:1059] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:58.563552 11745 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:58.685173  8399 rpcz_store.cc:276] 0709 13:02:57.272159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.587844 (+1315685us) service_pool.cc:224] Handling call
0709 13:02:58.629242 (+ 41398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.684534 (+ 55292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.684719 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:58.602811  9007 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:02:58.690267 11777 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.672897  8404 rpcz_store.cc:276] 0709 13:02:57.470702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.670926 (+1200224us) service_pool.cc:224] Handling call
0709 13:02:58.671340 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.672194 (+   854us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.672428 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:58.689500  8255 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.005s	sys 0.014s
W20260709 13:02:58.737105  8255 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.005s	sys 0.014s
I20260709 13:02:58.738111  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 499) took 1686 ms. Trace:
I20260709 13:02:58.738471  8255 rpcz_store.cc:276] 0709 13:02:57.051835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.051997 (+   162us) service_pool.cc:224] Handling call
0709 13:02:57.084651 (+ 32654us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:57.085074 (+   423us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:57.901504 (+816430us) log.cc:844] Serialized 14 byte log entry
0709 13:02:58.737519 (+836015us) log.cc:844] Serialized 14 byte log entry
0709 13:02:58.737741 (+   222us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:58.737802 (+    61us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:58.737858 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.737944 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:58.738056 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2182346,"raft.queue_time_us":25343,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":2191956,"thread_start_us":765999,"threads_started":2,"wal-append.queue_time_us":106174}
W20260709 13:02:58.693827  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 321965us
I20260709 13:02:58.718502  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:58.742529  8329 raft_consensus.cc:2393] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 8: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:58.626318 11882 consensus_queue.cc:1059] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:02:58.584964 11850 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:58.726133  8325 raft_consensus.cc:2368] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10.
I20260709 13:02:58.675669 11846 raft_consensus.cc:740] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:02:58.721202 11819 consensus_meta.cc:287] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.685s	user 0.006s	sys 0.005s
I20260709 13:02:58.750015 11819 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:58.751330 11850 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:58.732654  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2231) took 1311 ms. Trace:
I20260709 13:02:58.753687  8324 rpcz_store.cc:276] 0709 13:02:57.421781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.641553 (+1219772us) service_pool.cc:224] Handling call
0709 13:02:58.732575 (+ 91022us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:58.754947  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:58.756332  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:58.757902  8327 raft_consensus.cc:2368] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 7.
W20260709 13:02:58.720433 11685 consensus_meta.cc:287] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.666s	user 0.010s	sys 0.000s
I20260709 13:02:58.759495 11685 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:58.686383  8330 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:58.763012  9006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 765) took 1267 ms. Trace:
I20260709 13:02:58.763450  9006 rpcz_store.cc:276] 0709 13:02:57.495231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.495378 (+   147us) service_pool.cc:224] Handling call
0709 13:02:57.495708 (+   330us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:57.565149 (+ 69441us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:57.565182 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:57.594161 (+ 28979us) log.cc:844] Serialized 52 byte log entry
0709 13:02:57.964555 (+370394us) spinlock_profiling.cc:243] Waited 7.21 ms on lock 0x7b18003ced40. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e0d1 00007f8828779a0f
0709 13:02:58.029528 (+ 64973us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:58.029572 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.029635 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:58.762671 (+733036us) raft_consensus.cc:1685] finished
0709 13:02:58.762694 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:58.762947 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9011,"raft.queue_time_us":38738,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":25311,"spinlock_wait_cycles":16587008,"thread_start_us":64738,"threads_started":1,"wal-append.queue_time_us":435353}
I20260709 13:02:58.732383 11588 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.744439  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2218) took 1396 ms. Trace:
I20260709 13:02:58.766148 11588 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:58.768103 11685 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:58.768963 11685 raft_consensus.cc:3055] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:58.757258  8324 raft_consensus.cc:2393] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 7: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:58.736421 11376 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.735533 11848 raft_consensus.cc:2764] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:02:58.745375  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2219) took 1396 ms. Trace:
I20260709 13:02:58.748664 11846 consensus_queue.cc:260] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:58.776854  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:58.766347  8325 rpcz_store.cc:276] 0709 13:02:57.348276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.718205 (+1369929us) service_pool.cc:224] Handling call
0709 13:02:58.744363 (+ 26158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:58.594962 11852 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:58.721895 11825 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:02:58.772373  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:58.769971 11777 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:58.780982  9000 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:58.770413 11819 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:58.779045  8324 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:58.773820  8329 rpcz_store.cc:276] 0709 13:02:57.348546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.718205 (+1369659us) service_pool.cc:224] Handling call
0709 13:02:58.745317 (+ 27112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:58.769502 11685 raft_consensus.cc:740] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:02:58.777652 11870 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.000s	sys 0.015s
I20260709 13:02:58.788304 11852 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:58.789239 11363 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:58.745158  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 212728us
I20260709 13:02:58.798144  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:58.801175  8325 raft_consensus.cc:2376] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9. Re-sending same reply.
I20260709 13:02:58.801863  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2221) took 1452 ms. Trace:
W20260709 13:02:58.814972  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 334303us
W20260709 13:02:58.815910  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:58.816416  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.816769  8403 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:02:58.818265  8220 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:58.820119  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:58.769160 11588 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:58.800767  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:58.822075  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:58.733825 11895 consensus_queue.cc:1059] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
W20260709 13:02:58.799945 11870 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.000s	sys 0.015s
I20260709 13:02:58.830549  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:02:58.881877  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 63690.1us
I20260709 13:02:58.835592  8324 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 10. Current term is 11. Ops: []
W20260709 13:02:58.882301  8328 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 441) took 1639 ms (client timeout 1900 ms). Trace:
W20260709 13:02:58.882992  8328 rpcz_store.cc:269] 0709 13:02:57.243290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.243454 (+   164us) service_pool.cc:224] Handling call
0709 13:02:58.882001 (+1638547us) spinlock_profiling.cc:243] Waited 431 ms on lock 0x7b540037ad54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:58.882218 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":991510912}
I20260709 13:02:58.843822  8330 raft_consensus.cc:2410] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:02:58.847795  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:58.851748  9007 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:58.853070 11363 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:58.779047 11868 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:58.802278  8325 rpcz_store.cc:276] 0709 13:02:57.349103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.778040 (+1428937us) service_pool.cc:224] Handling call
0709 13:02:58.801785 (+ 23745us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:58.853807  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:58.887761  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2213) took 1540 ms. Trace:
I20260709 13:02:58.888008  8401 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:58.888177  8324 rpcz_store.cc:276] 0709 13:02:57.346911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.823628 (+1476717us) service_pool.cc:224] Handling call
0709 13:02:58.823988 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.887507 (+ 63519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.887706 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:58.859400  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 181669us
I20260709 13:02:58.821796 11852 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.773208 11376 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:58.871595  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 212261us
I20260709 13:02:58.834118 11819 raft_consensus.cc:3055] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:02:58.872973  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.875177  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2227) took 1471 ms. Trace:
I20260709 13:02:58.847558 11893 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:58.885381  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 288081us
I20260709 13:02:58.882387  8402 raft_consensus.cc:2410] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:02:58.854123  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:58.891628  8324 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:02:58.911084  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2226) took 1511 ms. Trace:
I20260709 13:02:58.911510  8324 rpcz_store.cc:276] 0709 13:02:57.399363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.889079 (+1489716us) service_pool.cc:224] Handling call
0709 13:02:58.889386 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.910738 (+ 21352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.910997 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:58.867172 11822 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.894078  8328 raft_consensus.cc:1275] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:02:58.905663  8972 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:58.918421  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.910187  8405 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
W20260709 13:02:58.923678  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.922392  8325 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:58.924758  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2217) took 1576 ms. Trace:
I20260709 13:02:58.925195  8325 rpcz_store.cc:276] 0709 13:02:57.347998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.887915 (+1539917us) service_pool.cc:224] Handling call
0709 13:02:58.922177 (+ 34262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.924536 (+  2359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.924709 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:58.913863 11891 consensus_queue.cc:1059] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:02:58.929351 11893 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:58.930233 11895 raft_consensus.cc:3055] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:58.930637 11895 raft_consensus.cc:740] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:58.931499 11895 consensus_queue.cc:260] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:58.932219 11376 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:58.932736 11376 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.921381 11729 consensus_queue.cc:1059] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:02:58.933498 11822 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:58.934544  8403 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:58.935272  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:58.936317  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:58.882829 11825 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:58.947475  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2214) took 1600 ms. Trace:
I20260709 13:02:58.947978  8328 rpcz_store.cc:276] 0709 13:02:57.347194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.884294 (+1537100us) service_pool.cc:224] Handling call
0709 13:02:58.884663 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.923248 (+ 38585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.947410 (+ 24162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:58.890036 11678 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:58.950307  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.969159  8403 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
W20260709 13:02:58.983840  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.933171 11376 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:59.021746  9000 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:02:59.022774  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:59.023804  8399 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 11: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:58.925835 11363 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.907521 11819 raft_consensus.cc:740] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:58.883805 11685 consensus_queue.cc:260] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:59.009701 11678 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:59.056969 11777 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:59.057610 11363 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:59.058934 11876 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:59.059923 11913 raft_consensus.cc:3055] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:59.066751  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:59.087874  9007 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 10: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
W20260709 13:02:59.073850  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 168275us
I20260709 13:02:59.074859  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:59.089239  8402 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:59.076179  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:58.908334  8329 rpcz_store.cc:276] 0709 13:02:57.404238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.794206 (+1389968us) service_pool.cc:224] Handling call
0709 13:02:58.875085 (+ 80879us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:59.074512  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 174521us
I20260709 13:02:59.094334  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:59.094933  8399 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:02:59.095762  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:59.079766  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 179537us
I20260709 13:02:59.098874 11697 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:58.908069 11852 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:59.098673 11891 consensus_queue.cc:1059] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:58.908243 11868 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:59.097080  8220 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:59.101696  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:59.111553  9007 raft_consensus.cc:2393] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 7: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:59.097016  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1857) took 4334 ms. Trace:
I20260709 13:02:59.108989  8402 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:02:59.112317  8397 rpcz_store.cc:276] 0709 13:02:54.762243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.791770 (+1029527us) service_pool.cc:224] Handling call
0709 13:02:58.684094 (+2892324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.684690 (+   596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.096850 (+412160us) spinlock_profiling.cc:243] Waited 2.87 s on lock 0x7b54000c8d50. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:59.096963 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6609932032}
I20260709 13:02:59.114509  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:59.115212  8402 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:02:59.116063  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:59.116753  8402 raft_consensus.cc:2393] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 9: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:59.117590  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:59.116775  8220 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:59.120678  8401 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:02:59.122138  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:59.084466 11895 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:59.126169  8402 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:02:59.128028  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:59.132963  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 210659us
I20260709 13:02:59.136150  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:59.137562  8401 raft_consensus.cc:2393] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:59.087925 11777 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.139242  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:59.143487  8397 raft_consensus.cc:2393] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 9: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:59.099426 11697 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.087411 11913 raft_consensus.cc:740] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:59.114194 11685 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:59.142836  8399 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
W20260709 13:02:59.149559  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 51517.6us
I20260709 13:02:59.124301 11729 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:59.152724  8297 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:59.124912 11917 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.159646 11930 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.160423 11933 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:59.161242  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:59.181854  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:59.182561  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:59.183588  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:59.184132  8404 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:02:59.184278  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:59.184827  8330 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:02:59.184901  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:59.186306  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:59.187356  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:59.163110  9005 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:02:59.119424 11745 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:59.158865 11924 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:59.124625 11796 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:59.099406 11825 consensus_queue.cc:1059] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:02:59.083182 11882 consensus_queue.cc:1059] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
W20260709 13:02:59.176060  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 89361.2us
I20260709 13:02:59.179617  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:59.187419  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:59.168272 11919 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:59.187978  8324 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:02:59.204982  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:59.196591  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:59.198756  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:59.202293  9007 raft_consensus.cc:2410] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:02:59.202680  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:59.153862 11711 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:59.194120 11930 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:59.214341 11928 consensus_queue.cc:1059] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:59.183763 11917 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:59.215360 11918 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.215853 11918 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:59.216243 11930 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:59.216627 11917 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:59.217021 11930 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:59.217597 11376 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:59.217872 11918 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:59.218333 11913 consensus_queue.cc:260] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:59.219710 11913 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:59.138015 11819 consensus_queue.cc:260] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:59.222271 11819 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:59.225483 11711 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:59.228150 11711 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:59.229720 11882 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.230396 11882 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:59.231563 11890 consensus_meta.cc:287] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.561s	user 0.010s	sys 0.000s
I20260709 13:02:59.232827 11882 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:59.233522 11876 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:59.235513 11363 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:59.150861 11777 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:59.204955 11893 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:59.155897 11697 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:59.238759 11678 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:59.239943 11697 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:59.244669  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2217) took 3687 ms. Trace:
I20260709 13:02:59.245092  8398 rpcz_store.cc:276] 0709 13:02:55.556773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.856571 (+2299798us) service_pool.cc:224] Handling call
0709 13:02:57.868671 (+ 12100us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.575010 (+706339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.244408 (+669398us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b54003a1bd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:59.244600 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1566425856}
I20260709 13:02:59.246660  8398 raft_consensus.cc:1240] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:59.247532  8249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1869) took 2187 ms. Trace:
I20260709 13:02:59.248535 11681 raft_consensus.cc:3055] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:02:59.248596  8327 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2224) took 1880 ms (client timeout 1616 ms). Trace:
W20260709 13:02:59.270589  8327 rpcz_store.cc:269] 0709 13:02:57.368444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.610443 (+241999us) service_pool.cc:224] Handling call
0709 13:02:59.248387 (+1637944us) spinlock_profiling.cc:243] Waited 961 ms on lock 0x7b540008a554. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:59.248511 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2211341056}
I20260709 13:02:59.271711  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:59.250665 11648 raft_consensus.cc:3055] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:59.252076  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:59.273159  9007 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
W20260709 13:02:59.252660 11887 log.cc:927] Time spent T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 1.305s	user 0.013s	sys 0.040s
I20260709 13:02:59.252846  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:59.253764  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:59.275841  9005 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 5.
I20260709 13:02:59.254854 11777 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:59.255599 11872 consensus_meta.cc:287] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.525s	user 0.009s	sys 0.000s
I20260709 13:02:59.258563 11864 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:59.264276  8372 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:59.214915  8401 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:02:59.264966  8249 rpcz_store.cc:276] 0709 13:02:57.060297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.529057 (+1468760us) service_pool.cc:224] Handling call
0709 13:02:58.549476 (+ 20419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.584436 (+ 34960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.247316 (+662880us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b540034c6d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:59.247474 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26780544}
W20260709 13:02:59.282544  8220 leader_election.cc:336] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.612s (SENT)
I20260709 13:02:59.273672  8404 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:59.283003  8220 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:59.290882  8398 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:59.207587 11933 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.088s
W20260709 13:02:59.298923 11927 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:02:59.304997 11723 raft_consensus.cc:3055] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:59.305444 11723 raft_consensus.cc:740] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:59.306263 11723 consensus_queue.cc:260] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:59.307271 11723 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:59.292518 11777 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:59.313581 11850 raft_consensus.cc:3055] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:59.314193 11850 raft_consensus.cc:740] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:59.315281 11850 consensus_queue.cc:260] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:59.318696 11868 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:59.318871  8254 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1870) took 2258 ms (client timeout 2061 ms). Trace:
I20260709 13:02:59.270378 11681 raft_consensus.cc:740] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:59.277076 11648 raft_consensus.cc:740] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:02:59.319453  8254 rpcz_store.cc:269] 0709 13:02:57.060556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.782441 (+721885us) service_pool.cc:224] Handling call
0709 13:02:59.318668 (+1536227us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b54002ee354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:59.318778 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":403118464}
I20260709 13:02:59.321471  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1837) took 2060 ms. Trace:
I20260709 13:02:59.337990  8253 rpcz_store.cc:276] 0709 13:02:57.261230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.539391 (+1278161us) service_pool.cc:224] Handling call
0709 13:02:58.562043 (+ 22652us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.772268 (+210225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:58.772298 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:58.772343 (+    45us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:58.772384 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.321185 (+548801us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b540034b2d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:02:59.321241 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:59.321386 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1866,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":7628,"spinlock_wait_cycles":479341440}
I20260709 13:02:59.322402  9001 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:59.325773  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:59.332420  8220 leader_election.cc:336] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.934s (SENT)
I20260709 13:02:59.340302  8220 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:59.346942  8372 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:59.344816 11777 raft_consensus.cc:697] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:59.349666 11893 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:59.350175 11893 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.350543 11893 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:59.350447 11777 consensus_queue.cc:237] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:59.351588 11648 consensus_queue.cc:260] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:59.352597 11648 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:59.353519 11926 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.354113 11926 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:59.333151 11924 consensus_queue.cc:1059] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:02:59.354849  8372 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:59.300292 11895 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:59.356683 11926 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:59.358191  8327 raft_consensus.cc:2410] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260709 13:02:59.359146 11836 consensus_meta.cc:287] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.778s	user 0.008s	sys 0.001s
I20260709 13:02:59.362632 11926 raft_consensus.cc:3055] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:59.365509 11740 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.365828 11926 raft_consensus.cc:740] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:59.366062 11740 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:59.366787 11926 consensus_queue.cc:260] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:59.367522 11852 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:59.281188 11872 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:02:59.367887 11926 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:59.369782 11872 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:59.370589 11689 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:02:59.371423  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.372411  8402 raft_consensus.cc:2393] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 9: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:02:59.356243 11681 consensus_queue.cc:260] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:59.375743  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:02:59.376452  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.381148  8325 raft_consensus.cc:2410] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:02:59.381220  8220 leader_election.cc:336] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.598s (SENT)
I20260709 13:02:59.381870  8220 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:59.383208  8220 leader_election.cc:336] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.611s (SENT)
I20260709 13:02:59.383656  8220 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:59.384984  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 183735us
I20260709 13:02:59.344702 11850 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:59.345166 11825 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:59.388351  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:59.390892  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:59.391734  8397 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
W20260709 13:02:59.396790  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 56267.6us
I20260709 13:02:59.399804  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:59.401546  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:59.403497  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:59.409592  8399 raft_consensus.cc:1240] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:02:59.404907  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:59.410385  8398 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:02:59.389485  8325 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:59.426640  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:59.427146  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.427403  8325 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
W20260709 13:02:59.428272  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 324380us
I20260709 13:02:59.435914  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:02:59.387578 11790 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.433s	user 0.019s	sys 0.004s
I20260709 13:02:59.381922 11890 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:59.430528 11685 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:59.434433 11878 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.008s	sys 0.026s
I20260709 13:02:59.425311 11681 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:59.425140 11891 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:59.384867 11836 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:59.454638  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:59.461234  8325 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
W20260709 13:02:59.442776 11790 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.500s	user 0.019s	sys 0.007s
W20260709 13:02:59.458474 11878 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.009s	sys 0.026s
I20260709 13:02:59.463773  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:59.419181 11933 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:02:59.464355  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:59.435863 11852 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:02:59.442116 11890 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:59.477839  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:59.478704  8325 raft_consensus.cc:2410] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:02:59.481653  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.483982  9003 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:02:59.498369  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:59.499018  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.506362 11892 raft_consensus.cc:3055] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:59.506261 11893 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:59.506909 11892 raft_consensus.cc:740] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:59.507783 11892 consensus_queue.cc:260] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:59.508970 11892 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:59.467883 11878 raft_consensus.cc:697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 7 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:02:59.510267 11878 consensus_queue.cc:237] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:59.513733  8397 raft_consensus.cc:2368] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 6. Current term is 7.
I20260709 13:02:59.513656  8369 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:02:59.516454  8296 service_pool.cc:136] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:57202 dropped due to backpressure: service queue is full with 50 items
W20260709 13:02:59.516837  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.516932  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:59.517635  9001 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:59.517668  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:59.523365 11789 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:59.516739  8296 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:57202 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2029)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2248)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2025)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2015)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2024)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2017)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2004)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2014)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2010)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2030)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2031)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2032)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2011)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2033)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2018)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2012)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2035)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2019)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2246)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2249)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2039)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 1999)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2008)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2022)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2021)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2006)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2003)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2041)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2042)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2250)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2251)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2040)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2038)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2034)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2036)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2020)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 1998)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2001)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 448)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2247)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 447)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 452)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2002)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2013)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2016)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2023)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2026)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2027)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2037)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 453)
I20260709 13:02:59.524997 11836 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.542209 11836 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:59.544380 11836 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:59.544684 11825 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.546420  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:59.526091  8406 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 2227) took 2597 ms (client timeout 1917 ms). Trace:
I20260709 13:02:59.547032  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:59.547271  8406 rpcz_store.cc:269] 0709 13:02:56.929044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.212970 (+283926us) service_pool.cc:224] Handling call
0709 13:02:59.525888 (+2312918us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54000fcdd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:59.526025 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2698151808}
I20260709 13:02:59.547645  8324 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:59.480099 11852 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:59.570885  8297 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:02:59.571796  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:59.535763  8400 raft_consensus.cc:1240] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:59.478518 11890 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:59.457676 11685 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:59.482463 11913 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:59.546761  8399 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:59.548753  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:59.535018 11846 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:59.526280  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:59.581080  8327 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:59.551537  8222 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:59.574273  8397 raft_consensus.cc:2955] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Committing config change with OpId 6.6: config changed from index 3 to 6, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:02:59.584547  8222 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:59.588272  8222 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:02:59.568056  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:59.575922  8369 leader_election.cc:336] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:57202 dropped due to backpressure: service queue is full with 50 items
I20260709 13:02:59.576107  8297 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:02:59.578518  9001 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:02:59.597913  9006 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:59.600402  8222 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:59.600989 11685 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:02:59.601032 11895 consensus_queue.cc:1048] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:59.604050 11745 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:59.604362 11931 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:59.591498 11934 consensus_queue.cc:1059] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:02:59.550086 11893 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:02:59.545217 11825 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:02:59.610594  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:59.596607  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.569238 11789 consensus_queue.cc:1059] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:02:59.619787  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:59.668861  8325 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:59.665464  8220 leader_election.cc:336] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 3.234s (SENT)
I20260709 13:02:59.620311  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:02:59.620642  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 44602us
I20260709 13:02:59.670346  8399 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
W20260709 13:02:59.620766  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 44871.4us
I20260709 13:02:59.624084  8372 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:59.612784 11928 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:02:59.626025  9005 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:02:59.671841  8222 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:02:59.622880 11850 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:59.587744 11723 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:59.677062  8406 raft_consensus.cc:1275] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:02:59.637756  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.597456 11748 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:59.629146 11821 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:02:59.678117  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 125325us
I20260709 13:02:59.673892  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:59.626904 11745 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:02:59.682879  8324 raft_consensus.cc:2410] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:02:59.673029 11850 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:59.593324 11796 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.085s
W20260709 13:02:59.681782  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.688802  8325 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:02:59.689355 11855 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:59.690325 11893 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:02:59.691265 11939 log.cc:927] Time spent T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260709 13:02:59.683720  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 59722.4us
I20260709 13:02:59.683378  8974 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:02:59.729032  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 45724.4us
I20260709 13:02:59.729851  8974 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:02:59.684185 11821 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.613903 11938 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.109s
I20260709 13:02:59.732230  8974 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:02:59.737694  8974 leader_election.cc:400] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 6. Current term is 7.
I20260709 13:02:59.737756  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:59.738162  8974 leader_election.cc:403] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:02:59.693997 11934 consensus_queue.cc:1243] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:02:59.706465 11745 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:59.717655 11850 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:59.747607  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.738535  8323 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:02:59.682915 11758 consensus_queue.cc:1059] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
W20260709 13:02:59.760176  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 88342.4us
I20260709 13:02:59.763185  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:59.735605 11821 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:59.680020 11928 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:59.758463  8972 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:02:59.783778  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.750272 11933 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:59.736969 11893 raft_consensus.cc:697] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:59.740289 11748 raft_consensus.cc:697] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:59.746155 11648 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:59.754362 11852 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:02:59.751871 11913 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:59.746471 11904 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.752684 11895 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:59.825994  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:59.736474 11711 consensus_queue.cc:1059] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 13:02:59.826990  9006 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:02:59.828831  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:59.831151  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:59.844954  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:02:59.764422 11911 consensus_queue.cc:1059] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
I20260709 13:02:59.833310  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:59.834389  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:59.839032 11959 consensus_queue.cc:1048] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:02:59.841830  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 83476.2us
I20260709 13:02:59.784322 11940 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 6. Current term is 7.
I20260709 13:02:59.845351  8253 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:59.845737  8255 raft_consensus.cc:2393] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 10: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:59.845890  8254 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:02:59.857796  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:59.831430 11748 consensus_queue.cc:237] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:59.844477  8251 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:02:59.841509 11723 raft_consensus.cc:697] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:02:59.851745  8256 raft_consensus.cc:2393] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 11: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:02:59.826972 11933 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:59.864234 11872 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.865079 11924 raft_consensus.cc:3055] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:59.851054 11937 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.849730 11648 raft_consensus.cc:697] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:02:59.866551  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:59.841593 11913 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.707139  8297 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:57202 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2017)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 455)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2004)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2014)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2010)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2030)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2031)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2032)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2011)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2033)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2018)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2012)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2035)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2019)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2246)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2249)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2039)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 1999)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2008)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 458)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2022)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2021)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2252)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2006)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2003)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2041)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2042)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2250)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2251)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2040)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2038)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 456)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2034)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2036)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2020)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 1998)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2001)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 448)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2247)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 447)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 452)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2002)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2013)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2016)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2023)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2026)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2027)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2037)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 453)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 457)
I20260709 13:02:59.849213 11848 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:59.877810  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:02:59.845791 11904 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:59.881549 11872 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:59.882416 11924 raft_consensus.cc:740] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:02:59.871299  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:02:59.850142 11895 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.885797  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:59.886765  8398 raft_consensus.cc:2376] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9. Re-sending same reply.
I20260709 13:02:59.889128  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:59.892118  9001 raft_consensus.cc:2393] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 10: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:02:59.893108  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:02:59.894145  9001 raft_consensus.cc:2410] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:02:59.889794  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:59.872692 11933 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:59.882920 11937 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:02:59.886590 11848 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.902760  8253 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:02:59.886260 11913 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:59.897505 11895 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:59.919236 11848 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:59.920686 11937 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:02:59.933840 11950 log.cc:927] Time spent T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.152s	user 0.000s	sys 0.003s
I20260709 13:02:59.935011 11924 consensus_queue.cc:260] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:59.922416 11850 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:02:59.947654 11954 log.cc:927] Time spent T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
W20260709 13:02:59.949210  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.949563  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:02:59.949968 11945 log.cc:927] Time spent T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 13:02:59.950613  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:59.950726  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:59.979234  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:02:59.979640  8328 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:02:59.980535  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:59.954648  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:59.977249  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:59.978091  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:59.951948  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1879) took 1100 ms. Trace:
I20260709 13:02:59.983150  8403 rpcz_store.cc:276] 0709 13:02:58.851297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.256282 (+404985us) service_pool.cc:224] Handling call
0709 13:02:59.322931 (+ 66649us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:59.323204 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:59.323240 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:59.337286 (+ 14046us) log.cc:844] Serialized 26 byte log entry
0709 13:02:59.470370 (+133084us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:59.470418 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.470505 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:59.496793 (+ 26288us) raft_consensus.cc:1685] finished
0709 13:02:59.496819 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:59.951701 (+454882us) spinlock_profiling.cc:243] Waited 66.3 ms on lock 0x7b54000221d0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:59.951889 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":152493952,"thread_start_us":59198,"threads_started":1,"wal-append.queue_time_us":100521}
I20260709 13:02:59.953989  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2215) took 4991 ms. Trace:
I20260709 13:02:59.986012  8400 rpcz_store.cc:276] 0709 13:02:54.962568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.571716 (+2609148us) service_pool.cc:224] Handling call
0709 13:02:59.535530 (+1963814us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:59.577322 (+ 41792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.953802 (+376480us) spinlock_profiling.cc:243] Waited 1.96 s on lock 0x7b54000fcdd0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:02:59.953932 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4515174784}
I20260709 13:02:59.958732  9005 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 13:02:59.987959  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:59.988554  8397 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:02:59.953260  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:59.957870  9001 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:02:59.985198  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:02:59.995844  8324 raft_consensus.cc:2410] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260709 13:03:00.056849  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.063983  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:59.994953  8328 raft_consensus.cc:2410] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:03:00.065546  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:00.066286  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:00.062996 11694 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:03:00.062664  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:00.076967  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:00.077593  8369 leader_election.cc:400] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 9. Current term is 10.
I20260709 13:03:00.077831  8332 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:03:00.078787  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:00.075217  8972 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:00.071208  8328 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:03:00.079952  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:00.080163  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:00.080611  8332 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:00.074405  8403 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:02:59.995671  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:00.074386  8295 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:00.073951  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.082237  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:00.072888  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:00.087033  8222 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:00.097517  8327 raft_consensus.cc:2368] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12.
I20260709 13:03:00.099821  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:00.101418  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:00.107367  8328 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:00.107455  8327 raft_consensus.cc:2393] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 7: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:00.105376  8972 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:59.950614 11866 raft_consensus.cc:3055] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:02:59.951004 11946 log.cc:927] Time spent T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 13:03:00.109047  9005 raft_consensus.cc:2368] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 7.
I20260709 13:02:59.903940 11648 consensus_queue.cc:237] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:00.109159  8972 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:02:59.975539 11825 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:00.112890  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:02:59.898015 11893 consensus_queue.cc:237] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:02:59.975085 11904 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:00.062945 11882 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:00.113574  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:00.008450 11924 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:00.115077  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:00.118690  8326 raft_consensus.cc:2393] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 6: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:02:59.995309 11740 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:00.119753  8327 raft_consensus.cc:2376] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10. Re-sending same reply.
I20260709 13:03:00.124161  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:00.126060  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:00.127403  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:00.127945  8327 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:03:00.108630 11938 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:00.108897 11919 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:00.116114 11681 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:00.113895  8223 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:00.063161 11790 consensus_queue.cc:1059] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:03:00.130752  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:00.079020 11931 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:00.112349 11866 raft_consensus.cc:740] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:00.115512 11892 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.136862  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:02:59.954584 11745 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:00.116874 11796 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:00.141067  8327 raft_consensus.cc:2368] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12.
I20260709 13:03:00.141100 11796 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:00.145116  8223 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:00.146780  8223 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:00.139561  8249 raft_consensus.cc:1240] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9. Ops: [8.9-8.9]
W20260709 13:03:00.149641  8252 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1873) took 3077 ms (client timeout 2209 ms). Trace:
W20260709 13:03:00.150229  8252 rpcz_store.cc:269] 0709 13:02:57.061238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.796569 (+735331us) service_pool.cc:224] Handling call
0709 13:03:00.138886 (+2342317us) spinlock_profiling.cc:243] Waited 55.1 ms on lock 0x7b5400178254. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:00.139080 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126787968}
I20260709 13:03:00.148311 11891 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:00.152945  8974 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:00.124126 11957 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:02:59.951153 11723 consensus_queue.cc:237] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:00.161710  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:00.154476  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:00.155982  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2247) took 1070 ms. Trace:
I20260709 13:03:00.161777  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:00.127231 11963 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:00.165274  8397 raft_consensus.cc:2393] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 9: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:00.165891  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:00.156315 11891 raft_consensus.cc:697] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:00.176628 11891 consensus_queue.cc:237] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:00.134784 11740 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:00.177379  9005 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:03:00.180781 11966 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:00.189848 11796 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:00.190089 11962 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:03:00.191139 11822 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:00.192912  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 1884) took 1339 ms. Trace:
I20260709 13:03:00.217989  8402 rpcz_store.cc:276] 0709 13:02:58.853139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.127551 (+274412us) service_pool.cc:224] Handling call
0709 13:03:00.192625 (+1065074us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54002be154. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:00.192811 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":559193984}
I20260709 13:03:00.219941  8405 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:59.953581 11913 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:00.162950  8401 rpcz_store.cc:276] 0709 13:02:59.085422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.292432 (+207010us) service_pool.cc:224] Handling call
0709 13:02:59.336249 (+ 43817us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:59.345299 (+  9050us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:59.345335 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:59.361657 (+ 16322us) log.cc:844] Serialized 26 byte log entry
0709 13:02:59.469098 (+107441us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:59.469139 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.469209 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:00.155699 (+686490us) raft_consensus.cc:1685] finished
0709 13:03:00.155730 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:00.155913 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15345,"raft.queue_time_us":9142,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":19543,"thread_start_us":107050,"threads_started":1,"wal-append.queue_time_us":311456}
I20260709 13:03:00.206581  8403 raft_consensus.cc:1275] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:00.205468  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:00.223724  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:00.157749 11890 consensus_meta.cc:287] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.571s	user 0.006s	sys 0.004s
I20260709 13:03:00.156868 11825 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:00.156605 11961 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:00.221146  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:00.235805  9004 raft_consensus.cc:2410] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:03:00.221115  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:00.221796  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:00.237258  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:00.225277  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:00.238477  8405 raft_consensus.cc:2410] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:03:00.238809  8403 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:03:00.128667 11882 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:00.232498  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:00.240302  8402 raft_consensus.cc:2393] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 8: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:00.206424 11740 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.152038 11892 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:00.242283  8400 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:03:00.237764  8398 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
I20260709 13:03:00.153694 11970 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.220016  8328 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:03:00.288200  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:00.289350  8328 raft_consensus.cc:2410] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:03:00.245563  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:00.290150  8403 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:00.291658  8403 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:00.240497  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:00.293833  8326 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:00.293659  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 146979us
I20260709 13:03:00.294932  8403 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:00.262506  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:00.295013  8223 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:00.296116  8402 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 11: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:00.268600  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:00.273274 11931 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.274040  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:00.306532  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:00.307641  8331 raft_consensus.cc:2368] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 7.
I20260709 13:03:00.298312  8223 leader_election.cc:400] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 7.
W20260709 13:03:00.280998  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.273300  8332 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
W20260709 13:03:00.282433  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.283161  9003 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:00.279778  8297 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:00.222787 11866 consensus_queue.cc:260] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:00.303375  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:00.522572  8328 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:00.523243  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2038) took 1050 ms. Trace:
I20260709 13:03:00.523633  8328 rpcz_store.cc:276] 0709 13:02:59.472807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.303076 (+830269us) service_pool.cc:224] Handling call
0709 13:03:00.523167 (+220091us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:00.524996  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:00.528254  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2034) took 1056 ms. Trace:
I20260709 13:03:00.332309  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:00.530071  8332 raft_consensus.cc:2393] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 12: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:00.530884  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:00.531581  8332 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:00.532135  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2036) took 1060 ms. Trace:
I20260709 13:03:00.532471  8332 rpcz_store.cc:276] 0709 13:02:59.472181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.530646 (+1058465us) service_pool.cc:224] Handling call
0709 13:03:00.532075 (+  1429us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:00.528796  8328 rpcz_store.cc:276] 0709 13:02:59.471543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.524676 (+1053133us) service_pool.cc:224] Handling call
0709 13:03:00.528198 (+  3522us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:00.534782  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:00.535774  8328 raft_consensus.cc:2410] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:03:00.536415  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 1998) took 1190 ms. Trace:
I20260709 13:03:00.536885  8328 rpcz_store.cc:276] 0709 13:02:59.345975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.534534 (+1188559us) service_pool.cc:224] Handling call
0709 13:03:00.536351 (+  1817us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:00.538162  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:00.447233  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2248) took 1047 ms. Trace:
W20260709 13:03:00.516896  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 237154us
I20260709 13:03:00.520641  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1835) took 3276 ms. Trace:
I20260709 13:03:00.539585  8250 rpcz_store.cc:276] 0709 13:02:57.244629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.532068 (+1287439us) service_pool.cc:224] Handling call
0709 13:02:59.270170 (+738102us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:59.270430 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:59.270456 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:59.278828 (+  8372us) log.cc:844] Serialized 26 byte log entry
0709 13:02:59.613336 (+334508us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:59.613375 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.613729 (+   354us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:59.840874 (+227145us) raft_consensus.cc:1685] finished
0709 13:02:59.840896 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:00.303047 (+462151us) spinlock_profiling.cc:243] Waited 710 ms on lock 0x7b540034c6d0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:00.520549 (+217502us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1632728320,"thread_start_us":293022,"threads_started":1,"wal-append.queue_time_us":439711}
W20260709 13:03:00.466782  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 131498us
I20260709 13:03:00.244953 11872 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:00.250625 11740 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:00.527562  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.538723  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:00.216599 11966 raft_consensus.cc:697] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:00.543246  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:00.575048  8248 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:03:00.576303  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:00.545756  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:00.539338  8331 rpcz_store.cc:276] 0709 13:02:59.400318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.148448 (+748130us) service_pool.cc:224] Handling call
0709 13:03:00.306187 (+157739us) spinlock_profiling.cc:243] Waited 56.6 ms on lock 0x7b58000e3498. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:03:00.447138 (+140951us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":130102016}
I20260709 13:03:00.217419 11777 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.099s
I20260709 13:03:00.194787 11957 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:00.549716  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2001) took 1202 ms. Trace:
I20260709 13:03:00.593765  8328 rpcz_store.cc:276] 0709 13:02:59.346779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.537910 (+1191131us) service_pool.cc:224] Handling call
0709 13:03:00.541414 (+  3504us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:00.564424  8326 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:03:00.594832 11926 raft_consensus.cc:3055] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:00.595304  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2040) took 1121 ms. Trace:
I20260709 13:03:00.595597 11957 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:00.595675  8326 rpcz_store.cc:276] 0709 13:02:59.479332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.225385 (+746053us) service_pool.cc:224] Handling call
0709 13:03:00.595239 (+369854us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:00.568630  8332 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:00.597132  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:00.597522  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2020) took 1231 ms. Trace:
I20260709 13:03:00.598047  8332 rpcz_store.cc:276] 0709 13:02:59.366583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.533201 (+1166618us) service_pool.cc:224] Handling call
0709 13:03:00.597455 (+ 64254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:00.599746  8332 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:03:00.600164  8332 pending_rounds.cc:85] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 6
I20260709 13:03:00.600445  8332 pending_rounds.cc:107] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:03:00.600754  8332 raft_consensus.cc:2889] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:00.595702  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:00.602617  8324 raft_consensus.cc:2376] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10. Re-sending same reply.
I20260709 13:03:00.603271  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2251) took 1201 ms. Trace:
I20260709 13:03:00.603755  8324 rpcz_store.cc:276] 0709 13:02:59.402362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.149612 (+747250us) service_pool.cc:224] Handling call
0709 13:03:00.595558 (+445946us) spinlock_profiling.cc:243] Waited 32.3 ms on lock 0x7b58000e3498. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:03:00.603200 (+  7642us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":74374016}
I20260709 13:03:00.595311 11926 raft_consensus.cc:740] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:00.606652 11926 consensus_queue.cc:260] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:00.429818 11970 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:00.607980 11926 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:00.609709  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2002) took 1254 ms. Trace:
I20260709 13:03:00.578598  8248 raft_consensus.cc:2410] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:03:00.582113  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:00.582947  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:00.611773  8331 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:03:00.612401  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 448) took 1391 ms. Trace:
I20260709 13:03:00.612809  8331 rpcz_store.cc:276] 0709 13:02:59.220967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.578388 (+1357421us) service_pool.cc:224] Handling call
0709 13:03:00.612327 (+ 33939us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:00.551577  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:00.614074  8332 rpcz_store.cc:276] 0709 13:02:59.347025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.598854 (+1251829us) service_pool.cc:224] Handling call
0709 13:03:00.599274 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.601185 (+  1911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.601359 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:00.583896  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:00.587672  8220 leader_election.cc:336] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.929s (SENT)
W20260709 13:03:00.616225  8220 leader_election.cc:336] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 3.308s (SENT)
I20260709 13:03:00.429332 11825 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:00.591127  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:00.610275  8324 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:03:00.622751  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2013) took 1258 ms. Trace:
I20260709 13:03:00.623243  8324 rpcz_store.cc:276] 0709 13:02:59.364736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.604800 (+1240064us) service_pool.cc:224] Handling call
0709 13:03:00.610055 (+  5255us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:00.622329 (+ 12274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.622674 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:00.583088 11729 raft_consensus.cc:3055] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:03:00.624151  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.616788  8220 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:00.590072  8327 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:00.625890  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2247) took 1407 ms. Trace:
I20260709 13:03:00.626379  8327 rpcz_store.cc:276] 0709 13:02:59.218659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.589442 (+1370783us) service_pool.cc:224] Handling call
0709 13:03:00.589821 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.625549 (+ 35728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.625814 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:00.243399 11890 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:00.641248  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:00.641535  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:00.642212  8252 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
W20260709 13:03:00.643262  8222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.096s	sys 0.209s
W20260709 13:03:00.643688  8222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.096s	sys 0.209s
W20260709 13:03:00.644208  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 557222us
I20260709 13:03:00.554566 11694 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:00.527063 11963 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.646821  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:00.652899  8327 raft_consensus.cc:1275] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:03:00.541539 11933 consensus_meta.cc:287] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.612s	user 0.006s	sys 0.004s
I20260709 13:03:00.648308  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:00.565848 11681 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.649631  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:00.677397  8252 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:03:00.653720  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2027) took 1208 ms. Trace:
I20260709 13:03:00.678560  8327 rpcz_store.cc:276] 0709 13:02:59.444897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.627356 (+1182459us) service_pool.cc:224] Handling call
0709 13:03:00.652528 (+ 25172us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.653376 (+   848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.653650 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:00.655607  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:00.279618 11848 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:00.657387  8251 raft_consensus.cc:2393] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 7: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:00.658771  8324 raft_consensus.cc:1240] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:03:00.668385  8403 raft_consensus.cc:2393] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 8: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:00.670004  9008 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:03:00.670925  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 54731.4us
I20260709 13:03:00.687654  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2026) took 1243 ms. Trace:
I20260709 13:03:00.688040  8324 rpcz_store.cc:276] 0709 13:02:59.444616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.624274 (+1179658us) service_pool.cc:224] Handling call
0709 13:03:00.658547 (+ 34273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:00.687363 (+ 28816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.687598 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:00.689393  8220 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:00.671865  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:00.674247  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:00.680011  8254 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:00.681566  8397 raft_consensus.cc:1240] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:00.695827  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:00.696760  8252 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:03:00.690976  8257 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:03:00.630440 11729 raft_consensus.cc:740] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:00.649341 11866 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:00.521270 11931 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:00.713687  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2037) took 1241 ms. Trace:
I20260709 13:03:00.714329  8327 rpcz_store.cc:276] 0709 13:02:59.472479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.679588 (+1207109us) service_pool.cc:224] Handling call
0709 13:03:00.713005 (+ 33417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.713202 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:00.713243 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:00.713301 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:00.713349 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.713440 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:00.713610 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:00.667991 11963 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:00.727166 11895 consensus_meta.cc:287] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.774s	user 0.007s	sys 0.003s
I20260709 13:03:00.727798 11895 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:00.728761 11949 log.cc:927] Time spent T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.901s	user 0.000s	sys 0.010s
I20260709 13:03:00.729367  8255 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 10: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:00.731402  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:00.732024  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:00.732609  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:00.734066  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:00.735212  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:00.736049  8255 raft_consensus.cc:2376] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11. Re-sending same reply.
I20260709 13:03:00.736879  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:00.737262 11890 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:00.737401  8255 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:00.739019 11821 consensus_meta.cc:287] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.924s	user 0.012s	sys 0.000s
I20260709 13:03:00.739446 11821 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:00.686951 11933 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:00.745898  9005 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:00.749672  8329 raft_consensus.cc:2393] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 10: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:03:00.751106  8329 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2030) took 1280 ms (client timeout 1632 ms). Trace:
W20260709 13:03:00.751596  8329 rpcz_store.cc:269] 0709 13:02:59.470227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.950378 (+480151us) service_pool.cc:224] Handling call
0709 13:03:00.750822 (+800444us) spinlock_profiling.cc:243] Waited 766 ms on lock 0x7b540014ffd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:00.751011 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1762834304}
I20260709 13:03:00.753784  8328 raft_consensus.cc:1240] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:03:00.733214  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:00.756635  8255 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:03:00.760361  8254 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:03:00.686573 11796 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:00.774889  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:00.776113  8254 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:00.777925  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 192164us
W20260709 13:03:00.783159  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.783308  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:00.785141  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 244364us
W20260709 13:03:00.798532  8369 leader_election.cc:336] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.912s (SENT)
I20260709 13:03:00.693050 11878 consensus_queue.cc:1059] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 13:03:00.699941 11681 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:00.775456 11745 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:00.832858  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 516) took 1427 ms. Trace:
I20260709 13:03:00.833417  8404 rpcz_store.cc:276] 0709 13:02:59.405783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.405953 (+   170us) service_pool.cc:224] Handling call
0709 13:02:59.406280 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:59.406682 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:59.406712 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:59.407267 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 13:02:59.480338 (+ 73071us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:59.480409 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.480520 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:00.728039 (+1247519us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:00.813222 (+ 85183us) raft_consensus.cc:1685] finished
0709 13:03:00.813255 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:00.832772 (+ 19517us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4370,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":10423,"spinlock_wait_cycles":4355456,"thread_start_us":72447,"threads_started":1,"wal-append.queue_time_us":320998}
I20260709 13:03:00.785611 11796 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:00.803442 11821 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:00.836966  8254 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:03:00.806154 11963 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:00.831561 11729 consensus_queue.cc:260] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:00.861537  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:00.862447  8254 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:03:00.859658  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:00.867972  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:00.871107  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:00.872044  9008 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 12: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:00.888518  9001 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:00.846814 11984 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.780033 11966 consensus_queue.cc:237] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:00.868047 11882 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:00.904670  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 215362us
W20260709 13:03:00.892140  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 202866us
I20260709 13:03:00.892388  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:00.908150  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:00.908852  9001 raft_consensus.cc:2368] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 7.
I20260709 13:03:00.909852  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:00.891628  8369 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:00.870951 11892 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:00.887247 11789 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:00.912986  8369 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:00.871275  8252 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:03:00.921098  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:00.921769  8252 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:03:00.895241 11872 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:00.923681  8220 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:00.930989  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:00.934798  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:00.935416  8252 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:00.937986  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:00.940068  8220 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:00.941397  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:00.942265  8252 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:00.945454  8220 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:00.946945  8220 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:03:00.945866  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:00.947217  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.948288  8220 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:00.911554 11895 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:00.949610  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:00.950312  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.952250  8252 raft_consensus.cc:2376] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12. Re-sending same reply.
I20260709 13:03:00.921962 11789 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.922096 11790 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:00.961588 11931 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:00.962466 11895 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:00.961578 11789 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:00.964172  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:00.964809  8252 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:03:00.966053  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:00.967356  8252 raft_consensus.cc:2393] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 9: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:00.968190  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:00.956324 11681 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:00.904289 11740 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:00.897905 11984 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:00.960093  8332 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:00.942593 11890 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.963241 11895 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:00.919445 11745 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.049s
W20260709 13:03:00.968708  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 182716us
I20260709 13:03:00.969026  8252 raft_consensus.cc:2393] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 9: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:00.937896 11937 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:00.983450  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:00.973423 11992 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:00.974545 11979 log.cc:927] Time spent T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 13:03:00.975050 11740 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:00.975613  8369 leader_election.cc:336] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.531s (SENT)
I20260709 13:03:00.976261  8248 raft_consensus.cc:2393] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 9: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:00.976783 11890 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:00.955883 11980 log.cc:927] Time spent T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:03:00.946719 11872 raft_consensus.cc:697] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:00.983325  8297 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:01.004336  8297 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:00.934412 11975 consensus_queue.cc:1048] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:03:00.985488  8249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 510) took 1154 ms. Trace:
I20260709 13:03:01.007810  8249 rpcz_store.cc:276] 0709 13:02:59.831873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.831995 (+   122us) service_pool.cc:224] Handling call
0709 13:03:00.139214 (+307219us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:00.147726 (+  8512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.981750 (+834024us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b5400178250. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:00.985394 (+  3644us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":705653632}
I20260709 13:03:00.980327 11895 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:00.988044  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:00.954617 11892 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:00.991185  8252 raft_consensus.cc:2368] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 9. Current term is 10.
I20260709 13:03:01.012204  8297 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:01.012451  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:01.010553  8330 raft_consensus.cc:2393] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 7: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:00.969502 11973 raft_consensus.cc:2764] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 13:03:01.013837  8330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2014) took 1648 ms (client timeout 1695 ms). Trace:
I20260709 13:03:01.014299  8252 raft_consensus.cc:2410] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:03:01.012924 11895 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:01.014464  8330 rpcz_store.cc:269] 0709 13:02:59.364979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.987716 (+1622737us) service_pool.cc:224] Handling call
0709 13:03:01.013779 (+ 26063us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:01.015377  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:01.016088  8249 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:00.936269 11918 consensus_queue.cc:1048] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:03:01.009689 11926 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:01.018478 11926 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:01.018918 11926 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:01.017093  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:01.023440  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:01.024016  8249 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:01.026057 11984 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:00.999995 11937 raft_consensus.cc:697] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 9 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:01.027204 11937 consensus_queue.cc:237] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:03:01.030648 11872 consensus_queue.cc:237] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:01.033794 11995 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:01.034163  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:01.034403 11821 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:00.919085 11825 raft_consensus.cc:2955] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Committing config change with OpId 3.5: config changed from index 2 to 5, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:01.035851  8297 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:01.037070 11924 consensus_meta.cc:287] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.909s	user 0.006s	sys 0.006s
I20260709 13:03:01.037238  8297 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:01.038265 11918 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:01.038709 11918 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:01.038935 11994 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:01.039131 11918 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:01.039563 11994 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:01.040011 11994 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:01.041644  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 453) took 1556 ms. Trace:
I20260709 13:03:01.042064  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:01.042656  8253 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:01.041992  8324 rpcz_store.cc:276] 0709 13:02:59.484852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.688800 (+1203948us) service_pool.cc:224] Handling call
0709 13:03:00.734048 (+ 45248us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:00.734477 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:00.734520 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:00.735147 (+   627us) log.cc:844] Serialized 52 byte log entry
0709 13:03:00.894374 (+159227us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:00.894423 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.894495 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:01.041481 (+146986us) raft_consensus.cc:1685] finished
0709 13:03:01.041498 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.041597 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":113323,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":13050,"thread_start_us":158817,"threads_started":1,"wal-append.queue_time_us":268556}
I20260709 13:03:01.043486  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:01.044476  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:01.044721 11890 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:01.045006  8253 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:01.046283  8250 raft_consensus.cc:2393] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 10: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:01.046944  9003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 771) took 1157 ms. Trace:
I20260709 13:03:01.047714  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:01.048813  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:01.047384  9003 rpcz_store.cc:276] 0709 13:02:59.889187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.890187 (+  1000us) service_pool.cc:224] Handling call
0709 13:02:59.906680 (+ 16493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.516710 (+610030us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.046661 (+529951us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b54005983d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:01.046881 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":865086208}
W20260709 13:03:01.015414 11981 log.cc:927] Time spent T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 13:03:01.055734  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:01.002287  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 54089.5us
I20260709 13:03:01.039202 11975 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.057044 11975 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:01.059147 11975 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:01.060262  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:01.060412 11821 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:01.061650  8220 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:01.056444  8252 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:01.062659  8325 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2017) took 1696 ms (client timeout 1673 ms). Trace:
W20260709 13:03:01.063062  8325 rpcz_store.cc:269] 0709 13:02:59.365779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.062465 (+1696686us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:01.062580 (+   115us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:01.063475 11975 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:01.064767  8220 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:01.066358  8253 raft_consensus.cc:1275] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:01.067813  8253 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:01.068547 11975 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:01.068990 11975 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:01.069332 11975 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:01.033262 11926 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:01.069384  8253 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:01.070652  8220 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:01.077420  8220 leader_election.cc:400] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12.
I20260709 13:03:01.075650  8256 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:01.080305  8220 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:01.081995  8220 leader_election.cc:400] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12.
I20260709 13:03:01.038733  8297 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:01.082762  8220 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:01.078979  8249 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:01.067880 11918 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:01.084472  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 45799.6us
I20260709 13:03:01.107919  8250 raft_consensus.cc:1240] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 13:03:01.086540  8369 leader_election.cc:336] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.632s (SENT)
I20260709 13:03:01.108455  8369 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:01.108922 11974 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:01.109040  8252 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:03:01.109450 11866 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:01.074062 11790 raft_consensus.cc:697] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 9 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:03:01.109863  8369 leader_election.cc:336] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.639s (SENT)
I20260709 13:03:01.110599  8252 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:01.056342 11789 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:01.104833 11694 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:01.106449 11992 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:01.110675  8297 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:01.083876 11796 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:01.119926 11895 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:01.121526  8253 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:03:01.080098 11961 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:03:01.123148  8296 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:57202 dropped due to backpressure: service queue is full with 50 items [suppressed 1 similar messages]
I20260709 13:03:01.153812 11961 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:01.104709 11745 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:01.154330 11961 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:01.154479 11745 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:01.155140 11745 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:01.137727  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 188210us
W20260709 13:03:01.158682  8323 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2010) took 1809 ms (client timeout 1734 ms). Trace:
I20260709 13:03:01.127032  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 457) took 1486 ms. Trace:
I20260709 13:03:01.151616 11895 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:01.165211 11745 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:01.153915  8296 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:57202 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
W20260709 13:03:01.159122  8323 rpcz_store.cc:269] 0709 13:02:59.349446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.158487 (+1809041us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:01.158607 (+   120us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:01.135484 11789 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:01.152132  8249 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:01.171799  8249 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "ff7a6fe533e94c1bae555b4cae427728"
 from {username='slave'} at 127.0.0.1:50420
I20260709 13:03:01.172407  8249 raft_consensus.cc:606] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Received request to transfer leadership to TS ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:01.172734  8249 raft_consensus.cc:614] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:03:01.159518  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 729284us
I20260709 13:03:01.160876  8297 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:01.150302  8252 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:01.163184 11928 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:01.176609  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:01.177326  8325 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:01.178025  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:01.177354  9009 raft_consensus.cc:3055] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:01.164438  8327 rpcz_store.cc:276] 0709 13:02:59.640379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.715629 (+1075250us) service_pool.cc:224] Handling call
0709 13:03:00.716006 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.716256 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:00.982625 (+266369us) log.cc:844] Serialized 14 byte log entry
0709 13:03:01.126656 (+144031us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:01.126708 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:01.126761 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.126833 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.126965 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":330271,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"raft.queue_time_us":95,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"thread_start_us":344001,"threads_started":2}
I20260709 13:03:01.160678 11821 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:01.178596  9009 raft_consensus.cc:740] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:01.179620  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:01.180275  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:01.180665  8330 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:03:01.185781  8220 leader_election.cc:400] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 7.
I20260709 13:03:01.194402  8223 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:01.196430  8295 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:01.196686  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:01.197343  8324 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:03:01.198961  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:01.182106  9009 consensus_queue.cc:260] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:01.200565  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.200716  9009 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:01.202189  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:01.206429  8249 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:01.207664  8223 leader_election.cc:400] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 7.
I20260709 13:03:01.208065  8223 leader_election.cc:403] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:03:01.207901  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:01.208631  8330 raft_consensus.cc:2376] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10. Re-sending same reply.
I20260709 13:03:01.121611 11790 consensus_queue.cc:237] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:01.211269  9007 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:01.209165  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:01.214054  9003 raft_consensus.cc:2393] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 7: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:01.180823 11984 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.217051  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:01.195677 11928 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:01.212180  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:01.219959  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:01.181033 11895 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:01.218545  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2016) took 1853 ms. Trace:
I20260709 13:03:01.223100  8331 rpcz_store.cc:276] 0709 13:02:59.365534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.613756 (+1248222us) service_pool.cc:224] Handling call
0709 13:03:00.654677 (+ 40921us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.654933 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:01.080061 (+425128us) log.cc:844] Serialized 14 byte log entry
0709 13:03:01.218199 (+138138us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:01.218260 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:01.218312 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.218382 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.218494 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":391390,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"thread_start_us":481740,"threads_started":2}
I20260709 13:03:01.223960 11973 consensus_queue.cc:1059] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:01.224937  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:01.219726 12001 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 7.
I20260709 13:03:01.224241  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:01.234767 11745 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:01.210088 11959 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:01.239948 11994 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:01.199009 11796 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:03:01.243697 11850 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:01.245543 11992 consensus_queue.cc:1059] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:01.225744  8324 raft_consensus.cc:2393] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 7: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:01.251202  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:01.250298  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:01.253286  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:01.253940  8249 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:03:01.254743  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:01.255474  8249 raft_consensus.cc:2410] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:03:01.256217  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:01.256778  8249 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:03:01.257576  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:01.260789  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:01.220378 11694 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:01.261509  8324 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:01.262418  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:01.262535 11918 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:01.263386  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:01.225167 11878 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:01.263958  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:01.264169  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:01.264492  8402 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:01.223145 11984 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:01.264818  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:01.265358  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:01.265710  8254 raft_consensus.cc:2410] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:03:01.265942  8405 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:03:01.266534  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:01.266685 11984 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:01.267272  8254 raft_consensus.cc:2393] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 9: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:01.267937  8325 raft_consensus.cc:2410] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:03:01.266713  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:01.270264  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:01.270447  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:01.271015  8398 raft_consensus.cc:2393] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 10: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:01.271605  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:01.271811  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:01.272154  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:01.272660  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:01.271195  8405 raft_consensus.cc:2410] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:03:01.274338  8326 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:03:01.274470 11933 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:01.273288  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:01.251538 11895 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:01.276518 11926 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:01.277030  8257 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:03:01.277870 11895 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:01.278177  8257 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:01.276511  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:01.278977  8403 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:01.279599  8222 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:01.280161  8257 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 13:03:01.275399 11850 raft_consensus.cc:3055] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:01.294211 11850 raft_consensus.cc:740] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:01.295094 11850 consensus_queue.cc:260] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:01.296092 11850 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:01.296769 11970 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:01.298332 11913 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:01.298643 11796 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:01.293607  8403 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:01.300053 11848 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:01.302976  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:01.291173  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:01.309504  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:01.309309  8223 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:01.310443  8399 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:03:01.311792  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:01.312467  8399 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:01.291481  8222 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:01.291888  8253 raft_consensus.cc:1240] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:01.272993  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:01.292644  8402 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:03:01.264336  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:01.290696  8297 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:01.316337  8324 raft_consensus.cc:2368] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 8. Current term is 9.
I20260709 13:03:01.318420  8297 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:01.318750  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1890) took 1158 ms. Trace:
I20260709 13:03:01.305234  8249 raft_consensus.cc:2393] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 9: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:01.319331  8397 rpcz_store.cc:276] 0709 13:03:00.160608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.165930 (+  5322us) service_pool.cc:224] Handling call
0709 13:03:00.166188 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.693978 (+527790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.318484 (+624506us) spinlock_profiling.cc:243] Waited 515 ms on lock 0x7b540028a0d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:01.318679 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1184492288}
I20260709 13:03:01.273945  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:01.322139  8222 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:01.319566  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:01.323701  8403 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:01.324942  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2023) took 1957 ms. Trace:
I20260709 13:03:01.325349  8332 rpcz_store.cc:276] 0709 13:02:59.367411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.617406 (+1249995us) service_pool.cc:224] Handling call
0709 13:03:00.617810 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.972185 (+354375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.319983 (+347798us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b5400058cd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:01.324883 (+  4900us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":786352512}
I20260709 13:03:01.331682  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:01.333233  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:01.333714  8220 leader_election.cc:400] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 8. Current term is 9.
I20260709 13:03:01.333931  8398 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:01.334034  8220 leader_election.cc:403] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:01.333417  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:01.334895  8402 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:01.336562  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:01.338291  8332 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:03:01.334095  8324 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
W20260709 13:03:01.341857  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 145667us
I20260709 13:03:01.300977 11994 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:01.343976  8399 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:03:01.371804  8399 raft_consensus.cc:1240] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:03:01.344800  8324 raft_consensus.cc:3055] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:01.373698  8324 raft_consensus.cc:740] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:01.342875  8295 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:01.355321  8403 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:01.368664  8398 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:03:01.381485  8297 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:01.382617  8403 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:01.373371  8399 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 13:03:01.385679 12006 log.cc:927] Time spent T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:03:01.324934 11895 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:01.374670  8324 consensus_queue.cc:260] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:01.391885  8324 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:01.329937 11938 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:01.332779 11796 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:01.336078 11970 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:01.300602 11975 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:01.394085  8399 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:03:01.398237  8402 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:01.371548 11745 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:03:01.370291 11748 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:01.412927 11748 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:01.413952 11961 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:01.370633 11878 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:01.414914 11878 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:01.415304 11878 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:01.418447  9009 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:03:01.418916  9009 pending_rounds.cc:85] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e: Aborting all ops after (but not including) 8
I20260709 13:03:01.419193  9009 pending_rounds.cc:107] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:03:01.419477  9009 raft_consensus.cc:2889] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:01.422698 11975 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:01.402128 11938 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:01.423703 11938 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:01.423506 11878 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:01.425597 11846 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:03:01.426263 11878 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:01.428077 11964 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.860s	user 0.000s	sys 0.005s
I20260709 13:03:01.429250  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 519) took 1668 ms. Trace:
I20260709 13:03:01.429656  8406 rpcz_store.cc:276] 0709 13:02:59.761279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.762847 (+  1568us) service_pool.cc:224] Handling call
0709 13:02:59.815409 (+ 52562us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:59.825872 (+ 10463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:59.825900 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:59.826331 (+   431us) log.cc:844] Serialized 26 byte log entry
0709 13:02:59.923952 (+ 97621us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:59.924001 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.924077 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:01.427525 (+1503448us) spinlock_profiling.cc:243] Waited 36.8 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:01.428953 (+  1428us) raft_consensus.cc:1685] finished
0709 13:03:01.428975 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.429191 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19471,"raft.queue_time_us":11214,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":19485,"spinlock_wait_cycles":84671232,"thread_start_us":60962,"threads_started":1,"wal-append.queue_time_us":394568}
I20260709 13:03:01.409824 11588 raft_consensus.cc:3055] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:01.435444 11836 raft_consensus.cc:3055] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:01.435992 11836 raft_consensus.cc:740] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:01.411064 12009 consensus_queue.cc:1059] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:03:01.437348 11836 consensus_queue.cc:260] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:01.431962 11588 raft_consensus.cc:740] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:01.432575 11938 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:01.326787  8222 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:01.439711  9000 raft_consensus.cc:1240] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:01.551359  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 452) took 2328 ms. Trace:
I20260709 13:03:01.552436  9000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 777) took 1468 ms. Trace:
I20260709 13:03:01.527148  8251 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:01.553054  9000 rpcz_store.cc:276] 0709 13:03:00.083684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.083882 (+   198us) service_pool.cc:224] Handling call
0709 13:03:01.439569 (+1355687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:01.550720 (+111151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.551849 (+  1129us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54006ce3d0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:01.552069 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3116776832}
I20260709 13:03:01.521714  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:01.552035  8326 rpcz_store.cc:276] 0709 13:02:59.222204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.596749 (+1374545us) service_pool.cc:224] Handling call
0709 13:03:00.597059 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:01.274793 (+677734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.550429 (+275636us) spinlock_profiling.cc:243] Waited 677 ms on lock 0x7b540037d054. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:01.551003 (+   574us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1556899072}
I20260709 13:03:01.560341  8256 raft_consensus.cc:1240] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:01.565181  8257 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:03:01.565759  8329 raft_consensus.cc:1240] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 6. Current term is 7. Ops: []
W20260709 13:03:01.566123  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 184615us
I20260709 13:03:01.566772 11928 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:01.567386 11928 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:01.567976 11928 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:01.568578  8297 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:01.439675  9006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56954 (request call id 657) took 2004 ms. Trace:
W20260709 13:03:01.569466  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 226452us
I20260709 13:03:01.569977 11855 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:01.570022  9006 rpcz_store.cc:276] 0709 13:02:59.435419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.435643 (+   224us) service_pool.cc:224] Handling call
0709 13:03:01.439414 (+2003771us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b54006ce3d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:01.439591 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":349579648}
I20260709 13:03:01.571218 11855 raft_consensus.cc:697] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:01.571595 11588 consensus_queue.cc:260] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:01.448657 11745 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 8. Current term is 9.
I20260709 13:03:01.573750 12009 consensus_queue.cc:1059] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:01.576691  8329 raft_consensus.cc:1275] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:03:01.463949 11968 log.cc:927] Time spent T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.910s	user 0.000s	sys 0.002s
I20260709 13:03:01.583835 11984 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:01.584677 11984 raft_consensus.cc:697] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:01.586562  9002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 666) took 1719 ms. Trace:
I20260709 13:03:01.588696 11777 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.587123  9002 rpcz_store.cc:276] 0709 13:02:59.867353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.867466 (+   113us) service_pool.cc:224] Handling call
0709 13:02:59.881991 (+ 14525us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:59.882420 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:59.882453 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:59.882979 (+   526us) log.cc:844] Serialized 26 byte log entry
0709 13:03:00.101503 (+218524us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:00.101565 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.101664 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:01.463226 (+1361562us) spinlock_profiling.cc:243] Waited 26.9 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:01.585710 (+122484us) raft_consensus.cc:1685] finished
0709 13:03:01.585752 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.586282 (+   530us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8754,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":6081,"spinlock_wait_cycles":61809408,"thread_start_us":218126,"threads_started":1,"wal-append.queue_time_us":353098}
I20260709 13:03:01.589720 11777 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:01.590915 11588 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:01.516458  8152 catalog_manager.cc:5697] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 4 to 6, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 6 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:01.592746 11836 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:01.474284 11961 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:01.454347 11976 log.cc:927] Time spent T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.565s	user 0.000s	sys 0.002s
I20260709 13:03:01.618141 11685 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.619309 11685 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:01.597023  9004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 791) took 1319 ms. Trace:
I20260709 13:03:01.599202  8972 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:01.601924  8401 raft_consensus.cc:1240] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:03:01.601918  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:01.610510 11648 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.627658  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1915) took 1038 ms. Trace:
I20260709 13:03:01.611374 11681 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.612699  9000 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:03:01.630075 11681 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:01.625260 11846 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.634778 11846 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:01.636197 11745 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:01.636952 11745 raft_consensus.cc:697] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:01.639250 11745 consensus_queue.cc:237] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:01.641710  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.641752 11969 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:03:01.644850  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 317946us
I20260709 13:03:01.646174  8222 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:01.628362 11648 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:01.451179 11878 raft_consensus.cc:3055] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:01.628171  8256 rpcz_store.cc:276] 0709 13:03:00.589391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.095506 (+506115us) service_pool.cc:224] Handling call
0709 13:03:01.095924 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:01.561176 (+465252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.627312 (+ 66136us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b540033e354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:01.627560 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1057084672}
I20260709 13:03:01.657500 11961 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:01.657732  8972 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:01.660475  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:01.669759 11969 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:01.709954  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:01.710225  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:01.710834  9006 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:01.712119  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:01.692476 11777 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:01.693056 11748 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:01.692847  8972 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:01.693691 11878 raft_consensus.cc:740] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:01.695586 11648 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:01.603919 11984 consensus_queue.cc:237] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:01.688553  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:01.702816  8324 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:03:01.706018  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 60087.8us
W20260709 13:03:01.707897  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.707641  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:01.573210 11855 consensus_queue.cc:237] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:01.676517 11681 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:01.622262  9004 rpcz_store.cc:276] 0709 13:03:00.277809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.278790 (+   981us) service_pool.cc:224] Handling call
0709 13:03:00.279153 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.279439 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:00.526316 (+246877us) log.cc:844] Serialized 14 byte log entry
0709 13:03:00.550683 (+ 24367us) log.cc:844] Serialized 14 byte log entry
0709 13:03:01.595703 (+1045020us) spinlock_profiling.cc:243] Waited 8.71 ms on lock 0x7b18003cea40. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882564bc0a 00007f882ed8b219 00007f88256d3a40 00007f88256ce4e4 00007f88256ce455 00007f88287b6060 00007f88287c8716 00007f88287d9287 00007f88287d9208
0709 13:03:01.596169 (+   466us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:01.596305 (+   136us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:01.596422 (+   117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.596554 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.596814 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1100265,"prepare.queue_time_us":253306,"prepare.run_cpu_time_us":125,"prepare.run_wall_time_us":7616,"raft.queue_time_us":1098093,"raft.run_cpu_time_us":387,"raft.run_wall_time_us":387,"spinlock_wait_cycles":20030208}
I20260709 13:03:01.725939  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:01.731318  8222 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:01.732290  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:01.733260  8400 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:03:01.734148 11846 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:01.737783 11983 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:03:01.738178  8295 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:01.739804 12009 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:03:01.741675 12019 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.076s	user 0.002s	sys 0.000s
I20260709 13:03:01.742455 11846 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:03:01.742520  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 49887.2us
I20260709 13:03:01.743355  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:01.748049 11878 consensus_queue.cc:260] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:01.749475 11846 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:01.750080 11846 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:01.751217  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:01.753732  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:01.754958  8254 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:01.751549 11729 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:01.741099  8295 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:01.759927  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:01.760561  8250 raft_consensus.cc:1240] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:01.756596  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 447) took 2536 ms. Trace:
W20260709 13:03:01.760771  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 413469us
I20260709 13:03:01.750082 11878 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:01.766752  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 44102.8us
I20260709 13:03:01.761174  8328 rpcz_store.cc:276] 0709 13:02:59.220560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.594507 (+1373947us) service_pool.cc:224] Handling call
0709 13:03:00.753461 (+158954us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:00.754375 (+   914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.756163 (+1001788us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b540014ffd0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:01.756390 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":364567936}
I20260709 13:03:01.760454  8152 catalog_manager.cc:5697] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 4 to 7, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 7 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:01.773239  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:01.769752 11748 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:01.780025  9006 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:03:01.782335  8401 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:01.774223 11681 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:01.781622  8251 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:03:01.781736  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:01.793273  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:01.794193  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 63166.2us
I20260709 13:03:01.795634  8222 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:01.797223 11961 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:01.797581 11926 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:01.799578  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:01.800263  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.803053  8222 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:01.789618 11938 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:01.805572  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:01.806474  8253 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
W20260709 13:03:01.809546  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 75134.3us
I20260709 13:03:01.811741  8298 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:01.813203 11931 consensus_queue.cc:1059] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:03:01.814124  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.820730  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:01.785907 11748 raft_consensus.cc:697] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:01.824381 11938 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:01.829329  8220 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:01.846683 11694 consensus_queue.cc:1059] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 13:03:01.848004  9000 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:03:01.850075  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 109198us
I20260709 13:03:01.851446  8295 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:01.779908 12009 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:03:01.854666 11748 consensus_queue.cc:237] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:01.861621 11961 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:01.866845  8248 raft_consensus.cc:1275] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:03:01.868362  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:01.870517  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:01.870951  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1864) took 1018 ms. Trace:
I20260709 13:03:01.871256  8253 raft_consensus.cc:2376] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7. Re-sending same reply.
I20260709 13:03:01.871469  8257 rpcz_store.cc:276] 0709 13:03:00.852392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.306684 (+454292us) service_pool.cc:224] Handling call
0709 13:03:01.328295 (+ 21611us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:01.565952 (+237657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.870646 (+304694us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b54003f1bd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:01.870888 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":522986752}
I20260709 13:03:01.829704  9004 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:01.869272  8248 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:03:01.902139 11777 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:01.904495  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 122256us
I20260709 13:03:01.903067 11777 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:01.915567  8254 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
I20260709 13:03:01.835958 11685 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:01.922408  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:01.919265 11938 raft_consensus.cc:697] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 7 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:01.824975 11681 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:01.929535  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 137236us
I20260709 13:03:01.917062 11745 raft_consensus.cc:2764] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:03:01.936592  8972 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:01.927608 11931 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:03:01.958259  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 155450us
I20260709 13:03:01.966074  8297 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:01.925901 11855 consensus_queue.cc:1059] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:03:01.971091  8250 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
I20260709 13:03:01.966826 12021 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:03:01.976008  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.980458 11882 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.982611 11882 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:01.983323 11777 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:01.984894 11882 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:01.985858 11933 consensus_queue.cc:1059] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:03:01.986960 11961 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:01.989583 11959 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.990064 11959 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:01.991150 11928 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:01.993054  8252 raft_consensus.cc:2410] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:03:01.980588 11848 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:01.999078 12034 raft_consensus.cc:3055] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:01.999729 12034 raft_consensus.cc:740] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:02.003863  8255 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:03:02.005517 11681 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:02.007061  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 70557.6us
I20260709 13:03:02.009696 11891 consensus_queue.cc:1059] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:02.010566  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:02.010703 11891 raft_consensus.cc:3055] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:02.011188 11891 raft_consensus.cc:740] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:02.011618  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:02.016734 12032 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:03:02.017226  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:02.020964  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:02.021075 11938 consensus_queue.cc:237] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:02.021938  8328 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:02.027156  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 56341.8us
I20260709 13:03:02.031121  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:01.977116 12021 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:02.033632 11685 raft_consensus.cc:3055] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:02.036408  8405 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:02.048746  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:03:02.034225  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 182903us
I20260709 13:03:02.012288 11891 consensus_queue.cc:260] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:02.055055  8400 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:03:02.055651 11891 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:02.045539 11850 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:03:02.056558  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:02.058053  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:02.058760  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1894) took 1510 ms. Trace:
I20260709 13:03:02.058972  8406 raft_consensus.cc:2410] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 13 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:03:02.059151  8255 rpcz_store.cc:276] 0709 13:03:00.548521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.782923 (+234402us) service_pool.cc:224] Handling call
0709 13:03:02.058537 (+1275614us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400309154. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:02.058684 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2803982720}
I20260709 13:03:02.055538  8400 pending_rounds.cc:85] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 9
I20260709 13:03:02.060237  8400 pending_rounds.cc:107] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:03:02.060549  8400 raft_consensus.cc:2889] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:02.037881  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:02.061689  9008 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:02.061915 11961 consensus_queue.cc:1048] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:02.064047 12033 raft_consensus.cc:3055] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:02.064531 12033 raft_consensus.cc:740] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:02.064239 11846 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:02.065415  9002 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:02.065472 12033 consensus_queue.cc:260] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:02.066954 12033 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:02.068001 11992 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.068508 11992 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:02.046487 11994 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:02.000885 12034 consensus_queue.cc:260] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:02.069957 11994 raft_consensus.cc:697] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:02.070183 12034 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:02.070732 11992 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:02.071102 11994 consensus_queue.cc:237] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:02.072458 11846 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:02.049319 11685 raft_consensus.cc:740] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:02.074717 11685 consensus_queue.cc:260] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:02.055472  8974 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:02.080528  8401 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
I20260709 13:03:02.038755  8372 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:02.079120  8332 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
W20260709 13:03:02.087652  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.050676 12021 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:02.100886  8974 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:02.101712  8405 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:02.103286  8328 raft_consensus.cc:2410] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:03:02.104846  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:02.112739  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:02.104590  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 66066.7us
I20260709 13:03:02.126016  8401 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:02.126873  9008 raft_consensus.cc:2410] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:03:02.125226  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:02.128188  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:02.128789  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:02.128788  8400 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:03:02.105209  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:02.045246 11729 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:03:02.099663 11846 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:02.138056 11924 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:02.104384 11928 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:02.132524  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:02.143649  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 195439us
I20260709 13:03:02.128041 11872 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:03:02.149206  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 48383.3us
I20260709 13:03:02.151762  8253 raft_consensus.cc:2393] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 7: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:02.156761 11911 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:02.157462 11911 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.158002 11911 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:02.154304  8372 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:02.160837  8152 catalog_manager.cc:5697] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 5 to 9, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 9 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:02.170073  8401 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:02.171977  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:02.173941  8328 raft_consensus.cc:2410] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:03:02.174216 11745 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:03:02.174778  8369 leader_election.cc:336] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.711s (SENT)
W20260709 13:03:02.175905  8369 leader_election.cc:336] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.734s (SENT)
I20260709 13:03:02.176290  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:02.176633 12009 raft_consensus.cc:3055] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:02.177155 12009 raft_consensus.cc:740] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:02.178165 12009 consensus_queue.cc:260] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:02.185798  8222 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:02.169231  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:02.188409  8372 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:02.130499 11836 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
W20260709 13:03:02.195667  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 105850us
I20260709 13:03:02.198937  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:02.200912  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.187150 11872 raft_consensus.cc:697] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:02.204234 11992 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:02.204730 11992 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.204466 11872 consensus_queue.cc:237] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:02.206528  8324 raft_consensus.cc:1275] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:02.208348  8330 raft_consensus.cc:1275] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:02.208705  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:02.209476  8325 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:02.205122 11992 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:02.217864  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.221539  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:02.197315 11685 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:02.240391  8328 raft_consensus.cc:2393] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 6: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:02.241688  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:03:02.243053  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.271983  8325 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
W20260709 13:03:02.277119  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 157418us
I20260709 13:03:02.285748  8295 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:02.286898  9008 raft_consensus.cc:1240] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:02.293823 12050 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:03:02.294277  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.263761 12003 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:02.291505  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:02.305868  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:02.308959  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.311295  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:02.312126  8324 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:02.318286  8330 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:02.322645  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 146760us
I20260709 13:03:02.294559 12050 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:02.317961  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 129621us
I20260709 13:03:02.302568 12003 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.301664 11848 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:02.330192  8325 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
W20260709 13:03:02.331681  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 187284us
I20260709 13:03:02.296205 11938 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:02.337181 11883 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:02.303488 11825 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.140s
I20260709 13:03:02.356206  8331 raft_consensus.cc:2410] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:03:02.338232  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:02.345485 11790 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:02.329154 12003 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:02.350999  8332 raft_consensus.cc:2410] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:03:02.325840 12050 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:02.368265  8972 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:02.368773  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:02.373346  8400 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:02.369915  8297 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:02.374923  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 131208us
I20260709 13:03:02.364568 11969 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.378189  8397 raft_consensus.cc:1275] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:02.366626 11928 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:02.379276  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:03:02.383260 12045 log.cc:927] Time spent T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:03:02.385547  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:02.390188 12041 log.cc:927] Time spent T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:03:02.389219  8295 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:03:02.393196 12046 log.cc:927] Time spent T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
W20260709 13:03:02.395217 12040 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:03:02.399636  8297 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:02.403671 12042 log.cc:927] Time spent T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:03:02.404914 12048 consensus_queue.cc:1059] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:03:02.404693  8152 catalog_manager.cc:5697] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:02.405833 12048 raft_consensus.cc:3055] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:02.407589  8330 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:03:02.378068 12034 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.412910  8400 raft_consensus.cc:2410] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:03:02.411794  8401 raft_consensus.cc:1275] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:02.406196 12048 raft_consensus.cc:740] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:02.372282 11961 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:02.414492  8223 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:02.406401  8152 catalog_manager.cc:5697] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:02.422941  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.401018 11883 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:02.406041 12033 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:02.415299 11928 raft_consensus.cc:697] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:02.382025 11938 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:02.439435  8220 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:02.426323 11790 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:03:02.447129  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 110749us
I20260709 13:03:02.452600  8220 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:02.453084  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:02.453495  8253 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:03:02.454000  8256 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:02.454344  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:02.454780 12037 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.426s	user 0.003s	sys 0.000s
I20260709 13:03:02.455168  8251 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:02.452634 12049 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:02.456709 12049 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:03:02.456856 12039 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.327s	user 0.004s	sys 0.000s
I20260709 13:03:02.457252 12049 raft_consensus.cc:697] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:02.430290 11883 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:02.457502  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:02.457989 11883 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:02.458334  8255 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:02.458866 11918 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:02.459401 11918 raft_consensus.cc:697] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:02.401409 11969 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:02.459720  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:02.459666 11790 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:02.460443 11918 consensus_queue.cc:237] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:02.461752  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:02.462059 11891 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:02.458297 12049 consensus_queue.cc:237] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:02.462723 12023 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:02.472681  8252 raft_consensus.cc:1275] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:03:02.488994  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:02.480443  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 91309.2us
I20260709 13:03:02.434219 11984 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:02.486363  8254 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 6.
I20260709 13:03:02.416953 12034 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:02.492353  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 119584us
I20260709 13:03:02.497290  8324 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:03:02.498766  8220 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:02.510486  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:02.445027 12033 raft_consensus.cc:697] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:02.433988 11926 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:03:02.443321 11825 consensus_queue.cc:1048] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:03:02.488021 11891 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:03:02.538285  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 76051.2us
I20260709 13:03:02.542939  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:02.540745  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:02.543843  8152 catalog_manager.cc:5697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:02.548316  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:02.538923  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.553990  8401 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:03:02.554432 11969 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:02.554859  8151 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45448 (request call id 283) took 25461 ms (client timeout 15000 ms). Trace:
I20260709 13:03:02.555610 12033 consensus_queue.cc:237] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:02.449683 11961 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:02.563791  8397 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
I20260709 13:03:02.568500  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:02.546473 11891 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:02.569137  8403 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:03:02.568174  8972 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:02.579408  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:02.579017  8152 catalog_manager.cc:5697] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:02.583405  9002 raft_consensus.cc:2393] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 10: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:02.570927 11790 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:02.587059  8295 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:02.579439  8972 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:03:02.592684  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.596079 11932 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:02.596573 11932 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.596942 11932 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:02.599730 11959 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:02.600885  8251 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:03:02.603224  8972 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:02.589130  8298 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:02.519234 12048 consensus_queue.cc:260] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:02.601358 11959 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:02.627306 12035 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:02.627561  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:02.628759  8152 catalog_manager.cc:5697] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 6 to 10, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4), now has a pending config: VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1), VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2), VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3), VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 10 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: NON_VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: true } health_report { overall_health: HEALTHY } } } pending_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:02.593968 12023 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:02.635766  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:02.638051  8405 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:02.602974 11969 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:02.648729 11685 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:02.641673 11959 raft_consensus.cc:697] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:02.651244  8152 catalog_manager.cc:5697] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 4 to 7, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 7 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:02.652328 11959 consensus_queue.cc:237] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:02.653774  9006 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:02.653013  8152 catalog_manager.cc:5697] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 1 to 5, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 5 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:02.657029 12066 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.664410  8369 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:02.664279 12066 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:02.665674  8369 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:02.666432 11777 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:02.666857 11777 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:02.667034 12066 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:02.667361 11777 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:02.555341  8151 rpcz_store.cc:269] 0709 13:02:37.093651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.093911 (+   260us) service_pool.cc:224] Handling call
0709 13:03:02.554469 (+25460558us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:53.092000 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:53.196000 (+104000us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:53.196068 (+    68us) write_op.cc:435] Acquired schema lock
0709 13:02:53.196091 (+    23us) tablet.cc:662] Decoding operations
0709 13:02:53.197722 (+  1631us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:53.197812 (+    90us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:53.197838 (+    26us) tablet.cc:685] Acquiring locks for 14 operations
0709 13:02:53.336507 (+138669us) tablet.cc:701] Row locks acquired
0709 13:02:53.347725 (+ 11218us) write_op.cc:260] PREPARE: finished
0709 13:02:53.347928 (+   203us) write_op.cc:270] Start()
0709 13:02:53.348151 (+   223us) write_op.cc:276] Timestamp: P: 1783602173347857 usec, L: 0
0709 13:02:53.348182 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:02:53.348899 (+   717us) log.cc:844] Serialized 5500 byte log entry
0709 13:02:53.453030 (+104131us) op_driver.cc:464] REPLICATION: finished
0709 13:02:53.938654 (+485624us) write_op.cc:301] APPLY: starting
0709 13:02:53.978341 (+ 39687us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:53.993401 (+ 15060us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:54.058210 (+ 64809us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:54.667535 (+609325us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:54.667998 (+   463us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=14
0709 13:02:54.668117 (+   119us) write_op.cc:312] APPLY: finished
0709 13:02:54.668663 (+   546us) log.cc:844] Serialized 101 byte log entry
0709 13:02:54.746268 (+ 77605us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:54.759364 (+ 13096us) write_op.cc:454] Released schema lock
0709 13:02:54.785047 (+ 25683us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:54.844550 (+ 59503us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:54.851089 (+  6539us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":54349,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":300173,"apply.run_cpu_time_us":14424,"apply.run_wall_time_us":991785,"num_ops":14,"prepare.queue_time_us":225674,"prepare.run_cpu_time_us":26346,"prepare.run_wall_time_us":341636,"raft.queue_time_us":26706,"raft.run_cpu_time_us":23170,"raft.run_wall_time_us":208579,"replication_time_us":104591,"thread_start_us":482231,"threads_started":5,"wal-append.queue_time_us":225155,"wal-append.run_cpu_time_us":11401,"wal-append.run_wall_time_us":2089952}]]}
W20260709 13:03:02.669173  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 170446us
I20260709 13:03:02.643611 12034 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:02.673596  8152 catalog_manager.cc:5697] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 4 to 6, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 6 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:02.645162 11969 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.651274 12048 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:02.682049  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:02.682925  8324 raft_consensus.cc:2410] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:03:02.655408 11926 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:02.612155 11928 consensus_queue.cc:237] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:02.680038 11969 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:02.690219  8152 catalog_manager.cc:5697] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 3 to 4, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 4 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:02.683411 11872 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:02.692065 12057 log.cc:927] Time spent T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 13:03:02.695849 11911 consensus_meta.cc:287] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.537s	user 0.011s	sys 0.001s
I20260709 13:03:02.708158 11911 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:02.709359  9003 raft_consensus.cc:2368] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11.
I20260709 13:03:02.717357 11911 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:02.718343  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:02.719570  8405 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
W20260709 13:03:02.725240  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 166956us
W20260709 13:03:02.736629  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 174675us
I20260709 13:03:02.737664  8222 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:02.739739  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:02.740597  8324 raft_consensus.cc:2393] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 9: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:02.740022 12062 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:02.745793 12062 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.746181 12062 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:02.774895  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 171728us
W20260709 13:03:02.782557  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 116971us
I20260709 13:03:02.646029 12050 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:02.736029 11748 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.805694  8400 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:03:02.778515 11926 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:02.808691  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:02.810158  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:02.790351 12009 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:02.824393  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:02.826100  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:02.837328  8252 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:03:02.838034 12049 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:02.838228  8251 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:03:02.838675 12049 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:02.829536 12064 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:02.853626  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.846138 12065 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:02.798202 11748 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:02.863855  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:02.868031  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 57991.4us
I20260709 13:03:02.852950 11790 raft_consensus.cc:2764] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:03:02.832803 11926 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:02.897336 12063 log.cc:927] Time spent T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:03:02.899533 12060 log.cc:927] Time spent T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:03:02.900328  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 108040us
I20260709 13:03:02.846583 11883 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:02.915083  8369 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:03:02.919603  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.935478 11790 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:02.893850 12064 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:02.936131 11790 raft_consensus.cc:697] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:02.937170 11790 consensus_queue.cc:237] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:02.937902 12064 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:02.940114 11850 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.940652 11850 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:02.942288  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:02.914896 12003 consensus_meta.cc:287] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.532s	user 0.011s	sys 0.000s
W20260709 13:03:02.946707  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 44301us
W20260709 13:03:02.914712 12021 consensus_meta.cc:287] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.785s	user 0.007s	sys 0.008s
I20260709 13:03:02.947703  8369 leader_election.cc:400] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 9. Current term is 10.
I20260709 13:03:02.947455 12021 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:02.948663 12062 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:02.949628  9000 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:02.949741 12021 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:02.950387 12049 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:02.902760 12033 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:02.943814  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:02.948695  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:02.946576 12003 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:02.946096  8403 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:02.952256  8397 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:03:02.956936  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:02.959636  8256 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:02.960216  8295 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:02.963272  8222 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:02.967269  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:02.969226 11938 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:02.972399  8330 raft_consensus.cc:1275] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:03:02.902617 12065 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:02.977923  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:02.942747 11850 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:02.983572  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:02.983930  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:02.984244  9008 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:02.972810  8330 pending_rounds.cc:85] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 5
I20260709 13:03:02.984967  8330 pending_rounds.cc:107] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:03:02.985203  8256 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:03:02.985354  8330 raft_consensus.cc:2889] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:02.981004  8222 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:02.981280  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:02.979300 12033 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:02.988534 11926 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:02.989933 11748 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:02.990320 11883 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:02.990720  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:02.991926  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:02.932108 11891 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:02.993149  8220 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:02.993803  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:02.997098  8324 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:03.004410  8972 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:02.972196 12044 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:03.007717  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:03.009663  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:03.010411  8256 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:02.992638 11825 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:02.992861 12021 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:02.995826 12050 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:03.017253 12068 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:03.017598 11963 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:03.017710 12077 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:03.026568 12062 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:03.018090 12021 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:03.027266 12021 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:03.001492 11938 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:03:03.025004  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:03.015514 11992 consensus_meta.cc:287] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.772s	user 0.008s	sys 0.005s
I20260709 13:03:03.008347 12044 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.025677  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:03.038897  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.016394 11825 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.044040  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:03.044891  8324 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:03.026607 12077 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.051508 12036 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:03.052995  8369 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:03.020004 11850 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:03.040546 11938 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:03.051371 12077 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:03.060611 11938 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:03.052335 11825 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:03.056311  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:03.043785 11992 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.043906 12044 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:03.025717 12068 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:03.068138 11992 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:03.064004 12070 log.cc:927] Time spent T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:03:03.077805 11790 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:03.052685 11932 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.078435 11777 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:03.069767  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:03.080657  8405 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:03.040154 11963 raft_consensus.cc:697] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:03.082203  8256 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:03:03.065662 12033 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:03.072173 11938 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.101119  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:03.103348  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:03.103811  9005 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:03.103946  8256 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:03.101207  8972 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:03.102661  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:03.104957 11846 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:03.080351 11932 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:03.047353 11983 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:03:03.083153 12038 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.108795 11926 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.112665  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:03.112778 11926 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.113324  8404 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
W20260709 13:03:03.108973  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 83706.4us
I20260709 13:03:03.114164  8399 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:03:03.114842  8369 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:03.115291 11926 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:03.117328 12050 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:03.110528 11938 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:03.118546 11790 consensus_queue.cc:1048] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:03.120208 11938 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:03.120473 11790 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:03.081115 12068 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:03.109125 11891 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:03.125593 12038 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.127638 12038 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:03.111510 11846 raft_consensus.cc:697] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:03.131348  8974 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:03.130919 11846 consensus_queue.cc:237] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:03.137209 12076 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.271s	user 0.000s	sys 0.001s
I20260709 13:03:03.143795  8220 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:03.146301  8974 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:03.146598  9005 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:03.148072  8405 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:03.148828  8256 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:03:03.141297  8401 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:03:03.157546  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:03.158360  8328 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:03.160722  8324 raft_consensus.cc:2410] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:03:03.139710 11777 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:03.153038  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 89330.5us
I20260709 13:03:03.169217  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:03.152835 11911 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:03.170680 11911 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.171043 11911 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:03.152942 12022 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:03.173666 12022 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.174023 12022 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:03.151608 12048 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:03:03.176604  8369 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:03.177412 11777 consensus_queue.cc:1059] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:03.179004 11777 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:03.179466 11777 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:03.180092 12077 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.180588 11872 raft_consensus.cc:3055] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:03.181157 11872 raft_consensus.cc:740] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:03:03.182036 12093 log.cc:927] Time spent T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:03:03.182320 11872 consensus_queue.cc:260] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:03.183853 11872 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:03.179953 11777 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:03.187173  8400 raft_consensus.cc:3055] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:03.187587  8400 raft_consensus.cc:740] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:03.188290  8400 consensus_queue.cc:260] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:03.189201 11961 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:03.189201  8400 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:03.161957 11932 consensus_queue.cc:1059] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:03:03.177634  8369 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:03.153306 12003 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:03.197335  8222 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:03.167413 11926 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:03:03.199885 11961 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.200418 11961 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.196805 11963 consensus_queue.cc:237] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.202401 11961 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:03.202910 11961 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:03.163175 11748 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:03:03.187176 12081 log.cc:927] Time spent T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:03:03.152416 11969 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.203441 11961 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:03.244069  8328 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
W20260709 13:03:03.247219  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 143989us
I20260709 13:03:03.217713 12048 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:03.255913  8222 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:03.250517  8298 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:03.255501  8295 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:03.257174  8400 raft_consensus.cc:1275] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:03.258545  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:03.262781  8369 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:03.264153 12082 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:03.259361 11748 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:03:03.269680 12082 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:03.270691 12077 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:03.271055 11961 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:03.271885 12048 consensus_queue.cc:1059] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:03.272639  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:03.273196  8399 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:03:03.257293 12003 consensus_queue.cc:1059] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:03:03.273629 11825 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:03.274405 12003 raft_consensus.cc:3055] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:03.274891 12003 raft_consensus.cc:740] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:03.275803 11825 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:03.257401 12047 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:03.276800 12047 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:03.277158 12047 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:03.280313 11911 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.282146 11850 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:03.282850 12068 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.284996 12068 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:03.289537  8399 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
W20260709 13:03:03.295972  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 192705us
I20260709 13:03:03.290256 12082 raft_consensus.cc:3055] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:03:03.296461  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 145313us
I20260709 13:03:03.298534  8223 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:03.303799  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:03.290931 11926 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:03.304880  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2063) took 1115 ms. Trace:
I20260709 13:03:03.290266 12036 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.303593  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:03.312441  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:03.330787  8324 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:03.327284  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:03.328316  9000 raft_consensus.cc:1240] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:03.312757  8325 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:03:03.328994  8332 rpcz_store.cc:276] 0709 13:03:02.184883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.192736 (+  7853us) service_pool.cc:224] Handling call
0709 13:03:03.299912 (+1107176us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b540008aa54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:03.300041 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347043072}
I20260709 13:03:03.333091  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:03.333879  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:03.297277 12077 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:03.340464 12077 raft_consensus.cc:697] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:03.341357 12077 consensus_queue.cc:237] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:03.342442 12065 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:03.300021 12082 raft_consensus.cc:740] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:03.331969  8325 pending_rounds.cc:85] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 4
I20260709 13:03:03.345598  8325 pending_rounds.cc:107] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:03:03.345925  8325 raft_consensus.cc:2889] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:03.327310  8972 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.572s (SENT) [suppressed 69 similar messages]
I20260709 13:03:03.335392  9003 raft_consensus.cc:2393] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 10: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:03.301826 11918 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:03.337402  8328 raft_consensus.cc:1275] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:03.337764 11959 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.337733 12036 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:03.338521 12092 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:03.338817  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:03.295423 11992 consensus_queue.cc:1059] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:03:03.296453 11961 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.358724  8324 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:03.303011 12050 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:03:03.359529  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 88627.7us
I20260709 13:03:03.365518  9005 raft_consensus.cc:1275] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:03:03.368407  8369 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:03.275821 12003 consensus_queue.cc:260] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:03.368956  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:03.373121  9002 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:03.371258  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 127896us
W20260709 13:03:03.373751  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.298329 12022 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.376886  8405 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:03.379001  8255 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:03.378925  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:03.379920  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:03.380497 11825 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.381840 11969 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:03.382485 11911 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:03.381868  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:03.400691  8251 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:03.390369  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:03.398836  9001 raft_consensus.cc:2368] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11.
I20260709 13:03:03.406549  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:03.407150  9007 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:03.409521  9006 raft_consensus.cc:2393] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 9: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:03.395082 11959 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:03.413779  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 45467.2us
I20260709 13:03:03.416097  8220 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:03.416314  8405 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:03.414649  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:03.410578 12003 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:03.417789 12094 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.418340 12094 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:03.407014 12065 raft_consensus.cc:3055] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:03.405472 12092 raft_consensus.cc:697] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:03.423686  9002 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:03.377085 11961 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.424988 12085 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.362792 12082 consensus_queue.cc:260] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:03.425624 12104 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:03.425576 11777 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:03.429420  8223 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:03:03.430922  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.443459  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:03.444072  9002 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:03.435952 12104 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:03.427112 12065 raft_consensus.cc:740] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:03.438750 12085 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:03.439379 11850 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:03.440359 11961 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:03.461694 11961 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:03.462154 11961 raft_consensus.cc:697] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:03.441056 11959 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:03.463070 11961 consensus_queue.cc:237] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.466152 11872 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:03.427220 12009 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.449769  8251 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:03.450933 12077 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.457201  8223 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:03.435529 12094 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:03.475133  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 64977.9us
I20260709 13:03:03.471735 12009 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:03.481632 11872 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:03.482201 11872 raft_consensus.cc:697] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:03.482462 12085 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:03.424108 12095 consensus_queue.cc:1048] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:03:03.430542 11825 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:03.460047 12104 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:03.494355 11926 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:03.483106 11872 consensus_queue.cc:237] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:03.502246  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:03.480460 11748 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:03.528929 11748 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.529156  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:03.529400 11748 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:03.529692  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:03.531150 12065 consensus_queue.cc:260] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.529695 12036 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:03.537607  8251 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:03.538393 11777 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:03.539529  8256 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:03.529901 12092 consensus_queue.cc:237] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.578197  9007 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:03.578799  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:03.583595  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:03.586010  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:03.586767  8252 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:03.587666  8222 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:03.577714 12021 consensus_meta.cc:287] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.550s	user 0.010s	sys 0.001s
I20260709 13:03:03.475805 12077 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:03.600921  8256 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:03.604303  8298 leader_election.cc:400] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11.
I20260709 13:03:03.611218  8298 leader_election.cc:403] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:03.618062  8251 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
W20260709 13:03:03.613658  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 286323us
W20260709 13:03:03.590658 12023 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.927s	user 0.015s	sys 0.006s
I20260709 13:03:03.595338 11984 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.640168  8256 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:03.503238 12047 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.620340  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:03.622294 12023 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.965s	user 0.015s	sys 0.007s
I20260709 13:03:03.645723  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:03.653218  8254 raft_consensus.cc:2393] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 9: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:03.601928 11983 consensus_queue.cc:1059] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:03:03.654803  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:03.659068  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:03.637800 12065 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:03.620868 12094 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.613864 12085 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:03.666105  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:03.671599  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:03.589998 12044 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:03.604068 11928 consensus_queue.cc:1059] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:03:03.612018 12021 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.612442 11878 consensus_queue.cc:1048] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:03:03.678645  8249 raft_consensus.cc:1240] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10. Ops: [8.9-9.10]
I20260709 13:03:03.649775 11926 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.692448  8254 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:03:03.691372  8253 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:03:03.680220  8248 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:03.685487  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:03.695155  8399 raft_consensus.cc:2393] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 10: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:03.695438  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:03.696002  8249 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:03.656867 12022 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:03.650215 12023 raft_consensus.cc:697] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:03.653535 11850 raft_consensus.cc:697] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:03.688489  8250 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 6. Current term is 7. Ops: [5.8-6.9]
I20260709 13:03:03.613415 12111 consensus_queue.cc:1048] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:03:03.690435  8257 raft_consensus.cc:1275] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:03:03.714733  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 112516us
I20260709 13:03:03.700767  8406 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:03.687058 11825 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:03.645679 11984 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:03.692193 11983 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:03.699910 12107 log.cc:927] Time spent T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 13:03:03.669991 11748 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.717985  9003 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:03.719249 11850 consensus_queue.cc:237] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.724540  8249 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:03.725221  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:03.726502  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:03:03.779191  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 52752.6us
I20260709 13:03:03.727161 12114 consensus_queue.cc:1059] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:03.727645  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:03.728060  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:03.686467 12094 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:03.729255  8222 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:03.792896  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 63711.2us
W20260709 13:03:03.732067 12091 log.cc:927] Time spent T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.631s	user 0.000s	sys 0.005s
I20260709 13:03:03.733152  8255 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:03:03.797923 11932 consensus_queue.cc:1059] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:03.736202  8398 raft_consensus.cc:1275] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:03.799510  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:03.800920  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:03.736613  8252 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:03.738816 12044 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:03.739729 12009 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:03.701927 12036 raft_consensus.cc:3055] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:03.801815 12082 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:03.802139 12036 raft_consensus.cc:740] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:03.802371  8251 raft_consensus.cc:1275] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:03.803030 12036 consensus_queue.cc:260] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.803988 12065 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:03.804230 12036 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:03.741979  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:03.805629  8398 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:03.806267  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:03.806901 12098 log.cc:927] Time spent T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.391s	user 0.003s	sys 0.000s
I20260709 13:03:03.806874  8399 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
I20260709 13:03:03.807984  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 535) took 1212 ms. Trace:
I20260709 13:03:03.808118  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:03.742606 11685 raft_consensus.cc:3055] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:03:03.747175 12088 log.cc:927] Time spent T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.691s	user 0.002s	sys 0.000s
I20260709 13:03:03.724929 12104 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:03.729220  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.698022 11928 consensus_queue.cc:1243] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 10.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:03:03.808892  8399 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:03.815056  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 331705us
I20260709 13:03:03.729858  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:03.816900  8974 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:03.780591  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:03.788934  8404 raft_consensus.cc:2393] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 10: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:03.789233  8402 raft_consensus.cc:2393] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 12: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:03.817587  9006 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:03.821640  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:03.808535  8401 rpcz_store.cc:276] 0709 13:03:02.595734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.611607 (+ 15873us) service_pool.cc:224] Handling call
0709 13:03:03.807818 (+1196211us) spinlock_profiling.cc:243] Waited 447 ms on lock 0x7b5400267fd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:03.807911 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1029105920}
I20260709 13:03:03.757391 12077 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:03.698495 11926 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:03.847332  8369 leader_election.cc:336] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:57202 dropped due to backpressure: service queue is full with 50 items
W20260709 13:03:03.849793  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 69286.6us
I20260709 13:03:03.789855 11963 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:03.853020  8256 raft_consensus.cc:2393] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 8: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:03.817667 12003 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.851763  8974 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:03.809278 11685 raft_consensus.cc:740] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:03.860734  8252 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 10. (index mismatch)
I20260709 13:03:03.830835 11729 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.830049 11959 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.830374 12114 raft_consensus.cc:3055] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:03.844513 12120 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:03.865770  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.865701  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:03.868556  8401 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:03.859193  8972 leader_election.cc:336] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.953s (SENT)
I20260709 13:03:03.847726 11883 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.869645  8251 raft_consensus.cc:1240] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:03.855309 11846 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:03.876186  8297 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:03.874536  8248 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:03:03.851646 11969 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:03.879287  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:03.888993  8405 raft_consensus.cc:2393] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 10: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:03.886543  8972 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:03.881717  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:03.893074  8297 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:03.895095  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:03.867489 12114 raft_consensus.cc:740] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:03.899466  8399 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:03.868108 11963 raft_consensus.cc:697] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:03.855626 11878 consensus_queue.cc:1059] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:03:03.878182 12009 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.891229 11846 raft_consensus.cc:697] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:03.918795  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:03.865252 12003 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:03.933756  8401 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:03:03.810010 12023 consensus_queue.cc:237] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.939582  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:03.875998 12120 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:03.936983  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 47202.9us
I20260709 13:03:03.943478 12117 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.919515 12038 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:03.944566 12021 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:03.907137 12022 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:03.884716 11729 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.947307  8398 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:03:03.947510  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:03.942374 12114 consensus_queue.cc:260] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.880956 11959 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:03.948764  8401 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:03.944073 12094 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:03.952332  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:03.978636  8404 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:03:03.891445 11883 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.924860 11928 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:03.979746  8403 raft_consensus.cc:2410] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:03:03.924479 11685 consensus_queue.cc:260] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.982131 11685 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:03.983062  8404 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:03:03.984329  8404 raft_consensus.cc:1275] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:03.946797 11969 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:03:03.986631 11883 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:03.981166 11846 consensus_queue.cc:237] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.986975 11969 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.987462 11928 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:03.987736 11969 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.989473 11928 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:03.989828 12038 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.990387 12038 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:03.990967 11729 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:03.991720 11969 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:03.992159  8398 raft_consensus.cc:1275] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:03:03.993710 11748 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:03.993891  8404 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:03:03.994287  8404 pending_rounds.cc:85] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 4
I20260709 13:03:03.994616  8404 pending_rounds.cc:107] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:03:03.994907  8404 raft_consensus.cc:2889] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:03.996390  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 103419us
I20260709 13:03:03.954550 11878 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:03:03.965322  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 113608us
I20260709 13:03:03.998582  8297 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:03.945894 12123 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.010632  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:03.955193 11872 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:03.972590 12104 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:04.014518  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:04.016284  8403 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:03.926342 12009 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:04.021826  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:04.001972 12120 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:04.005759 12100 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.004870 12022 raft_consensus.cc:697] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 13 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:04.012068 11883 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:03:04.039026  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 103748us
I20260709 13:03:04.026733  9006 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
W20260709 13:03:04.028394 12124 log.cc:927] Time spent T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:03:04.028782 11911 consensus_queue.cc:1059] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:04.025882 11878 consensus_queue.cc:1048] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:03:04.030390 12100 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:04.031032 12047 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:04.031497 12134 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:04.052074 12065 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:04.034796 11959 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:04.036402  9005 raft_consensus.cc:1275] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:04.037447 12009 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:04.025249  8403 raft_consensus.cc:1275] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:04.016295 12021 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:04.016186 12038 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:04.017590 12096 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:04.046756 12133 raft_consensus.cc:3055] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:04.048350 12100 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:04.044023 12094 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.955967 12117 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:04.066084 12094 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:04.061923 12049 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.067492 12117 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:04.053975 12022 consensus_queue.cc:237] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:04.019091 12114 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:04.068198 12094 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:04.043191 11883 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:04.020328 11963 consensus_queue.cc:237] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:04.067692 12049 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:04.026419 11984 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:04.069980  8369 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:04.071780  8220 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:04.087888 12125 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.089334 11883 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:04.088693 12125 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:04.069260 12133 raft_consensus.cc:740] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:04.069074 12096 raft_consensus.cc:697] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:04.020488 12123 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:04.111994  8220 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:04.122102  8332 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:03:04.105511 11872 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:03:04.125137  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 109935us
I20260709 13:03:04.126236  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.071841 12021 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:04.111963 12047 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:04.129727  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 69083.9us
I20260709 13:03:04.131536  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.132265  8332 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:03:04.134210  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:04.123468 12100 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:04.137601  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:04.138226  8328 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:04.140756  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.141463  8332 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
W20260709 13:03:04.145831  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:04.146648 12139 log.cc:927] Time spent T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:03:04.148084 12096 consensus_queue.cc:237] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:04.123678 12094 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:03:04.142150 12133 consensus_queue.cc:260] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:04.120985 11745 consensus_queue.cc:1048] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:03:04.152581  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.154934 12125 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:04.162547  8324 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:03:04.145622 12100 raft_consensus.cc:697] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:04.163996 11983 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11.
I20260709 13:03:04.165783 12021 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:04.169868  8332 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:04.174744  8328 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
I20260709 13:03:04.175638  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.176911  8324 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:03:04.156863 12134 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:04.186651  8331 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:04.186848  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.187925  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.188047  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:04.188654  8326 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:04.188906  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:04.190110  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:04.193205  8330 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:04.193773  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:04.194357  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:04.194832  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:04.194974  8330 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:04.195369  8329 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:04.195744  8326 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:03:04.196595  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:04.200177  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:04.204865  8330 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:03:04.206837  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:04.207743  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 50123.6us
W20260709 13:03:04.209237  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.211055  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:04.219436  8325 raft_consensus.cc:2410] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 7 because replica has last-logged OpId of term: 6 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20260709 13:03:04.220849  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.221508  8328 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
W20260709 13:03:04.212908 12145 log.cc:927] Time spent T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:03:04.222615  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.223359 12036 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.224156  8324 raft_consensus.cc:2376] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8. Re-sending same reply.
I20260709 13:03:04.224853 12049 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:04.224781 12036 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:04.226004 12123 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:04.226853 12036 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:04.164243 12100 consensus_queue.cc:237] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:04.214607 11872 raft_consensus.cc:2764] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:03:04.246055  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.246979  8324 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
W20260709 13:03:04.252427  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 253872us
I20260709 13:03:04.255753  8329 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:03:04.253468  8297 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:04.259465  8297 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:04.260385  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:04.262856  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.264431 12134 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:04.265846 11872 consensus_queue.cc:1059] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:04.266746 11872 consensus_queue.cc:1048] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:04.267345  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:04.268288 11872 raft_consensus.cc:3055] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:04.268689 11872 raft_consensus.cc:740] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:04.268869 12110 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:04.269290 12110 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:04.269742 12110 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:04.269820 11872 consensus_queue.cc:260] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:04.271323 11872 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:04.277562 12120 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:04.278039 12120 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:04.278416 12120 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:04.268694  8369 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:04.284394  8331 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:03:04.292035  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:04.295684  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.298311  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.300338  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:04.301102  8401 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 12: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:04.301739  8324 raft_consensus.cc:1275] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:03:04.304208  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:04.304962  8323 raft_consensus.cc:2376] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12. Re-sending same reply.
I20260709 13:03:04.305410 12120 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:04.306412  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:04.307022  8403 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:04.307739 12120 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:04.307904  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:04.308681  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:04.309322  8405 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:03:04.309792  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.309828  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:04.309057  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:04.292349 11872 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:04.294891 12134 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.317008  8323 raft_consensus.cc:3055] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:04.317574  8323 raft_consensus.cc:740] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:04.318470  8323 consensus_queue.cc:260] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:04.320219  8323 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:04.321959  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.316999 12104 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:03:04.323132  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.318678 12134 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:04.326730  8328 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:04.328572 12134 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:04.333235 12104 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:04.333876 12104 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:04.333259  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.336298  8222 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:04.342207  8406 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:04.342625  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:04.343164  8403 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:03:04.345901  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:04.346649  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:04.348315  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:04.348814  8401 raft_consensus.cc:1275] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:03:04.352119  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:04.354470  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:04.355096  8404 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:04.357316 11872 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:04.366269  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:04.374294  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.375218  8326 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:03:04.346143 11850 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:04.379527  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.380321  8329 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:04.384155  8326 raft_consensus.cc:1275] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:03:04.385772  8399 raft_consensus.cc:3055] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:04.386256  8399 raft_consensus.cc:740] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:04.384624  8326 pending_rounds.cc:85] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 4
I20260709 13:03:04.390307  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:04.390666  8326 pending_rounds.cc:107] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:03:04.393129  8326 raft_consensus.cc:2889] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:04.391754  8323 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:03:04.392328  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.393949  8330 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:03:04.391185  8398 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:04.394054  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:04.395506  8372 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.989s	user 0.005s	sys 0.030s
W20260709 13:03:04.395849  8372 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.996s	user 0.005s	sys 0.030s
I20260709 13:03:04.395941  8329 raft_consensus.cc:2376] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8. Re-sending same reply.
W20260709 13:03:04.396130  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 1.02168e+06us
I20260709 13:03:04.381950 11850 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.402324  8372 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:04.403438  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.404330  8326 raft_consensus.cc:2410] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:03:04.404214 11872 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:04.404958 11872 raft_consensus.cc:697] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:04.405230  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.405522  8220 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:04.405859 11872 consensus_queue.cc:237] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:04.407326 12092 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:04.408010 12092 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.408427 12092 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:04.409596 11848 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:04.410985  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:04.412094  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:04.413184  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:04.413712  9008 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:04.406332  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.417810  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:04.418105  8326 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:03:04.383793 12110 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:04.419297  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.420459 12110 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:04.421526  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:04.424212  8220 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:04.435040  8220 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:04.436187  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:04.437136  8398 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:04.439174  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:04.439332  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.439811  8255 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:03:04.437232  8248 raft_consensus.cc:2410] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:03:04.442029  9008 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:04.424654  8399 consensus_queue.cc:260] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:04.441692  8972 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:04.444136  8223 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:04.446619  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:04.446942  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.440003  8220 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:04.447510  8329 raft_consensus.cc:2376] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10. Re-sending same reply.
I20260709 13:03:04.458352  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:04.463660  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:04.478361  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.479100  8252 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:03:04.479267  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.478660  8297 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:04.480466 11928 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:04.480855 11928 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.481231 11928 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:04.480988  8297 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:04.482312 12147 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:04.482937 12147 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.483377 12147 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:04.489223 11928 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:04.491807 11928 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:04.493355  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:04.493672  8297 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:04.494720  8329 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 6.
I20260709 13:03:04.445638 12094 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:04.505811  8399 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:04.533218  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:04.534648  8256 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:04.536065  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 141149us
I20260709 13:03:04.436302 11745 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:04.430277 11850 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:04.538758  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 45209.6us
I20260709 13:03:04.521543  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:04.505954  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 66005.8us
I20260709 13:03:04.541807  8220 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:04.449450 11891 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:04.549175  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:04.445762 12033 consensus_queue.cc:1048] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.106s
I20260709 13:03:04.446308 12104 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:04.551934 12036 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.263s
I20260709 13:03:04.789547 11932 raft_consensus.cc:3055] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:03:04.619565  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.457559 12123 consensus_queue.cc:1059] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.131s
I20260709 13:03:04.547498 12120 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:04.767517 12119 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:03:04.630697  8329 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 7 index: 10. (index mismatch)
I20260709 13:03:04.901336 12119 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.607187 11891 raft_consensus.cc:697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:04.457537 11848 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:04.836510  9005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 690) took 1059 ms. Trace:
I20260709 13:03:04.901934 12119 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:04.627104  8399 raft_consensus.cc:1275] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:03:04.903450  8399 pending_rounds.cc:85] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 4
I20260709 13:03:04.902911 11891 consensus_queue.cc:237] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:04.903688  8399 pending_rounds.cc:107] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:03:04.903935  8399 raft_consensus.cc:2889] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:04.904587 12119 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:04.587803 11878 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.817092  8222 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:04.905534 11878 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:04.722632  8404 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:03:04.906405 12123 raft_consensus.cc:3055] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:03:04.649829  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 108187us
I20260709 13:03:04.906848 12123 raft_consensus.cc:740] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:03:04.675683  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 115188us
I20260709 13:03:04.907752 12123 consensus_queue.cc:260] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:04.908140  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:04.908560  8220 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:04.908906 12123 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:04.619565  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.909473 11745 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:04.909816  8220 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:04.910343  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.636193 12147 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:04.911072  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.911819  8331 raft_consensus.cc:2368] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9.
I20260709 13:03:04.703677 12003 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:04.912436 12110 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:04.912787 12003 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.913168 12110 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.913506 12021 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:04.913281 12003 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:04.594761 12047 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:04.914098 12021 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:04.914709 12047 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.915113 12047 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:04.437678 12049 consensus_queue.cc:1048] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.104s
I20260709 13:03:04.678964  8403 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:03:04.916334 12021 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:04.657210  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.633632  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:04.917284  8398 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:04.570016  8400 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
I20260709 13:03:04.532989 12134 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.090s
W20260709 13:03:04.714349 12150 log.cc:927] Time spent T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:03:04.533591 12114 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:04.684085  8972 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:04.788094 12094 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
W20260709 13:03:04.815912 12130 log.cc:927] Time spent T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.732s	user 0.003s	sys 0.000s
W20260709 13:03:04.905937  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 88900.5us
I20260709 13:03:04.948084  8222 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:04.626595  8256 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
W20260709 13:03:04.637045  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:04.950080  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.604609  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:04.911011  8369 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:04.902841  9005 rpcz_store.cc:276] 0709 13:03:03.776959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.777137 (+   178us) service_pool.cc:224] Handling call
0709 13:03:03.814320 (+ 37183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.060136 (+245816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.826229 (+766093us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b54005a8cd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:04.836433 (+ 10204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":509710592}
I20260709 13:03:04.953779  8406 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:03:04.537233 12065 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:03:04.933491  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.944532  8220 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:04.958166  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 274181us
I20260709 13:03:04.958220  8398 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:04.953315  8297 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:04.963196  8972 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:04.947263 12105 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.969944 12104 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:04.970539 12105 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:04.972685 12125 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.972820 12105 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:04.913882 12110 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:04.973589 12092 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:04.813074 11961 consensus_queue.cc:1048] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.153s
I20260709 13:03:04.946923 12131 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:04.974509 12131 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:04.974908 12131 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:04.975301 12110 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:04.977669  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 66727.5us
I20260709 13:03:04.977927  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:04.978960  8369 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:04.979113 11745 consensus_queue.cc:1048] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:04.979735 12104 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:04.980177  8369 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:04.970924  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:04.977717  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:04.973308 12125 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:04.919703 12021 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:04.920784 11932 raft_consensus.cc:740] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:04.913180 12147 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:04.938875 12100 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:05.005014  9008 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:04.925199 12062 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:04.784584 12064 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.967195 12065 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:05.016733  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 72272.9us
I20260709 13:03:04.982754  8222 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:05.033149 12062 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:05.033737 12062 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.033344 11932 consensus_queue.cc:260] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:05.034116 12062 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:05.034767 11932 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:05.019044 12077 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:05.039348 12003 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:05.021396 12085 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.025s	sys 0.058s
I20260709 13:03:05.041875 12064 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:05.044687 12064 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:05.045907 11932 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:05.046474 11932 raft_consensus.cc:697] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:05.047343 11932 consensus_queue.cc:237] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:05.050319  9005 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:03:04.964989 11926 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:05.055086  8297 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:05.057088 12141 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:05.057782 12141 raft_consensus.cc:697] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:05.058807 12141 consensus_queue.cc:237] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:05.059306 12131 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:04.607424 12096 consensus_queue.cc:1059] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.352s
W20260709 13:03:05.032759  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 50097.7us
W20260709 13:03:05.064217  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 84080.8us
I20260709 13:03:05.065403  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:05.065761  8369 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:04.977880 11848 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:05.006071 12120 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:05.074074  8398 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:05.016280 12079 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:05.078467  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:05.021639 11983 consensus_queue.cc:1059] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:03:05.081990  8222 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:05.062281 11872 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:05.019135 12036 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.070930 12065 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:05.090978  8369 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:05.071847 11848 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:05.126099  8324 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.024s	sys 0.040s
W20260709 13:03:05.126611  8324 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.024s	sys 0.040s
I20260709 13:03:05.018229 12100 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.077116 12049 consensus_queue.cc:1059] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:03:05.090325 11745 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:05.090116 12085 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
W20260709 13:03:05.130833  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 46320.7us
W20260709 13:03:05.087885 12022 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.005s	sys 0.015s
I20260709 13:03:05.077461 12082 raft_consensus.cc:2764] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
I20260709 13:03:04.957638 12114 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:05.134725  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:05.140990  8405 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:03:05.141940  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:05.085731 12120 raft_consensus.cc:697] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:05.002249 12110 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:05.135946  8295 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:05.135509 12100 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:05.138072  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:05.131358 12134 raft_consensus.cc:3055] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:03:05.150882  8251 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.017s	sys 0.020s
W20260709 13:03:05.185027  8251 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.017s	sys 0.020s
I20260709 13:03:05.152724  8405 raft_consensus.cc:1240] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 10. Current term is 11. Ops: [10.5-10.5]
W20260709 13:03:05.138203 12022 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.005s	sys 0.015s
I20260709 13:03:05.019184 12021 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:05.194988  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 104085us
W20260709 13:03:05.231017  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 50305.8us
I20260709 13:03:05.250452  8295 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:05.091506 12066 consensus_queue.cc:1048] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 13:03:05.147984 11745 raft_consensus.cc:697] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 8 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:05.087085 12033 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:03:05.137379 12003 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:05.147056 12131 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:05.255028  8399 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:03:05.130163 12036 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:05.139400 11983 consensus_queue.cc:1059] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.099s
W20260709 13:03:05.256848  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 293713us
I20260709 13:03:05.256444  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:05.184617 12134 raft_consensus.cc:740] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:05.240993 11926 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:05.266371  9007 raft_consensus.cc:1240] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 13 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:05.241729 12094 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.302086  8972 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:05.303622  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.303617  8399 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:03:05.297956 12003 consensus_queue.cc:1048] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.016s
W20260709 13:03:05.316109  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.303977  8972 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:05.334678  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.265975 12162 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.051s
W20260709 13:03:05.372350  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 66904.4us
I20260709 13:03:05.373152  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:05.283890 12094 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:05.318650 12147 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:03:05.396440  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:05.398170 12163 log.cc:927] Time spent T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:03:05.401046 12169 raft_consensus.cc:3055] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:05.300887 12104 consensus_queue.cc:1048] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:03:05.378268 12133 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:05.396658 12062 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:05.313375 11745 consensus_queue.cc:237] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:05.381778 12125 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:05.402562 12050 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:05.402783 12022 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:03:05.434168  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 61069.7us
I20260709 13:03:05.412459 12147 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.305421 12120 consensus_queue.cc:237] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:05.462042  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:05.415047 12169 raft_consensus.cc:740] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:03:05.466393  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 216003us
I20260709 13:03:05.469940  8295 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:05.473380  8400 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:05.324139 12134 consensus_queue.cc:260] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:05.410740 12023 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:03:05.421386 12173 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:05.482368  8295 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:05.465025 12147 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:05.488673  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:05.493901  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:05.498071  8324 raft_consensus.cc:1275] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:05.500248 12064 raft_consensus.cc:3055] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:05.500880 12064 raft_consensus.cc:740] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:05.501902 12064 consensus_queue.cc:260] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:05.502998  8331 raft_consensus.cc:1275] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:03:05.503728 12064 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:05.507511 11848 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:05.506207  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:05.508513  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:05.509289  8324 raft_consensus.cc:2393] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 7: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:03:05.510249 12165 log.cc:927] Time spent T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 13:03:05.514313  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:05.515079  8324 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 11: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:05.517947 12182 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:05.518422 12182 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.518424  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:05.518934 12182 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:05.519291  8329 raft_consensus.cc:2393] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 6: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:05.523957  9007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 808) took 1097 ms. Trace:
I20260709 13:03:05.524452  9007 rpcz_store.cc:276] 0709 13:03:04.426244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.426393 (+   149us) service_pool.cc:224] Handling call
0709 13:03:04.446129 (+ 19736us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:05.272447 (+826318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.523632 (+251185us) spinlock_profiling.cc:243] Waited 818 ms on lock 0x7b54006c2d54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:05.523880 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1881704832}
I20260709 13:03:05.526470  8329 raft_consensus.cc:1240] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: [5.3-8.5]
W20260709 13:03:05.527452  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.527333 12177 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.528275 12177 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:05.530568 12177 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:05.534446  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:05.487767 11850 consensus_meta.cc:287] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.894s	user 0.008s	sys 0.004s
W20260709 13:03:05.493348 12161 log.cc:927] Time spent T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.247s	user 0.000s	sys 0.001s
I20260709 13:03:05.489672 12050 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:05.484963 12173 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.546823  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:05.485359 12022 raft_consensus.cc:697] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:05.547709  8326 raft_consensus.cc:2393] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 10: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:05.550782  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:03:05.533797 12158 log.cc:927] Time spent T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.294s	user 0.001s	sys 0.000s
I20260709 13:03:05.553803  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:05.556422  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 74131.7us
I20260709 13:03:05.496954 12021 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:05.561751  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:05.546825 12134 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:05.569576  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:05.573555  8326 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:05.578141  9007 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:05.580785 12184 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.581307 12184 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:05.582408 12174 log.cc:927] Time spent T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:03:05.583415 12184 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:05.586202 12022 consensus_queue.cc:237] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:05.555807 12169 consensus_queue.cc:260] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:05.594404 12169 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:05.598891  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 553) took 1158 ms. Trace:
I20260709 13:03:05.599267  8250 rpcz_store.cc:276] 0709 13:03:04.440660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.440768 (+   108us) service_pool.cc:224] Handling call
0709 13:03:04.441028 (+   260us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:04.484008 (+ 42980us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:04.484038 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:04.484717 (+   679us) log.cc:844] Serialized 78 byte log entry
0709 13:03:04.918219 (+433502us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:04.976749 (+ 58530us) log.cc:844] Serialized 14 byte log entry
0709 13:03:04.977054 (+   305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.977130 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:05.598639 (+621509us) raft_consensus.cc:1685] finished
0709 13:03:05.598659 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.598809 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":41621,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":62844,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":21821,"thread_start_us":433128,"threads_started":1,"wal-append.queue_time_us":601849}
I20260709 13:03:05.552918 12104 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.602118  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2298) took 1260 ms. Trace:
W20260709 13:03:05.607462  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.552297 12173 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:05.610960  8325 rpcz_store.cc:276] 0709 13:03:04.342094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.342253 (+   159us) service_pool.cc:224] Handling call
0709 13:03:04.342568 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:04.347602 (+  5034us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:04.351222 (+  3620us) log.cc:844] Serialized 14 byte log entry
0709 13:03:04.809964 (+458742us) spinlock_profiling.cc:243] Waited 4.57 ms on lock 0x7b18002ed020. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:03:04.919228 (+109264us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b18002ed020. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e23a 00007f88287c8a2a
0709 13:03:05.420716 (+501488us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:05.421264 (+   548us) log.cc:844] Serialized 26 byte log entry
0709 13:03:05.421463 (+   199us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:05.421758 (+   295us) log.cc:844] Serialized 14 byte log entry
0709 13:03:05.421928 (+   170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.422006 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:05.601908 (+179902us) raft_consensus.cc:1685] finished
0709 13:03:05.601929 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.602060 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5085,"prepare.queue_time_us":4081,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":4267,"raft.queue_time_us":214466,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":10092,"spinlock_wait_cycles":33578240,"thread_start_us":493250,"threads_started":2,"wal-append.queue_time_us":1119014}
I20260709 13:03:05.599119 12181 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:05.619100  8972 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:05.547408 11850 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:05.573007 12114 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:05.547784 12177 consensus_queue.cc:1059] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:03:05.623489  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:05.626515  8974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.000s	sys 0.022s
I20260709 13:03:05.629117  8325 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:03:05.623488  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:05.576470 11848 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:05.629140  8974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.000s	sys 0.022s
I20260709 13:03:05.634488  8974 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:05.624382 12023 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:05.639890  8972 leader_election.cc:400] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9.
I20260709 13:03:05.641665  8972 leader_election.cc:403] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:03:05.602761 12050 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.028s
W20260709 13:03:05.631990 12171 log.cc:927] Time spent T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:03:05.563472 12141 consensus_queue.cc:1048] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.061s
W20260709 13:03:05.647281  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 71914.6us
W20260709 13:03:05.639604 12065 consensus_meta.cc:287] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.500s	user 0.005s	sys 0.005s
I20260709 13:03:05.638511 12114 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:05.617933 12003 consensus_queue.cc:1059] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:03:05.643667 12177 consensus_queue.cc:1243] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:03:05.618528 12104 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:05.661021  9007 raft_consensus.cc:1217] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Deduplicated request from leader. Original: 8.9->[9.10-9.10]   Dedup: 9.10->[]
I20260709 13:03:05.684465  9007 raft_consensus.cc:1240] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:03:05.684587  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:05.687084  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:05.660563 12114 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:05.687589  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:05.687983 12114 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:05.688704 12190 consensus_queue.cc:1048] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:03:05.691387  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:05.691708  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:05.692255  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:05.692993  9008 raft_consensus.cc:2393] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 13: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:05.693365  8401 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:03:05.692804  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:05.694681  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:05.695549  8400 raft_consensus.cc:2393] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 12: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:05.696447  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:05.697755  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:05.702491 11850 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:05.702713  8402 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:03:05.704310 12184 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:05.707504 12184 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:03:05.709920 12160 log.cc:927] Time spent T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.792s	user 0.003s	sys 0.000s
I20260709 13:03:05.711393  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2311) took 1306 ms. Trace:
I20260709 13:03:05.711764  8330 rpcz_store.cc:276] 0709 13:03:04.404659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.422355 (+ 17696us) service_pool.cc:224] Handling call
0709 13:03:04.422652 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:04.423012 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:04.423042 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:04.423515 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 13:03:04.726659 (+303144us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:04.726712 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.726807 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:05.631583 (+904776us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:05.710993 (+ 79410us) raft_consensus.cc:1685] finished
0709 13:03:05.711013 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.711334 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":87,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"spinlock_wait_cycles":6140288,"thread_start_us":193165,"threads_started":1,"wal-append.queue_time_us":342415}
I20260709 13:03:05.713730  8324 raft_consensus.cc:1240] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:05.646770 12062 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:05.648680 12094 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:05.719492  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:05.745128  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.746567  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:05.747624  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 97817.9us
I20260709 13:03:05.657847 11932 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:05.748056  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:05.748921  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:03:05.730754  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.753774  8972 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:05.754604  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 115739us
I20260709 13:03:05.757038  8369 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:05.644150 12131 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:03:05.658269 12178 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:05.761972  8403 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:05.660385 12036 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:05.677273 12187 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.762704  8369 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:05.769610  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:05.767656  8323 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:03:05.773641  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:05.759886 12167 log.cc:927] Time spent T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
I20260709 13:03:05.651211 12021 consensus_queue.cc:1059] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.113s
I20260709 13:03:05.677747 11848 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.091s
I20260709 13:03:05.659113 12065 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:05.783720  8972 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:05.784739 12175 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.775990 12201 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:05.786410 11878 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:05.719901 12177 consensus_queue.cc:1048] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.090s
W20260709 13:03:05.805305 12100 consensus_meta.cc:287] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.554s	user 0.007s	sys 0.001s
I20260709 13:03:05.798448 12189 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:05.824826 12100 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:05.827090 12100 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:05.827922 12189 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:03:05.808159 12123 consensus_meta.cc:287] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.899s	user 0.011s	sys 0.000s
I20260709 13:03:05.821604 12175 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:05.823045 11872 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:05.797823 12050 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:05.831012 12050 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.831419 12050 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:05.832089 12082 raft_consensus.cc:3055] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:05.832471 12082 raft_consensus.cc:740] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:05.780180 12178 raft_consensus.cc:697] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:05.833228 12082 consensus_queue.cc:260] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:05.834244 12175 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:05.834539 12082 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:05.834120 12178 consensus_queue.cc:237] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:05.835929  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:05.837659 12186 log.cc:927] Time spent T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 13:03:05.839788 12022 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:05.842177 12065 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:05.777870 12187 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:05.845115 12173 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:05.845961 12187 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:05.846933 12022 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.847527 12022 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:05.849335  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.850482  8329 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:03:05.852006  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:05.829339 11872 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.847085  9005 raft_consensus.cc:2393] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 7: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:05.853852 11872 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:05.856650 12050 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:05.852583 12110 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.857750 12110 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:05.858745 12050 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:05.859710 12110 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:05.823094 12047 consensus_meta.cc:287] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.907s	user 0.000s	sys 0.010s
I20260709 13:03:05.860735 12047 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:05.863029 12047 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:05.863947  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 549) took 1425 ms. Trace:
I20260709 13:03:05.866024  8328 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:03:05.869164  9002 raft_consensus.cc:1240] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:05.872210  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:05.875342  8404 raft_consensus.cc:2410] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:03:05.872991  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:05.850678 12104 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:05.874101  8974 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:05.882345  8405 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:03:05.887073  8331 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:05.881264 12182 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:05.888672  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:05.900096  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:05.911854  8327 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:05.913030  8222 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:05.900669  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:05.860788 12114 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:05.881518 12110 consensus_queue.cc:1059] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:03:05.873908  8251 rpcz_store.cc:276] 0709 13:03:04.438083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.438203 (+   120us) service_pool.cc:224] Handling call
0709 13:03:04.458158 (+ 19955us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:04.542290 (+ 84132us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:04.542318 (+    28us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:04.553146 (+ 10828us) log.cc:844] Serialized 78 byte log entry
0709 13:03:05.185403 (+632257us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:05.238070 (+ 52667us) log.cc:844] Serialized 14 byte log entry
0709 13:03:05.863291 (+625221us) spinlock_profiling.cc:243] Waited 4.67 ms on lock 0x7b1800180340. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882564bc0a 00007f882ed8b219 00007f88256d3a40 00007f88256ce4e4 00007f88256ce455 00007f88287b6060 00007f88287c8716 00007f88287d9287 00007f88287d9208
0709 13:03:05.863492 (+   201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.863571 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:05.863619 (+    48us) raft_consensus.cc:1685] finished
0709 13:03:05.863710 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.863879 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60583,"prepare.queue_time_us":52691,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":2935,"raft.queue_time_us":672592,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":11725,"spinlock_wait_cycles":10734464,"thread_start_us":567569,"threads_started":1,"wal-append.queue_time_us":702017}
I20260709 13:03:05.915841  8297 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:05.921491  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:05.922374  8328 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:05.919515  8972 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:05.925415  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:05.877645 12141 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:03:05.911788 12170 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.501s	user 0.004s	sys 0.000s
I20260709 13:03:05.905503 11848 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:05.904963 12033 raft_consensus.cc:3055] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:05.945710  8325 raft_consensus.cc:1240] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 11. Current term is 12. Ops: [9.6-11.7]
W20260709 13:03:05.945873  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.944579  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:05.956542  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:05.958302  8250 raft_consensus.cc:2410] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:03:05.921015 11992 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.945281 12123 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:05.899992 12187 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.909744 12181 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.951272  9008 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:05.951957  8251 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:03:05.952939  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:05.953270  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:05.990063  8253 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:05.914242 12190 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.958698  8256 raft_consensus.cc:2393] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 8: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:05.990732  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:05.956737  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:05.927462 11926 consensus_meta.cc:287] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.641s	user 0.010s	sys 0.000s
I20260709 13:03:05.993152  8248 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 12: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:05.952404 11848 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:05.987067  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 74143.9us
I20260709 13:03:05.987188  8257 raft_consensus.cc:1275] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:05.945358 12023 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:05.944617 12036 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:05.947657  8252 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:06.002734  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:05.990849  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:06.003005  8253 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:06.003665  8250 raft_consensus.cc:2368] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9.
I20260709 13:03:05.905790 12125 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.984066  8328 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:03:05.980635 12096 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.009094  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:06.011601  8249 raft_consensus.cc:2410] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 9 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:03:05.909552 12131 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:03:05.927600 12189 consensus_queue.cc:1059] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:03:06.037253  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:05.999169  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:06.038352  8250 raft_consensus.cc:2410] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:03:05.956032 12033 raft_consensus.cc:740] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:05.913885 12175 consensus_queue.cc:1059] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:03:05.980212 12022 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:05.948139 12104 raft_consensus.cc:3055] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:06.007424  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:06.008874  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.008965  8256 raft_consensus.cc:1275] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:03:06.017925  8248 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:05.995407 12187 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:05.995299 12123 raft_consensus.cc:697] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:06.020169 12036 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:06.021899 12065 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.021766  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:05.992645 12085 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:05.990725 11992 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:05.995003 11926 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:05.997291 12190 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:05.950963 12141 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:05.996583 12181 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:05.995740 12062 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:03:06.043764  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 274190us
I20260709 13:03:06.041728  8972 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:06.047863  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:06.084573  8404 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:05.997392 12003 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:06.053882  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:06.051250  8250 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:03:06.057560  8256 pending_rounds.cc:85] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 6
I20260709 13:03:06.068234 12085 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.021826 11748 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.069419  8298 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:06.086858 11926 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:06.103420 11748 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:06.103590 12062 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:06.037992 12120 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:06.105733 12120 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:06.106300 12062 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:06.107877 12120 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:06.056358 12104 raft_consensus.cc:740] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:06.109011 12104 consensus_queue.cc:260] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:06.110311 12104 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:06.087783  8256 pending_rounds.cc:107] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:03:05.997377 11848 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:06.052304 12172 consensus_queue.cc:1059] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:03:06.036296 12125 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:06.082679 12134 consensus_meta.cc:287] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.000s
I20260709 13:03:06.103663 11926 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:06.100996  8248 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:03:06.071051  8249 raft_consensus.cc:1240] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:03:06.137759  9009 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:06.174046  8256 raft_consensus.cc:2889] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:06.140923  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:06.120721  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:06.138535  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:06.138238  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:06.115670  8254 raft_consensus.cc:2410] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:03:06.129237  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:06.118094  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:06.176956  8331 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:03:06.177292  8406 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:06.176376  8328 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:03:06.175447  8400 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:06.129688  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 60342.5us
I20260709 13:03:06.179291  8298 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:06.180750  8404 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:06.165551  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:06.046789 12096 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:06.186758  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:06.176707  8250 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:03:06.052779 11872 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:06.185472  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:06.188100  9007 raft_consensus.cc:2393] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:03:06.188460  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.188966  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:06.193643  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:06.079677 12207 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:06.194368  8369 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:06.194381  8323 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:03:06.207264  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:06.207899  8332 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:03:06.186123  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:06.204936  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:06.208665  8331 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 6.
I20260709 13:03:06.205564  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:06.207978  8323 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:06.209498  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:06.072742 12196 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.027s
W20260709 13:03:06.206561  8327 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 472) took 2019 ms (client timeout 1740 ms). Trace:
I20260709 13:03:06.102409 12085 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:06.215477  8327 rpcz_store.cc:269] 0709 13:03:04.186712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.187552 (+   840us) service_pool.cc:224] Handling call
0709 13:03:06.206322 (+2018770us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b540008ff54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:06.206481 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3962781056}
I20260709 13:03:06.210078  8406 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:03:06.095674 12023 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:06.106603 11992 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:06.111761 11932 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9.
I20260709 13:03:06.220216  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:06.272097 12114 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:06.272492  8255 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:06.221191 12064 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.204219  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:06.122727 12182 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:06.118108 12036 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:06.273416 12064 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:06.226475  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:06.274163  9009 raft_consensus.cc:2410] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:03:06.197233 12201 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:06.272128  9004 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:06.219755 12207 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:06.221642 12196 consensus_queue.cc:1243] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:03:06.279913 12172 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.280047 11872 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:06.280579 12172 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:06.282256 12125 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:06.282891 12172 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:06.248383 11748 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:06.249949  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:06.220548 12134 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:06.074679 12065 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:06.127640 12181 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:06.101388 12003 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:06.294166 12003 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:06.294525 12065 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:06.097173 12033 consensus_queue.cc:260] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:06.120553 12123 consensus_queue.cc:237] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:06.296357 12065 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.269989 12023 raft_consensus.cc:697] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:06.296896 12065 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:06.298207 12023 consensus_queue.cc:237] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:06.206307 12131 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:06.299960 12131 raft_consensus.cc:697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:06.300832 12131 consensus_queue.cc:237] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:06.184729 12119 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:06.302345  8327 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:03:06.271188  8403 raft_consensus.cc:2393] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 10: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:06.221755  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:06.303277  8328 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:06.304049  8327 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:03:06.304804  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:06.221277  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:06.305470  9007 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:06.306347  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:06.306938  8249 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
W20260709 13:03:06.307479 12164 log.cc:927] Time spent T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.938s	user 0.003s	sys 0.000s
I20260709 13:03:06.288019 12207 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:06.288897 12206 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:06.290133  8323 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
W20260709 13:03:06.297266 12147 consensus_meta.cc:287] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.761s	user 0.009s	sys 0.000s
I20260709 13:03:06.307772  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:06.320482  9003 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
W20260709 13:03:06.825522  8400 consensus_meta.cc:287] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.647s	user 0.013s	sys 0.013s
I20260709 13:03:06.420156  9004 pending_rounds.cc:85] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e: Aborting all ops after (but not including) 7
I20260709 13:03:06.826882  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:06.834916  9001 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:06.832574  8249 raft_consensus.cc:2410] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:03:06.326786 12064 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:06.767928 12047 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:06.829785  8972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.014s	sys 0.036s
W20260709 13:03:06.845852  8972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.014s	sys 0.036s
W20260709 13:03:06.846421  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 705498us
W20260709 13:03:06.825520  8328 consensus_meta.cc:287] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.522s	user 0.023s	sys 0.025s
I20260709 13:03:06.348641  8255 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:03:06.834030  9004 pending_rounds.cc:107] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:03:06.330576 11685 raft_consensus.cc:3055] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:06.831502  8400 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:03:06.840108  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:06.476962  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 279067us
I20260709 13:03:06.847810  8328 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:06.850701  8323 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:03:06.848026  9004 raft_consensus.cc:2889] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:06.851015  8295 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:06.832545 12147 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:06.854087  8295 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:06.858136  8400 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:03:06.862885  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:06.864465  8327 raft_consensus.cc:1240] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:03:06.866319  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:06.866600  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:06.867836  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:06.323812 12187 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:06.873482  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:06.330219 12173 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 7 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:06.866329  8323 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:06.624847 11932 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.868944 12207 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:06.884161  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:06.884584 12207 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:06.885326  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:06.885921 12065 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:06.887132 12134 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:06.887401  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:06.888139  8399 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:06.888626 12134 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.889760  8405 raft_consensus.cc:1275] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:03:06.542016 12085 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:06.835311 12190 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:06.910892  8249 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:06.876323  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2074) took 1359 ms. Trace:
W20260709 13:03:06.852876 11848 consensus_meta.cc:287] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.648s	user 0.013s	sys 0.001s
I20260709 13:03:06.877120  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:06.882427  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:06.912281  8324 rpcz_store.cc:276] 0709 13:03:05.516820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.516950 (+   130us) service_pool.cc:224] Handling call
0709 13:03:05.713543 (+196593us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:05.714197 (+   654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.875833 (+1161636us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b540029aed0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:06.876103 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":451377024}
I20260709 13:03:06.916108  8400 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:06.883198 12147 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:06.883819  8326 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:03:06.897758  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:06.919976  8323 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:06.914412  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 60418.9us
I20260709 13:03:06.851512 11685 raft_consensus.cc:740] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:06.869647 12206 raft_consensus.cc:697] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:06.938303  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:06.938881  8327 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:06.858701 12047 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:06.889179 12134 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:06.947427 12134 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:06.948522  8223 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:06.929912 11848 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:06.950176 12065 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:06.951633  9006 raft_consensus.cc:2393] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 12: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:06.951534 12065 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:03:06.952136  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:06.952569  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:06.952934  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.933750  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:06.901146 12114 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:06.924144 12220 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:06.955214  8223 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:06.956732  9005 raft_consensus.cc:1240] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:06.947789  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:06.962247  8295 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:06.932637  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:06.924417 12050 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:06.965866  9006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56948 (request call id 814) took 1086 ms. Trace:
I20260709 13:03:06.923928 12173 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:06.968739  8223 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:06.966648  8255 raft_consensus.cc:2368] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8.
I20260709 13:03:06.974002  8295 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:06.970405  9006 rpcz_store.cc:276] 0709 13:03:05.867839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.878942 (+ 11103us) service_pool.cc:224] Handling call
0709 13:03:06.953823 (+1074881us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400598dd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:06.954039 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2384344832}
I20260709 13:03:06.953322 12065 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:06.980201  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:06.979712 12173 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:06.963869 12096 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:06.985630 12206 consensus_queue.cc:237] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:06.910079 11932 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:06.975596 12134 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:06.999125  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:07.000609  8323 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:06.971930 12064 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:06.968851 12114 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:06.976449 12050 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:06.991739 12220 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:07.024416  8323 raft_consensus.cc:1240] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:03:07.028198  8295 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:07.020815  8249 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:07.029541  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:07.030839  8323 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:03:07.034834 12228 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:07.035429 12228 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:07.048728  8295 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:07.051065 12231 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:07.052698  8295 leader_election.cc:400] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9.
W20260709 13:03:07.056217 12226 log.cc:927] Time spent T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:03:07.056901 12224 log.cc:927] Time spent T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:03:07.057063 12228 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:07.057798 12228 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:07.058542  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 506) took 1227 ms. Trace:
I20260709 13:03:07.004014 12162 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:03:07.059053  8331 rpcz_store.cc:276] 0709 13:03:05.830677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.274840 (+444163us) service_pool.cc:224] Handling call
0709 13:03:06.275296 (+   456us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:06.275717 (+   421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:06.275756 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:06.276450 (+   694us) log.cc:844] Serialized 52 byte log entry
0709 13:03:06.445925 (+169475us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:06.445968 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.446039 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:07.058291 (+612252us) raft_consensus.cc:1685] finished
0709 13:03:07.058314 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.058466 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1421,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":107,"thread_start_us":169006,"threads_started":1,"wal-append.queue_time_us":704112}
I20260709 13:03:07.081219  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:07.047428 12223 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.050414 12220 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:07.095441  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:07.094658  9001 raft_consensus.cc:1275] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:03:07.081205  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:06.993602 11685 consensus_queue.cc:260] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:07.061753 12064 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:07.149799  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:07.150920  8249 raft_consensus.cc:2393] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 12: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:03:07.160494  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 203273us
I20260709 13:03:07.163859  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:07.164803  8249 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:03:07.094516 12047 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:07.136979 11878 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.113379 12207 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:07.171861  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:07.172917  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:07.083214 12134 consensus_queue.cc:1059] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.079s
I20260709 13:03:07.149055 12190 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.149288 12064 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.143306 12147 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:07.187832  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:03:07.190891  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.193284  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:07.190220  9001 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:03:07.189302  9000 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:03:07.070104 12230 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.122s
I20260709 13:03:07.082561 12231 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.105s
I20260709 13:03:07.161523 12085 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:07.206646  8327 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:07.209746  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:07.229766  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.230695 12023 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.230574  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:07.175271 12232 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:07.231362 12023 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:07.231693 12232 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:07.232597  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:07.233321  8248 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:07.234592 12232 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:07.234957 12023 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:07.236794 12023 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.240841  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:07.241911  8249 raft_consensus.cc:2368] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7.
I20260709 13:03:07.239419  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1914) took 1203 ms. Trace:
I20260709 13:03:07.202386  8298 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:07.111815 12222 consensus_queue.cc:1048] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.126s
I20260709 13:03:07.251694  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:07.243180  8257 rpcz_store.cc:276] 0709 13:03:06.035638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.046252 (+ 10614us) service_pool.cc:224] Handling call
0709 13:03:06.072003 (+ 25751us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:06.080845 (+  8842us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:06.080877 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:06.103765 (+ 22888us) log.cc:844] Serialized 52 byte log entry
0709 13:03:06.357990 (+254225us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:06.830051 (+472061us) log.cc:844] Serialized 14 byte log entry
0709 13:03:06.995560 (+165509us) log.cc:844] Serialized 14 byte log entry
0709 13:03:07.238998 (+243438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.239103 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:07.239160 (+    57us) raft_consensus.cc:1685] finished
0709 13:03:07.239171 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.239342 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10770,"prepare.queue_time_us":424826,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":66,"raft.queue_time_us":248754,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":14094,"thread_start_us":1072189,"threads_started":4,"wal-append.queue_time_us":783896}
I20260709 13:03:07.256189  8220 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:07.199841 12207 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.146700 12223 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:07.209708 12064 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:07.268203  8369 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:07.256739  8372 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:07.264971  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:07.266667  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:07.284488  8257 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:07.285132  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:07.300608  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:07.191210 11878 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:07.311430  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:07.306348  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:07.244799 12233 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.307018  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:07.308815  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:07.310194  8249 raft_consensus.cc:1275] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:07.255740 12123 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:07.219875 12147 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:07.327934  8249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 556) took 1269 ms. Trace:
I20260709 13:03:07.328158  8257 raft_consensus.cc:2393] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 8: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:07.210795 12190 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:07.311931  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:07.285547 12206 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:07.289453 12047 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.344345  8249 rpcz_store.cc:276] 0709 13:03:06.058688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.301014 (+1242326us) service_pool.cc:224] Handling call
0709 13:03:07.301444 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:07.327697 (+ 26253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.327882 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:07.352023  8248 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:07.306154 12240 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.247671 12023 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:07.360195  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 104090us
W20260709 13:03:07.374143  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 106046us
I20260709 13:03:07.377875  8369 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:07.377854  8220 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:07.317533 12239 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:07.337267 12123 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.289651 12207 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:07.336243 11932 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:07.337607 12209 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:07.347451 12169 raft_consensus.cc:3055] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:07.347718 12184 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:07.412910 12184 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.358543 12141 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:07.386376  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 50084.3us
I20260709 13:03:07.415805  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:07.347708 12246 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:07.429842 12184 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:07.414116 12169 raft_consensus.cc:740] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:07.385533 12114 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:07.381419 12050 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:07.358418 12131 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 13:03:07.452011  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1969) took 2343 ms. Trace:
I20260709 13:03:07.431613 12209 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:07.381224 12240 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:07.452438  8398 rpcz_store.cc:276] 0709 13:03:05.108686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.182557 (+ 73871us) service_pool.cc:224] Handling call
0709 13:03:05.182993 (+   436us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:05.183508 (+   515us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:05.183549 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:05.858257 (+674708us) spinlock_profiling.cc:243] Waited 2.87 ms on lock 0x7b180004cd80. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882569f2a6 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a
0709 13:03:05.858869 (+   612us) spinlock_profiling.cc:243] Waited 59.2 ms on lock 0x7b2c000e1c48. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:03:06.845977 (+987108us) spinlock_profiling.cc:243] Waited 7.38 ms on lock 0x7b2c000e1c48. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:03:06.853484 (+  7507us) log.cc:844] Serialized 52 byte log entry
0709 13:03:07.333973 (+480489us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:07.334021 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.334133 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:07.451740 (+117607us) raft_consensus.cc:1685] finished
0709 13:03:07.451763 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.451923 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4788,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":3967,"spinlock_wait_cycles":159767296,"thread_start_us":422401,"threads_started":1,"wal-append.queue_time_us":576207}
I20260709 13:03:07.418515 12123 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:07.454682 12240 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:07.454418 12169 consensus_queue.cc:260] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:07.455237 12240 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:07.455551 12169 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:07.455730 12240 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:07.375026 12206 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:07.449815 12259 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:07.457702  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1912) took 1422 ms. Trace:
I20260709 13:03:07.457895 12259 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:07.449766 11878 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:07.385936 12247 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:07.411059 12239 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:07.449223 12244 log.cc:927] Time spent T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
W20260709 13:03:07.450039 12236 log.cc:927] Time spent T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:03:07.480588  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 60511us
I20260709 13:03:07.470029 12240 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:07.385737 12232 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:03:07.458377 12259 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:07.356088 12233 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:03:07.377972 12047 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:07.434273 11932 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:03:07.444981 12246 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:07.602384  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:07.458125  8253 rpcz_store.cc:276] 0709 13:03:06.034896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.035031 (+   135us) service_pool.cc:224] Handling call
0709 13:03:06.035397 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:06.035749 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:06.343784 (+308035us) log.cc:844] Serialized 14 byte log entry
0709 13:03:07.143689 (+799905us) log.cc:844] Serialized 14 byte log entry
0709 13:03:07.145525 (+  1836us) log.cc:844] Serialized 14 byte log entry
0709 13:03:07.229063 (+ 83538us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:07.229599 (+   536us) log.cc:844] Serialized 26 byte log entry
0709 13:03:07.229735 (+   136us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:07.229769 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.229848 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:07.457497 (+227649us) raft_consensus.cc:1685] finished
0709 13:03:07.457517 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.457646 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23324,"prepare.queue_time_us":836102,"prepare.run_cpu_time_us":164,"prepare.run_wall_time_us":3896,"raft.queue_time_us":215041,"raft.run_cpu_time_us":339,"raft.run_wall_time_us":34050,"spinlock_wait_cycles":29455872,"thread_start_us":1111923,"threads_started":4,"wal-append.queue_time_us":547787}
I20260709 13:03:07.491181  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:07.565110  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 83868.3us
I20260709 13:03:07.601787  8252 raft_consensus.cc:3055] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:07.617633  9006 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:07.700064  8252 raft_consensus.cc:740] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:03:07.701002 12220 consensus_meta.cc:287] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.558s	user 0.010s	sys 0.002s
I20260709 13:03:07.700945  8252 consensus_queue.cc:260] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:07.626468  8295 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:07.701601 12220 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:07.461206 12247 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:07.693693  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 436974us
I20260709 13:03:07.593216  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2319) took 1668 ms. Trace:
I20260709 13:03:07.583887  9008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 704) took 1824 ms. Trace:
I20260709 13:03:07.703043  8372 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:03:07.703292  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 76918.5us
I20260709 13:03:07.583333  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2086) took 1953 ms. Trace:
I20260709 13:03:07.585803  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 1970) took 1715 ms. Trace:
I20260709 13:03:07.696030  8256 raft_consensus.cc:2393] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 11: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:07.704118  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:07.476063 12050 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:07.707079  8250 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:07.578106  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2083) took 1999 ms. Trace:
I20260709 13:03:07.586901  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:07.573619  9004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 709) took 1274 ms. Trace:
I20260709 13:03:07.707983  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:07.571712  9003 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56954 (request call id 700) took 1887 ms (client timeout 1532 ms). Trace:
I20260709 13:03:07.708865  9004 rpcz_store.cc:276] 0709 13:03:06.298893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.375838 (+ 76945us) service_pool.cc:224] Handling call
0709 13:03:06.419856 (+ 44018us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:07.214539 (+794683us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:07.214589 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:07.215367 (+   778us) log.cc:844] Serialized 52 byte log entry
0709 13:03:07.376286 (+160919us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:07.376321 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.376387 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:07.573380 (+196993us) raft_consensus.cc:1685] finished
0709 13:03:07.573400 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.573563 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":255,"raft.queue_time_us":140873,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":3122,"thread_start_us":472865,"threads_started":2,"wal-append.queue_time_us":331406}
W20260709 13:03:07.581880  8254 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1911) took 1547 ms (client timeout 2001 ms). Trace:
I20260709 13:03:07.708416  8326 rpcz_store.cc:276] 0709 13:03:05.578599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.578778 (+   179us) service_pool.cc:224] Handling call
0709 13:03:05.579334 (+   556us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:06.925270 (+1345936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.577796 (+652526us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400058cd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:07.578028 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2999185152}
I20260709 13:03:07.693142  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:07.711694  8248 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
W20260709 13:03:07.569541 12242 log.cc:927] Time spent T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:03:07.573261  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:07.602324  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2352) took 1387 ms. Trace:
I20260709 13:03:07.705044  8325 rpcz_store.cc:276] 0709 13:03:05.630143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.630258 (+   115us) service_pool.cc:224] Handling call
0709 13:03:05.630613 (+   355us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:05.957463 (+326850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.583021 (+1625558us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b540011eed4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:07.583262 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":724019456}
I20260709 13:03:07.705853  8403 rpcz_store.cc:276] 0709 13:03:05.870617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.876310 (+  5693us) service_pool.cc:224] Handling call
0709 13:03:07.585580 (+1709270us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b54000febd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:07.585731 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":786239360}
I20260709 13:03:07.714558  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:07.724367  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.479723 12230 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:03:07.710669  8254 rpcz_store.cc:269] 0709 13:03:06.034506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.038881 (+  4375us) service_pool.cc:224] Handling call
0709 13:03:07.581634 (+1542753us) spinlock_profiling.cc:243] Waited 28 ms on lock 0x7b54001301d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:07.581809 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64485632}
W20260709 13:03:07.709947  9003 rpcz_store.cc:269] 0709 13:03:05.684170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.684313 (+   143us) service_pool.cc:224] Handling call
0709 13:03:07.571488 (+1887175us) spinlock_profiling.cc:243] Waited 589 ms on lock 0x7b54001b3fd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:07.571617 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1353853696}
I20260709 13:03:07.703982  9008 rpcz_store.cc:276] 0709 13:03:05.759418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.759569 (+   151us) service_pool.cc:224] Handling call
0709 13:03:05.773204 (+ 13635us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:05.989125 (+215921us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.583618 (+1594493us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b540057d5d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:07.583801 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":409037184}
I20260709 13:03:07.712277  8328 rpcz_store.cc:276] 0709 13:03:05.924430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.850718 (+926288us) service_pool.cc:224] Handling call
0709 13:03:06.851037 (+   319us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:06.981238 (+130201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:06.981268 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:06.984676 (+  3408us) log.cc:844] Serialized 78 byte log entry
0709 13:03:07.177502 (+192826us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:07.325074 (+147572us) log.cc:844] Serialized 14 byte log entry
0709 13:03:07.325476 (+   402us) log.cc:844] Serialized 14 byte log entry
0709 13:03:07.337032 (+ 11556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.337115 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:07.592950 (+255835us) raft_consensus.cc:1685] finished
0709 13:03:07.592974 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.593148 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34751,"prepare.queue_time_us":613537,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":73,"raft.queue_time_us":90799,"raft.run_cpu_time_us":361,"raft.run_wall_time_us":36295,"spinlock_wait_cycles":7424,"thread_start_us":365683,"threads_started":3,"wal-append.queue_time_us":296498}
I20260709 13:03:07.715655  8222 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:07.502907  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.707978  8252 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:07.606304 12033 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:07.527292 12240 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:07.718549  8397 rpcz_store.cc:276] 0709 13:03:06.214522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.214693 (+   171us) service_pool.cc:224] Handling call
0709 13:03:06.214991 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.277414 (+ 62423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:06.277479 (+    65us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:06.277533 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:06.277587 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.602104 (+1324517us) spinlock_profiling.cc:243] Waited 62.2 ms on lock 0x7b54001cc354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:07.602142 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.602249 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":143049472}
I20260709 13:03:07.506423 12147 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:07.599862 12198 raft_consensus.cc:3055] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:03:07.735896  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.654830 11685 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:07.743505  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:07.746337  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:07.757584  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 56641.1us
I20260709 13:03:07.702332 12247 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:07.604434 11848 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:07.651201 12221 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.778079 12221 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:07.778868  8257 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:03:07.779521  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 563) took 1648 ms. Trace:
I20260709 13:03:07.779822  8257 rpcz_store.cc:276] 0709 13:03:06.130845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.778252 (+1647407us) service_pool.cc:224] Handling call
0709 13:03:07.778549 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:07.779235 (+   686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.779465 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:07.780298 12221 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:07.702297 12134 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:07.782063 12240 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.782208 12223 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:07.727349 12261 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:07.782614 12240 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:07.784888 12240 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:07.607316 12239 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:07.785656  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 87677.8us
I20260709 13:03:07.785954 12264 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:07.786391  8972 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:03:07.787454  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.787500 12264 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:07.788074 12264 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:07.788259  8972 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:07.788467 12264 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:07.789050  8298 leader_election.cc:336] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825): Timed out: RequestConsensusVote RPC to 127.6.255.4:33825 timed out after 1.503s (SENT)
I20260709 13:03:07.752341  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:07.751886  8252 raft_consensus.cc:1275] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 5. (term mismatch)
I20260709 13:03:07.753513  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:07.789927  8254 raft_consensus.cc:1240] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 8 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:07.790247 12249 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:07.790509  8400 raft_consensus.cc:2410] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:03:07.790722 12249 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:07.790992 12190 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:07.754180  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:07.754653  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:07.755357 12267 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:07.791872  8402 raft_consensus.cc:2410] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:03:07.791823 12267 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:07.480334 12188 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.276s
I20260709 13:03:07.794327 12267 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:07.700435 12207 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:07.767259  8253 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
W20260709 13:03:07.589274 12065 consensus_meta.cc:287] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.606s	user 0.005s	sys 0.003s
I20260709 13:03:07.769244  8250 raft_consensus.cc:2410] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 8 because replica has last-logged OpId of term: 7 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 9.
I20260709 13:03:07.708060 11926 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:07.773172  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.591786 12184 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:07.719347 12243 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:07.774188 12064 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:07.756965 12263 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:07.771348 12050 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.746985 12230 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.765507 12198 raft_consensus.cc:740] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:07.835422 12047 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:07.835652 12198 consensus_queue.cc:260] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:07.837046 12198 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:07.751353  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:07.838783 12191 raft_consensus.cc:3055] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:07.839174 12191 raft_consensus.cc:740] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:07.783113 12261 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:07.840272 12261 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:07.840022 12191 consensus_queue.cc:260] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:07.789601 12237 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:07.841059 12191 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:07.718271  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 340428us
I20260709 13:03:07.848006  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:07.791224 12249 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:07.789824  8252 pending_rounds.cc:85] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 4
I20260709 13:03:07.790560  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 558) took 1688 ms. Trace:
I20260709 13:03:07.789358  8972 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:07.848845  8252 pending_rounds.cc:107] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 13:03:07.791786  8405 raft_consensus.cc:2393] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 10: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:07.718962 12206 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:07.797935  9001 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:03:07.527125 12085 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:07.852630 12085 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:07.719347 12238 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:07.827970 12243 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:07.853691 12238 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:07.855576 12243 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:07.855829 12238 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:07.583943 12123 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:07.855940 12050 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:07.857218 12184 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:07.858196 12050 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:07.804533  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:07.858551 12123 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:07.801712  8298 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:07.810506  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:07.811421  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:07.859759  8329 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:03:07.860060  8406 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:03:07.861083  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:07.829766 12195 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.859777  8401 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:03:07.862162  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:07.772591 12273 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:07.811885  8257 raft_consensus.cc:1240] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:07.863051 12222 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.863662  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1938) took 1578 ms. Trace:
I20260709 13:03:07.863514 12222 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:07.864017  8257 rpcz_store.cc:276] 0709 13:03:06.285051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.811281 (+1526230us) service_pool.cc:224] Handling call
0709 13:03:07.811657 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:07.863404 (+ 51747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.863606 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:07.812225  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:07.865800 12085 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:07.818707  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:07.869849  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:07.870559  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:07.870872  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:07.871368  8403 raft_consensus.cc:2410] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:03:07.865281  8257 raft_consensus.cc:1240] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:03:07.871640  8327 raft_consensus.cc:2410] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:03:07.819317  8331 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:03:07.822012  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.871848  8369 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:07.872236  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.862602 12237 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:07.826818  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:07.874194  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1946) took 1047 ms. Trace:
I20260709 13:03:07.828732  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:07.874565  8257 rpcz_store.cc:276] 0709 13:03:06.826621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.864767 (+1038146us) service_pool.cc:224] Handling call
0709 13:03:07.865075 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:07.871746 (+  6671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.874143 (+  2397us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:07.875221  8399 raft_consensus.cc:2410] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
W20260709 13:03:07.831808  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 74786.1us
I20260709 13:03:07.832613  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:07.833271  9009 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:03:07.832311 12220 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:07.834714 12230 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:07.789722  8256 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:07.846844  8297 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:07.848906  8254 rpcz_store.cc:276] 0709 13:03:06.102253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.749792 (+1647539us) service_pool.cc:224] Handling call
0709 13:03:07.789763 (+ 39971us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:07.790333 (+   570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.790499 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:07.849247  8252 raft_consensus.cc:2889] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:07.849452  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 60125.5us
I20260709 13:03:07.912561  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1913) took 1877 ms. Trace:
I20260709 13:03:07.912954  8252 rpcz_store.cc:276] 0709 13:03:06.035278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.036852 (+  1574us) service_pool.cc:224] Handling call
0709 13:03:06.054113 (+ 17261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:07.601693 (+1547580us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54003c49a0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f8828735dc0 00007f88287c01c0 00007f88287c1231 00007f88287bdd8f 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c
0709 13:03:07.912318 (+310625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.912506 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":384664320}
I20260709 13:03:07.861835 12195 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:07.862660  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:07.815702  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:07.922747  8325 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 13: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:07.865921 12064 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:07.927469  8401 raft_consensus.cc:2410] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:03:07.880867 12050 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:07.818223  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:07.931744  8330 raft_consensus.cc:2393] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 10: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:07.850657  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:07.934518  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:07.935042  8405 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:03:07.866230  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:07.937183  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:07.938624  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:07.939721  8404 raft_consensus.cc:2410] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:03:07.940711  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:07.817127  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:07.833618 12263 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:07.910864 12221 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:07.961588 12050 raft_consensus.cc:697] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:07.961905 12222 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:07.962532 12259 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:07.963155 12246 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:07.911755 12238 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:07.868184  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:03:07.872603  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:07.964811  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.964762  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:07.966298  8405 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
W20260709 13:03:07.824383  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 425297us
I20260709 13:03:07.862778  8406 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:07.827333  8250 raft_consensus.cc:1240] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:03:07.823711 12267 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:07.879012  8323 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:07.879307  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 509) took 1991 ms. Trace:
I20260709 13:03:07.968353  8332 rpcz_store.cc:276] 0709 13:03:05.887415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.295431 (+408016us) service_pool.cc:224] Handling call
0709 13:03:06.295766 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:06.827783 (+532017us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:06.827826 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:06.848412 (+ 20586us) log.cc:844] Serialized 26 byte log entry
0709 13:03:07.608023 (+759611us) spinlock_profiling.cc:243] Waited 5.83 ms on lock 0x7b1800225e20. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e0d1 00007f8828779a0f
0709 13:03:07.810216 (+202193us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:07.810246 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.810299 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:07.870911 (+ 60612us) raft_consensus.cc:1685] finished
0709 13:03:07.870931 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.879247 (+  8316us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28693,"raft.queue_time_us":531604,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":13907,"spinlock_wait_cycles":13406976,"thread_start_us":202021,"threads_started":1,"wal-append.queue_time_us":961459}
I20260709 13:03:07.969548  8329 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:07.879854  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:07.970367  8403 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:07.880084  8222 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:07.909159  8369 leader_election.cc:400] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8.
I20260709 13:03:07.971107  8369 leader_election.cc:403] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
W20260709 13:03:07.882081  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 93038.4us
W20260709 13:03:07.909979  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 112293us
I20260709 13:03:07.975766  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:07.830194 12023 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:07.978344  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1944) took 1152 ms. Trace:
I20260709 13:03:07.978900  8250 rpcz_store.cc:276] 0709 13:03:06.826069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.826613 (+1000544us) service_pool.cc:224] Handling call
0709 13:03:07.827012 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:07.977981 (+150969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.978271 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:07.979502  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 99518.8us
W20260709 13:03:07.925990  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 79246.6us
I20260709 13:03:07.982833  8297 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:07.983919  8332 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:03:07.984630  8297 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:07.984907  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:07.803762 12065 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:07.929636  8974 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:07.867568  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:07.945684 12195 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:07.946233 12260 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:07.913601 12237 raft_consensus.cc:697] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:07.949708  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:07.943325 12184 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:07.921924 12062 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:03:07.910310 12220 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:07.958326 12209 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:07.958865 12263 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:07.976734  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:07.976946  8325 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:07.928412  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:08.019243  8401 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:07.987807  8405 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:07.988279  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:08.020632  8332 raft_consensus.cc:2393] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 11: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:07.987916  8297 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:07.991520  8256 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
I20260709 13:03:07.980238  8222 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:07.999850  9005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 827) took 1803 ms. Trace:
I20260709 13:03:08.021870  9005 rpcz_store.cc:276] 0709 13:03:06.196607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.300507 (+103900us) service_pool.cc:224] Handling call
0709 13:03:06.956479 (+655972us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.965461 (+  8982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.999586 (+1034125us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b5400598dd0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:07.999766 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1507725056}
I20260709 13:03:05.985569  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:03:08.005159  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 96259.3us
I20260709 13:03:08.024053  8256 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:08.024652  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1915) took 1048 ms. Trace:
I20260709 13:03:08.024937  8256 rpcz_store.cc:276] 0709 13:03:06.976251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.023574 (+1047323us) service_pool.cc:224] Handling call
0709 13:03:08.023908 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.024431 (+   523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.024591 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:08.025409  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:08.006534  9002 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56954 (request call id 705) took 1819 ms (client timeout 1503 ms). Trace:
W20260709 13:03:08.026094  9002 rpcz_store.cc:269] 0709 13:03:06.187254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.188511 (+  1257us) service_pool.cc:224] Handling call
0709 13:03:08.006337 (+1817826us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400527bd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:08.006468 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2382568320}
I20260709 13:03:07.927831 12141 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:08.007886  9006 raft_consensus.cc:1240] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7. Ops: []
W20260709 13:03:08.010769  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 81213.9us
I20260709 13:03:08.018028  8330 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
W20260709 13:03:08.038363  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 58193.3us
I20260709 13:03:08.038739  8323 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:08.039095  8222 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:08.036146  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.040407  8295 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:08.003648  8151 catalog_manager.cc:5697] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 6 to 9, leader changed from <none> to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 9 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:08.035022  8406 raft_consensus.cc:2410] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:03:08.060791  8295 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:08.043869  8254 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:03:08.061659  8399 raft_consensus.cc:1275] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:03:08.044816  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:03:08.044329  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 56493.8us
I20260709 13:03:08.060948  8326 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:03:08.066020  8256 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:08.067981  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:07.992645 12085 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:08.074705  8974 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:07.981868 12246 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:08.073726  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1920) took 1075 ms. Trace:
I20260709 13:03:08.075933  8329 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
I20260709 13:03:08.065027  8297 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:08.078933  8255 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:08.015427 12221 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:08.007485 12206 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 7. Current term is 8.
I20260709 13:03:08.045110 12194 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:08.097965  8369 leader_election.cc:400] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7.
I20260709 13:03:08.068917 12195 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:08.172420  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:08.127306  8256 rpcz_store.cc:276] 0709 13:03:06.998175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.025628 (+1027453us) service_pool.cc:224] Handling call
0709 13:03:08.065863 (+ 40235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.066394 (+   531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.073658 (+  7264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:08.044134 12272 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:08.172587  8369 leader_election.cc:403] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:03:08.003403 12267 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:08.174049  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:08.180498  9002 raft_consensus.cc:2393] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 10: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:08.175595  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:08.060746 12243 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:08.077356 12188 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:08.177531  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:08.184229  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 86512.1us
I20260709 13:03:08.173305  9004 raft_consensus.cc:2410] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:03:08.182458 12233 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:08.182920 12224 log.cc:927] Time spent T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:03:07.962563 12050 consensus_queue.cc:237] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:08.040978 12184 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:08.185765  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:08.199196  8369 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:08.037125 12169 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.132s
I20260709 13:03:08.060792 12064 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.116s
I20260709 13:03:08.097150 12260 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:08.193969  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:08.214990  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.215266  8256 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:08.162294 11848 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:08.194999  8406 raft_consensus.cc:1275] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:03:08.226610  8406 pending_rounds.cc:85] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 7
I20260709 13:03:08.226907  8406 pending_rounds.cc:107] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:03:08.227144  8406 raft_consensus.cc:2889] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:08.227449  8404 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:08.039813 12264 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:08.167876 12209 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.065562 12273 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.123s
I20260709 13:03:08.216367  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:08.095371 12131 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.106s
I20260709 13:03:08.167157 12207 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:08.275240  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:08.170629 12238 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:08.225646  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:03:08.229460  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:08.230335  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 196074us
I20260709 13:03:08.172456 12194 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:03:08.252578  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.173588 12257 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:08.274794  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:08.279079  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:08.162317 12246 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:08.196458 11878 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.196606 12243 raft_consensus.cc:697] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:03:08.292557  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.196533 12123 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:08.346729  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 281769us
I20260709 13:03:08.350342  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:08.369637  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 299306us
W20260709 13:03:08.363822  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.226501 11848 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.226270 12240 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:08.371984  8256 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
I20260709 13:03:08.275053 12147 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:08.403177  8220 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:08.403770  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:08.366885 12207 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:08.435994  9004 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:08.377664 12224 log.cc:927] Time spent T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.099s	user 0.004s	sys 0.000s
I20260709 13:03:08.381876 11848 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:08.404639  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:08.522877  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:08.383432 12276 log.cc:927] Time spent T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.474s	user 0.000s	sys 0.001s
I20260709 13:03:08.382623 12240 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:08.346392 12065 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:08.512683  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:08.373082 11878 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:08.216153 12237 consensus_queue.cc:237] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:08.361724 12257 raft_consensus.cc:697] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:08.525993  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:08.360515 12194 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:08.193727 12272 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:08.276002 12260 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:08.279531 12209 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:08.555482  8397 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:03:08.534821 12190 consensus_queue.cc:1059] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:03:08.512629 12296 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:08.404722 12259 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:08.514176 12243 consensus_queue.cc:237] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:08.737747  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:08.775275  8295 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.004s	sys 0.005s
I20260709 13:03:08.585562  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:08.796150  8256 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
I20260709 13:03:08.775735  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:08.797360  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:08.762907  8403 consensus_meta.cc:287] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.780s	user 0.009s	sys 0.000s
I20260709 13:03:08.798326  8403 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 7.
W20260709 13:03:08.798759  8369 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.000s	sys 0.008s
I20260709 13:03:08.648548  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:08.583051  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 179937us
W20260709 13:03:08.799307  8369 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.000s	sys 0.008s
I20260709 13:03:08.798561  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:08.764479  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.593144  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:08.795666  8295 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.004s	sys 0.006s
I20260709 13:03:08.801708  8972 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:08.802778  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 741968us
I20260709 13:03:08.803153  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:08.803547  8295 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:08.805750  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:08.806336  8329 raft_consensus.cc:2376] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10. Re-sending same reply.
I20260709 13:03:08.807436  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:08.808466  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:08.808598  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 364297us
I20260709 13:03:08.809233  8222 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:08.809235  8330 raft_consensus.cc:2410] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 13:03:08.806269  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 607092us
I20260709 13:03:08.812137  8397 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:03:08.814363  8397 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:08.812835  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:08.816777  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:08.555621 12240 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:08.570737 12065 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:08.813364  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:08.814292  8403 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:08.620561 12233 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:08.820307  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:08.895670  8397 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:03:09.073935  9000 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.975s	user 0.014s	sys 0.023s
W20260709 13:03:09.119720  9000 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.020s	user 0.014s	sys 0.027s
I20260709 13:03:09.113008  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 523) took 1130 ms. Trace:
W20260709 13:03:08.904122  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:08.858093  8974 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.3:37367 timed out after 1.667s (SENT) [suppressed 16 similar messages]
I20260709 13:03:08.820997  8220 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:09.114913  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 565) took 1272 ms. Trace:
I20260709 13:03:09.137935  8403 rpcz_store.cc:276] 0709 13:03:07.842777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.813733 (+970956us) service_pool.cc:224] Handling call
0709 13:03:08.813999 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.114671 (+300672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.114861 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:09.138348  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 317467us
I20260709 13:03:08.981006  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:09.111829  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 308320us
I20260709 13:03:08.845034  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:09.141527  8329 raft_consensus.cc:2376] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8. Re-sending same reply.
I20260709 13:03:08.840752  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:08.594887 12277 log.cc:927] Time spent T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.416s	user 0.000s	sys 0.001s
W20260709 13:03:08.595322 12278 log.cc:927] Time spent T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.416s	user 0.001s	sys 0.000s
I20260709 13:03:09.119656  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49220 (request call id 579) took 1089 ms. Trace:
I20260709 13:03:09.143599  8256 rpcz_store.cc:276] 0709 13:03:08.030064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.810685 (+780621us) service_pool.cc:224] Handling call
0709 13:03:09.119602 (+308917us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:09.120147  8325 rpcz_store.cc:276] 0709 13:03:07.982490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.018912 (+ 36422us) service_pool.cc:224] Handling call
0709 13:03:08.060728 (+ 41816us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:08.086922 (+ 26194us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:08.086948 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:08.087394 (+   446us) log.cc:844] Serialized 26 byte log entry
0709 13:03:08.406482 (+319088us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:08.406528 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.406601 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:09.107879 (+701278us) raft_consensus.cc:1685] finished
0709 13:03:09.107905 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.112946 (+  5041us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64576,"raft.queue_time_us":40954,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":62005,"thread_start_us":220246,"threads_started":1,"wal-append.queue_time_us":909653}
I20260709 13:03:08.675071 12141 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:08.817371 12301 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:09.159314  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:09.160169  9002 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:03:08.972908 12309 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:08.995465 12264 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:09.125855 12287 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.052s
W20260709 13:03:09.126573 12191 consensus_meta.cc:287] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.279s	user 0.010s	sys 0.002s
I20260709 13:03:09.172704  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:09.107564 11848 consensus_meta.cc:287] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.552s	user 0.009s	sys 0.001s
I20260709 13:03:09.173753 11848 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:09.175941 11848 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:09.127519  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:09.128062 12291 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.178013 12291 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:09.130349 12272 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:08.822199  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:09.182163  8332 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 13: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:09.182055 12295 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:08.804314 12206 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7.
I20260709 13:03:09.120574 12246 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:09.133635  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 320294us
I20260709 13:03:09.173770  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1935) took 1141 ms. Trace:
I20260709 13:03:09.183707  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:09.183962  8256 rpcz_store.cc:276] 0709 13:03:08.032505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.144660 (+1112155us) service_pool.cc:224] Handling call
0709 13:03:09.173697 (+ 29037us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:09.184834  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:09.185496  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49220 (request call id 575) took 1156 ms. Trace:
I20260709 13:03:09.185829  8369 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:08.740903 12220 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:09.077486 12230 consensus_meta.cc:287] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.160s	user 0.003s	sys 0.009s
I20260709 13:03:09.186633 12230 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:09.188802 12230 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:08.646417 12207 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:09.181334 12194 consensus_meta.cc:287] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.542s	user 0.008s	sys 0.000s
I20260709 13:03:09.189715 12207 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:09.189965 12194 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:09.190915  8398 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:09.191313  8331 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:09.192502 12207 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:08.915959 12283 log.cc:927] Time spent T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.380s	user 0.001s	sys 0.000s
I20260709 13:03:09.193795  8252 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:09.194674 12194 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:09.194558 12259 consensus_queue.cc:1059] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:03:09.159461 12221 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.014s	sys 0.058s
I20260709 13:03:09.195552 12312 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.195781 12238 raft_consensus.cc:3055] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:09.196205 12312 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:08.896602 12279 log.cc:927] Time spent T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.681s	user 0.001s	sys 0.001s
I20260709 13:03:09.193380 12207 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:09.198639  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1945) took 2372 ms. Trace:
W20260709 13:03:08.916541 12308 log.cc:927] Time spent T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:03:09.162145  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:09.200351  8403 raft_consensus.cc:2410] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:03:09.199054  8248 rpcz_store.cc:276] 0709 13:03:06.826347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.838002 (+1011655us) service_pool.cc:224] Handling call
0709 13:03:07.838370 (+   368us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:07.838730 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:07.838760 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:07.839470 (+   710us) log.cc:844] Serialized 52 byte log entry
0709 13:03:07.964260 (+124790us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:08.443930 (+479670us) log.cc:844] Serialized 14 byte log entry
0709 13:03:08.513283 (+ 69353us) log.cc:844] Serialized 14 byte log entry
0709 13:03:08.535701 (+ 22418us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.535790 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:09.198427 (+662637us) raft_consensus.cc:1685] finished
0709 13:03:09.198452 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.198585 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":241227,"prepare.queue_time_us":166016,"prepare.run_cpu_time_us":122,"prepare.run_wall_time_us":12155,"raft.queue_time_us":165750,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":29122,"thread_start_us":524585,"threads_started":2,"wal-append.queue_time_us":248623}
I20260709 13:03:09.157857  8295 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:09.195444 12221 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.015s	sys 0.058s
W20260709 13:03:09.201762  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 44017.5us
I20260709 13:03:09.202142 12220 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:09.202454  8295 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:09.202628 12220 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.203138 12220 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:09.203495 12062 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:09.203912  8295 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:08.800061 12296 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:09.205515 12296 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:09.206678  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2366) took 1409 ms. Trace:
I20260709 13:03:09.207042  8400 rpcz_store.cc:276] 0709 13:03:07.797751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.205894 (+1408143us) service_pool.cc:224] Handling call
0709 13:03:09.206195 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.206383 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:09.206408 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:09.206451 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:09.206483 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.206536 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.206628 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:09.157693  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.018918 12209 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:09.134335 11878 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:09.165493  8325 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:03:09.120402 12233 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.163339 12301 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:09.169322  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:09.169651 12310 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:09.170707 12065 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:09.171957 12287 consensus_queue.cc:1059] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:09.182967 12295 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:09.185787  8256 rpcz_store.cc:276] 0709 13:03:08.028945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.184572 (+1155627us) service_pool.cc:224] Handling call
0709 13:03:09.185446 (+   874us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:08.752029 12022 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:03:09.111220 12198 consensus_meta.cc:287] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 1.266s	user 0.008s	sys 0.004s
W20260709 13:03:08.973670 12288 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.334s	user 0.001s	sys 0.000s
I20260709 13:03:09.197026  8369 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:03:09.360111  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 163145us
I20260709 13:03:09.225165 12309 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:09.198836 12207 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:09.074173 12305 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:08.917675 12294 raft_consensus.cc:3055] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:09.201196  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:09.210439  8295 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:09.166225 12141 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:09.213598  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 404380us
W20260709 13:03:09.402263  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 544129us
I20260709 13:03:09.169157 12260 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:09.300439  8372 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:09.196213 12238 raft_consensus.cc:740] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:03:09.299750 12064 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.016s	sys 0.019s
I20260709 13:03:09.225092 12194 consensus_queue.cc:1048] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.133s
I20260709 13:03:09.358384 12022 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:09.223279 12190 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:09.540881 12238 consensus_queue.cc:260] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:09.195505 12259 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:03:09.351188 12247 consensus_meta.cc:287] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.568s	user 0.011s	sys 0.001s
I20260709 13:03:09.400003  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:09.225356 12301 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:09.400702  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 190309us
I20260709 13:03:09.626680  8295 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:09.350158 12310 raft_consensus.cc:697] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:09.413024  8222 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:09.630271  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 217301us
I20260709 13:03:09.630882 12120 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:09.631330 12120 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.631650 12120 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:09.632189  8222 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:09.413813 12287 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:03:09.414285  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 300092us
W20260709 13:03:09.413825  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 298908us
I20260709 13:03:09.408077 12221 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:03:09.453033  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 313688us
I20260709 13:03:09.405342 12198 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:03:09.403154 12294 raft_consensus.cc:740] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:03:09.453598  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 92437.9us
I20260709 13:03:09.299621 12191 consensus_queue.cc:1059] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.153s
W20260709 13:03:09.532676 12261 consensus_meta.cc:287] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 1.692s	user 0.012s	sys 0.001s
I20260709 13:03:09.514853 12173 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:09.509141  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 519) took 1532 ms. Trace:
I20260709 13:03:09.378443 12209 consensus_queue.cc:1059] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.091s
I20260709 13:03:09.510746  9009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 834) took 2428 ms. Trace:
W20260709 13:03:09.501063  8324 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.928s	user 0.020s	sys 0.053s
I20260709 13:03:09.223464 12272 consensus_queue.cc:1059] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.155s
I20260709 13:03:09.376610 12312 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:09.377897 12246 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:03:09.503091 12249 consensus_meta.cc:287] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 1.622s	user 0.007s	sys 0.004s
I20260709 13:03:09.619395  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:09.635748 12313 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:09.505384  8144 maintenance_manager.cc:419] P b3bd0e8d7a084e649e387e729c0adabc: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033376
I20260709 13:03:09.544556  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 566) took 1701 ms. Trace:
I20260709 13:03:09.764727  8398 rpcz_store.cc:276] 0709 13:03:07.843067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.811979 (+968912us) service_pool.cc:224] Handling call
0709 13:03:08.812382 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.191322 (+378940us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.544228 (+352906us) spinlock_profiling.cc:243] Waited 378 ms on lock 0x7b54001cbe54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:09.544480 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":869923584}
I20260709 13:03:09.766050  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:09.632536 12247 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:09.308065 12233 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:08.978797 12257 consensus_queue.cc:237] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:09.467156  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:09.771483  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1931) took 1740 ms. Trace:
I20260709 13:03:09.772148  8248 rpcz_store.cc:276] 0709 13:03:08.031426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.407893 (+1376467us) service_pool.cc:224] Handling call
0709 13:03:09.771405 (+363512us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:09.416066 12305 raft_consensus.cc:697] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 8 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:09.448428 12243 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 13:03:09.409642 12309 raft_consensus.cc:697] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:09.399250 12207 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:09.360499  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.632869  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49220 (request call id 569) took 1606 ms. Trace:
I20260709 13:03:09.781325  8256 rpcz_store.cc:276] 0709 13:03:08.026915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.356210 (+1329295us) service_pool.cc:224] Handling call
0709 13:03:09.632808 (+276598us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:09.623696 12064 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.892s	user 0.024s	sys 0.032s
I20260709 13:03:09.639199 12022 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:09.639671  9001 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:09.632498 12220 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:09.634923 12302 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:09.642167 12238 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:09.786612  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:09.670284 12221 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.642863 12310 consensus_queue.cc:237] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:09.643421  9000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 840) took 1596 ms. Trace:
I20260709 13:03:09.643548 12210 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.643993 11878 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.788470 12333 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:09.644937 12222 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.788928 11878 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:09.788126  9000 rpcz_store.cc:276] 0709 13:03:08.047298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.047409 (+   111us) service_pool.cc:224] Handling call
0709 13:03:08.096136 (+ 48727us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:09.136101 (+1039965us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:09.136146 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:09.136662 (+   516us) log.cc:844] Serialized 26 byte log entry
0709 13:03:09.377942 (+241280us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:09.377982 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.378053 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:09.643210 (+265157us) raft_consensus.cc:1685] finished
0709 13:03:09.643232 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.643363 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":177135,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":69936,"thread_start_us":1275535,"threads_started":2,"wal-append.queue_time_us":475785}
I20260709 13:03:09.789685 12222 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:09.791635 11878 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:09.792217 12222 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:09.646981 12240 consensus_meta.cc:287] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.509s	user 0.009s	sys 0.001s
I20260709 13:03:09.793136 12240 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:09.795264 12240 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:09.648195  8253 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 10. Ops: []
I20260709 13:03:09.796252  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1941) took 3509 ms. Trace:
I20260709 13:03:09.796690  8253 rpcz_store.cc:276] 0709 13:03:06.289764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.647636 (+3357872us) service_pool.cc:224] Handling call
0709 13:03:09.648011 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.795937 (+147926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.796185 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:09.797688  8403 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:09.649168  8220 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:09.657572 12295 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:09.663465  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 1989) took 1846 ms. Trace:
I20260709 13:03:09.637266  8222 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:03:09.798981  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 149910us
I20260709 13:03:09.646988 12319 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.799995 12319 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:09.798801  8404 rpcz_store.cc:276] 0709 13:03:07.816912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.940425 (+123513us) service_pool.cc:224] Handling call
0709 13:03:09.657387 (+1716962us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b5400276354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:09.663373 (+  5986us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":657708416}
W20260709 13:03:09.657164 12123 consensus_meta.cc:287] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.275s	user 0.007s	sys 0.003s
I20260709 13:03:09.648466 12141 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:09.801698  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:09.751784  8324 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.179s	user 0.052s	sys 0.128s
I20260709 13:03:09.751638  9009 rpcz_store.cc:276] 0709 13:03:07.082571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.592337 (+509766us) service_pool.cc:224] Handling call
0709 13:03:07.592729 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:07.878955 (+286226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.510457 (+1631502us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54004132d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:09.510677 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":552573312}
I20260709 13:03:09.802112 12123 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:09.661713 12133 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:09.762977 12332 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:09.799997  8220 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:09.802868 12133 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:09.759069 12249 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:09.767123  8398 raft_consensus.cc:2410] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:03:09.376812 12291 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:09.805124  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:09.805478  9007 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:09.805864  8398 raft_consensus.cc:2393] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 10: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:09.806622  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:09.763190  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:09.807390  8406 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:03:09.808153  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:09.809023  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:03:09.775548  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.787406  8248 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:09.789160 12333 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:09.643113 11926 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:09.811707 11926 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:09.813807 11926 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:09.813962  9007 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
W20260709 13:03:09.646288 12235 log.cc:927] Time spent T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 2.510s	user 0.011s	sys 0.012s
I20260709 13:03:09.814486  8406 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:09.803206 12332 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:09.788800 12210 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:09.802811  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:09.818107  8404 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:09.818441 12210 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:09.818918  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:09.819496  8404 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:03:09.820772  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:09.750253  8328 rpcz_store.cc:276] 0709 13:03:07.976227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.980925 (+  4698us) service_pool.cc:224] Handling call
0709 13:03:07.981186 (+   261us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:08.001577 (+ 20391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:08.001620 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:08.043039 (+ 41419us) log.cc:844] Serialized 26 byte log entry
0709 13:03:08.229903 (+186864us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:08.229943 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.230004 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:09.508906 (+1278902us) raft_consensus.cc:1685] finished
0709 13:03:09.508928 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.509078 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":131682,"raft.queue_time_us":36959,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":22043,"thread_start_us":63006,"threads_started":1,"wal-append.queue_time_us":329029}
I20260709 13:03:09.828192  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:09.835312  8406 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:09.831115  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:09.839077  8404 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:09.843703  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:09.844678  8406 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:03:09.747108 12261 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:09.782933 12191 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.855046 12133 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:09.856572 12333 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:09.858695 12332 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:09.774230 12305 consensus_queue.cc:237] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:09.867220  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:09.870273  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:09.872669 12313 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:09.873260 12313 raft_consensus.cc:697] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:09.875262  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:09.833640 12221 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:09.880105  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:09.880739  8406 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:09.882565  8398 raft_consensus.cc:2393] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 10: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:09.884423  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:09.887529  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:09.870762 12287 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:09.899283  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:09.894294  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 94391.4us
I20260709 13:03:09.898434  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:09.900949  8256 raft_consensus.cc:2410] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:03:09.899874  8249 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:09.902340  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:09.903065  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:09.903314  8220 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:09.903647  8406 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:09.781733 12294 consensus_queue.cc:260] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:09.905531 12285 log.cc:927] Time spent T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 1.244s	user 0.003s	sys 0.003s
W20260709 13:03:09.907352 12331 log.cc:927] Time spent T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
W20260709 13:03:09.908273 12329 log.cc:927] Time spent T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:03:09.910029  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1947) took 3083 ms. Trace:
I20260709 13:03:09.910408  8257 rpcz_store.cc:276] 0709 13:03:06.826858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.886356 (+1059498us) service_pool.cc:224] Handling call
0709 13:03:07.886755 (+   399us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:07.887211 (+   456us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:07.887254 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:07.910975 (+ 23721us) log.cc:844] Serialized 52 byte log entry
0709 13:03:08.171337 (+260362us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:08.171374 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.171440 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:09.774594 (+1603154us) spinlock_profiling.cc:243] Waited 7.53 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:09.909694 (+135100us) raft_consensus.cc:1685] finished
0709 13:03:09.909714 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.909968 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15566,"raft.queue_time_us":334,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":5510,"spinlock_wait_cycles":17309568,"thread_start_us":179842,"threads_started":1,"wal-append.queue_time_us":405277}
I20260709 13:03:09.912264  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:09.888528 12305 consensus_queue.cc:1059] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:03:09.913355  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:09.916472  8257 raft_consensus.cc:2410] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:03:09.917665  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 521) took 1935 ms. Trace:
I20260709 13:03:09.919346 12120 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:09.920451  9007 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56940 (request call id 844) took 1737 ms (client timeout 1883 ms). Trace:
I20260709 13:03:09.920532  9001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56948 (request call id 828) took 1161 ms. Trace:
W20260709 13:03:09.920950  9007 rpcz_store.cc:269] 0709 13:03:08.183465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.186602 (+  3137us) service_pool.cc:224] Handling call
0709 13:03:09.920283 (+1733681us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54004ec854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:09.920397 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3507091840}
I20260709 13:03:09.921056  9001 rpcz_store.cc:276] 0709 13:03:08.759515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.759705 (+   190us) service_pool.cc:224] Handling call
0709 13:03:09.920364 (+1160659us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b54005a8cd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:09.920473 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1922117888}
I20260709 13:03:09.923462  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1926) took 2632 ms. Trace:
I20260709 13:03:09.923903  8257 rpcz_store.cc:276] 0709 13:03:07.291451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.917023 (+2625572us) service_pool.cc:224] Handling call
0709 13:03:09.917307 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.917501 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:09.917530 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:09.917572 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:09.923194 (+  5622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.923279 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.923405 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:09.918020  8324 rpcz_store.cc:276] 0709 13:03:07.982039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.991452 (+  9413us) service_pool.cc:224] Handling call
0709 13:03:07.991797 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.076876 (+ 85079us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:08.571787 (+494911us) log.cc:844] Serialized 14 byte log entry
0709 13:03:09.802438 (+1230651us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:09.802486 (+    48us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:09.802525 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.917453 (+114928us) spinlock_profiling.cc:243] Waited 84.8 ms on lock 0x7b540037b254. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:09.917502 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.917612 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44212,"prepare.queue_time_us":93774,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":308,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":73,"spinlock_wait_cycles":195147008,"thread_start_us":1696710,"threads_started":2,"wal-append.queue_time_us":1054296}
I20260709 13:03:09.924935  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:09.925565  8257 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:09.881805 12302 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:09.927541  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:09.925945  8223 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:03:09.930214  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 295586us
I20260709 13:03:09.867266 12191 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:09.933401  8249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 570) took 1906 ms. Trace:
I20260709 13:03:09.933491  8220 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:09.933964  8249 rpcz_store.cc:276] 0709 13:03:08.027486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.932680 (+1905194us) service_pool.cc:224] Handling call
0709 13:03:09.932951 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.933116 (+   165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:09.933139 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:09.933179 (+    40us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:09.933215 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.933265 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.933353 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:09.935627  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:09.936213  8406 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:03:09.874150 12313 consensus_queue.cc:237] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:09.947523 12085 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:09.947917 12120 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:09.947999 12085 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.948959 12085 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:09.949199 12147 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:09.949641 12147 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:09.952087  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:09.952270  9006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 821) took 2796 ms. Trace:
I20260709 13:03:09.952946  8405 raft_consensus.cc:2410] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 9 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:03:09.952689  9006 rpcz_store.cc:276] 0709 13:03:07.157929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.708075 (+550146us) service_pool.cc:224] Handling call
0709 13:03:08.007716 (+299641us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.037022 (+ 29306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.952016 (+1914994us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b5400527bd0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:09.952175 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":688388992}
I20260709 13:03:09.954021  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:09.954375  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2341) took 1232 ms. Trace:
I20260709 13:03:09.954808  8405 raft_consensus.cc:2410] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:03:09.954736  8331 rpcz_store.cc:276] 0709 13:03:08.721662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.722829 (+  1167us) service_pool.cc:224] Handling call
0709 13:03:08.723234 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.191680 (+468446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.954179 (+762499us) spinlock_profiling.cc:243] Waited 468 ms on lock 0x7b5400365954. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:09.954326 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1076025600}
I20260709 13:03:09.955719  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:09.956913 12085 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:09.948122 12247 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:09.964998  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:09.957361  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1997) took 1091 ms. Trace:
I20260709 13:03:09.966409  8405 rpcz_store.cc:276] 0709 13:03:08.866003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.956638 (+1090635us) service_pool.cc:224] Handling call
0709 13:03:09.956914 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.957085 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:09.957110 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:09.957150 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:09.957179 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.957228 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.957316 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:09.968868  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:09.969411  8405 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:03:09.979460 12085 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:09.983577  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 50233.3us
I20260709 13:03:09.965090 12033 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.987555 12261 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:09.999253 12033 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:09.974458 12319 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:09.976125 12249 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:10.012637 12033 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:10.014042 12338 raft_consensus.cc:3055] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:10.014496 12337 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:03:10.018723 12325 log.cc:927] Time spent T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.508s	user 0.003s	sys 0.000s
W20260709 13:03:10.022600  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 91703.3us
I20260709 13:03:09.988412 12263 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:10.037379  8220 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:10.047070  8405 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:03:10.035216 12195 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.034354 12338 raft_consensus.cc:740] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:10.055294  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:10.056803 12195 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:10.053671 12220 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:10.068822  8405 raft_consensus.cc:2393] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 12: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:10.091639  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:10.092098  8405 raft_consensus.cc:2376] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9. Re-sending same reply.
I20260709 13:03:10.050124 12233 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:10.051237 12337 raft_consensus.cc:697] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:03:10.101909 12315 log.cc:927] Time spent T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.579s	user 0.000s	sys 0.009s
W20260709 13:03:10.099582 12237 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.001s	sys 0.008s
W20260709 13:03:10.126983 12237 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.001s	sys 0.009s
I20260709 13:03:09.943874 12309 consensus_queue.cc:237] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:10.119468  8257 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:03:10.112641 12147 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:10.122205 12345 raft_consensus.cc:3055] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:03:10.125990 12296 consensus_meta.cc:287] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.920s	user 0.011s	sys 0.000s
I20260709 13:03:10.128489  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:10.135452  8257 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:10.129657  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:10.148466  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:10.158037  8405 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:10.159049  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:10.159627  8405 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:10.159869  8257 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:03:10.161895 12147 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:10.169616  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:10.131173 12345 raft_consensus.cc:740] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:10.134780 12191 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:10.134589 12207 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:10.170991  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:10.134088 12296 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:10.171427 12301 consensus_meta.cc:287] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.539s	user 0.008s	sys 0.003s
I20260709 13:03:10.177138  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:10.185838  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:10.191536  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2352) took 1328 ms. Trace:
I20260709 13:03:10.192013  8329 rpcz_store.cc:276] 0709 13:03:08.863538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.170295 (+306757us) service_pool.cc:224] Handling call
0709 13:03:09.170704 (+   409us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:09.171186 (+   482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:09.171212 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:09.171907 (+   695us) log.cc:844] Serialized 52 byte log entry
0709 13:03:09.618195 (+446288us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:09.635968 (+ 17773us) log.cc:844] Serialized 14 byte log entry
0709 13:03:09.776041 (+140073us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.776158 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:10.018011 (+241853us) spinlock_profiling.cc:243] Waited 25.8 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:10.187603 (+169592us) raft_consensus.cc:1685] finished
0709 13:03:10.191282 (+  3679us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.191473 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14969,"prepare.queue_time_us":23260,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":110056,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":3645,"spinlock_wait_cycles":59386752,"thread_start_us":573440,"threads_started":2,"wal-append.queue_time_us":335997}
W20260709 13:03:10.202911 12307 log.cc:927] Time spent T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.733s	user 0.000s	sys 0.002s
I20260709 13:03:10.188006 12301 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:10.209702 12296 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:10.210435 12338 consensus_queue.cc:260] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:10.226821 12338 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:10.230177  8326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 516) took 2252 ms (client timeout 2071 ms). Trace:
W20260709 13:03:10.230613  8326 rpcz_store.cc:269] 0709 13:03:07.975407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.975525 (+   118us) service_pool.cc:224] Handling call
0709 13:03:10.227581 (+2252056us) spinlock_profiling.cc:243] Waited 84.5 ms on lock 0x7b540014ffd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:10.227727 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":194425600}
I20260709 13:03:10.263652 12296 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.264566 12296 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:10.267901 12296 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:10.292603  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 255364us
I20260709 13:03:10.294377  8220 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:10.310490  8220 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:10.311950  8220 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:10.289732 12282 log.cc:927] Time spent T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.766s	user 0.001s	sys 0.001s
W20260709 13:03:10.309051 12325 log.cc:927] Time spent T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:03:10.325093  8324 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:10.325273  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:10.327390  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:10.328447  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.315869 12245 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:10.334233  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:10.336215  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:10.344044  8328 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:10.311235 12313 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:10.332017  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:10.334900  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:10.346702  8331 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:10.346833  8330 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:10.317824 12335 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:10.353312 12301 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:10.317582 12345 consensus_queue.cc:260] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:10.353840 12343 log.cc:927] Time spent T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.254s	user 0.000s	sys 0.001s
I20260709 13:03:10.355405  8328 raft_consensus.cc:2410] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:03:10.191638 12337 consensus_queue.cc:237] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:10.332652 12305 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:10.368338  8220 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:10.321215 12233 consensus_queue.cc:1059] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:03:10.367686 12195 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:10.384999  8331 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
W20260709 13:03:10.386008  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 713680us
W20260709 13:03:10.389230  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.386327 12209 consensus_queue.cc:1059] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:10.386716 12246 raft_consensus.cc:697] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:10.387274 12272 raft_consensus.cc:3055] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:10.387993  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1992) took 2017 ms. Trace:
I20260709 13:03:10.391798  8397 rpcz_store.cc:276] 0709 13:03:08.370796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.815543 (+444747us) service_pool.cc:224] Handling call
0709 13:03:08.815992 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.119417 (+303425us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.387722 (+1268305us) spinlock_profiling.cc:243] Waited 77.5 ms on lock 0x7b5400276854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:10.387914 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":178324352}
I20260709 13:03:10.392803  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:10.393153  8397 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:10.365495 12346 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:03:10.376545 12305 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:10.395563 12286 log.cc:927] Time spent T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 1.951s	user 0.002s	sys 0.011s
W20260709 13:03:10.396828  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 1.09632e+06us
I20260709 13:03:10.397461 12050 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:10.399189  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1995) took 1534 ms. Trace:
I20260709 13:03:10.415683  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:10.415985  8403 rpcz_store.cc:276] 0709 13:03:08.864429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.200979 (+336550us) service_pool.cc:224] Handling call
0709 13:03:09.201240 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.798165 (+596925us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.398980 (+600815us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b540019a5d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:10.399136 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1371069184}
I20260709 13:03:10.417156  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:10.418205  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:10.400302  8251 raft_consensus.cc:1275] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:10.418390  8223 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:10.419003  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1939) took 4133 ms. Trace:
I20260709 13:03:10.419356  8251 rpcz_store.cc:276] 0709 13:03:06.285318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.399661 (+4114343us) service_pool.cc:224] Handling call
0709 13:03:10.400010 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.418778 (+ 18768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.418952 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:10.401369 12184 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:10.401592 12328 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:10.402168  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:10.403949  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:10.404780 12209 raft_consensus.cc:3055] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:10.429563 12209 raft_consensus.cc:740] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:10.429383 12328 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:10.430404 12209 consensus_queue.cc:260] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:10.431535 12316 log.cc:927] Time spent T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.918s	user 0.002s	sys 0.000s
I20260709 13:03:10.406294 12233 raft_consensus.cc:3055] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:10.406769 12207 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:10.407959  8397 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
I20260709 13:03:10.438817  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:10.439529  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:10.439831  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:10.440112  8328 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:03:10.440400  8397 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:10.391081  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 577) took 2361 ms. Trace:
I20260709 13:03:10.441603  8257 rpcz_store.cc:276] 0709 13:03:08.029522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.195050 (+2165528us) service_pool.cc:224] Handling call
0709 13:03:10.230279 (+ 35229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.230556 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:10.230857 (+   301us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.373296 (+142439us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.390761 (+ 17465us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:10.390817 (+    56us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:10.390862 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.390926 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.391030 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24007,"prepare.queue_time_us":4128,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":43679,"raft.queue_time_us":36837,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":2762,"spinlock_wait_cycles":9459584,"thread_start_us":131647,"threads_started":1}
I20260709 13:03:10.409065  8220 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:10.410265  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1996) took 1545 ms. Trace:
I20260709 13:03:10.431654 12328 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:10.444238  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:10.443186  8400 rpcz_store.cc:276] 0709 13:03:08.865104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.207914 (+342810us) service_pool.cc:224] Handling call
0709 13:03:09.208181 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.208377 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:09.208645 (+   268us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.382163 (+1173518us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.397311 (+ 15148us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:10.397364 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:10.397407 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.410106 (+ 12699us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.410213 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1105609,"prepare.queue_time_us":1529,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":12122,"raft.queue_time_us":982003,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":159,"spinlock_wait_cycles":9653504,"thread_start_us":209930,"threads_started":1,"wal-append.queue_time_us":301863}
I20260709 13:03:10.444842  8257 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:10.408740  8295 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:10.445911  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:10.446732  8257 raft_consensus.cc:2393] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 11: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:10.447595  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:10.425287 12245 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:03:10.393841 12221 consensus_meta.cc:287] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.001s
I20260709 13:03:10.432947  8372 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:10.420111  9006 raft_consensus.cc:1240] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: [11.11-11.11]
W20260709 13:03:10.405805  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.461229  8295 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:10.434996 12265 raft_consensus.cc:2764] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:03:10.435626 12335 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:10.433736  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:10.433219  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:10.435850  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:10.437070  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:10.431654 12209 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:10.441200  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:10.448887 12120 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:10.426106  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:10.436733 12233 raft_consensus.cc:740] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:10.427850  9001 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:03:10.391635 12272 raft_consensus.cc:740] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:03:10.459447  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 50427.4us
I20260709 13:03:10.461014  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:10.422269 12339 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:10.445235  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:10.461736  8257 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:03:10.461810 12341 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:10.462670  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:10.421489 12050 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.463636  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:10.475309  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:10.475968  8330 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:10.475253  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:10.476490  8331 raft_consensus.cc:2410] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:03:10.475751  8323 raft_consensus.cc:2393] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 10: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:03:10.480690  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.480779  8327 raft_consensus.cc:2393] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 8: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:10.475742  8295 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:10.484547  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:10.484668  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:10.493467  8403 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:10.494760  8251 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:03:10.507134  8400 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:10.502556  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:10.510211  9006 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:10.499032  9000 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:10.513522  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:10.513728  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:10.513548  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:10.514335  8327 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:10.514271  8330 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:03:10.507220  8325 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:10.515076  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:10.515388  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:10.516335  8328 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:03:10.516649  8330 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:10.517613  8222 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:10.509444  8332 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:10.519326  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:10.519876  8328 raft_consensus.cc:1240] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:10.523654  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:10.524394  8323 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:10.483978 12184 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.526674  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:10.526572  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:10.578899  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 43919.3us
I20260709 13:03:10.578116  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2365) took 2822 ms. Trace:
I20260709 13:03:10.578191  8323 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:10.582470  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:03:10.583002  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 56774.3us
I20260709 13:03:10.584522  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:10.579536  8402 rpcz_store.cc:276] 0709 13:03:07.755538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.577324 (+2821786us) service_pool.cc:224] Handling call
0709 13:03:10.577652 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.577820 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:10.577844 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:10.577885 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:10.577914 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.577965 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.578064 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:10.590164  8327 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:10.483382 12328 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:10.492897 12120 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:10.493971 12195 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:10.595817  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:10.613358  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:10.614188  8327 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:10.482695 12221 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:10.516599 12162 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:10.618201  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1951) took 3755 ms. Trace:
I20260709 13:03:10.494081 12207 consensus_queue.cc:1059] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.092s
W20260709 13:03:10.577127 12316 log.cc:927] Time spent T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:03:10.622884  8330 raft_consensus.cc:1275] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:10.632241  8403 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:10.517608 12339 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:10.527077 12350 log.cc:927] Time spent T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:03:10.635528  8254 rpcz_store.cc:276] 0709 13:03:06.846276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.911801 (+1065525us) service_pool.cc:224] Handling call
0709 13:03:08.023065 (+111264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:08.067158 (+ 44093us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.601846 (+2534688us) spinlock_profiling.cc:243] Waited 94.6 ms on lock 0x7b540016f650. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:10.602048 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":217522944}
I20260709 13:03:10.510026 12050 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:10.636724  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 118070us
I20260709 13:03:10.639459  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:10.521593 12023 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:10.521700 12238 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:10.611474 12120 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:10.604416 12328 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:10.647156  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:10.582324 12341 raft_consensus.cc:697] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:03:10.647534  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 130036us
I20260709 13:03:10.643218 12207 consensus_queue.cc:1243] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:03:10.614189 12313 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:10.642935 12162 raft_consensus.cc:697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:10.634788 12319 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:10.678894  8250 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:10.679286 12319 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:10.680622 12348 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.212s	user 0.000s	sys 0.000s
I20260709 13:03:10.681887  8249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 576) took 2652 ms. Trace:
I20260709 13:03:10.682804 12319 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:10.684067  8369 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:10.685827  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:10.686117  8369 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:10.682235  8249 rpcz_store.cc:276] 0709 13:03:08.029223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.934888 (+1905665us) service_pool.cc:224] Handling call
0709 13:03:09.935187 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:09.983533 (+ 48346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:09.983579 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:09.984124 (+   545us) log.cc:844] Serialized 26 byte log entry
0709 13:03:10.078712 (+ 94588us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:10.078799 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.078890 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:10.681683 (+602793us) raft_consensus.cc:1685] finished
0709 13:03:10.681704 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.681822 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51769,"raft.queue_time_us":53791,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":10934,"thread_start_us":94248,"threads_started":1,"wal-append.queue_time_us":429574}
I20260709 13:03:10.688104  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:10.651149  8972 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:10.602953 12249 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:03:10.647758  8326 raft_consensus.cc:2393] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:10.690164  8972 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:10.516083 12246 consensus_queue.cc:237] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:10.593709 12184 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:10.645247 12339 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:10.663336  8324 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:10.673717  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:10.647167 12245 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.676416  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:10.653973 12085 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:10.750069  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 572) took 1233 ms. Trace:
I20260709 13:03:10.677574 12341 consensus_queue.cc:237] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:10.750759  8398 rpcz_store.cc:276] 0709 13:03:09.516859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.806388 (+289529us) service_pool.cc:224] Handling call
0709 13:03:10.749829 (+943441us) spinlock_profiling.cc:243] Waited 75.3 ms on lock 0x7b54002689d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:10.749994 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":173154048}
I20260709 13:03:10.677747 12162 consensus_queue.cc:237] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:10.678174  8255 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 1943) took 3852 ms (client timeout 1851 ms). Trace:
I20260709 13:03:10.637241 12195 raft_consensus.cc:697] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:10.664155  8222 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:10.689190  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:10.668754 12310 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.666252 12328 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:10.662403 12238 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:10.701385  8972 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:10.695205 12237 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:10.704869  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:10.706338  8401 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:10.810283  8401 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:10.710039  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1953) took 3835 ms. Trace:
I20260709 13:03:10.716593  9005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 841) took 2654 ms. Trace:
I20260709 13:03:10.812227  8252 rpcz_store.cc:276] 0709 13:03:06.874262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.915608 (+1041346us) service_pool.cc:224] Handling call
0709 13:03:07.915906 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.194183 (+1278277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.709719 (+1515536us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540034a8d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:10.709961 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2938302336}
I20260709 13:03:10.812408  9005 rpcz_store.cc:276] 0709 13:03:08.061703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.061898 (+   195us) service_pool.cc:224] Handling call
0709 13:03:08.169976 (+108078us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.888890 (+718914us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:08.888925 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:08.888970 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:08.889010 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.716285 (+1827275us) spinlock_profiling.cc:243] Waited 719 ms on lock 0x7b54006dd654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:10.716340 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.716518 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1652675840}
I20260709 13:03:10.725754  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:10.726687  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:10.727550  9000 raft_consensus.cc:1240] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:10.814090  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:10.728267  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:10.729844  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:10.814740  8252 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:03:10.739022  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:10.815071  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:10.815335  8248 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:10.742977  8151 catalog_manager.cc:5697] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 1 to 7, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 7 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:10.815989  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:10.816975  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:10.816815  8151 catalog_manager.cc:5697] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 1 to 10. New cstate: current_term: 10 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:10.817488  8252 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:10.750176  8406 raft_consensus.cc:3055] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:10.500195 12233 consensus_queue.cc:260] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:10.752715  8255 rpcz_store.cc:269] 0709 13:03:06.825789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.998884 (+173095us) service_pool.cc:224] Handling call
0709 13:03:10.677964 (+3679080us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400408154. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:10.678083 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2394870528}
I20260709 13:03:10.822439  8406 raft_consensus.cc:740] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:10.823155  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:10.824245  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:10.684818 12206 consensus_queue.cc:1048] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:03:10.695554 12261 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:10.790057  8405 raft_consensus.cc:1240] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:10.695832 12198 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:10.699982  8323 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 8. Current term is 9. Ops: [6.6-8.7]
I20260709 13:03:10.701710 11685 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:03:10.802124  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 138029us
I20260709 13:03:10.840598  8222 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:10.774226 12305 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:10.806478  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:10.807031  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 44388.9us
W20260709 13:03:10.818324  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 116982us
I20260709 13:03:10.818351  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:10.847837  8254 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:10.849793  8324 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:10.818495  8151 catalog_manager.cc:5697] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:10.729075  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:10.819221  9001 raft_consensus.cc:2410] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:03:10.819815  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:10.752283 11932 consensus_queue.cc:1059] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.083s
I20260709 13:03:10.827580  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:10.807626 12237 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.831545  8401 raft_consensus.cc:1275] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:10.820713  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 572) took 2792 ms. Trace:
W20260709 13:03:10.834651  8152 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55754 (request call id 199) took 27772 ms (client timeout 15000 ms). Trace:
I20260709 13:03:10.839646  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:10.838586  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:10.845718  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:10.852129  8406 consensus_queue.cc:260] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:10.643028 12272 consensus_queue.cc:260] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:10.839236 12355 log.cc:927] Time spent T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:03:10.829070  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:10.807783 12230 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:10.857995  8151 catalog_manager.cc:5697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 5 to 7, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 7 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:10.794252 12296 consensus_queue.cc:1048] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:03:10.861358  8252 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:03:10.840389 12261 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:10.830143 12263 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:10.835397 12114 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:10.859225 12062 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:10.801997 12169 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:03:10.755925 12245 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:10.870613  8249 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:10.849210 12335 consensus_queue.cc:1059] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:03:10.873211  8151 catalog_manager.cc:5697] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 6 to 7, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 7 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:10.862033  9005 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:03:10.839097 11685 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.873042  8974 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:10.875401  8330 raft_consensus.cc:1275] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:03:10.838860 12198 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.860464  8256 rpcz_store.cc:276] 0709 13:03:08.028120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.938174 (+1910054us) service_pool.cc:224] Handling call
0709 13:03:09.938423 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.938629 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:09.938866 (+   237us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.742455 (+803589us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b18002e8ee0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88283664bc 00007f88256b448b 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e23a
0709 13:03:10.820260 (+ 77805us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.820421 (+   161us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:10.820466 (+    45us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:10.820506 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.820572 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.820664 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":89957,"prepare.queue_time_us":15654,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":88088,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":9368,"spinlock_wait_cycles":38366080,"thread_start_us":31475,"threads_started":1,"wal-append.queue_time_us":68831}
I20260709 13:03:10.882517  8257 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:10.923249  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 580) took 2893 ms. Trace:
I20260709 13:03:10.923748  8257 rpcz_store.cc:276] 0709 13:03:08.030329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.851973 (+2821644us) service_pool.cc:224] Handling call
0709 13:03:10.882300 (+ 30327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.922927 (+ 40627us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.923179 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:10.890729  8253 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:10.853927  8222 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:10.860596  8152 rpcz_store.cc:269] 0709 13:02:43.046190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.060864 (+ 14674us) service_pool.cc:224] Handling call
0709 13:03:10.818076 (+27757212us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:03.070253 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:03.078187 (+  7934us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:03.078240 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:03:03.078258 (+    18us) tablet.cc:662] Decoding operations
0709 13:03:03.079527 (+  1269us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:03.079605 (+    78us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:03.079630 (+    25us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:03:03.140045 (+ 60415us) tablet.cc:701] Row locks acquired
0709 13:03:03.140088 (+    43us) write_op.cc:260] PREPARE: finished
0709 13:03:03.140275 (+   187us) write_op.cc:270] Start()
0709 13:03:03.140421 (+   146us) write_op.cc:276] Timestamp: P: 1783602183140199 usec, L: 0
0709 13:03:03.140438 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:03:03.199325 (+ 58887us) log.cc:844] Serialized 4982 byte log entry
0709 13:03:03.719315 (+519990us) op_driver.cc:464] REPLICATION: finished
0709 13:03:04.027531 (+308216us) write_op.cc:301] APPLY: starting
0709 13:03:04.042659 (+ 15128us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:04.090042 (+ 47383us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:04.090198 (+   156us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:04.154603 (+ 64405us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:04.154973 (+   370us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0709 13:03:04.155031 (+    58us) write_op.cc:312] APPLY: finished
0709 13:03:04.155435 (+   404us) log.cc:844] Serialized 89 byte log entry
0709 13:03:04.157876 (+  2441us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:04.161644 (+  3768us) write_op.cc:454] Released schema lock
0709 13:03:04.163455 (+  1811us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:04.163965 (+   510us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:04.163995 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1348196,"reported_tablets":30,"child_traces":[["op",{"apply.queue_time_us":277376,"apply.run_cpu_time_us":5611,"apply.run_wall_time_us":146500,"num_ops":12,"prepare.queue_time_us":158763,"prepare.run_cpu_time_us":19748,"prepare.run_wall_time_us":293903,"raft.queue_time_us":211829,"raft.run_cpu_time_us":21712,"raft.run_wall_time_us":206909,"replication_time_us":571604,"thread_start_us":479032,"threads_started":4,"wal-append.queue_time_us":234058,"wal-append.run_cpu_time_us":67816,"wal-append.run_wall_time_us":1737152}]]}
W20260709 13:03:10.925758  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 71897us
I20260709 13:03:10.866242 12237 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:10.800944 12310 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:10.901816  8248 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:10.940039  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1929) took 2909 ms. Trace:
I20260709 13:03:10.865792 12207 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:10.837491 12260 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:10.915099  8254 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:10.918119  8151 catalog_manager.cc:5697] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 4 to 7, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 7 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:03:10.890682 12261 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:10.929010  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:10.929174  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:10.937959  8325 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:10.940491  8253 rpcz_store.cc:276] 0709 13:03:08.030897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.866150 (+2835253us) service_pool.cc:224] Handling call
0709 13:03:10.890565 (+ 24415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.939749 (+ 49184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.939979 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:10.941927  8256 raft_consensus.cc:1240] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:03:10.942988  8257 raft_consensus.cc:1240] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:03:10.959067  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1934) took 2926 ms. Trace:
I20260709 13:03:10.907974 12169 consensus_queue.cc:1243] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:03:10.949301  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.906353 12062 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.950435  8151 catalog_manager.cc:5697] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 2 to 10, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 10 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:10.952999  8254 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:10.922258 12356 log.cc:927] Time spent T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:03:10.956951  8249 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:03:10.875077 12229 consensus_queue.cc:1048] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:03:10.959553  8257 rpcz_store.cc:276] 0709 13:03:08.032228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.942317 (+2910089us) service_pool.cc:224] Handling call
0709 13:03:10.942700 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:10.958741 (+ 16041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.958998 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:10.947875  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 74919us
I20260709 13:03:10.978541  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1940) took 1224 ms. Trace:
I20260709 13:03:10.978790  8974 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:10.978922  8253 rpcz_store.cc:276] 0709 13:03:09.754102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.958329 (+1204227us) service_pool.cc:224] Handling call
0709 13:03:10.978092 (+ 19763us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.978249 (+   157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:10.978271 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:10.978309 (+    38us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:10.978337 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.978395 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.978492 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:10.979955  8974 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:10.890282 12249 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:03:10.882010 11848 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:03:10.966883  8401 raft_consensus.cc:2393] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:10.967609  8405 raft_consensus.cc:1275] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:10.969511  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1932) took 2937 ms. Trace:
I20260709 13:03:10.988093  8256 rpcz_store.cc:276] 0709 13:03:08.031697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.941304 (+2909607us) service_pool.cc:224] Handling call
0709 13:03:10.941690 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:10.969198 (+ 27508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.969420 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:10.970301  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:10.970472  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.919512 12114 raft_consensus.cc:697] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:10.972008  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1938) took 1218 ms. Trace:
I20260709 13:03:10.991909  8253 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:10.890645 12195 consensus_queue.cc:237] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:10.921618 12221 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:10.931248 11685 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:10.931118 12198 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:10.981032  8151 catalog_manager.cc:5697] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 5 to 9, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 9 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:10.999354  8151 catalog_manager.cc:5697] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:10.948697 12207 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:11.003268 12207 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:10.956375 12257 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:03:10.989295  8298 connection.cc:256] RPC call timeout handler was delayed by -2.1336718439999913s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1532s for a response
I20260709 13:03:11.005666 12301 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:10.991469  8254 rpcz_store.cc:276] 0709 13:03:09.753510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.949282 (+1195772us) service_pool.cc:224] Handling call
0709 13:03:10.952784 (+  3502us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.971744 (+ 18960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.971952 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:10.974507 12023 consensus_queue.cc:1059] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 13:03:10.974781 12062 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:10.972458 12184 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:11.002599 12249 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:11.008816  8332 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:10.955302 12368 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:11.005698  8298 connection.cc:256] RPC call timeout handler was delayed by -1.9275718439999991s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1761s for a response
I20260709 13:03:11.001268  8151 catalog_manager.cc:5697] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 6 to 8, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 8 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:03:10.988004 12296 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:11.046775  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:11.054556  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 278743us
W20260709 13:03:11.053296  8298 connection.cc:256] RPC call timeout handler was delayed by -1.5366035270000111s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1542s for a response
W20260709 13:03:11.054497  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 365329us
I20260709 13:03:11.054733  8151 catalog_manager.cc:5697] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 5 to 9, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 9 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:11.078279  8220 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:11.079475  8298 connection.cc:256] RPC call timeout handler was delayed by -1.0749035270000604s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20550000000000002s for a response
I20260709 13:03:11.072785  8297 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:11.090317  8297 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:11.080346  8151 catalog_manager.cc:5697] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:11.043919 12162 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:11.071491 12238 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:11.052017 12378 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:11.084486 12377 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.100977 12221 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:11.053859 12296 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:11.102494 12287 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:11.135115 12377 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:11.102670  8151 catalog_manager.cc:5697] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: config changed from index 3 to 6, term changed from 10 to 12, f865c4d761f5446eb12b3a029234772e (127.6.255.4) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1), VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2), VOTER e64761f43e4240319c41117cf886d22f (127.6.255.3), VOTER f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 12 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:11.148602 12120 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:11.148687  8151 catalog_manager.cc:5697] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 5 to 7, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 7 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:11.104550 12267 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:11.151475  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 61272.5us
W20260709 13:03:11.152069  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 71936.3us
W20260709 13:03:11.155028  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 175148us
W20260709 13:03:11.154794  8400 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.005s	sys 0.013s
I20260709 13:03:11.114552 12162 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:11.106644 12114 consensus_queue.cc:237] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:11.157790  8401 raft_consensus.cc:1275] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:11.164721  8151 catalog_manager.cc:5697] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 5 to 10, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 10 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:11.193099  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.125135 12383 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:11.213830  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 581) took 2626 ms. Trace:
I20260709 13:03:11.271733  8974 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:11.132767 12221 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:11.271397  8297 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:11.268962  8151 catalog_manager.cc:5697] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 7 to 9, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 9 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:11.273526  8369 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:11.271199  8248 rpcz_store.cc:276] 0709 13:03:08.587205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.939249 (+2352044us) service_pool.cc:224] Handling call
0709 13:03:10.939676 (+   427us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:10.940079 (+   403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:10.940119 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:10.940834 (+   715us) log.cc:844] Serialized 52 byte log entry
0709 13:03:11.120384 (+179550us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:11.120445 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.120543 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:11.213460 (+ 92917us) raft_consensus.cc:1685] finished
0709 13:03:11.213489 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:11.213742 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1017,"raft.queue_time_us":8337,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":21552,"thread_start_us":129265,"threads_started":1,"wal-append.queue_time_us":233131}
I20260709 13:03:11.274345  8151 catalog_manager.cc:5697] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 5 to 8, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 8 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:11.276715  8974 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:11.268930  8298 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:11.214181 12385 log.cc:927] Time spent T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 13:03:11.210171 12370 log.cc:927] Time spent T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:03:11.226969 12120 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:11.278350  8151 catalog_manager.cc:5697] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 2 to 9. New cstate: current_term: 9 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:11.314333  8974 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:11.315250  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:11.356352  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 87517.8us
I20260709 13:03:11.222486 12310 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:11.128367 12378 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:03:11.225673 12245 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:11.228853 12377 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:11.260830 12382 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:03:11.358455  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 56571.4us
I20260709 13:03:11.375628  8369 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:11.358495  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 44293.1us
I20260709 13:03:11.278699 12368 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:11.373855  8151 catalog_manager.cc:5697] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 3 to 7, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 7 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:11.362952 12364 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:03:11.372586 12394 log.cc:927] Time spent T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.220s	user 0.000s	sys 0.003s
I20260709 13:03:11.280288 12386 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:11.377341  8974 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:11.305179 12376 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:11.275413 12221 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:11.304728 12383 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:03:11.357051 12324 log.cc:927] Time spent T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 1.739s	user 0.000s	sys 0.004s
I20260709 13:03:11.411814  8369 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:11.361398 12296 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:11.437891  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 166568us
W20260709 13:03:11.409032 12209 consensus_meta.cc:287] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.901s	user 0.012s	sys 0.000s
I20260709 13:03:11.415616  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:11.441123  8297 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:11.428773 12169 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:03:11.407778 12339 consensus_meta.cc:287] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.636s	user 0.006s	sys 0.005s
I20260709 13:03:11.446753  8369 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:11.392679 12267 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:11.280158 12198 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:11.409832 12377 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:11.399472 12368 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:11.450739  8297 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:11.425676 12310 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:11.414825 12376 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:03:11.439986 12370 log.cc:927] Time spent T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:03:11.413697 12383 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:03:11.448172 12050 consensus_meta.cc:287] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.786s	user 0.012s	sys 0.000s
I20260709 13:03:11.426985 12378 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:11.448282 12380 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:11.451206 12339 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:11.476765  8297 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:11.436733 12390 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:11.450999 12238 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:11.448084 12169 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:11.489746  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 118977us
W20260709 13:03:11.451264 12328 consensus_meta.cc:287] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.632s	user 0.011s	sys 0.000s
I20260709 13:03:11.495203  8297 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:11.499152  8297 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:11.413527 12120 consensus_queue.cc:1059] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:03:11.476763 12368 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:11.515019  8256 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:11.408883 12382 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:11.477595 12383 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:11.501052 12169 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:11.583575 12339 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:11.477134 12376 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:11.494508 12390 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:11.497879 12238 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:03:11.562772 12370 log.cc:927] Time spent T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
W20260709 13:03:11.611491 12327 log.cc:927] Time spent T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 2.101s	user 0.004s	sys 0.000s
I20260709 13:03:11.507418 12328 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:11.600142 12062 consensus_meta.cc:287] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.581s	user 0.010s	sys 0.001s
I20260709 13:03:11.594489 12380 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.492172 12050 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:11.627862 12296 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:11.640523 12238 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:11.630426 12390 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:11.751061 12190 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:11.751539 12190 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:11.751911 12190 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:11.668715 12162 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:11.773859  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 299461us
I20260709 13:03:11.774067  8974 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:11.775184  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:11.775892 12399 log.cc:927] Time spent T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.366s	user 0.001s	sys 0.000s
W20260709 13:03:11.755077 12207 consensus_meta.cc:287] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.751s	user 0.011s	sys 0.001s
I20260709 13:03:11.787923  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:11.788866  8331 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:03:11.713506 12062 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:11.775563 12296 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:11.818496  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:11.798107 12261 consensus_meta.cc:287] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.819s	user 0.004s	sys 0.008s
W20260709 13:03:11.800210 12398 log.cc:927] Time spent T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.379s	user 0.001s	sys 0.000s
I20260709 13:03:11.830873  8257 raft_consensus.cc:1240] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12. Ops: [11.5-11.5]
I20260709 13:03:11.715322 12380 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:11.819595  9004 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56948 (request call id 837) took 1345 ms (client timeout 1662 ms). Trace:
W20260709 13:03:11.832031  9004 rpcz_store.cc:269] 0709 13:03:10.474431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.474622 (+   191us) service_pool.cc:224] Handling call
0709 13:03:11.818276 (+1343654us) spinlock_profiling.cc:243] Waited 1.03 ms on lock 0x7b5800175898. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:03:11.819516 (+  1240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2361216}
I20260709 13:03:11.782917 12162 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:03:11.816416 12296 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:11.814605 12312 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:11.862036 12050 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:11.864892 12272 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:11.794498 12207 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:11.844599 12198 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:11.871757 12296 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:11.835460 12261 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:11.871531 12312 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:11.882599 12050 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:11.886237 12195 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:03:11.893234 12237 consensus_meta.cc:287] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.935s	user 0.013s	sys 0.000s
I20260709 13:03:11.891465 12382 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:11.895627 12312 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:11.932865 12358 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 1.024s	user 0.000s	sys 0.003s
I20260709 13:03:11.933063 12221 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:11.935416 12221 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:11.909698 12368 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:11.940804 12261 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:11.922258 12404 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.942137 12404 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:11.926117 12328 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:11.943944 12386 consensus_meta.cc:287] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.504s	user 0.010s	sys 0.002s
I20260709 13:03:11.944363 12386 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:11.914268 12237 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:11.946995 12389 log.cc:927] Time spent T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.790s	user 0.002s	sys 0.000s
W20260709 13:03:11.974476 12397 log.cc:927] Time spent T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.821s	user 0.001s	sys 0.002s
I20260709 13:03:11.974825 12380 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:11.977761 12401 log.cc:927] Time spent T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.563s	user 0.002s	sys 0.000s
I20260709 13:03:11.980305 12376 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:11.982470 12376 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:11.983930 12404 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:12.016929  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 517604us
I20260709 13:03:12.020577  8297 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:12.024623  8297 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:12.027971  8297 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:12.037243  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2381) took 1635 ms. Trace:
I20260709 13:03:12.055709  9003 raft_consensus.cc:2368] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12.
W20260709 13:03:12.056756  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 282610us
I20260709 13:03:11.905064 12050 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.071079 12062 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:12.072706 12114 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:12.073588 12120 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:11.908496 12195 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:12.091514 12237 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:12.095358 12207 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:12.115345 12368 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:12.118227 12403 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:12.127034 12345 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:12.140326  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1981) took 1660 ms. Trace:
I20260709 13:03:12.133769 12114 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.144879 12114 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:12.108202  8398 rpcz_store.cc:276] 0709 13:03:10.401379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.751792 (+350413us) service_pool.cc:224] Handling call
0709 13:03:10.755981 (+  4189us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:10.977148 (+221167us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:11.015624 (+ 38476us) log.cc:844] Serialized 14 byte log entry
0709 13:03:11.153827 (+138203us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:11.154317 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 13:03:11.154456 (+   139us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:11.154491 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.154558 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:11.941238 (+786680us) spinlock_profiling.cc:243] Waited 31.3 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:12.036083 (+ 94845us) raft_consensus.cc:1685] finished
0709 13:03:12.036389 (+   306us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:12.036878 (+   489us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10791,"prepare.queue_time_us":29630,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":96352,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":20656,"spinlock_wait_cycles":71957760,"thread_start_us":272243,"threads_started":2,"wal-append.queue_time_us":81396}
I20260709 13:03:12.147068  8249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1930) took 4115 ms. Trace:
I20260709 13:03:12.146700 12207 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:12.147704  8249 rpcz_store.cc:276] 0709 13:03:08.031150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.908764 (+2877614us) service_pool.cc:224] Handling call
0709 13:03:10.952269 (+ 43505us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:10.976774 (+ 24505us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.146500 (+1169726us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b5400346cd0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:12.146827 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34099456}
I20260709 13:03:12.149646 12050 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:12.153947 12207 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:12.158190 12207 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:12.140960  8256 rpcz_store.cc:276] 0709 13:03:10.480138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.995358 (+515220us) service_pool.cc:224] Handling call
0709 13:03:10.995646 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.557281 (+561635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.139552 (+582271us) spinlock_profiling.cc:243] Waited 519 ms on lock 0x7b5400178754. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:12.140010 (+   458us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1193796864}
I20260709 13:03:12.153192  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1944) took 2370 ms. Trace:
I20260709 13:03:12.160835  8257 rpcz_store.cc:276] 0709 13:03:09.782476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.977920 (+1195444us) service_pool.cc:224] Handling call
0709 13:03:10.997246 (+ 19326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:11.837076 (+839830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.152497 (+315421us) spinlock_profiling.cc:243] Waited 833 ms on lock 0x7b5400407c54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:12.152970 (+   473us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1916431232}
I20260709 13:03:12.121884 12195 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:12.162456 12403 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:12.157042 12390 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.173328  8974 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:12.178570 12238 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:12.202843  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 175205us
I20260709 13:03:12.204905 12368 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:12.207742 12294 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:12.210511 12296 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:12.197075 12267 consensus_meta.cc:287] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.733s	user 0.009s	sys 0.013s
I20260709 13:03:12.212034  8972 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:12.187440 12390 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:12.226079  8297 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:12.228461  8297 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:03:12.229022 11685 consensus_meta.cc:287] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 1.209s	user 0.018s	sys 0.007s
I20260709 13:03:12.279794 11685 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.231941  8403 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:12.236483  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1956) took 5291 ms. Trace:
I20260709 13:03:12.240280 12391 raft_consensus.cc:3055] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:03:12.300743  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 89067.7us
W20260709 13:03:12.243328 12383 consensus_meta.cc:287] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.618s	user 0.007s	sys 0.013s
I20260709 13:03:12.245103 12296 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:12.246285  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 578) took 4216 ms. Trace:
W20260709 13:03:12.247546 12169 consensus_meta.cc:287] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.667s	user 0.022s	sys 0.001s
I20260709 13:03:12.252169  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:12.252965  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:12.254195  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 1986) took 4445 ms. Trace:
I20260709 13:03:12.254257  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:12.255199  8402 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:12.256506  8398 raft_consensus.cc:1240] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:12.258402 12188 raft_consensus.cc:3055] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:12.259503  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:12.259709  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:12.260423  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:12.261559  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:12.305383  8251 rpcz_store.cc:276] 0709 13:03:08.029800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.424866 (+2395066us) service_pool.cc:224] Handling call
0709 13:03:10.425299 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:10.510030 (+ 84731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.245613 (+1735583us) spinlock_profiling.cc:243] Waited 69.2 ms on lock 0x7b54001306d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:12.246079 (+   466us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":159237504}
I20260709 13:03:12.261853  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:12.261018  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:12.270550 12267 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:12.331792  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:12.332726  8397 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:12.273243 12312 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.334391  9007 raft_consensus.cc:1240] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:12.335569 12368 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:12.308239 12188 raft_consensus.cc:740] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:12.338124 11685 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:12.274910 12190 consensus_meta.cc:287] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.010s
I20260709 13:03:12.276226 12262 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.278752 12233 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:12.284010 12395 log.cc:927] Time spent T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.869s	user 0.003s	sys 0.001s
I20260709 13:03:12.287356  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 535) took 1776 ms. Trace:
I20260709 13:03:12.286844 12050 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.288445 12114 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:12.289741 12373 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.231108 12363 raft_consensus.cc:3055] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:12.371215 12363 raft_consensus.cc:740] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:12.293049 12403 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:12.373689 12403 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:12.374236 12403 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:12.294010 12120 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:12.377787 12120 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:12.378328 12120 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:12.347302 12190 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.384003 12190 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:12.385900 12373 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:12.388792 12373 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:12.348326 12383 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:12.391862  8323 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:12.392238 12383 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:12.393693  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:12.394461  8331 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:12.395653 12050 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:12.295468  8406 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:12.296841  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 68610.4us
I20260709 13:03:12.300575  8250 rpcz_store.cc:276] 0709 13:03:06.945245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.979949 (+1034704us) service_pool.cc:224] Handling call
0709 13:03:10.678532 (+2698583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.679356 (+   824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.235878 (+1556522us) spinlock_profiling.cc:243] Waited 2.7 s on lock 0x7b5400408150. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:12.236191 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6204899200}
I20260709 13:03:12.310447 12209 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.311460  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:12.311616 12260 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:12.312728  8401 raft_consensus.cc:2410] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:03:12.314060  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:12.314756  9002 raft_consensus.cc:2393] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 12: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:12.315110  9003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56954 (request call id 722) took 1590 ms. Trace:
I20260709 13:03:12.315227 12169 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.403200  9007 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:12.404469  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:12.405393 12260 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:12.410346  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:12.411255  8401 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:12.413699  8402 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:12.416059  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1928) took 4385 ms. Trace:
I20260709 13:03:12.316148  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:12.317219  9000 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:12.317684  9005 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:12.318557  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 586) took 1799 ms. Trace:
I20260709 13:03:12.302011 12391 raft_consensus.cc:740] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:12.322307 12247 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:03:12.323423  8403 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 2389) took 1395 ms (client timeout 1817 ms). Trace:
W20260709 13:03:12.330467  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.330358  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:12.338229 12312 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:12.273867 12238 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:12.306986  8399 rpcz_store.cc:276] 0709 13:03:07.808825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.105004 (+296179us) service_pool.cc:224] Handling call
0709 13:03:08.170162 (+ 65158us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.347353 (+177191us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:08.891364 (+544011us) log.cc:844] Serialized 14 byte log entry
0709 13:03:09.377382 (+486018us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.020658 (+643276us) spinlock_profiling.cc:243] Waited 27.3 ms on lock 0x7b180045fd60. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:03:10.064417 (+ 43759us) spinlock_profiling.cc:243] Waited 3.86 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:10.372394 (+307977us) spinlock_profiling.cc:243] Waited 23 ms on lock 0x7b180045fd60. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882564bc0a 00007f882ed8b219 00007f88256d3a40 00007f88256ce4e4 00007f88256ce455 00007f88287b6060 00007f88287c8716 00007f88287d9287 00007f88287d9208
0709 13:03:10.505752 (+133358us) spinlock_profiling.cc:243] Waited 4.74 ms on lock 0x7b180045fd60. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256ce455 00007f88287b6060 00007f88287c8716 00007f88287d9287 00007f88287d9208
0709 13:03:10.834852 (+329100us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.856745 (+ 21893us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.856898 (+   153us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:10.856942 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:10.856991 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.253654 (+1396663us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54000fc8d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:12.253727 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:12.253970 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1095036,"prepare.queue_time_us":406596,"prepare.run_cpu_time_us":189,"prepare.run_wall_time_us":18997,"raft.queue_time_us":1087508,"raft.run_cpu_time_us":359,"raft.run_wall_time_us":24800,"spinlock_wait_cycles":542150912,"thread_start_us":1244379,"threads_started":3,"wal-append.queue_time_us":619567,"wal-append.run_cpu_time_us":12196,"wal-append.run_wall_time_us":2594780}
I20260709 13:03:12.338686 12188 consensus_queue.cc:260] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.340754  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:12.349294  8332 rpcz_store.cc:276] 0709 13:03:10.510793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.518754 (+  7961us) service_pool.cc:224] Handling call
0709 13:03:10.519023 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.052539 (+533516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.286677 (+1234138us) spinlock_profiling.cc:243] Waited 490 ms on lock 0x7b540037ad54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:12.287095 (+   418us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1125947648}
W20260709 13:03:12.308677  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 360306us
I20260709 13:03:12.370019 12262 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.400190  8397 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:12.398613  9003 rpcz_store.cc:276] 0709 13:03:10.724595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.728797 (+  4202us) service_pool.cc:224] Handling call
0709 13:03:12.314845 (+1586048us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54004b87d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:12.315017 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2763719552}
I20260709 13:03:12.416430  8255 rpcz_store.cc:276] 0709 13:03:08.030620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.861652 (+2831032us) service_pool.cc:224] Handling call
0709 13:03:10.863690 (+  2038us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:10.894666 (+ 30976us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:11.047171 (+152505us) log.cc:844] Serialized 14 byte log entry
0709 13:03:11.261184 (+214013us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:11.261685 (+   501us) log.cc:844] Serialized 26 byte log entry
0709 13:03:11.261829 (+   144us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:11.262101 (+   272us) log.cc:844] Serialized 14 byte log entry
0709 13:03:11.262332 (+   231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.262414 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:11.947887 (+685473us) spinlock_profiling.cc:243] Waited 8.34 ms on lock 0x7b18001a7280. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882569f2a6 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882d6e7aa1 00007f882d6deb37 00007f882d6de9ca 00007f882d7d626f
0709 13:03:12.187800 (+239913us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b18001a7280. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d1522 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:03:12.282762 (+ 94962us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:12.415679 (+132917us) raft_consensus.cc:1685] finished
0709 13:03:12.415702 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:12.415896 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":515797,"prepare.queue_time_us":261064,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":75,"raft.queue_time_us":3873,"raft.run_cpu_time_us":2935,"raft.run_wall_time_us":694834,"spinlock_wait_cycles":88234624,"thread_start_us":323418,"threads_started":2,"wal-append.queue_time_us":250150}
I20260709 13:03:12.405288 12169 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:12.405297  9007 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 14: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:12.433641  8405 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:12.434083 12251 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:12.372612 12363 consensus_queue.cc:260] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:12.429709  8298 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.001s	sys 0.010s
I20260709 13:03:12.447485  9002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56954 (request call id 723) took 1722 ms. Trace:
W20260709 13:03:12.422266  8403 rpcz_store.cc:269] 0709 13:03:10.928440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.928636 (+   196us) service_pool.cc:224] Handling call
0709 13:03:12.323151 (+1394515us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b540020fb54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:12.323323 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2929296768}
W20260709 13:03:12.455771  8298 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.957s	user 0.001s	sys 0.010s
I20260709 13:03:12.455972  9009 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:12.451560  9002 rpcz_store.cc:276] 0709 13:03:10.724910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.729532 (+  4622us) service_pool.cc:224] Handling call
0709 13:03:12.422984 (+1693452us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b540057f8d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:12.447425 (+ 24441us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3425853696}
I20260709 13:03:12.460994  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:12.462093  8397 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 14: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:12.393052 12383 consensus_queue.cc:1059] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:03:12.420416  8253 rpcz_store.cc:276] 0709 13:03:10.519007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.995114 (+476107us) service_pool.cc:224] Handling call
0709 13:03:11.016192 (+ 21078us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:11.021292 (+  5100us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:11.202186 (+180894us) log.cc:844] Serialized 14 byte log entry
0709 13:03:11.312821 (+110635us) log.cc:844] Serialized 14 byte log entry
0709 13:03:11.424829 (+112008us) spinlock_profiling.cc:243] Waited 36 ms on lock 0x7b1800148180. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882564bc0a 00007f882ed8b219 00007f88256d3a40 00007f88256ce4e4 00007f88256ce455 00007f88287b6060 00007f88287c8716 00007f88287d9287 00007f88287d9208
0709 13:03:11.425058 (+   229us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:11.434651 (+  9593us) log.cc:844] Serialized 26 byte log entry
0709 13:03:11.435213 (+   562us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:11.435575 (+   362us) log.cc:844] Serialized 14 byte log entry
0709 13:03:11.947645 (+512070us) spinlock_profiling.cc:243] Waited 35.6 ms on lock 0x7b1800148180. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:03:11.948538 (+   893us) spinlock_profiling.cc:243] Waited 4.35 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:12.291574 (+343036us) spinlock_profiling.cc:243] Waited 5.25 ms on lock 0x7b1800148180. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d1522 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:03:12.317409 (+ 25835us) spinlock_profiling.cc:243] Waited 1.02 ms on lock 0x7b1800148180. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882564bc0a 00007f882ed8b219 00007f88256d3a40 00007f88256ce4e4 00007f88256ce455 00007f88287b6060 00007f88287c8716 00007f88287d9287 00007f88287d9208
0709 13:03:12.317901 (+   492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.318048 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:12.318126 (+    78us) raft_consensus.cc:1685] finished
0709 13:03:12.318140 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:12.318344 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":430501,"prepare.queue_time_us":137670,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":105,"raft.queue_time_us":1015583,"raft.run_cpu_time_us":550,"raft.run_wall_time_us":16704,"spinlock_wait_cycles":190960000,"thread_start_us":220342,"threads_started":2,"wal-append.queue_time_us":146957}
I20260709 13:03:12.421794 12247 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.483776  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:12.420768 12195 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.480335 12363 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:12.481987  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 983049us
I20260709 13:03:12.500420  8972 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:12.496569  8406 raft_consensus.cc:1275] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:03:12.499837  8297 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:12.434744 12251 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:12.517894  8323 raft_consensus.cc:1240] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:12.522686  8404 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 581) took 2914 ms (client timeout 2155 ms). Trace:
I20260709 13:03:12.506431  8406 pending_rounds.cc:85] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 7
I20260709 13:03:12.528730  8331 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
W20260709 13:03:12.525224  8404 rpcz_store.cc:269] 0709 13:03:09.608549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.840295 (+231746us) service_pool.cc:224] Handling call
0709 13:03:12.522306 (+2682011us) spinlock_profiling.cc:243] Waited 2.59 s on lock 0x7b5400267fd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:12.522481 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5956088704}
I20260709 13:03:12.456888 12209 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.537696  8401 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:03:12.525900  8406 pending_rounds.cc:107] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:03:12.539541  8406 raft_consensus.cc:2889] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:12.532855  8402 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:03:12.544312 12188 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:12.545639 12190 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:12.546088  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 78732us
I20260709 13:03:12.545922 12260 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.547720  9006 raft_consensus.cc:2393] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 14: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:12.547492 12190 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:12.548235  8297 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:12.548583  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 59987.7us
I20260709 13:03:12.549045 12312 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:12.549883  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:12.550498  8327 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:12.550544  8332 raft_consensus.cc:1240] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:12.551793  8399 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:12.554697 12346 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:12.555775 12346 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:12.556998 12417 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:03:12.558211  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 576) took 2996 ms. Trace:
I20260709 13:03:12.558580  8406 rpcz_store.cc:276] 0709 13:03:09.561381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.966001 (+404620us) service_pool.cc:224] Handling call
0709 13:03:09.978467 (+ 12466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.750085 (+771618us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b54000df820. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f8828735dc0 00007f88287c01c0 00007f88287c1231 00007f88287bdd8f 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c
0709 13:03:12.294594 (+1544509us) spinlock_profiling.cc:243] Waited 967 ms on lock 0x7b58007391d0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f882871a0aa 00007f882879c447 00007f88287b4fe7 00007f88287b67de 00007f88287c0b49 00007f88287c1416 00007f88287bdd8f 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4
0709 13:03:12.557864 (+263270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.558158 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3165381120}
W20260709 13:03:12.548763  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.549180  8325 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:12.574676  9005 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:12.581362  9008 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:12.582837 12190 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:12.583348 12262 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:12.583438 12190 raft_consensus.cc:697] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:12.584668 12195 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:12.584740 12190 consensus_queue.cc:237] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.582882 12418 consensus_queue.cc:1059] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:03:12.586566 12418 raft_consensus.cc:3055] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:12.586915 12418 raft_consensus.cc:740] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:12.590449 12373 consensus_queue.cc:1059] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:12.591449 12373 raft_consensus.cc:3055] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:12.591822 12373 raft_consensus.cc:740] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:12.592810 12373 consensus_queue.cc:260] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:12.593623 12259 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:03:12.593791  8372 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:12.594129  8404 raft_consensus.cc:1240] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:03:12.594820 12267 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:12.595454 12267 raft_consensus.cc:697] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:12.573858 12312 raft_consensus.cc:697] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:12.598150  8330 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:12.597927 12368 consensus_queue.cc:1059] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:12.598441  8398 raft_consensus.cc:1217] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[5.6-13.8]   Dedup: 5.6->[11.7-13.8]
I20260709 13:03:12.599072  8398 raft_consensus.cc:1240] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 13. Current term is 14. Ops: [5.6-13.8]
I20260709 13:03:12.600257 12247 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:12.600088 12131 consensus_queue.cc:1059] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:12.601091 12299 raft_consensus.cc:3055] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:12.601517 12299 raft_consensus.cc:740] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:12.602309  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2354) took 2278 ms. Trace:
I20260709 13:03:12.602550 12299 consensus_queue.cc:260] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.603881 12299 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:12.602679  8329 rpcz_store.cc:276] 0709 13:03:10.323790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.326210 (+  2420us) service_pool.cc:224] Handling call
0709 13:03:10.326516 (+   306us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:10.326879 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:11.584297 (+1257418us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b18003a0fc0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256ce455 00007f882c1b63c7 00007f88287c89a7 00007f88287d80bf 00007f88287d8038
0709 13:03:11.833008 (+248711us) log.cc:844] Serialized 14 byte log entry
0709 13:03:12.298606 (+465598us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:12.374165 (+ 75559us) log.cc:844] Serialized 52 byte log entry
0709 13:03:12.374394 (+   229us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:12.374445 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.374586 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:12.562221 (+187635us) raft_consensus.cc:1685] finished
0709 13:03:12.601916 (+ 39695us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:12.602100 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3212,"prepare.queue_time_us":479345,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":5132,"raft.queue_time_us":29346,"raft.run_cpu_time_us":480,"raft.run_wall_time_us":20794,"spinlock_wait_cycles":23474432,"thread_start_us":368348,"threads_started":2,"wal-append.queue_time_us":176960}
I20260709 13:03:12.604581 12251 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:12.604957 12411 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.606001 12368 raft_consensus.cc:3055] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:12.606482 12368 raft_consensus.cc:740] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:12.606932 12260 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:12.605580 12411 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:12.608687 12368 consensus_queue.cc:260] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:12.598196  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 65961.7us
I20260709 13:03:12.596585 12267 consensus_queue.cc:237] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.594233 12373 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:12.583264  8369 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:12.621706  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 110352us
I20260709 13:03:12.612389 12246 consensus_queue.cc:1059] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:03:12.631218 12275 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.630937 12391 consensus_queue.cc:260] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.631955 12275 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.636590  8399 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
W20260709 13:03:12.635767  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 169180us
I20260709 13:03:12.598518 12312 consensus_queue.cc:237] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.615032 12050 consensus_queue.cc:1048] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:03:12.650830 12275 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:12.651675  8298 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:12.641739 12345 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.640642 12391 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:12.655050  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:12.655857  8329 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:12.587896 12418 consensus_queue.cc:260] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:12.670233  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.670593  8330 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:12.673651  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.675755 12425 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.676609 12414 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:12.655555 12368 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:12.661038 12346 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:12.679243  9009 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:12.681828  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:12.682690  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:12.683120 12419 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 13:03:12.683954 12377 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:12.683980  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2111) took 1887 ms. Trace:
I20260709 13:03:12.697871  9001 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:12.700456  8327 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
I20260709 13:03:12.708493 12418 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:12.710075  8329 raft_consensus.cc:2410] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:03:12.708925 12319 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.716857  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:12.717926 12319 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:12.720641 12319 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:12.721477 12232 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.706439 12246 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.707031 12414 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:12.700779 12247 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:12.732689  8331 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:12.732894  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:12.690312 12345 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:12.769037  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:12.709718 12383 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:12.713368  8326 rpcz_store.cc:276] 0709 13:03:10.796458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.796631 (+   173us) service_pool.cc:224] Handling call
0709 13:03:10.796970 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.797224 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:12.251620 (+1454396us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b1800000d00. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88283664bc 00007f88256b448b 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256ce455 00007f882c1b63c7 00007f88287c89a7 00007f88287d80bf
0709 13:03:12.251830 (+   210us) log.cc:844] Serialized 14 byte log entry
0709 13:03:12.645030 (+393200us) log.cc:844] Serialized 14 byte log entry
0709 13:03:12.666139 (+ 21109us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:12.683439 (+ 17300us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:12.683512 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.683639 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:12.683820 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77117,"prepare.queue_time_us":285437,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":87,"raft.queue_time_us":72272,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":21865,"spinlock_wait_cycles":53964544,"thread_start_us":426128,"threads_started":3,"wal-append.queue_time_us":360266}
I20260709 13:03:12.722007 12251 consensus_queue.cc:1059] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:03:12.784718  8974 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:12.783053  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:12.786912  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 150545us
I20260709 13:03:12.791290  8330 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
I20260709 13:03:12.777458  8331 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:03:12.778075 12247 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.782423 12310 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.798218  8405 raft_consensus.cc:2393] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 10: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:03:12.795722  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.740034 12414 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:12.799154  8406 raft_consensus.cc:1240] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12. Ops: [11.5-11.5]
I20260709 13:03:12.814637  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:12.815027 12345 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:12.816102  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:12.787367 12425 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.818387 12425 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:12.830864  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:12.831964  8406 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:12.799906  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:12.800669 12230 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.801272  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:12.791945 12232 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.786553 12319 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.802879  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:12.807839 12403 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:12.795723 12363 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:12.810456 12120 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:12.838296  9004 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
I20260709 13:03:12.845968 12230 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:12.846961 12319 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:12.848336  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:03:12.848760  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.850389  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:12.850817  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 117157us
I20260709 13:03:12.851586  8251 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:12.851678  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:12.852022  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:12.852223  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:12.852998  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:12.853586 12243 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.870507 12243 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.871444 12247 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:12.854502 12259 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.856266 12238 raft_consensus.cc:3055] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:12.872184 12238 raft_consensus.cc:740] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:12.872566 12243 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:12.857839 12246 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:12.858644 12330 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:12.859266  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:12.861881  8974 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:12.862838 12189 consensus_queue.cc:1048] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:12.852034  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:12.861949  8401 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260709 13:03:12.868860  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.872254 12259 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:12.739730 12346 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.118s
I20260709 13:03:12.873562  8404 raft_consensus.cc:2368] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12.
I20260709 13:03:12.798202 12310 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:12.864060 12363 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:12.851651 12233 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:12.882349  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:12.865849 12376 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:12.887099  8406 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:12.868207 12345 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.850509 12251 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:12.895192  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:12.897569 12330 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.898916 12259 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:12.899818 12330 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:12.888326 12262 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:12.900548 12262 raft_consensus.cc:697] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:12.901126 12345 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.903882  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:12.901166  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:12.904758  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:12.888173 12383 consensus_queue.cc:1059] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:03:12.909703  8248 raft_consensus.cc:2393] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 11: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:12.879138 12267 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:12.912422  9005 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:12.909757  8253 raft_consensus.cc:2376] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11. Re-sending same reply.
I20260709 13:03:12.889881 12050 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.890177 12363 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:12.915001  8405 raft_consensus.cc:2410] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:03:12.916262  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:12.917520  8249 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:12.922287  8331 raft_consensus.cc:1240] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:12.923100 12251 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.923345 12267 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:12.923642 12131 consensus_queue.cc:1059] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:12.925745  8256 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:12.925869  8974 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:12.926352  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:12.926988  8252 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:03:12.918773 12376 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:12.932606  8325 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:12.908628 12312 consensus_queue.cc:1059] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:03:12.940354  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:12.943434  8251 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:12.944113  8327 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:12.948254 12209 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:12.948390 12411 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:12.925662 12259 consensus_queue.cc:1059] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:03:12.949759 12232 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:12.951056  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 62821.5us
I20260709 13:03:12.928133 11685 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.916100 12233 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:12.932056 12251 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:12.961534  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:12.963133 11932 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:12.963519 12330 consensus_queue.cc:1059] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:03:12.959052  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 51849us
I20260709 13:03:12.949975 12259 consensus_queue.cc:1243] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:03:12.969753 12050 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:12.970705 12296 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.928455 12131 consensus_queue.cc:1243] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:03:12.991058  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 49504.2us
I20260709 13:03:12.939239 12246 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:12.950762 12383 raft_consensus.cc:3055] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:12.995306  8298 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:12.974339 12319 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:13.009351 12424 log.cc:927] Time spent T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.461s	user 0.000s	sys 0.002s
I20260709 13:03:13.006631 12411 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:13.002543 12383 raft_consensus.cc:740] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:13.011121 12383 consensus_queue.cc:260] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.012509 12383 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:12.994599  8252 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:50420
I20260709 13:03:13.014005  8252 raft_consensus.cc:606] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:03:13.009472  8404 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:13.001855  8256 raft_consensus.cc:2410] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:03:12.933822 12267 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:13.016801  9004 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:13.009907  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:12.993903  8249 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:03:13.009635  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:13.019582  8253 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:12.921777 12238 consensus_queue.cc:260] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:12.997483 12262 consensus_queue.cc:237] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.025878  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:13.026746 12427 log.cc:927] Time spent T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
I20260709 13:03:13.027069  8298 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:13.027640  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:13.028353  8248 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:03:13.028486 12162 raft_consensus.cc:2764] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:03:13.026958 12319 consensus_queue.cc:1048] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:13.028990  8298 leader_election.cc:400] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12.
I20260709 13:03:13.008116 11932 raft_consensus.cc:697] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:13.031647  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:13.031945 11932 consensus_queue.cc:237] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.032771  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:13.033313  8248 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
W20260709 13:03:13.033562  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.985841 11685 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.034219  8404 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:13.035640  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:13.036309  8332 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:13.037747  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:13.038432  8330 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:03:13.039143 11685 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:13.039228  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:13.039987  8330 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 11: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:13.040812  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:13.041489  8400 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:03:13.042114 12238 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:13.042243  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 147421us
I20260709 13:03:13.043377  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:13.043612  8369 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:13.044718  8324 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:13.045836  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:13.046782 12050 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:13.047351 12050 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:13.047725 12050 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:13.048120 12050 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:13.048171  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:13.026768  8257 raft_consensus.cc:2393] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 11: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:13.049794  8328 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:13.050076 12310 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:13.050150 12232 raft_consensus.cc:3055] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:13.050693 12232 raft_consensus.cc:740] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:13.051941  8329 raft_consensus.cc:1240] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:03:13.051651 12232 consensus_queue.cc:260] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.052891 12232 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:13.055220  8372 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:13.056183  8372 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:13.056943 12267 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:13.057312 12267 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.057677 12267 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:13.058085  8331 raft_consensus.cc:1275] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:13.058391 12376 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:13.058938 12376 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:13.059398 12376 raft_consensus.cc:697] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:13.060583  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:13.038439  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:13.060338 12376 consensus_queue.cc:237] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.064983  8251 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:13.073647  8325 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:03:13.079452  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:13.075196  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:13.083009  8254 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:13.083040  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:13.084650  8325 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 14: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:13.084770  8252 raft_consensus.cc:2410] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:03:13.087534  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.087425  8972 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:13.126746  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:03:13.137205  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 55362us
I20260709 13:03:13.140055  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:13.143224  8248 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
W20260709 13:03:13.149861  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 121116us
I20260709 13:03:13.150422  8369 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:13.150856  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:13.151139  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 63797.7us
I20260709 13:03:13.089735 12403 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:13.158128  8325 raft_consensus.cc:1240] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:13.161849 12131 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:03:13.167148 12431 log.cc:927] Time spent T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.277s	user 0.001s	sys 0.001s
W20260709 13:03:13.169710  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 99596.9us
I20260709 13:03:13.169893 12310 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.171777 12414 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:13.173959  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:13.174719  8331 raft_consensus.cc:1275] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:03:13.174950  8329 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:13.175606  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:13.176277  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:13.177488  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:13.178277  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:13.179006 12259 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:13.182968  8324 raft_consensus.cc:1240] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:13.183817 12345 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:13.132983 12368 consensus_queue.cc:1059] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:03:13.185456 12233 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:13.133396 11685 consensus_queue.cc:1059] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:03:13.127892 12120 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:13.190388  9009 raft_consensus.cc:1275] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:03:13.193790  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.193943  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:13.193991  8298 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:13.204546  8253 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:13.077926 12296 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:13.209095  8332 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:13.226361  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:13.227586  8327 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:13.208648  8331 pending_rounds.cc:85] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 5
I20260709 13:03:13.228562  8331 pending_rounds.cc:107] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:03:13.228971  8331 raft_consensus.cc:2889] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:13.219733  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:13.229981  8257 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 14: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:13.220258  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:13.231185  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:13.231559  8325 raft_consensus.cc:2393] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 12: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:13.231190  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:13.233871  8330 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:13.236686  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:13.239972  8250 raft_consensus.cc:2410] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 12 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
W20260709 13:03:13.243788 12423 log.cc:927] Time spent T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.584s	user 0.000s	sys 0.005s
I20260709 13:03:13.243952 12259 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:13.245384 12272 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:13.246459 12259 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:13.262228  8251 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
W20260709 13:03:13.262771  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 228578us
I20260709 13:03:13.266502  8323 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
W20260709 13:03:13.268150  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 74281.6us
I20260709 13:03:13.268153  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:13.215718 12299 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:03:13.269701  8325 raft_consensus.cc:2410] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:03:13.264930  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:13.274224  8328 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:13.231427 12373 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:13.272696  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:13.279116  9001 raft_consensus.cc:2393] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 12: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:13.206071 12312 consensus_queue.cc:1059] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:03:13.253280 12403 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:13.274883  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:13.276288  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:13.299268  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.299340  8331 raft_consensus.cc:2410] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:03:13.277348  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 133287us
I20260709 13:03:13.274546  8324 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:03:13.227394 12330 consensus_queue.cc:1048] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:03:13.216166 12310 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.282959  8330 raft_consensus.cc:1275] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:13.305917  8399 raft_consensus.cc:2410] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:03:13.311067  8257 raft_consensus.cc:1240] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:13.318049  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:13.338716  8369 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:13.341727 12432 log.cc:927] Time spent T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.539s	user 0.003s	sys 0.000s
I20260709 13:03:13.345053  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 591) took 1058 ms. Trace:
I20260709 13:03:13.345630  8403 rpcz_store.cc:276] 0709 13:03:12.286701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.499529 (+212828us) service_pool.cc:224] Handling call
0709 13:03:12.516459 (+ 16930us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:12.517075 (+   616us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:12.517119 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:12.608944 (+ 91825us) log.cc:844] Serialized 52 byte log entry
0709 13:03:12.662957 (+ 54013us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:12.663012 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.663108 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:13.340694 (+677586us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:13.344695 (+  4001us) raft_consensus.cc:1685] finished
0709 13:03:13.344714 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.344975 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":701,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":130,"spinlock_wait_cycles":26131840,"thread_start_us":53592,"threads_started":1,"wal-append.queue_time_us":128979}
I20260709 13:03:13.350276  8298 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:13.356065  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:13.356734  9005 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:13.312470 12238 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.366027  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:13.366807  8331 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:13.373337  9001 raft_consensus.cc:1240] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:13.312407 12233 consensus_queue.cc:1048] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:03:13.379416  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:13.379799  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 75642.6us
I20260709 13:03:13.380803  8325 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 13: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:13.386752  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:13.351977 12267 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.363593 12120 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:13.394503  8328 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:13.398128  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:13.400604  8324 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:13.358518 12262 consensus_queue.cc:1059] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:03:13.415266  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:13.416185  9001 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:13.416520 12267 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:13.417820 12233 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:13.418610  8222 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:13.419823  8222 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:13.420589  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:13.420755 12247 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:13.421252  8398 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
W20260709 13:03:13.422070  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 262530us
I20260709 13:03:13.422811 12392 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:13.423260 12392 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:13.423611 12392 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:13.368727 12373 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:13.422179 12247 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:13.425969  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:13.430333  8325 raft_consensus.cc:2393] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:03:13.432663  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 120072us
I20260709 13:03:13.433274  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:13.431754  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:13.435086  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 217972us
I20260709 13:03:13.439369  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:13.439611  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:13.429122 12418 consensus_meta.cc:287] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.696s	user 0.012s	sys 0.000s
I20260709 13:03:13.441622  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:13.410720 12414 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:13.440119  8330 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:13.440138  8398 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:13.443725  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:13.450660  8402 raft_consensus.cc:2410] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:03:13.443064  8972 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:13.387670 12238 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.456511  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:13.429045 12131 raft_consensus.cc:697] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:13.464566  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:13.465044  8972 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:13.464598  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:13.468735  8406 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:13.461898  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:13.404699 12050 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:13.469686  8402 raft_consensus.cc:2393] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:13.420607 12321 consensus_queue.cc:1059] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:03:13.475760  8403 raft_consensus.cc:2393] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 13: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:03:13.479058  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.477077  8405 raft_consensus.cc:1240] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:13.480897  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:13.482244  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:13.491372  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:13.504343  8325 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:03:13.509174  8298 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:13.454952 12346 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:13.442610 12233 consensus_queue.cc:1059] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:03:13.504542  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:13.512539  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:13.515913  8403 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:13.527673  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:13.528458  9005 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:13.529383  8405 raft_consensus.cc:2410] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:03:13.511137 12312 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:13.503002 12418 consensus_queue.cc:1059] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:03:13.532770  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:13.532621 12346 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:13.534294 12443 log.cc:927] Time spent T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:03:13.536361 12383 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:13.536962 12383 raft_consensus.cc:697] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:13.538177 12190 raft_consensus.cc:3055] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:13.538017 12383 consensus_queue.cc:237] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.538573 12190 raft_consensus.cc:740] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:13.539402 12190 consensus_queue.cc:260] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:13.540848 12190 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:13.542069 12310 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:13.541895 12189 raft_consensus.cc:3055] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:13.546955  8254 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:13.513615 12452 raft_consensus.cc:3055] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:13.551934  8402 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:13.513469 12373 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:13.515527 12296 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:13.515268 12448 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:03:13.563755  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 61726.8us
I20260709 13:03:13.541375 12050 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:13.568338  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:13.569484  8402 raft_consensus.cc:2393] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 9: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:13.553222 12247 raft_consensus.cc:697] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:13.556725 12452 raft_consensus.cc:740] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:13.570828  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
W20260709 13:03:13.571053 12439 log.cc:927] Time spent T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
W20260709 13:03:13.571426  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 112756us
W20260709 13:03:13.571794 12438 log.cc:927] Time spent T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:03:13.547849 12189 raft_consensus.cc:740] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:03:13.579228 12449 log.cc:927] Time spent T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:03:13.564478 12411 consensus_queue.cc:1059] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:03:13.541925 12131 consensus_queue.cc:237] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.590493  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:13.580725 12275 raft_consensus.cc:3055] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:13.593588 12392 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.607808 12392 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:13.611279 11932 raft_consensus.cc:3055] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:13.611689 11932 raft_consensus.cc:740] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:13.611555 12368 consensus_queue.cc:1059] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:13.612591 11932 consensus_queue.cc:260] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.604609 12451 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.553911 12447 consensus_queue.cc:1059] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:03:13.610296 12262 raft_consensus.cc:3055] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:13.610745  8972 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:03:13.616861 12363 consensus_meta.cc:287] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.684s	user 0.010s	sys 0.000s
I20260709 13:03:13.623263 12346 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:13.566612 12454 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:03:13.644853  8297 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:13.645942  8297 leader_election.cc:400] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12.
I20260709 13:03:13.645807 12189 consensus_queue.cc:260] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.646216 12418 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:13.646857 12189 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:13.625697 12209 raft_consensus.cc:3055] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:13.629690  8402 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:13.631029  8295 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:13.631875  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.640028 12247 consensus_queue.cc:237] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:13.642266  8369 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:13.659651  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 48983.3us
I20260709 13:03:13.644268  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:13.646227  8297 leader_election.cc:403] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:03:13.647514  8405 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:03:13.668547  8405 pending_rounds.cc:85] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 6
I20260709 13:03:13.668875  8405 pending_rounds.cc:107] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:03:13.669163  8405 raft_consensus.cc:2889] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:13.603133 12275 raft_consensus.cc:740] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:13.630409 12238 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:13.692245  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.633333 12262 raft_consensus.cc:740] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:03:13.689010 12420 log.cc:927] Time spent T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:03:13.698717  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:13.653223 12209 raft_consensus.cc:740] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:13.689363 12312 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:13.638876 12296 consensus_queue.cc:1059] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:03:13.705940  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:13.690799 11932 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:13.715240  9006 raft_consensus.cc:2410] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 14 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:03:13.705997  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:13.704589  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:13.715927  9005 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:13.718176  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 75975.1us
I20260709 13:03:13.649019 12363 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.730484 12363 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:13.731577 12445 log.cc:927] Time spent T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 13:03:13.697973 12050 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:13.733335  9004 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:13.733409  8405 raft_consensus.cc:1240] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:13.741055  9005 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:13.648154 12451 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.705735 12411 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12.
I20260709 13:03:13.703357 12230 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:13.746932  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:13.739079 12403 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:13.746886  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:13.702994 12452 consensus_queue.cc:260] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:13.749619  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 50995.6us
I20260709 13:03:13.750054  8369 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:13.751166 12450 raft_consensus.cc:3055] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:13.751961 12262 consensus_queue.cc:260] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.802316  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:13.802942  8397 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
W20260709 13:03:13.748472  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 102659us
I20260709 13:03:13.769874  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:13.773717  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:13.778855  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:13.806892  8406 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:13.780006  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:13.762360  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:13.784739  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.790982  8248 raft_consensus.cc:1240] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:13.793942  9003 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:13.790798 12403 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.782495 12363 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:13.812637  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:03:13.813092  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 218416us
I20260709 13:03:13.814728  8295 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:13.791884 12450 raft_consensus.cc:740] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:13.808416 12418 raft_consensus.cc:697] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:13.820924  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:13.794456 12209 consensus_queue.cc:260] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.822935 12209 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:13.814095 12363 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.825712 12363 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:13.828913 12467 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:13.814250 12411 consensus_queue.cc:1059] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:03:13.830386 12411 raft_consensus.cc:3055] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:03:13.831188  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 96303.2us
I20260709 13:03:13.815043 12403 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:13.811194 12238 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:13.810881 12373 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:13.811769 12448 raft_consensus.cc:697] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:13.836083 12464 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.838174 12418 consensus_queue.cc:237] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:13.832041 12262 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:13.834546 12452 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:13.859321  8405 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:13.854180  9007 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:03:13.830734 12411 raft_consensus.cc:740] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:13.872273 12411 consensus_queue.cc:260] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.873571 12411 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:13.860170  8295 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:13.871414  9007 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:03:13.882265  9007 pending_rounds.cc:85] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e: Aborting all ops after (but not including) 7
I20260709 13:03:13.882534  9007 pending_rounds.cc:107] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:03:13.882788  9007 raft_consensus.cc:2889] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:13.893003  9007 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:03:13.849675 12468 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.888672 12363 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.904047  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 2017) took 1122 ms. Trace:
I20260709 13:03:13.849359 12450 consensus_queue.cc:260] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.904582  8399 rpcz_store.cc:276] 0709 13:03:12.782042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.858987 (+ 76945us) service_pool.cc:224] Handling call
0709 13:03:13.903833 (+1044846us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b54001dfe54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:13.903983 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":980457344}
I20260709 13:03:13.905320 12450 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:13.908799  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:13.909634  8400 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 11: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:13.910408  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:13.909582  8406 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
W20260709 13:03:13.903923  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 47628.5us
I20260709 13:03:13.847627 12373 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.925776  8297 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:13.912959  8406 pending_rounds.cc:85] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 4
I20260709 13:03:13.927106  8406 pending_rounds.cc:107] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:03:13.927507  8406 raft_consensus.cc:2889] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:13.914376 12190 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:13.929796 12190 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:13.930230 12190 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:13.931543  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:13.934407 12238 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.935050 12238 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.935532 12373 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:13.936874  8400 raft_consensus.cc:1275] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:13.938287  8400 raft_consensus.cc:1217] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[5.6-13.8]   Dedup: 5.6->[11.7-13.8]
I20260709 13:03:13.938843  8400 raft_consensus.cc:1240] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 13. Current term is 14. Ops: [5.6-13.8]
I20260709 13:03:13.939213  8401 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:03:13.939293  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 79206.5us
I20260709 13:03:13.940208  8400 raft_consensus.cc:1275] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:13.940707  8399 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:03:13.940407 12190 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.952595  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:13.952966  9006 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:13.954519  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:13.955718  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:13.956836  8369 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:13.959067  8399 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:13.960443  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.937297 12238 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:13.961643  9006 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:03:13.941578 12451 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:13.960391  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:13.981158  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:13.981964 12238 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:14.003324 12238 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:14.003813 12238 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:13.990334 12190 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:13.995540 12448 consensus_queue.cc:237] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.865389 12464 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:13.953734 12468 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:14.010241 12464 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:14.010743  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 1985) took 1163 ms. Trace:
I20260709 13:03:13.998009 12299 raft_consensus.cc:3055] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:13.998449 12237 raft_consensus.cc:3055] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:14.001631  8400 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:14.013861  8295 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:14.002183  9008 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:13.967000 12383 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:03:14.022369  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 530272us
I20260709 13:03:14.011173  8255 rpcz_store.cc:276] 0709 13:03:12.847430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.863383 (+ 15953us) service_pool.cc:224] Handling call
0709 13:03:12.875409 (+ 12026us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:12.945565 (+ 70156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:12.945599 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:12.946240 (+   641us) log.cc:844] Serialized 52 byte log entry
0709 13:03:13.135852 (+189612us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:13.429195 (+293343us) log.cc:844] Serialized 14 byte log entry
0709 13:03:13.456093 (+ 26898us) log.cc:844] Serialized 14 byte log entry
0709 13:03:14.010133 (+554040us) spinlock_profiling.cc:243] Waited 5.53 ms on lock 0x7b18002a0e80. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882564bc0a 00007f882ed8b219 00007f88256d3a40 00007f88256ce4e4 00007f88256ce455 00007f88287b6060 00007f88287c8716 00007f88287d9287 00007f88287d9208
0709 13:03:14.010302 (+   169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.010393 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:14.010533 (+   140us) raft_consensus.cc:1685] finished
0709 13:03:14.010550 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.010685 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76546,"prepare.queue_time_us":482928,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":48,"raft.queue_time_us":71762,"raft.run_cpu_time_us":377,"raft.run_wall_time_us":14538,"spinlock_wait_cycles":12721024,"thread_start_us":445489,"threads_started":2,"wal-append.queue_time_us":331865}
I20260709 13:03:14.024637  9002 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:03:14.013517 12237 raft_consensus.cc:740] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:14.014966 12299 raft_consensus.cc:740] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:14.011041 12468 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:14.037622  8295 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:03:14.040664  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.019742 12383 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:14.042519  8252 raft_consensus.cc:1240] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:14.042544  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.047806  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:14.058637  9007 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:14.034757 12467 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:14.059854  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:14.032655 12050 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:14.078753  8295 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:14.055723 12456 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.097816 12456 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:14.042593 12452 consensus_queue.cc:1059] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:03:14.067154 12418 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:14.086719 12050 raft_consensus.cc:697] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:14.087534 12363 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:14.088950  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 52572.9us
I20260709 13:03:14.093753 12209 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:03:14.093921  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 182371us
I20260709 13:03:14.094370 12247 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:14.044723 12383 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:14.099721  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 60053us
I20260709 13:03:14.106524  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:14.107189  8406 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
W20260709 13:03:14.108461  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.114503  8402 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:14.115963  8399 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:14.118713  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:14.119458  8327 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:14.119541  8297 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:14.113989 12456 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:14.118789  8223 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:14.121770  8327 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:03:14.074116 12392 consensus_queue.cc:1048] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:03:14.100723 12418 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.125136  8223 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:14.120795 12321 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.146766  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:14.118042 12209 raft_consensus.cc:697] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:14.147507  8254 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:14.147861  8248 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:14.148638  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:14.120791 12482 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:14.123802 12418 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:14.155684  8253 raft_consensus.cc:1275] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:03:14.130785 12456 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:14.159864  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:14.166366  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.160764  8255 raft_consensus.cc:3055] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:14.167440  8255 raft_consensus.cc:740] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:14.169227  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:14.124958 12467 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.130273 12481 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.169869  8397 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:03:14.132853 12468 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:14.167027  8298 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:14.174674  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:14.177567  9003 raft_consensus.cc:2410] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 10 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:03:14.177385  8325 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:14.154114 12425 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:14.179147  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:14.179617  8402 raft_consensus.cc:1240] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:03:14.175853  8255 consensus_queue.cc:260] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:14.185282  9007 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:03:14.180063  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:14.188717  8330 raft_consensus.cc:1275] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:03:14.134852 12251 consensus_queue.cc:1059] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:03:14.186208  8399 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:14.172945 12456 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.177870 12467 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:14.194718 12471 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:14.195446 12452 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:14.196044 12448 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:14.147671 12411 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:03:14.183820 12425 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.193899  8400 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
I20260709 13:03:14.187992  8220 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:14.125110 12247 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:14.176345 12330 consensus_queue.cc:1059] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:03:14.148010 12190 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:03:14.204558  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:14.153100 12479 consensus_queue.cc:1048] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:03:14.120151 12299 consensus_queue.cc:260] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:14.198915 12456 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:14.177362 12321 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:14.225759 12467 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:14.197263  8255 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:14.230957  8256 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:14.227733 12425 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:14.175520 12403 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:14.208503  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:14.247637  8222 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:14.248899  8253 raft_consensus.cc:2393] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 11: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:14.239213  8249 raft_consensus.cc:1275] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:14.239344  8250 raft_consensus.cc:1240] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12. Ops: [11.5-11.5]
I20260709 13:03:14.241639  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:14.191744 12230 consensus_queue.cc:1059] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:03:14.235899  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:14.246359  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 58455.3us
I20260709 13:03:14.246369  8295 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:03:14.238570 12477 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 13:03:14.258214  8254 raft_consensus.cc:2393] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 12: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:14.250906  9003 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:14.252668 12275 consensus_queue.cc:260] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:14.261104  8327 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 10. Ops: []
I20260709 13:03:14.183411 12481 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:14.260175  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:14.263435  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 96473.4us
I20260709 13:03:14.255219  8220 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:14.264356  8298 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:14.199826 12050 consensus_queue.cc:237] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:14.271843  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:14.272430  8251 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:14.273486  8254 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:03:14.277603  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 90240.6us
I20260709 13:03:14.254295 12209 consensus_queue.cc:237] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:14.260185 12486 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:14.292274  8220 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:14.265861 12299 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:14.261704 12467 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:14.262861 12482 raft_consensus.cc:697] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:14.287166 12263 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.371658 12263 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:14.290297 12296 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:14.371969 12482 consensus_queue.cc:237] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:14.290055 12230 raft_consensus.cc:3055] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:14.374868 12230 raft_consensus.cc:740] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:03:14.304059  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 56532.7us
I20260709 13:03:14.315112 12487 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:14.292516 12488 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:14.292778 12392 consensus_queue.cc:1048] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:03:14.270910 12373 consensus_queue.cc:1059] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
W20260709 13:03:14.330470 12483 log.cc:927] Time spent T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.106s	user 0.003s	sys 0.000s
I20260709 13:03:14.331547 12237 consensus_queue.cc:260] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:14.347023 12480 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
W20260709 13:03:14.348680  8972 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.526s (SENT) [suppressed 7 similar messages]
I20260709 13:03:14.365984  8372 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:14.366325  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:14.392656  8325 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:14.287767 12330 raft_consensus.cc:3055] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:14.374361 12403 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:14.377986  8974 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:14.278249 12275 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:14.396705  8974 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:14.397631  8974 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:14.314144  8220 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:03:14.398604  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 84554.4us
I20260709 13:03:14.399302  8220 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:14.380280  8297 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:14.381042  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:14.400314  8249 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:14.387223  8222 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:14.388784  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 102311us
I20260709 13:03:14.374405 12251 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:14.374361 12247 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:14.391007 12488 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.425568  8298 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:14.395360 12330 raft_consensus.cc:740] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:14.375864 12230 consensus_queue.cc:260] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:14.405972 12345 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:14.410972 12312 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:14.418545 12448 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:14.405412 12452 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:03:14.478528  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.409886 12492 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.414796 12251 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:14.395334 12418 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:14.414147 12450 consensus_queue.cc:1175] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e64761f43e4240319c41117cf886d22f" responder_term: 9 status { last_received { term: 7 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9. Ops: [8.8-8.8]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:03:14.419304 12363 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:03:14.488946  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:14.488380  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 101235us
I20260709 13:03:14.565371  8325 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
W20260709 13:03:14.753690  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:14.481822  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 115922us
I20260709 13:03:14.589661  8397 raft_consensus.cc:1240] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:14.755363  8222 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:14.423395 12471 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:03:14.762698  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:14.763764  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:14.751250  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:14.552762  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 134909us
I20260709 13:03:14.766827  8222 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:14.458997  8249 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:14.446396 12247 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:14.428596 12487 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:14.588636  9004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 860) took 1313 ms. Trace:
W20260709 13:03:14.667286  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 268017us
W20260709 13:03:14.600126 12472 log.cc:927] Time spent T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.722s	user 0.002s	sys 0.000s
W20260709 13:03:14.777716  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.799048  8220 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:14.590857 12457 raft_consensus.cc:3055] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:14.745990 12448 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.481668 12447 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.810781 12502 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.811335 12502 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:14.811154 12447 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:14.806241  8372 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:14.549701 12403 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:14.551895 12468 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:14.553844 12481 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:14.584741 12120 raft_consensus.cc:3055] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:14.872850 12120 raft_consensus.cc:740] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:14.466264 12312 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.800619  9004 rpcz_store.cc:276] 0709 13:03:13.275178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.275319 (+   141us) service_pool.cc:224] Handling call
0709 13:03:13.300876 (+ 25557us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.740418 (+439542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.588284 (+847866us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b54005983d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:14.588524 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":994076544}
I20260709 13:03:14.446262 12488 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:14.752857 12251 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:14.459947 12190 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.349s
W20260709 13:03:14.817201 12456 consensus_meta.cc:287] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.559s	user 0.008s	sys 0.002s
I20260709 13:03:14.810070 12469 consensus_queue.cc:1059] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:03:14.583178 12413 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.479733 12345 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:14.607715 12491 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:14.762135 12492 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:14.807060 12232 consensus_queue.cc:1048] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.128s
I20260709 13:03:14.601965 12330 consensus_queue.cc:260] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:14.858718 12131 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:14.862154 12238 consensus_meta.cc:287] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.858s	user 0.012s	sys 0.000s
I20260709 13:03:14.869614 12403 raft_consensus.cc:697] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:14.870332 12468 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:14.872200 12447 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:14.762796 12392 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:14.878731  8255 consensus_meta.cc:287] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.634s	user 0.005s	sys 0.006s
I20260709 13:03:14.758963 12321 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:14.882149  8220 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:15.101619  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 219566us
I20260709 13:03:15.103300  8220 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:14.882261  8222 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:15.104527  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 222433us
I20260709 13:03:14.763209 12471 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.576408 12050 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.241s
I20260709 13:03:14.809473 12448 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:14.583420 12490 consensus_queue.cc:1059] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.428s
W20260709 13:03:14.884148  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 115490us
I20260709 13:03:14.942184  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:15.060738 12373 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.010s	sys 0.067s
I20260709 13:03:14.804677 12457 raft_consensus.cc:740] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:14.762904 12452 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:15.054613  8369 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.004s	sys 0.066s
W20260709 13:03:15.052114  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 115358us
I20260709 13:03:14.488179 12479 consensus_queue.cc:1059] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.331s
W20260709 13:03:15.073441  8399 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.042s	sys 0.046s
I20260709 13:03:14.884974 12312 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:14.873903 12502 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:15.054456 12345 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:15.108318 12500 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:15.109643 12468 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:15.096740 12238 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.055181 12413 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.140565 12503 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:14.882740 12247 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:15.117417  8255 raft_consensus.cc:1275] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:15.195438  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 602) took 1050 ms. Trace:
I20260709 13:03:14.948894 12456 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.195843  8255 rpcz_store.cc:276] 0709 13:03:14.145215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.145348 (+   133us) service_pool.cc:224] Handling call
0709 13:03:14.160660 (+ 15312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.195137 (+1034477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.195375 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.127647  8220 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:15.069340 12481 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.116s
I20260709 13:03:15.137115  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 561) took 1014 ms. Trace:
I20260709 13:03:15.148623  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2417) took 1515 ms. Trace:
I20260709 13:03:15.199064  8327 rpcz_store.cc:276] 0709 13:03:14.123126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.123506 (+   380us) service_pool.cc:224] Handling call
0709 13:03:14.124081 (+   575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:14.262364 (+138283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.136764 (+874400us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b540037d054. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:15.137028 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":314702592}
I20260709 13:03:14.819967 12411 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.308s
I20260709 13:03:15.106000 12447 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:15.116909 12471 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:15.154403  8369 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.004s	sys 0.067s
W20260709 13:03:15.212383  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 907998us
W20260709 13:03:15.157423  8399 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.042s	sys 0.046s
I20260709 13:03:15.162637 12513 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:15.085270 12487 consensus_queue.cc:1059] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:03:15.116581 12469 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:15.145951 12373 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.011s	sys 0.068s
I20260709 13:03:15.121892 12321 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:15.203227 12500 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:15.186234  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:15.243345  9005 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:15.186234  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:15.198110 12490 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.134054 12363 raft_consensus.cc:697] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:15.095865 12131 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.109575  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:14.950327  8972 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.059s	sys 0.108s
W20260709 13:03:15.248449  8972 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.856s	user 0.059s	sys 0.108s
W20260709 13:03:15.197497  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 92152.4us
I20260709 13:03:15.152865  8402 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:15.149466 12505 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:03:15.199216  8398 rpcz_store.cc:276] 0709 13:03:13.633681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.634254 (+   573us) service_pool.cc:224] Handling call
0709 13:03:13.686928 (+ 52674us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:13.697228 (+ 10300us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:13.697273 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:13.723661 (+ 26388us) log.cc:844] Serialized 52 byte log entry
0709 13:03:13.831252 (+107591us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:14.033223 (+201971us) log.cc:844] Serialized 14 byte log entry
0709 13:03:14.040415 (+  7192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.040491 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:14.599285 (+558794us) spinlock_profiling.cc:243] Waited 1.74 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:15.124164 (+524879us) raft_consensus.cc:1685] finished
0709 13:03:15.148295 (+ 24131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.148537 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4071,"prepare.queue_time_us":241715,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":2820,"raft.queue_time_us":6398,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":8990,"spinlock_wait_cycles":3992832,"thread_start_us":265439,"threads_started":2,"wal-append.queue_time_us":134028}
I20260709 13:03:15.106455 12120 consensus_queue.cc:260] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:15.200497 12275 consensus_meta.cc:287] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.796s	user 0.006s	sys 0.004s
I20260709 13:03:15.153774  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:15.152870  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.213955  9004 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:03:15.268816  9009 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:15.266028  8222 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:15.197091 12247 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:15.197794 12520 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:03:15.240615  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.234088 12392 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:03:15.192135  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.245633  9001 raft_consensus.cc:2393] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 11: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:15.200654 12468 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:15.248489  8401 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
W20260709 13:03:15.249351  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 900584us
W20260709 13:03:15.280179  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:15.201293  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 73779.6us
I20260709 13:03:15.201201 12418 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:15.250001 12373 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:03:15.273938  8222 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:03:15.274622  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.274454  9005 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:15.285382  8257 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:03:15.215590 12513 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:15.281059  8298 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:15.242857 12411 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:15.204268 12503 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
W20260709 13:03:15.283601 12425 consensus_meta.cc:287] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.027s	user 0.013s	sys 0.000s
I20260709 13:03:15.214164 12447 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.275493 12247 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:15.331534  8295 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:03:15.330015  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 56164.5us
I20260709 13:03:15.333410  8222 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:15.278964 12467 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:15.320701  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.293749  8220 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:15.333076  8295 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:03:15.365842  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.255730 12490 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:15.370188 12363 consensus_queue.cc:237] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.275285 12232 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:03:15.294677 12120 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:15.387480  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 54235us
I20260709 13:03:15.301797 12513 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:15.243124 12403 consensus_queue.cc:237] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:15.393258  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 99589.3us
I20260709 13:03:15.276754 12500 consensus_queue.cc:1059] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.111s
I20260709 13:03:15.350451 12418 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:15.375241 12275 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:15.292256 12411 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:15.393925  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 60963.9us
I20260709 13:03:15.459226  8295 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:03:15.392298 12299 consensus_meta.cc:287] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 1.019s	user 0.011s	sys 0.001s
W20260709 13:03:15.432847  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 174829us
I20260709 13:03:15.464107  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:15.288897 12468 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:15.432864 12479 consensus_queue.cc:1243] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:03:15.349642 12425 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.452936  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2045) took 1073 ms. Trace:
I20260709 13:03:15.284246 12520 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.384363 12467 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:15.432428 12492 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:15.460289  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 68652.8us
I20260709 13:03:15.465088  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2043) took 1185 ms. Trace:
W20260709 13:03:15.466209  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.466135  8369 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:15.396256 12392 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:03:15.468238 12484 log.cc:927] Time spent T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 1.076s	user 0.000s	sys 0.012s
I20260709 13:03:15.475234 12296 consensus_queue.cc:1059] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:03:15.484210 12263 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:15.486192  8369 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:15.487370 12505 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:15.488052 12505 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:15.488543 12505 raft_consensus.cc:697] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 9 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:15.468601 12511 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:15.489302 12531 consensus_queue.cc:1048] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:15.489871 12511 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:15.490339 12511 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:15.490800 12425 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:15.491955  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.492357  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:15.469019 12521 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:15.493713 12521 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:15.494143 12521 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:15.494232  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:15.494889  8327 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:03:15.497071  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 617) took 1117 ms. Trace:
I20260709 13:03:15.497413 12237 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:15.497756  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:15.469540 12510 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:15.500236 12503 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:15.469981 12524 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:15.472848  8399 rpcz_store.cc:276] 0709 13:03:14.379244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.379369 (+   125us) service_pool.cc:224] Handling call
0709 13:03:14.400883 (+ 21514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:15.213203 (+812320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.213233 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:15.213743 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 13:03:15.329259 (+115516us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:15.329312 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.329410 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:15.452712 (+123302us) raft_consensus.cc:1685] finished
0709 13:03:15.452733 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.452872 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":181179,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":125,"thread_start_us":910619,"threads_started":2,"wal-append.queue_time_us":220802}
W20260709 13:03:15.455745  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.398077 12469 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 13:03:15.458094 12275 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.987119  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:15.501731  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:15.517200  9008 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:15.508252  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:15.497460  8405 rpcz_store.cc:276] 0709 13:03:14.379575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.379700 (+   125us) service_pool.cc:224] Handling call
0709 13:03:14.400970 (+ 21270us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:14.487907 (+ 86937us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:14.487938 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:14.565907 (+ 77969us) log.cc:844] Serialized 26 byte log entry
0709 13:03:15.088856 (+522949us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:15.088914 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.089010 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:15.496872 (+407862us) raft_consensus.cc:1685] finished
0709 13:03:15.496892 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.497014 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":181406,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":117,"thread_start_us":391189,"threads_started":2,"wal-append.queue_time_us":682477}
I20260709 13:03:15.476751  8400 rpcz_store.cc:276] 0709 13:03:14.279278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.279423 (+   145us) service_pool.cc:224] Handling call
0709 13:03:14.279825 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:14.431804 (+151979us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:14.431853 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:14.449007 (+ 17154us) log.cc:844] Serialized 26 byte log entry
0709 13:03:14.926987 (+477980us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:14.927042 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.927136 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:15.464875 (+537739us) raft_consensus.cc:1685] finished
0709 13:03:15.464894 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.465030 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":259984,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":13819,"thread_start_us":509674,"threads_started":2,"wal-append.queue_time_us":903195}
I20260709 13:03:15.471272 12209 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:03:15.521736  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 62611us
I20260709 13:03:15.521991  8330 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:03:15.521922  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:15.522756  8295 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:15.516687  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:15.455369 12452 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:15.523507  8332 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:15.524900  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:15.526235  8327 raft_consensus.cc:2393] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 13: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:15.526115  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:15.526500  8325 raft_consensus.cc:2393] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 11: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:15.528654  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:15.529681  9002 raft_consensus.cc:2410] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:03:15.526799  8326 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:15.532645 12411 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:15.471594 12418 consensus_queue.cc:1048] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:03:15.585626  8332 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:15.518461 12531 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:15.521600 12275 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:15.523149 12456 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:15.525861 12209 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:03:15.600268  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 147280us
I20260709 13:03:15.521512 12450 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.528620 12413 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:15.606910  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:15.593850 12533 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:03:15.534957 12528 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:15.550509 12479 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:15.534409 12524 raft_consensus.cc:697] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:15.582463 12411 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:15.658649 12486 consensus_meta.cc:287] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.269s	user 0.012s	sys 0.000s
I20260709 13:03:15.659106 12486 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:15.662724 12523 log.cc:927] Time spent T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
I20260709 13:03:15.664458 12521 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:15.666174 12530 log.cc:927] Time spent T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:03:15.611794 12383 consensus_meta.cc:287] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.494s	user 0.009s	sys 0.005s
I20260709 13:03:15.502156 12505 consensus_queue.cc:237] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.600229 12447 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:15.666054 12238 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:15.599220  8369 connection.cc:256] RPC call timeout handler was delayed by -1.1702454900000134s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1879s for a response
I20260709 13:03:15.579831 12251 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:15.669481  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.613722 12209 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:15.677328  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 199915us
I20260709 13:03:15.607442 12531 raft_consensus.cc:697] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:15.626055 12526 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:15.676371  8369 connection.cc:256] RPC call timeout handler was delayed by -1.1648454900000047s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1885s for a response
W20260709 13:03:15.678403  8369 connection.cc:256] RPC call timeout handler was delayed by -1.1351454900000135s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1918s for a response
W20260709 13:03:15.679702  8369 connection.cc:256] RPC call timeout handler was delayed by -1.1252454899999975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19290000000000002s for a response
W20260709 13:03:15.682117  8369 connection.cc:256] RPC call timeout handler was delayed by -1.1234454899999946s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19310000000000002s for a response
W20260709 13:03:15.682471  8369 connection.cc:256] RPC call timeout handler was delayed by -1.069445490000021s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19910000000000003s for a response
I20260709 13:03:15.580224 12510 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.092s
W20260709 13:03:15.684633  8406 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.000s	sys 0.012s
I20260709 13:03:15.667918 12131 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:15.667586 12502 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:15.678900 12518 log.cc:927] Time spent T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.289s	user 0.001s	sys 0.000s
I20260709 13:03:15.686159  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:15.685781  8406 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.000s	sys 0.012s
I20260709 13:03:15.697626  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:15.698385  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:15.698570  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:15.699487  8332 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 13:03:15.699831  8332 pending_rounds.cc:85] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 7
I20260709 13:03:15.700037  8332 pending_rounds.cc:107] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:03:15.700300  8332 raft_consensus.cc:2889] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:15.629662 12450 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.676875 12479 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.708074 12319 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.708959 12251 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:15.709609 12251 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.710039 12251 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.708771 12319 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:15.715545  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 185822us
I20260709 13:03:15.707058  8329 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
W20260709 13:03:15.717768  8257 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49220 (request call id 606) took 1566 ms (client timeout 1964 ms). Trace:
I20260709 13:03:15.684209 12392 raft_consensus.cc:697] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:03:15.718200  8257 rpcz_store.cc:269] 0709 13:03:14.146887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.148345 (+  1458us) service_pool.cc:224] Handling call
0709 13:03:15.712879 (+1564534us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400177854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:15.712980 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2597880576}
I20260709 13:03:15.720515  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:15.676167 12411 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:15.678364 12383 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:15.732272  8369 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:15.714921 12528 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.735636  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:15.744817  8331 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:03:15.703574 12502 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:15.771345  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:15.772234  8397 raft_consensus.cc:2393] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 10: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:15.729878 12533 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:15.726429 12418 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:03:15.776113  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 47857.4us
I20260709 13:03:15.734280 12490 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:15.725683 12263 consensus_queue.cc:1059] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:03:15.744740 12238 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:15.749725 12251 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:15.780427  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:15.783939  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:15.686264 12526 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.776937 12533 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:15.781869  8403 raft_consensus.cc:1240] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:15.786942  8398 raft_consensus.cc:1240] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:15.787145  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:15.804275  8402 raft_consensus.cc:2410] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:03:15.727775 12535 consensus_queue.cc:1059] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:03:15.792309 12539 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.792388 12520 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:15.720085 12479 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.792363 12545 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.785568  8404 raft_consensus.cc:1275] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 12 index: 5. (index mismatch)
I20260709 13:03:15.780864 12413 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.804281 12392 consensus_queue.cc:237] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:15.830204 12413 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.830823  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 616) took 1564 ms. Trace:
I20260709 13:03:15.831234  8406 rpcz_store.cc:276] 0709 13:03:14.266517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.266710 (+   193us) service_pool.cc:224] Handling call
0709 13:03:14.267155 (+   445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:14.269899 (+  2744us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:14.304160 (+ 34261us) log.cc:844] Serialized 14 byte log entry
0709 13:03:15.659099 (+1354939us) spinlock_profiling.cc:243] Waited 58.8 ms on lock 0x7b18002f7ac0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:03:15.695594 (+ 36495us) log.cc:844] Serialized 14 byte log entry
0709 13:03:15.695807 (+   213us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:15.704595 (+  8788us) log.cc:844] Serialized 26 byte log entry
0709 13:03:15.704871 (+   276us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:15.704935 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.705139 (+   204us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:15.830614 (+125475us) raft_consensus.cc:1685] finished
0709 13:03:15.830634 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.830766 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54111,"prepare.queue_time_us":611089,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":74,"raft.queue_time_us":526959,"raft.run_cpu_time_us":460,"raft.run_wall_time_us":65254,"spinlock_wait_cycles":135255424,"thread_start_us":1325656,"threads_started":3,"wal-append.queue_time_us":165280}
I20260709 13:03:15.832751 12413 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:15.750535 12468 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:15.833937 12498 log.cc:927] Time spent T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.848s	user 0.004s	sys 0.000s
W20260709 13:03:15.834127 12544 log.cc:927] Time spent T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:03:15.834712 12519 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.835232 12468 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:15.835404 12534 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.835801 12468 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:15.836203 12534 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.835235 12519 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.804796  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:15.840014 12482 raft_consensus.cc:3055] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:15.840548 12482 raft_consensus.cc:740] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:15.781200 12299 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.841600 12482 consensus_queue.cc:260] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:15.803246 12490 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.807394 12533 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:15.795121 12383 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:15.826306  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:15.788956 12467 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:15.841699 12299 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.770428 12131 consensus_queue.cc:1048] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:03:15.793286 12502 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:15.846508  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 47751.7us
I20260709 13:03:15.935834  8369 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:15.893826  8399 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
W20260709 13:03:15.835750 12536 log.cc:927] Time spent T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 13:03:15.788435 12418 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.902350 12319 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:15.902968 12526 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:15.735122 12524 consensus_queue.cc:237] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.805120 12233 consensus_queue.cc:1059] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
W20260709 13:03:15.912500 12488 consensus_meta.cc:287] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.855s	user 0.005s	sys 0.006s
W20260709 13:03:15.916527 12345 consensus_meta.cc:287] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.721s	user 0.006s	sys 0.002s
I20260709 13:03:15.749552 12528 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.789106 12542 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.168s
I20260709 13:03:15.929678 12490 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:15.972311 12312 consensus_meta.cc:287] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.776s	user 0.008s	sys 0.001s
I20260709 13:03:15.972862 12050 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:15.973459 12050 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.973992 12411 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:15.771189 12531 consensus_queue.cc:237] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.934873 12521 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:15.931293 12534 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:15.821925 12517 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:15.937836  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 115457us
I20260709 13:03:15.937084  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:15.980753  8404 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:15.947535  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:15.900933 12471 consensus_meta.cc:287] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.658s	user 0.008s	sys 0.002s
I20260709 13:03:15.946498 12383 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:15.924616 12511 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.929188 12513 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.963487 12488 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:15.968289 12533 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.969697 12495 raft_consensus.cc:3055] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:15.969390 12345 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:16.011765 12515 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.012293 12515 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.946275 12209 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:16.005347 12517 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:16.014590 12488 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:16.014933 12209 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:16.015432 12517 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:15.976279  8974 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:15.977169  8298 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:16.016693 12533 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:16.018042 12120 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:03:16.018496 12522 log.cc:927] Time spent T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.702s	user 0.000s	sys 0.004s
I20260709 13:03:16.018592 12120 raft_consensus.cc:697] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:16.013244 12513 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:16.019608 12120 consensus_queue.cc:237] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:16.023284 12467 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:16.023774  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:16.024879  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:15.989593 12534 consensus_queue.cc:1048] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:15.983058 12457 consensus_queue.cc:260] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:15.992066 12418 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:15.891496 12545 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.994143 12275 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:15.972748 12312 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:15.975669 12050 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:15.995702  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 48244.9us
I20260709 13:03:16.033030 12521 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.042256 12521 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:16.042852  8369 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:15.996304 12514 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:15.818998 12539 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:16.009984 12495 raft_consensus.cc:740] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:15.982671  8222 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:16.052673 12299 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:16.005725 12383 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
W20260709 13:03:16.056318  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 73719.4us
W20260709 13:03:16.028647  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.062428  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:16.063066  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:16.063859  8255 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
I20260709 13:03:16.065559  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:16.066226  8249 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:03:16.066478  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:16.067114  8253 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
W20260709 13:03:16.029336 12491 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.920s	user 0.016s	sys 0.012s
I20260709 13:03:16.013238 12345 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:16.026648 12457 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:15.997143 12471 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:15.977739 12542 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.976688 12479 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:16.031802  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 55600.6us
I20260709 13:03:16.084753  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:16.023186 12513 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.044620  8369 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:15.996728 12548 consensus_queue.cc:1059] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.083s
I20260709 13:03:16.048512 12514 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:16.050029 12539 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:16.050328 12275 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:16.046685 12050 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:16.050796  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:16.039222 12533 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:16.082016  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:16.080426 12519 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:16.148880  8253 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:03:16.145361  8404 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:16.115343  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:16.044973 12521 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:16.097879 12502 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:16.098160 12383 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:16.143452 12448 consensus_meta.cc:287] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.990s	user 0.010s	sys 0.001s
I20260709 13:03:16.085644  8250 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
W20260709 13:03:16.095724  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:16.073899 12491 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.969s	user 0.016s	sys 0.012s
W20260709 13:03:16.100021  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 121785us
W20260709 13:03:16.101287  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 56721.8us
I20260709 13:03:16.068943 12535 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:16.106618  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:16.182806  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:16.183046 12535 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.183374  8249 raft_consensus.cc:2393] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 11: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:16.183557  8250 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:16.156915 12533 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.169189  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:16.090022  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:16.171582  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:16.189158  8254 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:16.172258  9008 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:03:16.141999 12471 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:16.145648 12330 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:16.178558  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:16.212675  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.151098  9004 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:03:16.139978 12545 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:16.085517 12450 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:16.215752  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:16.148880 12557 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:16.170189 12490 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:16.254274  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:03:16.258976  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 80491.5us
I20260709 13:03:16.260412  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:16.268667  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:16.269209  8249 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:03:16.269551  8406 raft_consensus.cc:2393] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 10: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:16.183574 12535 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:16.177619 12491 raft_consensus.cc:697] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:16.273643  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:16.278944  8399 raft_consensus.cc:2410] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:03:16.275735  8369 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:16.277618  8250 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:16.278180  8398 raft_consensus.cc:2393] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 11: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:16.195856 12533 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:16.098297 12542 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:16.186645 12448 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:16.188835 12495 consensus_queue.cc:260] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:16.102980 12513 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:16.316763  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:16.310496  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:16.880225  8249 raft_consensus.cc:2393] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 13: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:16.880173  8403 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:16.881292  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:16.625938  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.291482  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:16.439896  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:16.688098  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.304986  8254 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:16.531996  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:16.891656  8406 raft_consensus.cc:2410] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:03:16.255375 12479 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.888882  8252 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:16.896927  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:16.903126  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:16.904731  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:16.441531 12392 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.020s	sys 0.022s
I20260709 13:03:16.905421  8406 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
W20260709 13:03:16.877923 12564 log.cc:927] Time spent T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.624s	user 0.001s	sys 0.000s
I20260709 13:03:16.910410  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:16.915871  8252 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:16.910451  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:16.918497  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:16.912248  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:16.912650  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:16.897409 12120 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.866s	user 0.005s	sys 0.026s
I20260709 13:03:16.269950 12050 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:16.914153  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:17.016037  8401 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:16.907372 12209 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:03:16.937695 12120 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.911s	user 0.006s	sys 0.026s
W20260709 13:03:16.917754  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 642023us
I20260709 13:03:16.920305 12495 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:16.305935 12557 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:03:16.900501 12247 consensus_meta.cc:287] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.522s	user 0.004s	sys 0.009s
I20260709 13:03:16.543444 12511 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:16.913956 12392 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.981s	user 0.027s	sys 0.033s
I20260709 13:03:17.007647  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:16.641206 12312 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:17.008529  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:16.917734  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:17.017084  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:16.912093  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:16.919262  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:17.024390  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:17.026708  8369 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:17.092839  8255 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.039s	sys 0.054s
I20260709 13:03:17.093511  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:17.110738  8249 raft_consensus.cc:2393] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 10: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:17.111068  8401 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:17.111124  8406 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:03:16.906011 12479 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:16.662133 12491 consensus_queue.cc:237] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:17.111644  9008 raft_consensus.cc:2393] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 12: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
W20260709 13:03:17.112035  8255 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.039s	sys 0.054s
I20260709 13:03:17.112259  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:17.115844  9003 raft_consensus.cc:2393] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 13: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:17.119266  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:17.112989  8406 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:03:17.125061  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:17.127066  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 2442) took 1055 ms. Trace:
I20260709 13:03:17.126535  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:17.127727  8406 rpcz_store.cc:276] 0709 13:03:16.072155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.124800 (+1052645us) service_pool.cc:224] Handling call
0709 13:03:17.127003 (+  2203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:17.098296 12209 raft_consensus.cc:697] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:17.129302  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:17.132121  8406 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:17.104972 12557 raft_consensus.cc:697] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:17.134063  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:17.094558 12247 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:17.122750 12312 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:17.183440 12535 consensus_meta.cc:287] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.878s	user 0.008s	sys 0.004s
W20260709 13:03:17.298810  8330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 567) took 1553 ms (client timeout 1526 ms). Trace:
I20260709 13:03:17.127064 12450 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:17.441121  8406 raft_consensus.cc:2410] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:03:17.442691  8401 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
W20260709 13:03:17.535470  8220 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.007s	sys 0.014s
I20260709 13:03:17.535586  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 2445) took 1459 ms. Trace:
W20260709 13:03:17.536031  8220 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.007s	sys 0.014s
W20260709 13:03:17.536538  8222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.007s	sys 0.015s
I20260709 13:03:17.535583  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 2443) took 1463 ms. Trace:
W20260709 13:03:17.536974  8222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.007s	sys 0.015s
W20260709 13:03:17.538388  8296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.010s	sys 0.011s
W20260709 13:03:17.538893  8296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.011s	sys 0.011s
I20260709 13:03:17.536170  8406 rpcz_store.cc:276] 0709 13:03:16.076267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.129076 (+1052809us) service_pool.cc:224] Handling call
0709 13:03:17.535518 (+406442us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:17.541042  8251 raft_consensus.cc:1240] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:17.541302  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:17.541901  8406 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:17.498296  8403 consensus_meta.cc:287] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.617s	user 0.009s	sys 0.004s
W20260709 13:03:17.516352 12573 log.cc:927] Time spent T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.390s	user 0.001s	sys 0.008s
I20260709 13:03:17.450698 12243 raft_consensus.cc:3055] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:17.441529 12545 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:17.598568  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 571784us
W20260709 13:03:17.531777  8330 rpcz_store.cc:269] 0709 13:03:15.741527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.294374 (+1552847us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:17.294557 (+   183us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:17.536998  8401 rpcz_store.cc:276] 0709 13:03:16.072515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.024074 (+951559us) service_pool.cc:224] Handling call
0709 13:03:17.535505 (+511431us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:17.581262  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:17.600047  8249 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
W20260709 13:03:17.543077  8972 leader_election.cc:336] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.526s (SENT)
I20260709 13:03:17.600919  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:17.500456 12330 consensus_meta.cc:287] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.275s	user 0.011s	sys 0.007s
W20260709 13:03:17.571210  8974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.007s	sys 0.015s
W20260709 13:03:17.602105  8974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.007s	sys 0.015s
I20260709 13:03:17.552168  8403 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
W20260709 13:03:17.604091  8403 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 2449) took 1504 ms (client timeout 1731 ms). Trace:
W20260709 13:03:17.604492  8403 rpcz_store.cc:269] 0709 13:03:16.099518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.304231 (+204713us) service_pool.cc:224] Handling call
0709 13:03:17.604025 (+1299794us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:17.496665 12392 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:17.594954  9004 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.017s	sys 0.008s
W20260709 13:03:17.536021 12457 consensus_meta.cc:287] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.441s	user 0.013s	sys 0.010s
I20260709 13:03:17.583850  8406 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:17.128173 12050 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:17.608786  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:17.584771 12542 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:17.605732  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:17.612669  8402 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:03:17.613188  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 627) took 1349 ms. Trace:
I20260709 13:03:17.593660 12448 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:17.610252  9004 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.017s	sys 0.008s
I20260709 13:03:17.541108 12312 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:17.610515  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 67472.9us
I20260709 13:03:17.611720  8248 raft_consensus.cc:1275] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:03:17.613627  8973 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.024s	sys 0.079s
W20260709 13:03:17.620443  8973 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.025s	sys 0.080s
W20260709 13:03:17.606818  8253 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.006s	sys 0.017s
W20260709 13:03:17.622459  8253 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.007s	sys 0.017s
I20260709 13:03:17.592160 12527 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:03:17.613194  8330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2414) took 1593 ms (client timeout 1566 ms). Trace:
W20260709 13:03:17.634542  8330 rpcz_store.cc:269] 0709 13:03:16.019901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.612982 (+1593081us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:17.613122 (+   140us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:17.609783  8401 raft_consensus.cc:2410] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 10 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:03:17.613550  8402 rpcz_store.cc:276] 0709 13:03:16.263853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.589849 (+1325996us) service_pool.cc:224] Handling call
0709 13:03:17.613138 (+ 23289us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:17.615161  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:17.611604 12263 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.606685  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:17.637077  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 2060) took 1533 ms. Trace:
I20260709 13:03:17.638756  8401 rpcz_store.cc:276] 0709 13:03:16.103847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.608510 (+1504663us) service_pool.cc:224] Handling call
0709 13:03:17.637013 (+ 28503us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:17.634965  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.641007  8251 raft_consensus.cc:2393] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 11: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:17.636379  8406 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:03:17.617748 12542 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:17.598276 12243 raft_consensus.cc:740] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:17.615008 12575 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.645272  8401 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:17.538141 12535 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:17.619486  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:17.630996  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:17.648857  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:17.649860  8251 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 11: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:17.632678  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:17.616186  8975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.004s	sys 0.019s
W20260709 13:03:17.650981  8975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.004s	sys 0.020s
I20260709 13:03:17.612031 12050 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:03:17.642979  8330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 573) took 1659 ms (client timeout 1618 ms). Trace:
I20260709 13:03:17.652660  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:03:17.652863  8330 rpcz_store.cc:269] 0709 13:03:15.978296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.637596 (+1659300us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:17.637695 (+    99us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:17.653461  8402 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:17.653970  8330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 574) took 1674 ms (client timeout 1623 ms). Trace:
W20260709 13:03:17.654429  8330 rpcz_store.cc:269] 0709 13:03:15.979948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.653769 (+1673821us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:17.653901 (+   132us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:17.617223 12513 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:17.647207  8369 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:17.663129  8369 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:17.665176  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 621) took 1622 ms. Trace:
I20260709 13:03:17.665844  8401 rpcz_store.cc:276] 0709 13:03:16.023342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.639594 (+1616252us) service_pool.cc:224] Handling call
0709 13:03:17.644870 (+  5276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:17.645774 (+   904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.646017 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:17.609772 12545 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:17.657938 12570 log.cc:927] Time spent T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
W20260709 13:03:17.658164 12565 log.cc:927] Time spent T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:03:17.658587 12495 consensus_meta.cc:287] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.616s	user 0.010s	sys 0.001s
I20260709 13:03:17.658914  9001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 748) took 1573 ms. Trace:
I20260709 13:03:17.659948 12243 consensus_queue.cc:260] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:17.667845  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:17.584664 12450 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:17.636085 12457 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.702316 12247 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:17.637336 12527 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:17.661648 12050 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:17.700408  8330 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:03:17.698287  9001 rpcz_store.cc:276] 0709 13:03:16.085542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.086091 (+   549us) service_pool.cc:224] Handling call
0709 13:03:16.105866 (+ 19775us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:16.106267 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:16.162120 (+ 55853us) log.cc:844] Serialized 14 byte log entry
0709 13:03:16.494177 (+332057us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:16.494887 (+   710us) log.cc:844] Serialized 26 byte log entry
0709 13:03:16.495043 (+   156us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:16.495088 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.495177 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:17.658708 (+1163531us) raft_consensus.cc:1685] finished
0709 13:03:17.658729 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:17.658858 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22893,"prepare.queue_time_us":55866,"raft.queue_time_us":8793,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":50024,"spinlock_wait_cycles":14474624,"thread_start_us":277860,"threads_started":1,"wal-append.queue_time_us":840246}
W20260709 13:03:17.704912  8330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2408) took 1744 ms (client timeout 1712 ms). Trace:
I20260709 13:03:17.655786 12448 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.674758  8401 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:17.661161 12583 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:17.678946  8257 raft_consensus.cc:2393] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 11: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:17.679869 12479 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:17.680035 12535 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:17.681918  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:17.710527  8252 raft_consensus.cc:2393] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 13: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:17.682834  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:17.643213 12209 consensus_queue.cc:237] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:17.615646 12392 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:17.686120  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:17.686585 12584 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.667587 12575 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:17.701565 12495 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:17.637012 12581 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:17.703392  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 82424.1us
W20260709 13:03:17.670612 12568 log.cc:927] Time spent T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:03:17.673519 12513 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:03:17.705641  8330 rpcz_store.cc:269] 0709 13:03:15.960173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.655254 (+1695081us) service_pool.cc:224] Handling call
0709 13:03:17.704853 (+ 49599us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:17.707185  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 44145us
W20260709 13:03:17.724977  8330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 578) took 1529 ms (client timeout 1519 ms). Trace:
I20260709 13:03:17.725175  8369 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:17.725358  8330 rpcz_store.cc:269] 0709 13:03:16.195556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.724811 (+1529255us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:17.724923 (+   112us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:17.707588  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 624) took 1683 ms. Trace:
I20260709 13:03:17.643985 12263 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:17.726161  8401 rpcz_store.cc:276] 0709 13:03:16.024040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.667202 (+1643162us) service_pool.cc:224] Handling call
0709 13:03:17.674326 (+  7124us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:17.707269 (+ 32943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.707509 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:17.709065  8295 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.084s	sys 0.102s
I20260709 13:03:17.711182  8249 raft_consensus.cc:2393] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 13: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:17.712157  8402 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
W20260709 13:03:17.728986  8295 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.084s	sys 0.102s
I20260709 13:03:17.730975  8369 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:17.713675  8221 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.011s	sys 0.026s
I20260709 13:03:17.716949  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:17.717213  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:17.737727  8221 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.011s	sys 0.026s
I20260709 13:03:17.718235  8223 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:17.610321  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:03:17.738503  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.704474 12457 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:17.739326  8403 raft_consensus.cc:2410] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:03:17.741365  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 2051) took 1716 ms. Trace:
I20260709 13:03:17.741879  8403 rpcz_store.cc:276] 0709 13:03:16.024998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.609944 (+1584946us) service_pool.cc:224] Handling call
0709 13:03:17.741297 (+131353us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:17.743319  8403 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:17.744092  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2057) took 1687 ms. Trace:
I20260709 13:03:17.744529  8403 rpcz_store.cc:276] 0709 13:03:16.056914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.742747 (+1685833us) service_pool.cc:224] Handling call
0709 13:03:17.743105 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:17.743829 (+   724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.744033 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:17.713166 12247 consensus_queue.cc:1059] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 13:03:17.716602 12579 consensus_queue.cc:1059] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:03:17.715406 12479 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.709682 12583 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:17.709270 12448 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:18.137955  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 611) took 2073 ms. Trace:
I20260709 13:03:17.726783 12581 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:18.090277 12232 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.160229  8255 rpcz_store.cc:276] 0709 13:03:16.064814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.064940 (+   126us) service_pool.cc:224] Handling call
0709 13:03:16.086089 (+ 21149us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:16.099265 (+ 13176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.099310 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:16.099885 (+   575us) log.cc:844] Serialized 52 byte log entry
0709 13:03:16.483787 (+383902us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:17.112581 (+628794us) log.cc:844] Serialized 14 byte log entry
0709 13:03:17.112785 (+   204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.112874 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:18.137626 (+1024752us) raft_consensus.cc:1685] finished
0709 13:03:18.137655 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:18.137870 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63573,"prepare.queue_time_us":1113837,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"raft.queue_time_us":500040,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":25354,"thread_start_us":922318,"threads_started":2,"wal-append.queue_time_us":1420633}
I20260709 13:03:18.084002 12535 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.904951 12330 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:18.105584 12577 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.321s	user 0.001s	sys 0.000s
I20260709 13:03:18.163619 12581 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:17.933979  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:18.152814  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:17.745728  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:18.168833 12545 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:18.168936 12580 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:18.169169  8251 raft_consensus.cc:2393] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 13: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:18.162719 12232 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:18.169488 12580 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:18.170028 12580 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:18.171500 12232 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:17.934839 12495 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:03:18.179167  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 245215us
I20260709 13:03:17.781872 12457 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:17.934697  8401 raft_consensus.cc:1275] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:18.180176  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:18.180485  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2048) took 2156 ms. Trace:
I20260709 13:03:18.163568 12535 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:17.780320 12263 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:18.147749  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 416724us
I20260709 13:03:18.165623  9004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 754) took 1903 ms. Trace:
I20260709 13:03:18.181352  8972 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:17.740718 12557 consensus_queue.cc:237] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:18.183003 12575 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:18.186048  8326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2394) took 2238 ms (client timeout 1793 ms). Trace:
I20260709 13:03:17.779050  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:18.197767  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:17.945797  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:18.209293  9007 raft_consensus.cc:2393] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:17.771211  8402 raft_consensus.cc:1240] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12. Ops: [11.5-11.5]
W20260709 13:03:18.206640  8326 rpcz_store.cc:269] 0709 13:03:15.948268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.185877 (+2237609us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:18.185987 (+   110us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:17.748009  8330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2401) took 1794 ms (client timeout 1762 ms). Trace:
W20260709 13:03:18.211907  8330 rpcz_store.cc:269] 0709 13:03:15.953750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.747773 (+1794023us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:17.747928 (+   155us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:18.212548  8220 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:18.213160  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2049) took 2188 ms. Trace:
I20260709 13:03:18.213583  8402 rpcz_store.cc:276] 0709 13:03:16.024505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.770626 (+1746121us) service_pool.cc:224] Handling call
0709 13:03:17.770985 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:18.212895 (+441910us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.213108 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:18.216387  8330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2395) took 2267 ms (client timeout 1877 ms). Trace:
I20260709 13:03:18.215049  9004 rpcz_store.cc:276] 0709 13:03:16.261761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.261914 (+   153us) service_pool.cc:224] Handling call
0709 13:03:16.879237 (+617323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:16.893953 (+ 14716us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.893991 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:16.894458 (+   467us) log.cc:844] Serialized 26 byte log entry
0709 13:03:17.616034 (+721576us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:17.616086 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.616172 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:18.165331 (+549159us) raft_consensus.cc:1685] finished
0709 13:03:18.165367 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:18.165548 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14284,"raft.queue_time_us":27143,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":4548,"thread_start_us":721290,"threads_started":1,"wal-append.queue_time_us":889874}
I20260709 13:03:18.203790  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1988) took 1747 ms. Trace:
W20260709 13:03:18.252300  8330 rpcz_store.cc:269] 0709 13:03:15.948517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.216173 (+2267656us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:18.216311 (+   138us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:18.219477 12487 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:18.252749  8248 rpcz_store.cc:276] 0709 13:03:16.456225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.456419 (+   194us) service_pool.cc:224] Handling call
0709 13:03:16.846593 (+390174us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:17.620379 (+773786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.203495 (+583116us) spinlock_profiling.cc:243] Waited 765 ms on lock 0x7b540033e354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:18.203718 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1758606592}
W20260709 13:03:18.211318  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:18.219477  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 487748us
I20260709 13:03:18.180899  8401 rpcz_store.cc:276] 0709 13:03:16.024267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.746044 (+1721777us) service_pool.cc:224] Handling call
0709 13:03:17.907862 (+161818us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.180236 (+272374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.180420 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:18.227126  8326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 570) took 2484 ms (client timeout 2085 ms). Trace:
W20260709 13:03:18.254829  8326 rpcz_store.cc:269] 0709 13:03:15.742410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.226945 (+2484535us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:18.227054 (+   109us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:17.777835 12533 consensus_meta.cc:287] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.872s	user 0.017s	sys 0.000s
W20260709 13:03:18.011152 12586 log.cc:927] Time spent T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
W20260709 13:03:18.192721 12497 log.cc:927] Time spent T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 3.241s	user 0.000s	sys 0.019s
I20260709 13:03:18.089951 12488 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:17.718266 12584 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:18.190800 12494 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 3.076s	user 0.008s	sys 0.012s
W20260709 13:03:18.246923 12050 consensus_meta.cc:287] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.533s	user 0.009s	sys 0.002s
I20260709 13:03:18.244352  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:18.012888 12585 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.179157 12495 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.181612 12545 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:18.183703 12312 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:18.214763 12452 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:03:18.253229  8330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2411) took 2291 ms (client timeout 1910 ms). Trace:
W20260709 13:03:18.245788 12491 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.007s	sys 0.004s
I20260709 13:03:18.160140  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:18.283421  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:18.283806  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:18.270018 12593 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.284662  8403 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:18.282490 12452 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:18.285601 12452 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:18.286763 12537 raft_consensus.cc:3055] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:18.287197 12537 raft_consensus.cc:740] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:18.288045 12537 consensus_queue.cc:260] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:18.289120 12537 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:18.260933  8326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 577) took 2071 ms (client timeout 1695 ms). Trace:
I20260709 13:03:18.263327  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:18.245196 12450 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:18.201743 12580 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:18.275504  8249 raft_consensus.cc:2393] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 11: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:18.295761  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 562) took 4172 ms. Trace:
I20260709 13:03:18.295045  8402 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
W20260709 13:03:18.281297  8330 rpcz_store.cc:269] 0709 13:03:15.961357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.253051 (+2291694us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:18.253175 (+   124us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:18.297252  8330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 572) took 2526 ms (client timeout 2128 ms). Trace:
W20260709 13:03:18.297598  8330 rpcz_store.cc:269] 0709 13:03:15.772048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.297096 (+2525048us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:18.297197 (+   101us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:18.298555  8330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2406) took 2339 ms (client timeout 1945 ms). Trace:
W20260709 13:03:18.298949  8330 rpcz_store.cc:269] 0709 13:03:15.959584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.298364 (+2338780us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:18.298492 (+   128us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:18.284202  8248 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 13: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:18.253295 12487 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:18.260176  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:18.299953  8401 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:18.209968 12583 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:18.289635  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:18.282594 12491 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.008s	sys 0.004s
W20260709 13:03:18.304729  8220 leader_election.cc:336] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.115s (SENT)
W20260709 13:03:18.304297  8406 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.008s	sys 0.008s
W20260709 13:03:18.305301  8406 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.008s	sys 0.008s
I20260709 13:03:18.279858 12495 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:18.291312 12502 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:03:18.294101  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:18.294633  8326 rpcz_store.cc:269] 0709 13:03:16.189746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.259916 (+2070170us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:18.260872 (+   956us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:18.309871  8326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 580) took 2055 ms (client timeout 1742 ms). Trace:
I20260709 13:03:18.279563 12545 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:18.273638 12581 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:18.303824  8330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 569) took 2561 ms (client timeout 2204 ms). Trace:
I20260709 13:03:18.296154  8328 rpcz_store.cc:276] 0709 13:03:14.123403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.129615 (+  6212us) service_pool.cc:224] Handling call
0709 13:03:14.129955 (+   340us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:14.159991 (+ 30036us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:14.160021 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:14.192716 (+ 32695us) log.cc:844] Serialized 52 byte log entry
0709 13:03:14.447969 (+255253us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:14.669105 (+221136us) log.cc:844] Serialized 14 byte log entry
0709 13:03:14.669348 (+   243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.669460 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:18.190080 (+3520620us) spinlock_profiling.cc:243] Waited 93 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:18.295432 (+105352us) raft_consensus.cc:1685] finished
0709 13:03:18.295540 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:18.295703 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2133,"prepare.queue_time_us":629606,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":3266,"raft.queue_time_us":97024,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":29913,"spinlock_wait_cycles":214013696,"thread_start_us":402257,"threads_started":2,"wal-append.queue_time_us":852839}
I20260709 13:03:18.305202  8220 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:18.291126  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:18.314029  8330 rpcz_store.cc:269] 0709 13:03:15.742112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.303615 (+2561503us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:18.303758 (+   143us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:18.311533  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:18.314850  8403 raft_consensus.cc:2393] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:18.315099  8401 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:18.315191  8330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2410) took 2354 ms (client timeout 2203 ms). Trace:
W20260709 13:03:18.310369  8326 rpcz_store.cc:269] 0709 13:03:16.254203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.309665 (+2055462us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:18.309791 (+   126us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:18.314451  8328 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2393) took 2366 ms (client timeout 2115 ms). Trace:
I20260709 13:03:18.312409  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:18.315598  8330 rpcz_store.cc:269] 0709 13:03:15.960843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.315038 (+2354195us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:18.315136 (+    98us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:18.316272  8326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2409) took 2355 ms (client timeout 2228 ms). Trace:
W20260709 13:03:18.316071  8328 rpcz_store.cc:269] 0709 13:03:15.947726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.314284 (+2366558us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:18.314397 (+   113us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:18.316886  8326 rpcz_store.cc:269] 0709 13:03:15.960489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.316117 (+2355628us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:18.316221 (+   104us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:18.219008 12479 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:18.324673  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 176277us
I20260709 13:03:18.293969 12535 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:18.327307  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:18.272208 12488 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:18.278012 12050 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:18.329010  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:18.264701 12533 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:18.331331  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:18.332340  8249 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:03:18.337683  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:18.338122  8403 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:18.305183 12487 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:18.294091 12513 raft_consensus.cc:697] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:18.341199  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:18.345978  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:18.346191  8328 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 581) took 2075 ms (client timeout 2232 ms). Trace:
I20260709 13:03:18.346647  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:18.346781  8326 raft_consensus.cc:2410] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
W20260709 13:03:18.346697  8328 rpcz_store.cc:269] 0709 13:03:16.270509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.317168 (+2046659us) service_pool.cc:224] Handling call
0709 13:03:18.346114 (+ 28946us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:18.347474  8330 raft_consensus.cc:2393] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 13: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:18.309314 12502 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:18.348850  8326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2413) took 2386 ms (client timeout 2849 ms). Trace:
I20260709 13:03:18.327510 12545 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:18.349292  8326 rpcz_store.cc:269] 0709 13:03:15.962212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.327881 (+2365669us) service_pool.cc:224] Handling call
0709 13:03:18.348792 (+ 20911us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:18.353652  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1992) took 1526 ms. Trace:
I20260709 13:03:18.337289 12581 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:18.356968  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:18.281975 12585 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:18.366627 12585 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:18.356487  8254 rpcz_store.cc:276] 0709 13:03:16.827418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.888089 (+ 60671us) service_pool.cc:224] Handling call
0709 13:03:16.888413 (+   324us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:16.895930 (+  7517us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.895961 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:16.907257 (+ 11296us) log.cc:844] Serialized 52 byte log entry
0709 13:03:17.615345 (+708088us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:17.994915 (+379570us) log.cc:844] Serialized 14 byte log entry
0709 13:03:17.995287 (+   372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.995386 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:18.353346 (+357960us) raft_consensus.cc:1685] finished
0709 13:03:18.353370 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:18.353591 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17862,"prepare.queue_time_us":43563,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":291,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":11593,"thread_start_us":848002,"threads_started":2,"wal-append.queue_time_us":769065}
I20260709 13:03:18.370733  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:18.371783  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:18.372316  8254 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:18.373087  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:18.359156  9006 raft_consensus.cc:2393] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:18.359648  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:18.359860  8369 connection.cc:256] RPC call timeout handler was delayed by -1.3094953599999712s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15660000000000002s for a response
W20260709 13:03:18.378150  8369 connection.cc:256] RPC call timeout handler was delayed by -1.2806953599999815s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15980000000000003s for a response
I20260709 13:03:18.378186  8248 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
W20260709 13:03:18.378595  8369 connection.cc:256] RPC call timeout handler was delayed by -1.1807953599999905s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17090000000000002s for a response
W20260709 13:03:18.378952  8369 connection.cc:256] RPC call timeout handler was delayed by -1.1177953600000023s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1779s for a response
W20260709 13:03:18.379271  8369 connection.cc:256] RPC call timeout handler was delayed by -1.0511953599999515s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18530000000000002s for a response
I20260709 13:03:18.379828  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:18.360184 12584 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:18.301298 12593 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:18.360736 12581 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:18.382278  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:18.382892  8248 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:03:18.383718  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:18.386730 12566 log.cc:927] Time spent T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.887s	user 0.000s	sys 0.001s
I20260709 13:03:18.361857 12527 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:18.345589 12535 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:18.387990 12535 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:18.381662  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:18.389504  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:18.390152  8405 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:18.390885  8249 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:18.389508  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:18.392030  8248 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:18.373828  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:18.383847 12572 log.cc:927] Time spent T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.851s	user 0.000s	sys 0.001s
I20260709 13:03:18.363489 12598 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:18.349267  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2415) took 2329 ms. Trace:
I20260709 13:03:18.407253  8330 rpcz_store.cc:276] 0709 13:03:16.020223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.328549 (+2308326us) service_pool.cc:224] Handling call
0709 13:03:18.349216 (+ 20667us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:18.375566 12479 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:18.394052  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:18.415066  8328 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:18.415688  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2407) took 2455 ms. Trace:
I20260709 13:03:18.416064  8328 rpcz_store.cc:276] 0709 13:03:15.959882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.358855 (+2398973us) service_pool.cc:224] Handling call
0709 13:03:18.415617 (+ 56762us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:18.389307 12593 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:18.395715 12508 raft_consensus.cc:993] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b: : Instructing follower e64761f43e4240319c41117cf886d22f to start an election
I20260709 13:03:18.395596  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:18.430284  8402 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
W20260709 13:03:18.399037  8257 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49220 (request call id 608) took 2337 ms (client timeout 2736 ms). Trace:
I20260709 13:03:18.400846 12050 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:18.431005  8257 rpcz_store.cc:269] 0709 13:03:16.061960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.062148 (+   188us) service_pool.cc:224] Handling call
0709 13:03:18.398805 (+2336657us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b5400178254. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:18.398951 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3715106048}
I20260709 13:03:18.402637 12580 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:18.403566  9009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 749) took 2317 ms. Trace:
W20260709 13:03:18.403664 12562 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 1.387s	user 0.001s	sys 0.001s
W20260709 13:03:18.404134 12578 log.cc:927] Time spent T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.619s	user 0.000s	sys 0.010s
I20260709 13:03:18.404701 12502 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:18.392787  8326 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:03:18.437592  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 576) took 2251 ms. Trace:
I20260709 13:03:18.339087 12491 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:03:18.348953 12589 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:18.414261  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 208416us
I20260709 13:03:18.426951  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:18.427469  8401 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:18.427467  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:18.437942  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:18.428748  8298 connection.cc:256] RPC call timeout handler was delayed by -1.3371433020000154s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:03:18.401728  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:18.433090  9009 rpcz_store.cc:276] 0709 13:03:16.085895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.086167 (+   272us) service_pool.cc:224] Handling call
0709 13:03:16.086480 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:16.086828 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.086868 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:16.107115 (+ 20247us) log.cc:844] Serialized 26 byte log entry
0709 13:03:16.457770 (+350655us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:16.457828 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.457917 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:18.383328 (+1925411us) spinlock_profiling.cc:243] Waited 6.01 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:18.403251 (+ 19923us) raft_consensus.cc:1685] finished
0709 13:03:18.403278 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:18.403486 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6455,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":2690,"spinlock_wait_cycles":13828352,"thread_start_us":273019,"threads_started":1,"wal-append.queue_time_us":954215}
I20260709 13:03:18.432320  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:18.437935  8326 rpcz_store.cc:276] 0709 13:03:16.185871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.356287 (+2170416us) service_pool.cc:224] Handling call
0709 13:03:18.437532 (+ 81245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:18.408213 12606 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.428399 12581 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.429291 12585 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.445003  8330 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:18.456668  8405 raft_consensus.cc:2393] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 11: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:18.460122  8298 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:18.457034  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:18.437038 12452 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:18.462657 12604 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:18.462868  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:18.478600  8328 raft_consensus.cc:2393] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 12: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:18.463279  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:18.463346 12502 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:18.435982 12610 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:18.488327  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 106895us
I20260709 13:03:18.465119  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 623) took 2441 ms. Trace:
I20260709 13:03:18.467831  8222 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:18.490882  8222 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:18.490973  8369 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:18.491982  8222 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:18.490885  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:18.484795  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:18.433341 12479 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.495857 12582 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:18.481288 12581 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:18.496080 12479 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:18.498018 12479 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:18.464931 12595 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:18.498322 12487 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:03:18.498713 12479 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:18.496352 12582 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.499810 12582 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:18.500084 12479 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:18.479776  8257 raft_consensus.cc:2393] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 12: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:18.481139 12606 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:18.500468 12487 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:18.501235  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:18.501896  8257 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:03:18.502858  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:18.503621  8257 raft_consensus.cc:2410] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 14 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:03:18.504041 12528 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:18.504351  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:18.505076 12275 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:18.505609 12275 raft_consensus.cc:697] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:18.510540 12275 consensus_queue.cc:237] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:18.482662  8248 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:18.512369 12346 raft_consensus.cc:3055] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:18.512565  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:18.512830 12346 raft_consensus.cc:740] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:18.457348 12503 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.477912 12605 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.444653 12602 raft_consensus.cc:1081] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Signalling peer e64761f43e4240319c41117cf886d22f to start an election
I20260709 13:03:18.489542  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:18.530835  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:18.514423  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:18.456552 12050 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:18.513316  8248 raft_consensus.cc:2393] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 11: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:18.493011  8406 rpcz_store.cc:276] 0709 13:03:16.023817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.642888 (+1619071us) service_pool.cc:224] Handling call
0709 13:03:17.646323 (+  3435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:17.646568 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:18.307502 (+660934us) log.cc:844] Serialized 14 byte log entry
0709 13:03:18.464698 (+157196us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:18.464749 (+    51us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:18.464791 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.464950 (+   159us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:18.465069 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19796,"prepare.queue_time_us":714656,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"thread_start_us":739617,"threads_started":3}
I20260709 13:03:18.410375 12598 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:18.537009 12598 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:18.537777 12533 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:18.539319 12611 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:18.540282  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:18.515296 12485 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:18.541831 12452 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:18.534096  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:18.542294  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2397) took 2593 ms. Trace:
I20260709 13:03:18.542689  8327 rpcz_store.cc:276] 0709 13:03:15.948979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.530593 (+2581614us) service_pool.cc:224] Handling call
0709 13:03:18.542234 (+ 11641us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:18.543013  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.543983  8327 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:18.544639  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 566) took 2803 ms. Trace:
I20260709 13:03:18.544934  8327 rpcz_store.cc:276] 0709 13:03:15.741168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.543439 (+2802271us) service_pool.cc:224] Handling call
0709 13:03:18.543713 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.544420 (+   707us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.544589 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:18.546123  8327 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:18.546622  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 568) took 2804 ms. Trace:
I20260709 13:03:18.542851  8330 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 13: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:18.546908  8327 rpcz_store.cc:276] 0709 13:03:15.741826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.545633 (+2803807us) service_pool.cc:224] Handling call
0709 13:03:18.545864 (+   231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.546426 (+   562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.546576 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:18.549015  8327 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:18.549546  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2396) took 2600 ms. Trace:
I20260709 13:03:18.550848  8327 rpcz_store.cc:276] 0709 13:03:15.948758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.548449 (+2599691us) service_pool.cc:224] Handling call
0709 13:03:18.548874 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.549340 (+   466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.549503 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:18.552263  8406 raft_consensus.cc:1240] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:18.553072  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2058) took 2477 ms. Trace:
I20260709 13:03:18.553489  8406 rpcz_store.cc:276] 0709 13:03:16.075696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.544176 (+2468480us) service_pool.cc:224] Handling call
0709 13:03:18.552033 (+  7857us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.552793 (+   760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.553012 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:18.436431 12513 consensus_queue.cc:237] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:18.498019 12581 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:18.556710 12612 consensus_queue.cc:1059] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:18.529732 12604 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:18.561689  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:18.500485 12479 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.570768  8402 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:18.513823 12346 consensus_queue.cc:260] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:18.572631  8332 raft_consensus.cc:1275] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:18.572672 12346 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:18.573338  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2147) took 2879 ms. Trace:
I20260709 13:03:18.573729  8332 rpcz_store.cc:276] 0709 13:03:15.701382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.542403 (+2841021us) service_pool.cc:224] Handling call
0709 13:03:18.572326 (+ 29923us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.573086 (+   760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.573289 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:18.574874  8332 raft_consensus.cc:1240] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:18.576292 12495 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:18.576738 12606 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:18.577294 12468 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:18.579263  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:18.579881  8255 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:18.580705  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:18.581984 12535 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:18.582050  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:18.582722  8251 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:18.588174 12583 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:18.590708 12479 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:18.572804  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1987) took 2166 ms. Trace:
I20260709 13:03:18.594633  8253 rpcz_store.cc:276] 0709 13:03:16.401599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.401750 (+   151us) service_pool.cc:224] Handling call
0709 13:03:16.402176 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:16.402602 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.402642 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:16.406164 (+  3522us) log.cc:844] Serialized 26 byte log entry
0709 13:03:17.622984 (+1216820us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:17.623038 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.623129 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:18.403632 (+780503us) spinlock_profiling.cc:243] Waited 92.9 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:18.543211 (+139579us) raft_consensus.cc:1685] finished
0709 13:03:18.543233 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:18.567720 (+ 24487us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":419081,"raft.queue_time_us":101,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":67564,"spinlock_wait_cycles":213607808,"thread_start_us":705745,"threads_started":1,"wal-append.queue_time_us":828682}
I20260709 13:03:18.596678  8253 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:18.581236  8255 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:18.541847 12485 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:18.568811  8297 connection.cc:256] RPC call timeout handler was delayed by -1.0020355560000098s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1547s for a response
I20260709 13:03:18.604018  8251 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:18.604646  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2399) took 2655 ms. Trace:
I20260709 13:03:18.488600 12585 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:18.611433  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:18.612888  8253 raft_consensus.cc:1275] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:18.612488  8332 rpcz_store.cc:276] 0709 13:03:15.953181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.574420 (+2621239us) service_pool.cc:224] Handling call
0709 13:03:18.574721 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.604320 (+ 29599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.604565 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:18.613940  8251 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:03:18.615384  8248 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:18.620536  8402 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:18.621361  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2059) took 2529 ms. Trace:
I20260709 13:03:18.622025  8402 rpcz_store.cc:276] 0709 13:03:16.092243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.571616 (+2479373us) service_pool.cc:224] Handling call
0709 13:03:18.620289 (+ 48673us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.621066 (+   777us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.621294 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:18.524066  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:18.531298 12503 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:18.623946  8253 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:18.569784 12488 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:18.600445 12545 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:18.657541  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 631) took 1742 ms. Trace:
I20260709 13:03:18.670017  8404 rpcz_store.cc:276] 0709 13:03:16.915087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.656731 (+1741644us) service_pool.cc:224] Handling call
0709 13:03:18.657017 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.657221 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:18.657246 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:18.657287 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:18.657317 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.657369 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:18.657490 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:18.625476  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:18.671644  8332 raft_consensus.cc:2393] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:18.655155 12583 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:18.506958 12610 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:18.611006 12581 consensus_queue.cc:1059] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:03:18.640090  8330 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:18.613042 12485 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:18.534168 12605 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:18.640865  8251 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:03:18.637687  8972 leader_election.cc:336] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.964s (SENT)
W20260709 13:03:18.665784  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 141788us
I20260709 13:03:18.672340  8253 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 11. Current term is 12. Ops: [11.8-11.8]
W20260709 13:03:18.674125  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 207212us
W20260709 13:03:18.683650  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 46012.3us
I20260709 13:03:18.684707  8369 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:18.675104  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2446) took 2594 ms. Trace:
W20260709 13:03:18.674573  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 137622us
I20260709 13:03:18.676719  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2441) took 2616 ms. Trace:
I20260709 13:03:18.679741  8248 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
I20260709 13:03:18.690524  8251 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 11. Current term is 12. Ops: [9.9-11.10]
I20260709 13:03:18.690129  8405 rpcz_store.cc:276] 0709 13:03:16.060586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.541370 (+2480784us) service_pool.cc:224] Handling call
0709 13:03:18.541659 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.541840 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:18.542085 (+   245us) log.cc:844] Serialized 14 byte log entry
0709 13:03:18.676296 (+134211us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:18.676368 (+    72us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:18.676427 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.676511 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:18.676650 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5497,"prepare.queue_time_us":460,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":7730,"raft.queue_time_us":14299,"thread_start_us":122083,"threads_started":1}
I20260709 13:03:18.685554  8406 rpcz_store.cc:276] 0709 13:03:16.080345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.557872 (+2477527us) service_pool.cc:224] Handling call
0709 13:03:18.558281 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:18.559066 (+   785us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:18.559108 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:18.559628 (+   520us) log.cc:844] Serialized 26 byte log entry
0709 13:03:18.665079 (+105451us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:18.665149 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.665240 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:18.674793 (+  9553us) raft_consensus.cc:1685] finished
0709 13:03:18.674820 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:18.675014 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":153,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":100,"thread_start_us":54892,"threads_started":1,"wal-append.queue_time_us":113154}
I20260709 13:03:18.695139  8254 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:18.690212  8295 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:18.702431  8405 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:03:18.704059  8253 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:03:18.704494  8406 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:18.705050  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:18.703384  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 632) took 1770 ms. Trace:
I20260709 13:03:18.654202 12495 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:18.708036  8406 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:18.708839  8402 raft_consensus.cc:1240] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:18.707820  8405 rpcz_store.cc:276] 0709 13:03:16.932715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.692464 (+1759749us) service_pool.cc:224] Handling call
0709 13:03:18.692820 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.703006 (+ 10186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.703316 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:18.711433  8406 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:18.712081  8405 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:18.712306  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:18.717509  8253 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:18.723040 12501 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.457s	user 0.011s	sys 0.035s
I20260709 13:03:18.723271  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:18.723419  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:18.724052  8332 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:18.724556  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:18.719175 12468 consensus_queue.cc:1048] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:03:18.728931  8254 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:18.730897  8332 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
I20260709 13:03:18.732203  8404 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:18.706008 12583 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:18.734418  8406 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:18.723448 12501 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.577s	user 0.011s	sys 0.036s
I20260709 13:03:18.710263 12621 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:18.753780  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:18.756217  8325 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
I20260709 13:03:18.757150  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:18.758701 12616 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:18.762907  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:18.764159  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:18.764720  8325 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:18.765509  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:18.766188  8253 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:18.766741  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:18.767370  8325 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:03:18.768931  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:18.767175  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:18.770651  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:18.772351  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:18.771338  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:18.736161  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:18.772317  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:18.818600  8325 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:18.827262  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:18.828203  9001 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:18.831410  8332 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:18.832336  9005 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:18.832607  8253 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:03:18.832412  8972 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:18.835711  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 134271us
I20260709 13:03:18.836414  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:18.842981  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:18.843865  9000 raft_consensus.cc:2410] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:03:18.755985 12621 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:18.843665  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:18.848136  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:19.103497  8325 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
W20260709 13:03:19.103914  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.026503  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:19.104733  9007 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
W20260709 13:03:19.104960  9006 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56940 (request call id 880) took 2092 ms (client timeout 1843 ms). Trace:
I20260709 13:03:18.918053  8323 raft_consensus.cc:3055] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:19.105558  8323 raft_consensus.cc:740] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:18.918977  9002 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:18.926862  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2450) took 2768 ms. Trace:
I20260709 13:03:18.959007  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:19.107297  9004 raft_consensus.cc:2393] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 12: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:18.987938  8220 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:18.932873  8398 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 2050) took 2908 ms (client timeout 1556 ms). Trace:
W20260709 13:03:19.108764  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:19.108959  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:19.109349  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:19.109782  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:19.105406  9006 rpcz_store.cc:269] 0709 13:03:16.894286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.894701 (+   415us) service_pool.cc:224] Handling call
0709 13:03:18.915968 (+2021267us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54006e23d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:18.986814 (+ 70846us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3283359104}
W20260709 13:03:18.929064  8397 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 2433) took 3149 ms (client timeout 1891 ms). Trace:
I20260709 13:03:18.917150  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:18.968868 12503 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:19.112645  8397 rpcz_store.cc:269] 0709 13:03:15.779635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.779798 (+   163us) service_pool.cc:224] Handling call
0709 13:03:18.928873 (+3149075us) spinlock_profiling.cc:243] Waited 2.52 s on lock 0x7b54003a20d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:18.928996 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5802379008}
I20260709 13:03:18.958408  8974 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:18.933723  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:18.988461  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:18.936259 12605 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:18.923839  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:18.987268  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:18.966279 12620 raft_consensus.cc:3055] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:03:19.128814  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.128783  8329 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:18.978050 12545 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:18.944808  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:18.750957 12582 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:18.921604  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:19.139600  8252 raft_consensus.cc:2393] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 12: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:18.959359 12232 raft_consensus.cc:3055] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:18.960563 12392 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:18.923978 12448 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:18.988507  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:18.919901  8331 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2148) took 3218 ms (client timeout 1926 ms). Trace:
I20260709 13:03:18.930030 12485 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:19.042546  8223 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:18.919867 12585 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:18.846076 12527 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:19.149932 12527 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.150321 12527 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:19.150825  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:19.151929  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:19.152465  8254 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:19.153198  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:19.154211  8254 raft_consensus.cc:2393] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 11: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:19.155076  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:19.155619  8254 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:19.156540  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:18.921715 12624 log.cc:927] Time spent T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.168s	user 0.000s	sys 0.004s
I20260709 13:03:19.158069 12545 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:19.158485  9000 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:19.158569 12545 raft_consensus.cc:697] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:18.917582 12580 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:19.159514 12545 consensus_queue.cc:237] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:19.160157  8372 connection.cc:256] RPC call timeout handler was delayed by -2.3065733739999814s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:03:19.160319 12580 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:19.160688 12580 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.161021 12580 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:19.162024 12243 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:18.918291 12450 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:03:19.165869  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 123395us
I20260709 13:03:18.940811  8328 raft_consensus.cc:1240] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:19.167014  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2402) took 3213 ms. Trace:
I20260709 13:03:19.167342  8328 rpcz_store.cc:276] 0709 13:03:15.953965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.940350 (+2986385us) service_pool.cc:224] Handling call
0709 13:03:18.940650 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:19.166738 (+226088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.166936 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29312}
I20260709 13:03:19.168426  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:19.169005  8328 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:19.102207  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 255028us
I20260709 13:03:19.169720  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:19.170145  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:19.170449  8254 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:19.171268  8251 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 12: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:19.141974  8323 consensus_queue.cc:260] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:19.172118  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1993) took 2292 ms. Trace:
W20260709 13:03:18.954339 12626 log.cc:927] Time spent T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:03:18.950567  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:19.106865  8402 rpcz_store.cc:276] 0709 13:03:16.158745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.650910 (+2492165us) service_pool.cc:224] Handling call
0709 13:03:18.678014 (+ 27104us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.709268 (+ 31254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.926616 (+217348us) spinlock_profiling.cc:243] Waited 30.6 ms on lock 0x7b54003a1bd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:18.926801 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":70321408}
I20260709 13:03:19.174881  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:19.175216  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:19.175513  8332 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:19.176472  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:19.112944  8324 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:18.871977 12621 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:19.114881  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:19.107201 12479 consensus_meta.cc:287] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.516s	user 0.010s	sys 0.000s
I20260709 13:03:19.188457  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:19.189038  8254 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
W20260709 13:03:19.144644  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 156756us
I20260709 13:03:19.135015 12620 raft_consensus.cc:740] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:19.190032  8220 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:19.147804  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:19.144651  8331 rpcz_store.cc:269] 0709 13:03:15.701795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.726226 (+ 24431us) service_pool.cc:224] Handling call
0709 13:03:18.919695 (+3193469us) spinlock_profiling.cc:243] Waited 8.45 ms on lock 0x7b54003ca854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:18.919798 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19445120}
I20260709 13:03:19.149325 12485 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:19.191102  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:19.157608 12345 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:19.172235  9004 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:19.113763  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:19.193681  9002 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:19.193835  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:19.194393  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.194594  8253 raft_consensus.cc:2376] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10. Re-sending same reply.
I20260709 13:03:19.181207  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:19.182554  9001 raft_consensus.cc:2393] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:19.183120  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:19.185979  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:19.196951  9009 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:19.197705  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:19.198094  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:19.198853  8254 raft_consensus.cc:2393] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 12: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:03:19.108973  8398 rpcz_store.cc:269] 0709 13:03:16.024750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.036562 (+ 11812us) service_pool.cc:224] Handling call
0709 13:03:18.932650 (+2896088us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54000fc3d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:18.932799 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":281908608}
I20260709 13:03:19.172093 12230 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:19.172652  8250 rpcz_store.cc:276] 0709 13:03:16.879936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.900165 (+ 20229us) service_pool.cc:224] Handling call
0709 13:03:16.931986 (+ 31821us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:17.021306 (+ 89320us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:18.293020 (+1271714us) spinlock_profiling.cc:243] Waited 4.16 ms on lock 0x7b180035dd00. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256ce455 00007f882c1b63c7 00007f88287c89a7 00007f88287d80bf 00007f88287d8038
0709 13:03:18.456694 (+163674us) log.cc:844] Serialized 14 byte log entry
0709 13:03:18.665363 (+208669us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:19.155094 (+489731us) spinlock_profiling.cc:243] Waited 4.83 ms on lock 0x7b2c000014c8. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:03:19.155461 (+   367us) log.cc:844] Serialized 26 byte log entry
0709 13:03:19.155600 (+   139us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:19.155633 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.155699 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.171918 (+ 16219us) raft_consensus.cc:1685] finished
0709 13:03:19.171936 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.172058 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21128,"prepare.queue_time_us":621426,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":7967,"raft.queue_time_us":14894,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":12944,"spinlock_wait_cycles":20677760,"thread_start_us":298088,"threads_started":2,"wal-append.queue_time_us":271659}
I20260709 13:03:19.179234  8323 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:19.143837 12232 raft_consensus.cc:740] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:19.199623  8220 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:19.199950  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:19.202620  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 571) took 3460 ms. Trace:
I20260709 13:03:19.227238  8330 rpcz_store.cc:276] 0709 13:03:15.742700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.548050 (+2805350us) service_pool.cc:224] Handling call
0709 13:03:18.550481 (+  2431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.677889 (+127408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.202394 (+524505us) spinlock_profiling.cc:243] Waited 89.4 ms on lock 0x7b54002c9f54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:19.202561 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":205538560}
W20260709 13:03:19.186878  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.203462 12606 raft_consensus.cc:3055] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:19.220523  8328 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:19.175088 12534 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:19.233668  8324 raft_consensus.cc:1240] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:19.223697  8329 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:19.223996  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:19.230751  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 116426us
I20260709 13:03:19.225231  8254 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:03:19.229894  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.230696  8328 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:19.234596  8332 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:19.238354  8250 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:03:19.237399  9005 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:19.244724  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:19.197700 12503 raft_consensus.cc:2764] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:03:19.240985  8222 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:19.239831  8369 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:19.245568  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:19.246091  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:19.247898  8254 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:03:19.240227  8252 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:19.249770  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2412) took 3272 ms. Trace:
I20260709 13:03:19.240836  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2405) took 3281 ms. Trace:
I20260709 13:03:19.242967  8223 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:19.258419  8328 rpcz_store.cc:276] 0709 13:03:15.959298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.230291 (+3270993us) service_pool.cc:224] Handling call
0709 13:03:19.230552 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:19.240616 (+ 10064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.240786 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:19.250938  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:19.260272  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:19.263847  8323 raft_consensus.cc:1275] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:19.264757  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2139) took 3748 ms. Trace:
I20260709 13:03:19.269620  8223 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:19.258335  8324 rpcz_store.cc:276] 0709 13:03:15.961913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.231091 (+3269178us) service_pool.cc:224] Handling call
0709 13:03:19.233511 (+  2420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:19.234041 (+   530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.234196 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:19.265151  8323 rpcz_store.cc:276] 0709 13:03:15.515915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.516050 (+   135us) service_pool.cc:224] Handling call
0709 13:03:15.516421 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.911210 (+3394789us) spinlock_profiling.cc:243] Waited 31.2 ms on lock 0x7b540033aa20. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f8828735dc0 00007f88287c01c0 00007f88287c1231 00007f88287bdd8f 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c
0709 13:03:19.264423 (+353213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.264686 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71699200}
I20260709 13:03:19.271893  8369 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:19.278254  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:19.282480  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:19.283293  8324 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 13:03:19.283960  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2155) took 1023 ms. Trace:
I20260709 13:03:19.273037  8372 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:19.284349  8324 rpcz_store.cc:276] 0709 13:03:18.260153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.282728 (+1022575us) service_pool.cc:224] Handling call
0709 13:03:19.283064 (+   336us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:19.283675 (+   611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.283906 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:19.287257  8331 raft_consensus.cc:1275] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:19.288811  8331 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:19.233151 12606 raft_consensus.cc:740] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:19.290305 12582 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:19.290449 12620 consensus_queue.cc:260] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:19.292163 12620 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:19.294046 12491 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:19.294611 12491 raft_consensus.cc:697] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:19.295470 12491 consensus_queue.cc:237] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:19.238754 12485 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:19.192361 12479 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:19.298257 12479 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:19.283805 12524 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:19.299247 12524 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:19.299579 12604 consensus_meta.cc:287] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.687s	user 0.007s	sys 0.003s
I20260709 13:03:19.300297 12604 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:19.301115 12604 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:19.301945  8327 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 13. Current term is 15. Ops: []
I20260709 13:03:19.302327  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:19.302459 12548 consensus_queue.cc:1059] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:19.283881 12539 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:19.286598 12232 consensus_queue.cc:260] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:19.305620  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:19.305665  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2459) took 1047 ms. Trace:
I20260709 13:03:19.307648  8253 raft_consensus.cc:1240] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
W20260709 13:03:19.283653 12190 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
W20260709 13:03:19.336131  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 377730us
I20260709 13:03:19.341007  8405 rpcz_store.cc:276] 0709 13:03:18.258148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.721609 (+463461us) service_pool.cc:224] Handling call
0709 13:03:18.721999 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:18.722349 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:18.722376 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:18.722796 (+   420us) log.cc:844] Serialized 26 byte log entry
0709 13:03:18.870496 (+147700us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:18.870542 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.870619 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.305379 (+434760us) raft_consensus.cc:1685] finished
0709 13:03:19.305404 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.305596 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7374,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":3588,"thread_start_us":108710,"threads_started":1,"wal-append.queue_time_us":484359}
I20260709 13:03:19.285247 12508 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:19.342573  8974 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:19.349723  8974 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:19.254702 12521 consensus_queue.cc:1059] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 13:03:19.277391 12501 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:03:19.284245 12537 consensus_meta.cc:287] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.982s	user 0.006s	sys 0.006s
I20260709 13:03:19.285481 12616 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:19.299604 12524 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:19.398705  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 128892us
I20260709 13:03:19.281235 12610 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:19.364260  8974 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:03:19.377506  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 533763us
I20260709 13:03:19.333030 12321 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:19.379696  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 441248us
W20260709 13:03:19.344406  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 239596us
I20260709 13:03:19.400051 12321 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:19.402158  8256 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:19.292484 12580 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:19.338315 12604 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:19.378553 12450 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:19.389458  9008 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:19.389261 12533 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:19.366590 12452 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:19.400058  8972 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:19.285110 12583 consensus_queue.cc:1048] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 13:03:19.415380  8298 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:19.415763  8295 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:19.406100  8248 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:19.414561  8297 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:19.364043 12508 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.346624 12539 raft_consensus.cc:697] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:19.424167  8972 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:19.386384 12501 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:03:19.432456  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 160601us
I20260709 13:03:19.435225  8297 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:19.436618  8295 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:19.438683  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 582) took 3165 ms. Trace:
I20260709 13:03:19.438953  8298 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:19.344393 12392 consensus_queue.cc:1059] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.079s
I20260709 13:03:19.290315 12606 consensus_queue.cc:260] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:19.450819  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:19.451473  9008 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:19.417960 12450 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.418341 12604 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:19.449460  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 176494us
I20260709 13:03:19.427819 12616 raft_consensus.cc:697] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:19.441708  8328 rpcz_store.cc:276] 0709 13:03:16.273182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.259388 (+2986206us) service_pool.cc:224] Handling call
0709 13:03:19.259649 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:19.259823 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:19.304766 (+ 44943us) log.cc:844] Serialized 14 byte log entry
0709 13:03:19.438368 (+133602us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:19.438414 (+    46us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:19.438458 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.438521 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.438629 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2437,"prepare.queue_time_us":45129,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"thread_start_us":133274,"threads_started":1}
I20260709 13:03:19.452491  8295 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:19.436008 12508 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:19.463284  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:19.421145 12637 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:19.422106 12634 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:03:19.453852 12452 raft_consensus.cc:697] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:19.465510 12606 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:19.465667 12583 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.465881  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:19.503408  8328 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:19.504596  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:19.465562 12616 consensus_queue.cc:237] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:19.466360 12539 consensus_queue.cc:237] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:19.470014 12527 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:19.447497 12501 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:19.432036 12582 consensus_queue.cc:1059] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:03:19.434348 12636 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:19.451685 12521 consensus_queue.cc:1059] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:03:19.473160  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:19.522977  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 59200.1us
I20260709 13:03:19.524238  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:19.458662 12450 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:19.500448  8972 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:19.508507  8328 raft_consensus.cc:1275] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 5. (term mismatch)
I20260709 13:03:19.530087  8328 pending_rounds.cc:85] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 4
I20260709 13:03:19.530432  8328 pending_rounds.cc:107] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 11.5
I20260709 13:03:19.530716  8328 raft_consensus.cc:2889] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:19.457790 12418 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:19.472956 12621 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:03:19.472544  9008 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:03:19.513345 12637 raft_consensus.cc:697] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:19.447194 12537 consensus_queue.cc:1059] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:03:19.511158 12640 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:19.502346 12634 consensus_queue.cc:1243] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:03:19.531239  8972 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:19.533735  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.511111 12607 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:19.510913 12548 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.473260 12580 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:19.478291 12633 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:19.563534 12633 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:19.564013 12633 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:19.565898  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2400) took 3612 ms. Trace:
I20260709 13:03:19.566375  8326 rpcz_store.cc:276] 0709 13:03:15.953503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.940350 (+2986847us) service_pool.cc:224] Handling call
0709 13:03:18.940640 (+   290us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:19.110160 (+169520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:19.110206 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:19.134204 (+ 23998us) log.cc:844] Serialized 52 byte log entry
0709 13:03:19.339407 (+205203us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:19.369955 (+ 30548us) log.cc:844] Serialized 14 byte log entry
0709 13:03:19.370554 (+   599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.370660 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.565633 (+194973us) raft_consensus.cc:1685] finished
0709 13:03:19.565660 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.565829 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":161011,"prepare.queue_time_us":30478,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":171579,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":23357,"spinlock_wait_cycles":19840,"thread_start_us":151875,"threads_started":1,"wal-append.queue_time_us":338496}
I20260709 13:03:19.568704 12515 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:19.570225  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 135059us
I20260709 13:03:19.518419 12636 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.562456 12634 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:19.562050 12637 consensus_queue.cc:237] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:19.563083 12545 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:19.571342  8328 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:03:19.543533 12418 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.559746 12640 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.584504  8328 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:03:19.584546  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 145681us
I20260709 13:03:19.556811 12642 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:19.543690 12533 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:19.513477 12583 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:19.545313 12243 raft_consensus.cc:3055] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:19.561456 12610 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:19.560096 12607 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:19.574781  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:19.588766  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.586591  8298 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:19.577903 12636 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:19.580816 12545 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.617290  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:19.585233 12418 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:19.586861 12642 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.586347 12640 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:19.573956 12548 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:19.624720  9008 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:19.593768 12639 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:19.602089 12243 raft_consensus.cc:740] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:19.578789 12452 consensus_queue.cc:237] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:19.627568  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 175086us
I20260709 13:03:19.612340 12611 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:19.627199  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 72158.2us
I20260709 13:03:19.626030 12642 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:19.622807 12545 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:19.682183 12660 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:19.612581 12607 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:19.688850  8295 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:19.612586 12635 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:03:19.687122 12645 log.cc:927] Time spent T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:03:19.708082  8295 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:19.696591  9008 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:19.630556 12534 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:19.701090  8257 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:19.729102  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 575) took 3745 ms. Trace:
W20260709 13:03:19.689767 12648 log.cc:927] Time spent T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.062s	user 0.000s	sys 0.004s
W20260709 13:03:19.705457  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 181224us
I20260709 13:03:19.706620  8972 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:19.714409 12580 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:19.711534 12243 consensus_queue.cc:260] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:19.715756 12583 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:19.711772 12468 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:19.721580 12621 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:19.723011  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.706789 12633 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:19.729528  8329 rpcz_store.cc:276] 0709 13:03:15.983773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.235816 (+3252043us) service_pool.cc:224] Handling call
0709 13:03:19.236200 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:19.247790 (+ 11590us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:19.247827 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:19.255326 (+  7499us) log.cc:844] Serialized 26 byte log entry
0709 13:03:19.425215 (+169889us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:19.425270 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.425358 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.728890 (+303532us) raft_consensus.cc:1685] finished
0709 13:03:19.728910 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.729046 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":194,"raft.queue_time_us":34877,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":27536,"thread_start_us":123207,"threads_started":1,"wal-append.queue_time_us":217412}
W20260709 13:03:19.731712  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.733845  8972 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:19.727036 12230 raft_consensus.cc:3055] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:19.754941  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:19.739785  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 165235us
I20260709 13:03:19.699349 12660 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.750319  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:19.739189  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 153209us
I20260709 13:03:19.758486 12230 raft_consensus.cc:740] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:19.760982 12580 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:19.761258 12667 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:19.761849 12667 raft_consensus.cc:697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:19.761629 12508 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:19.762526  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:19.762899 12667 consensus_queue.cc:237] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:19.763800  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:19.764308  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2403) took 3810 ms. Trace:
I20260709 13:03:19.764619 12312 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:19.761790  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:19.758522 12243 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:19.758303 12583 consensus_queue.cc:1048] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:03:19.767172  8252 raft_consensus.cc:2376] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10. Re-sending same reply.
I20260709 13:03:19.767899 12527 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:19.766536  8254 raft_consensus.cc:2368] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14.
I20260709 13:03:19.769925  8369 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:19.766551  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:19.766057 12610 consensus_queue.cc:1059] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:03:19.770663  8250 raft_consensus.cc:2376] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12. Re-sending same reply.
I20260709 13:03:19.743361 12533 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:19.769042  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2424) took 1031 ms. Trace:
I20260709 13:03:19.765255 12312 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:19.769063  8297 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:19.764667  8325 rpcz_store.cc:276] 0709 13:03:15.958650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.111402 (+3152752us) service_pool.cc:224] Handling call
0709 13:03:19.111734 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:19.336550 (+224816us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:19.336594 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:19.337809 (+  1215us) log.cc:844] Serialized 26 byte log entry
0709 13:03:19.583644 (+245835us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:19.583683 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.583756 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.764123 (+180367us) raft_consensus.cc:1685] finished
0709 13:03:19.764142 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.764256 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14467,"raft.queue_time_us":27555,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":23930,"thread_start_us":389627,"threads_started":2,"wal-append.queue_time_us":383855}
W20260709 13:03:19.809129  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.848529  9002 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:19.761528 12580 raft_consensus.cc:697] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 13 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:19.775123  8324 rpcz_store.cc:276] 0709 13:03:18.737477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.401235 (+663758us) service_pool.cc:224] Handling call
0709 13:03:19.401615 (+   380us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:19.401962 (+   347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:19.401988 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:19.402511 (+   523us) log.cc:844] Serialized 52 byte log entry
0709 13:03:19.685365 (+282854us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:19.706277 (+ 20912us) log.cc:844] Serialized 14 byte log entry
0709 13:03:19.708908 (+  2631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.709010 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.768848 (+ 59838us) raft_consensus.cc:1685] finished
0709 13:03:19.768865 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.768990 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32536,"prepare.queue_time_us":22321,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":1056,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":7583,"thread_start_us":282456,"threads_started":1,"wal-append.queue_time_us":356194}
W20260709 13:03:19.856235  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 87226.6us
I20260709 13:03:19.766244 12593 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:19.726882 12534 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:19.727592 12604 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:19.882211  8297 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:19.770105 12660 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:19.852196 12312 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:19.900333 12580 consensus_queue.cc:237] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:19.755409 12491 consensus_queue.cc:1059] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.106s
I20260709 13:03:19.768487 12655 consensus_queue.cc:1048] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.124s
I20260709 13:03:19.906590  8297 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:19.857951 12479 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:19.951005  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 242952us
I20260709 13:03:19.861016 12533 raft_consensus.cc:697] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:19.776553 12583 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.126s
I20260709 13:03:19.881224 12593 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.894323 12615 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:19.766332 12636 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:19.761526 12230 consensus_queue.cc:260] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:19.910916 12611 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:03:19.962316  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 43673.4us
W20260709 13:03:19.908805 12275 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.000s	sys 0.026s
W20260709 13:03:20.024770  8222 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.3:37367 timed out after 1.517s (SENT) [suppressed 61 similar messages]
W20260709 13:03:20.100001  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:20.166628  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:20.184813  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:20.283123  8399 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 2055) took 4244 ms (client timeout 1872 ms). Trace:
W20260709 13:03:20.283694  8399 rpcz_store.cc:269] 0709 13:03:16.038907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.909351 (+870444us) service_pool.cc:224] Handling call
0709 13:03:20.270914 (+3361563us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54002675d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:20.282863 (+ 11949us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3246297984}
I20260709 13:03:20.292269  8295 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:19.989781 12593 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:20.310823  8295 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:20.313226  8295 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:19.919960 12635 raft_consensus.cc:697] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:03:20.257313 12275 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.142s	user 0.002s	sys 0.034s
W20260709 13:03:20.256443 12524 consensus_meta.cc:287] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.861s	user 0.008s	sys 0.003s
W20260709 13:03:20.330083  8972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.051s	sys 0.109s
I20260709 13:03:20.350746  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 613) took 4285 ms. Trace:
W20260709 13:03:20.392690  8972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.051s	sys 0.109s
I20260709 13:03:20.392760  8256 rpcz_store.cc:276] 0709 13:03:16.065478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.066912 (+  1434us) service_pool.cc:224] Handling call
0709 13:03:16.084790 (+ 17878us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:19.402489 (+3317699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.350479 (+947990us) spinlock_profiling.cc:243] Waited 3.32 s on lock 0x7b5400076cd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:20.350680 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7627347200}
W20260709 13:03:20.393337  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 638342us
I20260709 13:03:20.048394 12611 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.011338 12662 raft_consensus.cc:3055] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:03:20.364483 12666 log.cc:927] Time spent T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.377s	user 0.000s	sys 0.000s
I20260709 13:03:20.035931 12604 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:19.917599 12659 consensus_queue.cc:1059] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.400s
I20260709 13:03:20.034482 12508 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:20.460101  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 1.20274e+06us
I20260709 13:03:20.460958 12636 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:20.384459 12296 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:20.461751 12296 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.462117 12296 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:20.465725 12659 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:20.379416 12633 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:20.382443 12488 raft_consensus.cc:3055] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:20.383307 12582 raft_consensus.cc:3055] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:03:20.383486 12658 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 13:03:20.384158  8369 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.028s	sys 0.109s
W20260709 13:03:20.495373  8369 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.028s	sys 0.109s
W20260709 13:03:20.495734  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 718248us
I20260709 13:03:20.497208  8369 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:20.043551 12479 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:19.987023 12615 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:20.276237 12230 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:20.276752 12667 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:20.013911 12610 consensus_queue.cc:1059] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.371s
I20260709 13:03:20.446379  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:20.446677  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:20.509963  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 159530us
I20260709 13:03:20.510633  8252 raft_consensus.cc:2376] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11. Re-sending same reply.
I20260709 13:03:20.446440  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:20.511989  8295 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:20.451198  8974 leader_election.cc:336] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Timed out: RequestConsensusVote RPC to 127.6.255.3:37367 timed out after 1.623s (SENT)
W20260709 13:03:20.513238  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 62095.1us
W20260709 13:03:20.385231 12620 consensus_meta.cc:287] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.063s	user 0.010s	sys 0.006s
I20260709 13:03:20.457831  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:20.495985 12501 consensus_meta.cc:287] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.955s	user 0.009s	sys 0.001s
I20260709 13:03:20.458952 12533 consensus_queue.cc:237] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:20.482381  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.424439 12611 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:20.487895 12582 raft_consensus.cc:740] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:20.500099 12615 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:20.449771  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.424834 12524 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:20.458483  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:20.481889 12508 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:20.489873 12450 consensus_meta.cc:287] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.940s	user 0.008s	sys 0.002s
W20260709 13:03:20.524143  8974 leader_election.cc:336] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Timed out: RequestConsensusVote RPC to 127.6.255.3:37367 timed out after 1.695s (SENT)
I20260709 13:03:20.563395  8974 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:20.488070 12488 raft_consensus.cc:740] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:20.518527  8253 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:20.555275  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.456911 12662 raft_consensus.cc:740] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:20.538574  8249 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:20.506384 12654 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:20.575580  8974 leader_election.cc:336] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Timed out: RequestConsensusVote RPC to 127.6.255.3:37367 timed out after 1.742s (SENT)
W20260709 13:03:20.529928 12469 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.065s	user 0.033s	sys 0.054s
W20260709 13:03:20.603502 12469 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.139s	user 0.038s	sys 0.067s
W20260709 13:03:20.616760  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 591879us
W20260709 13:03:20.564508 12491 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.004s	sys 0.027s
W20260709 13:03:20.571998 12418 consensus_meta.cc:287] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.943s	user 0.007s	sys 0.002s
I20260709 13:03:20.629323 12679 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:20.570276 12633 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:20.532497 12501 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:20.641757 12491 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.006s	sys 0.028s
I20260709 13:03:20.651032 12678 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:20.560599 12667 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 13:03:20.642812 12676 raft_consensus.cc:3055] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:20.670248  8249 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:03:20.603766 12582 consensus_queue.cc:260] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:20.674077  8222 leader_election.cc:336] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Timed out: RequestConsensusVote RPC to 127.6.255.3:37367 timed out after 1.566s (SENT)
W20260709 13:03:20.674625  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.601590 12508 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:20.599670 12450 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:20.676328  8253 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
I20260709 13:03:20.663997 12534 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:20.562076 12635 consensus_queue.cc:237] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:20.651197 12679 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:20.701774  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 201122us
W20260709 13:03:20.704072  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 189741us
W20260709 13:03:20.677402 12312 consensus_meta.cc:287] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.776s	user 0.008s	sys 0.001s
I20260709 13:03:20.707072  8369 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:20.710186  8220 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:20.648250 12418 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:20.676764 12676 raft_consensus.cc:740] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:20.710795  8256 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:03:20.714931  8220 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:20.712868  8253 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:20.703601 12582 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:20.718577  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.699030 12670 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:20.673552 12583 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:03:20.718194 12275 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:20.715811 12633 consensus_queue.cc:1059] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 7.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:03:20.746215 12683 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:20.704635 12488 consensus_queue.cc:260] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:20.741689 12275 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.749907 12275 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:20.752722 12545 consensus_meta.cc:287] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 1.040s	user 0.009s	sys 0.000s
I20260709 13:03:20.758685 12508 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:20.759414 12321 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:20.759802 12524 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:20.761234 12501 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:20.730458 12656 log.cc:927] Time spent T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:03:20.762018 12501 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:20.764559 12450 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:20.714221 12312 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:20.767953 12312 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:20.769479 12677 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
W20260709 13:03:20.775064 12644 log.cc:927] Time spent T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 1.049s	user 0.012s	sys 0.012s
W20260709 13:03:20.778262 12651 log.cc:927] Time spent T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 1.061s	user 0.010s	sys 0.008s
W20260709 13:03:20.778525  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.779889  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 579) took 4546 ms. Trace:
I20260709 13:03:20.711340 12491 consensus_queue.cc:1059] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:03:20.782019  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2159) took 1676 ms. Trace:
I20260709 13:03:20.782269 12491 raft_consensus.cc:3055] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:20.782823 12491 raft_consensus.cc:740] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:20.780267  8332 rpcz_store.cc:276] 0709 13:03:16.233692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.244486 (+3010794us) service_pool.cc:224] Handling call
0709 13:03:19.279825 (+ 35339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:19.304934 (+ 25109us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:19.304979 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:19.335913 (+ 30934us) log.cc:844] Serialized 26 byte log entry
0709 13:03:19.521263 (+185350us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:19.521319 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.521409 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:20.777698 (+1256289us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:20.779602 (+  1904us) raft_consensus.cc:1685] finished
0709 13:03:20.779625 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.779828 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":26142,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":28499,"spinlock_wait_cycles":34276096,"thread_start_us":135094,"threads_started":1,"wal-append.queue_time_us":342116}
I20260709 13:03:20.724282  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2398) took 4775 ms. Trace:
I20260709 13:03:20.785040  8327 rpcz_store.cc:276] 0709 13:03:15.949214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.552187 (+2602973us) service_pool.cc:224] Handling call
0709 13:03:18.552423 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:19.338392 (+785969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.724050 (+1385658us) spinlock_profiling.cc:243] Waited 749 ms on lock 0x7b5400365454. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:20.724221 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1722997504}
I20260709 13:03:20.715852 12534 consensus_queue.cc:1059] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
W20260709 13:03:20.725885 12674 log.cc:927] Time spent T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 13:03:20.801882  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:20.802847  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:20.720132 12539 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:20.730544 12482 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:20.731145 12681 log.cc:927] Time spent T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:03:20.671622 12672 consensus_queue.cc:1059] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 13:03:20.814510  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:20.815096  8400 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:20.815201  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:20.720584  8369 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:20.736328  8220 leader_election.cc:336] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.566s (SENT)
I20260709 13:03:20.817936  8220 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:20.806346 12595 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:03:20.742347  8143 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.860s	user 0.057s	sys 0.087s
I20260709 13:03:20.818542 12595 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:20.710498 12548 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:20.782464  8326 rpcz_store.cc:276] 0709 13:03:19.105492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.568749 (+463257us) service_pool.cc:224] Handling call
0709 13:03:19.569149 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:19.686948 (+117799us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:19.687138 (+   190us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:19.691193 (+  4055us) log.cc:844] Serialized 26 byte log entry
0709 13:03:20.781418 (+1090225us) spinlock_profiling.cc:243] Waited 4.13 ms on lock 0x7b1800392740. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88283664bc 00007f88256b448b 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e0d1
0709 13:03:20.781603 (+   185us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:20.781643 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.781714 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:20.781758 (+    44us) raft_consensus.cc:1685] finished
0709 13:03:20.781769 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.781961 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95320,"raft.queue_time_us":166413,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":8985,"spinlock_wait_cycles":9508736,"thread_start_us":262777,"threads_started":2,"wal-append.queue_time_us":297184}
I20260709 13:03:20.801064  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2404) took 4842 ms. Trace:
I20260709 13:03:20.825690  8330 rpcz_store.cc:276] 0709 13:03:15.959010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.228106 (+3269096us) service_pool.cc:224] Handling call
0709 13:03:19.246879 (+ 18773us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:19.269416 (+ 22537us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:19.269480 (+    64us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:19.282633 (+ 13153us) log.cc:844] Serialized 52 byte log entry
0709 13:03:19.499774 (+217141us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:19.514950 (+ 15176us) log.cc:844] Serialized 14 byte log entry
0709 13:03:19.760272 (+245322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.760364 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:20.774501 (+1014137us) spinlock_profiling.cc:243] Waited 2.53 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:20.784391 (+  9890us) raft_consensus.cc:1685] finished
0709 13:03:20.800787 (+ 16396us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.800938 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38435,"prepare.queue_time_us":14850,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":14791,"raft.queue_time_us":216668,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":44821,"spinlock_wait_cycles":5815680,"thread_start_us":427577,"threads_started":2,"wal-append.queue_time_us":405880}
W20260709 13:03:20.788828 12243 consensus_meta.cc:287] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.020s	user 0.011s	sys 0.000s
W20260709 13:03:20.844544 12640 consensus_meta.cc:287] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.199s	user 0.008s	sys 0.003s
W20260709 13:03:20.824033 12606 consensus_meta.cc:287] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.250s	user 0.004s	sys 0.004s
I20260709 13:03:20.848450 12604 raft_consensus.cc:3055] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:20.848860 12539 raft_consensus.cc:697] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:20.849880 12539 consensus_queue.cc:237] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:20.851107 12634 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:20.852548 12684 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:20.853672  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 117337us
I20260709 13:03:20.812250  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:20.697917 12662 consensus_queue.cc:260] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:20.854642  8405 raft_consensus.cc:2376] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11. Re-sending same reply.
I20260709 13:03:20.813141  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:20.825258  8406 raft_consensus.cc:1240] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:20.811203  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:20.725024 12296 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:20.833945  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:20.806502 12501 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:20.834412 12689 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:03:20.838524 12607 consensus_meta.cc:287] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 1.125s	user 0.007s	sys 0.003s
I20260709 13:03:20.838199 12676 consensus_queue.cc:260] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:20.841805  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:20.837308 12643 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:03:20.842911 12642 consensus_meta.cc:287] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 1.138s	user 0.009s	sys 0.001s
I20260709 13:03:20.753149 12545 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:20.823113 12491 consensus_queue.cc:260] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:03:20.809715  8403 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:39308
I20260709 13:03:20.848310  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:20.865553  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:20.865784  8403 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:20.867440  8403 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:20.841847  9007 raft_consensus.cc:1240] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
W20260709 13:03:20.819800  8222 leader_election.cc:336] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Timed out: RequestConsensusVote RPC to 127.6.255.3:37367 timed out after 1.713s (SENT)
I20260709 13:03:20.853132 12684 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:20.861218  8401 raft_consensus.cc:2410] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:03:20.868744  9007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 889) took 1755 ms. Trace:
I20260709 13:03:20.865214 12676 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:20.873054 12691 consensus_queue.cc:1048] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:20.865396 12642 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:20.875823 12642 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:20.863601 12643 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.876801 12643 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:20.869170  9007 rpcz_store.cc:276] 0709 13:03:19.113719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.113854 (+   135us) service_pool.cc:224] Handling call
0709 13:03:19.114171 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:20.841689 (+1727518us) spinlock_profiling.cc:243] Waited 54.5 ms on lock 0x7b54006dd920. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f8828735dc0 00007f88287c01c0 00007f88287c1231 00007f88287bdd8f 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c
0709 13:03:20.868474 (+ 26785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.868673 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":125345024}
I20260709 13:03:20.880784 12275 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:20.881747  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:20.882642  8253 raft_consensus.cc:2393] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 10: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:20.882943  8398 raft_consensus.cc:2393] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:20.857770 12392 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:20.848878 12604 raft_consensus.cc:740] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:03:20.894155  8220 leader_election.cc:336] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.731s (SENT)
W20260709 13:03:20.895143  8220 leader_election.cc:336] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.737s (SENT)
I20260709 13:03:20.896880  8397 raft_consensus.cc:2376] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11. Re-sending same reply.
W20260709 13:03:20.897694  8220 leader_election.cc:336] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.762s (SENT)
I20260709 13:03:20.898336  8220 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:20.868669  8222 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:20.890022 12491 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:20.892472  8405 pending_rounds.cc:85] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 7
I20260709 13:03:20.909107  8405 pending_rounds.cc:107] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:03:20.909520  8405 raft_consensus.cc:2889] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:20.863504 12689 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:20.913586 12611 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:20.914805 12684 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:20.857969 12683 consensus_queue.cc:1059] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:03:20.916036 12611 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:20.916998 12679 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:20.917965 12321 raft_consensus.cc:3055] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:20.918367 12321 raft_consensus.cc:740] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:20.918612 12595 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:20.919041  8403 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:20.919274 12321 consensus_queue.cc:260] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:20.920344  8406 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 10. Current term is 12. Ops: []
I20260709 13:03:20.882254 12692 consensus_queue.cc:1048] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:03:20.921669 12275 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:20.887945 12479 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:03:20.924813  8248 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2009) took 1981 ms (client timeout 1714 ms). Trace:
W20260709 13:03:20.925885  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 106093us
W20260709 13:03:20.925285  8248 rpcz_store.cc:269] 0709 13:03:18.943552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.020822 (+ 77270us) service_pool.cc:224] Handling call
0709 13:03:20.924606 (+1903784us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b5400309654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:20.924734 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":519021952}
I20260709 13:03:20.928990 12545 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:20.929713  8222 leader_election.cc:336] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Timed out: RequestConsensusVote RPC to 127.6.255.3:37367 timed out after 1.731s (SENT)
I20260709 13:03:20.930508  8222 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:20.931457 12582 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:20.927103 12321 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:20.934864 12593 consensus_meta.cc:287] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.544s	user 0.006s	sys 0.002s
I20260709 13:03:20.927860 12232 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:20.935585 12418 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:20.935359 12593 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:20.936161 12689 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:20.937907 12243 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:20.940527  9002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 865) took 1680 ms. Trace:
W20260709 13:03:20.940757  8220 leader_election.cc:336] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.793s (SENT)
I20260709 13:03:20.857478 12501 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:20.940971  9002 rpcz_store.cc:276] 0709 13:03:19.260644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.623479 (+362835us) service_pool.cc:224] Handling call
0709 13:03:19.645899 (+ 22420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:19.854688 (+208789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.940271 (+1085583us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b54001b3fd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:20.940462 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":465299200}
I20260709 13:03:20.943485 12501 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:20.944226  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:03:20.945842  8257 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2005) took 2606 ms (client timeout 3201 ms). Trace:
I20260709 13:03:20.946501  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:03:20.946292  8257 rpcz_store.cc:269] 0709 13:03:18.339012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.504133 (+165121us) service_pool.cc:224] Handling call
0709 13:03:20.945669 (+2441536us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54000762d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:20.945779 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2750350720}
I20260709 13:03:20.947389  8404 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:20.949815 12671 log.cc:927] Time spent T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.355s	user 0.003s	sys 0.000s
I20260709 13:03:20.949729  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:20.950229 12637 raft_consensus.cc:3055] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:20.950699  8397 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:20.890347 12312 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:20.841136 12615 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:20.951668  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:20.956699  9008 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:03:20.957491  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:20.958274  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:20.964852  8397 raft_consensus.cc:2393] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 9: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:20.965032  9007 raft_consensus.cc:2410] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:03:20.958163  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:20.965904  9009 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:20.962296  8257 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:20.966336  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:20.967352  9001 raft_consensus.cc:2368] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13.
I20260709 13:03:20.968156  8403 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:20.964108  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:20.969103  9008 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
W20260709 13:03:20.970242  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 156581us
I20260709 13:03:20.972144  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:20.973245  8404 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:20.974710  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:20.976297  8220 leader_election.cc:336] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.877s (SENT)
W20260709 13:03:20.977895  8220 leader_election.cc:336] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.910s (SENT)
I20260709 13:03:20.978408  8220 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:20.984102  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:20.984933  8249 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:20.966511  8222 leader_election.cc:336] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Timed out: RequestConsensusVote RPC to 127.6.255.3:37367 timed out after 1.737s (SENT)
I20260709 13:03:20.989146  8222 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:20.990911  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:20.992113  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:20.993058  9000 raft_consensus.cc:2393] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 15: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:20.980890  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:20.832731  8400 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:21.066039  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:20.950716 12637 raft_consensus.cc:740] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:21.054797  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:21.055574  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:21.057233  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:21.058830  8402 raft_consensus.cc:2393] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 10: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:21.098384  9001 raft_consensus.cc:2368] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11.
I20260709 13:03:21.058672  8397 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:20.845186 12640 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:20.863940 12607 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:21.059363  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 359665us
I20260709 13:03:21.109455  9000 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:20.953923 12319 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:03:21.066079  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 92169.6us
I20260709 13:03:21.069589  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:21.110816  8372 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:21.111526  8253 raft_consensus.cc:2393] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 15: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:21.070533  9007 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:03:21.059391  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:21.114322  8257 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 14: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:21.079881  8249 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
I20260709 13:03:21.080883  8248 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10. Ops: []
W20260709 13:03:21.082257  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 115745us
W20260709 13:03:21.119060  8222 leader_election.cc:336] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Timed out: RequestConsensusVote RPC to 127.6.255.3:37367 timed out after 1.891s (SENT)
I20260709 13:03:21.119726  8222 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:20.993593 12312 raft_consensus.cc:697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:03:21.051450  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 73571.6us
W20260709 13:03:21.123273  8220 leader_election.cc:336] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.945s (SENT)
W20260709 13:03:21.100126 12580 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.043s	user 0.002s	sys 0.012s
I20260709 13:03:21.099548  9003 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:03:21.121582  8372 leader_election.cc:400] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13.
I20260709 13:03:21.058701 12692 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:21.123695  8220 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:21.109637  8974 leader_election.cc:336] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Timed out: RequestConsensusVote RPC to 127.6.255.3:37367 timed out after 2.000s (SENT)
I20260709 13:03:21.070230 12508 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:21.125222  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.919130 12604 consensus_queue.cc:260] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:21.129473  9003 pending_rounds.cc:85] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: Aborting all ops after (but not including) 7
I20260709 13:03:21.129549  8372 leader_election.cc:403] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:03:21.103515 12275 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:03:21.162763  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:21.131557  8222 leader_election.cc:336] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Timed out: RequestConsensusVote RPC to 127.6.255.3:37367 timed out after 2.228s (SENT)
W20260709 13:03:21.133077  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.067981 12593 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:21.134076  9005 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
W20260709 13:03:21.129086 12580 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.090s	user 0.002s	sys 0.012s
I20260709 13:03:21.151430  9003 pending_rounds.cc:107] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: Aborting uncommitted NO_OP operation due to leader change: 8.8
W20260709 13:03:21.157135  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:21.174176  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.173893  9003 raft_consensus.cc:2889] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:21.157230  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 44373.9us
I20260709 13:03:21.175324  8372 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:21.177574  8372 leader_election.cc:400] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11.
I20260709 13:03:21.177920  8372 leader_election.cc:403] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:03:21.155097 12638 log.cc:927] Time spent T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 1.681s	user 0.011s	sys 0.028s
I20260709 13:03:21.159260  9006 raft_consensus.cc:1240] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:21.159960  8397 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:03:21.121160 12610 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:21.163379  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:21.186889  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.160265  9000 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:21.170837  8253 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:03:21.132952 12583 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.156145 12243 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:21.139091 12679 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:21.156877 12582 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.138048 12693 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:21.125830 12501 consensus_queue.cc:1059] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:03:21.154616 12595 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:03:21.187394  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:21.157619  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 437006us
I20260709 13:03:21.127142 12528 consensus_queue.cc:1059] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:03:21.128715 12319 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:21.126924 12689 consensus_queue.cc:1059] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:03:21.155949 12508 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:21.140089 12643 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:21.252519  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 48420.1us
I20260709 13:03:21.159255 12534 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:21.283739  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:21.169082 12684 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:21.310225 12706 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:03:21.311249 12508 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:21.160851 12642 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:21.231024  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.291220  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:21.314890  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 137481us
I20260709 13:03:21.230986  9004 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:21.291100  8222 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:21.303117 12588 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.306025 12643 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:21.277694 12548 consensus_queue.cc:1059] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
W20260709 13:03:21.275648 12700 log.cc:927] Time spent T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:03:21.184888 12604 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:21.159578 12636 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.149s
I20260709 13:03:21.165122 12230 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.143s
I20260709 13:03:21.169454 12637 consensus_queue.cc:260] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 7.10, Last appended by leader: 10, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:21.232489 12312 consensus_queue.cc:237] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:21.317205  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:21.184053 12392 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:03:21.432834  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.231978 12691 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.320564 12501 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:03:21.330020  8974 leader_election.cc:336] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Timed out: RequestConsensusVote RPC to 127.6.255.3:37367 timed out after 2.204s (SENT)
I20260709 13:03:21.466305  8222 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:03:21.380774  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.466614  8974 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:21.466522  8254 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 13:03:21.467864  8254 pending_rounds.cc:85] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 10
I20260709 13:03:21.468209  8254 pending_rounds.cc:107] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 10.11
I20260709 13:03:21.468549  8254 raft_consensus.cc:2889] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:21.460891  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:21.342839 12508 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:21.331121 12607 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:21.282863 12612 consensus_queue.cc:1059] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.171s
I20260709 13:03:21.473045  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 642) took 1062 ms. Trace:
W20260709 13:03:21.490162  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 160164us
W20260709 13:03:21.497579  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:21.498159  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.442313 12243 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:21.293277 12595 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:21.472162 12702 log.cc:927] Time spent T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:03:21.520290  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 204510us
I20260709 13:03:21.486342  8252 rpcz_store.cc:276] 0709 13:03:20.410893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.511433 (+100540us) service_pool.cc:224] Handling call
0709 13:03:20.522463 (+ 11030us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:20.675838 (+153375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:20.675866 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:20.708381 (+ 32515us) log.cc:844] Serialized 26 byte log entry
0709 13:03:20.953715 (+245334us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:20.953772 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.954047 (+   275us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:21.472673 (+518626us) raft_consensus.cc:1685] finished
0709 13:03:21.472694 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:21.472971 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":120524,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":7274,"thread_start_us":362086,"threads_started":2,"wal-append.queue_time_us":353819}
I20260709 13:03:21.293706 12679 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:21.316108 12582 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:21.315815 12583 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:21.390470 12642 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:21.533969  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.531797  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:21.472363 12501 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.493266 12637 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:21.499590 12718 raft_consensus.cc:3055] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:21.389400 12588 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:21.547720  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 232412us
I20260709 13:03:21.562307  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:21.562283  8372 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:21.493367 12643 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:03:21.527129 12615 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:21.293915 12693 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:21.597668  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.532444 12533 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:21.601495  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:03:21.532625 12660 consensus_meta.cc:287] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.568s	user 0.012s	sys 0.000s
I20260709 13:03:21.601140  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:21.546802 12607 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.602602  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:21.621214  8406 raft_consensus.cc:2393] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 9: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:21.532171 12296 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:21.586052 12501 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:21.562556 12612 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.540510 12243 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:21.590091 12640 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:21.526840 12508 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:21.623092  8323 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:03:21.623698  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:21.660277  8330 raft_consensus.cc:2368] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13.
I20260709 13:03:21.624312  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:21.661180  8327 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:21.628757  8255 raft_consensus.cc:1275] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:21.585843 12718 raft_consensus.cc:740] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:21.635845  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:21.466054 12691 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:21.646765  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:21.670027  9003 raft_consensus.cc:2393] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 13: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:21.648439  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:21.671123  9004 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 14: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:21.650926  8252 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:21.658229  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:03:21.663416 12714 log.cc:927] Time spent T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 13:03:21.663516 12689 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:21.682480  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:21.683516  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:21.684574  8406 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 14: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:21.664274 12508 raft_consensus.cc:697] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:21.665690  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:21.666169  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.646039  8324 raft_consensus.cc:1275] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:21.692133  8329 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:21.674329  8255 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:03:21.689955  8328 raft_consensus.cc:2393] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 10: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:21.688134  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:21.694265  8406 raft_consensus.cc:2393] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 15: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:21.695317  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:21.607914 12719 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:03:21.700174  8974 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:21.609684 12724 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 13:03:21.696115  8331 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:21.626417 12660 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:21.702651  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:21.704262  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:21.714025  8327 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:21.716185  8406 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 12: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:21.718295  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:21.719691  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:21.638206 12607 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:21.728214  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:21.707683 12691 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:21.731705  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:21.733801  8331 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:21.731102  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:21.654351 12612 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:21.741598  8327 raft_consensus.cc:1240] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:21.703634 12580 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:03:21.756595 12321 consensus_meta.cc:287] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.824s	user 0.006s	sys 0.004s
W20260709 13:03:21.757702 12232 consensus_meta.cc:287] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.822s	user 0.011s	sys 0.000s
I20260709 13:03:21.767716  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:21.771435 12707 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.772109 12707 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:21.774508 12691 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.774806 12707 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:21.775027 12691 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:21.775629 12679 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:21.776095  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:21.776813  9006 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:21.777221 12691 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:21.778352  8372 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:21.719878 12643 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:21.780459  8331 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:21.765864 12508 consensus_queue.cc:237] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:21.767979  8251 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:21.763087  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:21.769215  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:21.791394 12595 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:21.783962 12718 consensus_queue.cc:260] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:21.792821 12718 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:21.732124 12615 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.794781  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:21.804214  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:21.805236  8404 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.008s	sys 0.026s
W20260709 13:03:21.805650  8404 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.008s	sys 0.026s
I20260709 13:03:21.807920  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:21.808749  8331 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:21.811640  8251 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:03:21.786934 12679 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:21.820652  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:21.795712 12684 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13.
I20260709 13:03:21.838429  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:21.801364 12296 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:21.841141  8401 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:21.848063  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:21.847584 12642 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:21.854312 12296 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:21.855952 12583 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:21.856999 12296 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:21.857194 12660 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:21.858560 12607 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:21.854884 12612 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:21.822257 12679 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.868510  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:21.872282  9005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 769) took 1984 ms. Trace:
I20260709 13:03:21.874590 12662 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:21.879230  9005 rpcz_store.cc:276] 0709 13:03:19.888323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.888446 (+   123us) service_pool.cc:224] Handling call
0709 13:03:19.910668 (+ 22222us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:21.165133 (+1254465us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.872006 (+706873us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b540054c9d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:21.872217 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2812321664}
W20260709 13:03:21.875957  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.806912 12615 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:21.878800  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.882620  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:21.889282  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:21.890856  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:21.883181  8331 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:21.870767 12679 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:21.899291  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:21.899801  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:21.900137  8324 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: [7.7-12.10]
I20260709 13:03:21.901458  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:21.903365  8327 raft_consensus.cc:2393] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 13: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:21.906502  8369 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:21.913417  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:21.914225  8331 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:21.914696  8369 leader_election.cc:400] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14.
I20260709 13:03:21.915030  8369 leader_election.cc:403] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:03:21.920166  8328 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:03:21.923561  8327 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:03:21.925261  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:21.925872  8323 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:21.929844  8327 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: []
W20260709 13:03:21.930943  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.932399  8369 leader_election.cc:400] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13.
I20260709 13:03:21.938313  8972 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:21.941919  8295 leader_election.cc:336] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.714s (SENT)
I20260709 13:03:21.899375 12684 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.949988  8323 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
I20260709 13:03:21.962942  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:21.963563  8323 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 7.
I20260709 13:03:21.963699  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:21.964725  8327 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 14: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:21.913689 12607 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.935721  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:21.902351 12583 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.913558 12604 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.970930  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:21.959527  8331 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
W20260709 13:03:21.962598 12491 consensus_meta.cc:287] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 1.011s	user 0.006s	sys 0.006s
I20260709 13:03:21.932004 12689 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:21.968261  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:21.914119 12716 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:21.973408  8369 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:21.931877 12642 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14.
I20260709 13:03:21.961984 12545 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:21.976903  8327 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:21.979184  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:21.979980  8323 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 15: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:21.979990  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:21.982313  8404 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 7.
I20260709 13:03:21.973043 12719 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:21.945071 12726 consensus_queue.cc:1059] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:03:21.987895  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:21.982579  8369 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:21.972636 12610 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:21.992108  8331 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:03:21.992383 12501 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:21.996448  8397 raft_consensus.cc:2368] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12.
I20260709 13:03:21.997422 12501 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:21.977205 12383 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:21.998132 12383 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:21.998646  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:21.999353  8397 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 14: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:22.000098  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:22.000466 12383 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:21.757143  8143 maintenance_manager.cc:643] P b3bd0e8d7a084e649e387e729c0adabc: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 11.975s	user 0.254s	sys 0.200s Metrics: {"bytes_written":30337,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":606383,"dirs.run_cpu_time_us":471,"dirs.run_wall_time_us":13815,"drs_written":1,"lbm_read_time_us":38904,"lbm_reads_10-100_ms":2,"lbm_reads_lt_1ms":2,"lbm_write_time_us":472409,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":11,"lbm_writes_gt_100_ms":1,"lbm_writes_lt_1ms":21,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":588358,"threads_started":2,"update_count":20,"wal-append.queue_time_us":815597,"wal-append.run_cpu_time_us":548,"wal-append.run_wall_time_us":1055584}
I20260709 13:03:22.004076  8144 maintenance_manager.cc:419] P b3bd0e8d7a084e649e387e729c0adabc: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9271 bytes on disk
I20260709 13:03:22.001088  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:22.004815  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 648) took 1017 ms. Trace:
I20260709 13:03:22.007664  8143 maintenance_manager.cc:643] P b3bd0e8d7a084e649e387e729c0adabc: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":292,"lbm_reads_lt_1ms":4}
I20260709 13:03:22.010069  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:22.011219 12321 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.011886 12321 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:21.976347 12707 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:22.013798 12321 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:22.014039 12707 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:22.011375  8399 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:22.014660 12707 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:22.011086  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:22.015892  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:22.016863  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:22.017488  8397 raft_consensus.cc:2393] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 12: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:21.960855 12724 consensus_queue.cc:1048] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:03:22.018136  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:22.019961  8397 raft_consensus.cc:2393] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 12: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:22.005340  8253 rpcz_store.cc:276] 0709 13:03:20.987176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.189547 (+202371us) service_pool.cc:224] Handling call
0709 13:03:21.233728 (+ 44181us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:21.252723 (+ 18995us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.252763 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:21.286012 (+ 33249us) log.cc:844] Serialized 26 byte log entry
0709 13:03:21.876963 (+590951us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:21.877006 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.877075 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.004639 (+127564us) raft_consensus.cc:1685] finished
0709 13:03:22.004655 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.004765 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18480,"raft.queue_time_us":59469,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":23479,"thread_start_us":434856,"threads_started":1,"wal-append.queue_time_us":689702}
I20260709 13:03:21.966760 12633 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.015s
W20260709 13:03:22.031448  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.980082 12722 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:22.027971  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:22.026382  8327 raft_consensus.cc:3055] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:03:22.043756  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 61232.2us
I20260709 13:03:22.047036  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:22.047348  8369 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:21.969523 12684 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:21.983278 12689 raft_consensus.cc:697] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:21.974862 12604 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:22.047822  8257 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:21.974845 12583 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:22.042907  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:22.019590 12719 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.051432  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:21.973927 12607 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:22.065811  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:22.070348 12678 raft_consensus.cc:3055] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:22.057996  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:21.988026 12491 consensus_queue.cc:1059] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:03:22.076732  8397 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 14: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:22.052886 12501 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:22.074664  8369 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:22.077767  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:22.074699  8257 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:22.046670  8327 raft_consensus.cc:740] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:22.084450  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:22.062772 12719 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:22.085522  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:22.090866  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:22.092257  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:22.092952  8253 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:22.085394  8257 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:22.072376 12722 raft_consensus.cc:697] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:22.076184 12678 raft_consensus.cc:740] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:22.075708 12620 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:22.081995 12612 raft_consensus.cc:3055] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:22.077889 12716 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:22.096218  8369 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:22.085058  8327 consensus_queue.cc:260] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:22.087563 12706 consensus_meta.cc:287] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.775s	user 0.005s	sys 0.008s
I20260709 13:03:22.106884  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:22.110463  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:22.108601 12612 raft_consensus.cc:740] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:22.111321  8257 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:22.111903 12612 consensus_queue.cc:260] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:22.112869 12612 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:22.114636 12491 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:22.115163 12491 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:22.115612 12491 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:22.118458 12706 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11.
I20260709 13:03:22.119227 12582 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:22.119378 12706 consensus_queue.cc:1059] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:22.121558  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:22.122716  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:22.123664 12684 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:22.124516 12684 raft_consensus.cc:3055] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:22.122716  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:22.124934 12684 raft_consensus.cc:740] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:22.126078  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:22.126855  8248 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 14: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:22.126955  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:22.127574  8257 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:22.127681  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:22.128402  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:22.128713  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:22.129108  8257 raft_consensus.cc:2393] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 11: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:03:22.101207 12637 consensus_meta.cc:287] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.001s
I20260709 13:03:22.103327 12724 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:22.136656  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:22.147017  8257 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 15: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:22.149684  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:22.156992  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:22.157881  8248 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:22.157785  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:22.158828  8257 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:03:22.109501 12604 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:22.130019 12615 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:22.163105  8257 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:03:22.170152  8406 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:22.183243  8255 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:22.183570 12707 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:22.185827  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:22.186450  8332 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:22.193171  8332 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:22.194200  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:22.194351 12583 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:22.132325 12620 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:22.197247 12707 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:22.199085 12683 raft_consensus.cc:3055] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:22.199492 12683 raft_consensus.cc:740] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:22.197084 12620 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:22.202039  8253 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:22.145344 12724 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:03:22.212589  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 269738us
I20260709 13:03:22.219916  8248 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:22.200253 12683 consensus_queue.cc:260] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:22.220723  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.209820  8253 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:03:22.223073  8295 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:22.232896  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:22.214493 12724 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:22.238047  8256 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49220 (request call id 636) took 1840 ms (client timeout 2073 ms). Trace:
I20260709 13:03:22.132184 12689 consensus_queue.cc:237] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:22.249509  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 135358us
W20260709 13:03:22.221062 12616 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.499s	user 0.049s	sys 0.049s
I20260709 13:03:22.240612  8255 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:03:22.218322 12620 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.253655  8253 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
W20260709 13:03:22.248246  8256 rpcz_store.cc:269] 0709 13:03:20.397734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.397868 (+   134us) service_pool.cc:224] Handling call
0709 13:03:22.237872 (+1840004us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b540033ed54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:22.237989 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":460881792}
I20260709 13:03:22.245596 12716 raft_consensus.cc:697] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:22.246465  8332 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:22.257613  8972 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:22.219925 12665 consensus_queue.cc:1059] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:03:22.241789 12683 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:22.251138 12616 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.529s	user 0.050s	sys 0.050s
I20260709 13:03:22.247826 12659 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:03:22.329280  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 71731.5us
I20260709 13:03:22.338272  8972 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:22.245298 12607 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:22.344574  8400 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:22.340245  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:22.345912  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:22.346135  8972 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:22.347170  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:22.347859  8332 raft_consensus.cc:2393] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:22.348212 12637 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:22.348676  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:22.349076 12637 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:22.342514 12633 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:22.343145 12659 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:22.214496 12722 consensus_queue.cc:237] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:22.340545 12716 consensus_queue.cc:237] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:22.213201 12678 consensus_queue.cc:260] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:22.257831 12620 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:22.359934 12501 raft_consensus.cc:697] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:22.354399 12659 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:22.355190 12633 raft_consensus.cc:697] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 15 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:22.337567 12312 consensus_queue.cc:1059] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:03:22.367955 12637 raft_consensus.cc:697] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:03:22.451357 12737 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
W20260709 13:03:22.453373 12739 log.cc:927] Time spent T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:03:22.458377 12637 consensus_queue.cc:237] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:22.459125 12633 consensus_queue.cc:237] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:22.461722 12501 consensus_queue.cc:237] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:22.470835 12491 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:22.474072 12741 log.cc:927] Time spent T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 13:03:22.475132  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:22.532099 12662 consensus_meta.cc:287] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.653s	user 0.007s	sys 0.004s
I20260709 13:03:22.532882 12730 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.533473 12730 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:22.534996 12679 consensus_meta.cc:287] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.608s	user 0.011s	sys 0.000s
I20260709 13:03:22.535573 12679 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:22.539260 12620 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:22.547659 12491 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:22.489894 12719 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:22.622838 12679 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:22.624696 12719 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:22.628703 12724 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:22.634181 12724 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:21.362345  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:03:22.689096  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:22.690387  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:22.693329  8252 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.002s	sys 0.022s
W20260709 13:03:22.693960  8252 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.003s	sys 0.022s
I20260709 13:03:22.699144  8248 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:03:22.701211  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:22.703996  8253 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 11. Current term is 12. Ops: [9.9-11.10]
I20260709 13:03:22.704191  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:22.705457  8248 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:03:22.705621  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:22.706758  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:22.707015  8253 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 12: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:22.707783  8248 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
I20260709 13:03:22.708525  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:22.708982  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:22.709676  8253 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:03:22.710160  8248 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:22.711122  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:22.713357  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:22.716081  8256 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: [8.7-10.8]
W20260709 13:03:22.718950  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:22.721312  8257 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.012s	sys 0.035s
W20260709 13:03:22.721781  8257 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.012s	sys 0.035s
I20260709 13:03:22.729118  8253 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:22.734390  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:22.746824  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:22.747422  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:22.747812  8328 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:22.748425  8326 raft_consensus.cc:2393] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:22.749181  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:22.750392  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:22.750813  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:22.751587  8326 raft_consensus.cc:2393] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 10: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:22.753208  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:22.754587  8326 raft_consensus.cc:2410] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:03:22.755836  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:22.760679  8326 raft_consensus.cc:2393] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 12: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:22.751585  8328 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:22.765236  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:22.767305  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:22.773171  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:22.778687  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:22.781121  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:22.782912  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:22.784443  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:22.774442  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:22.786536  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:22.776578  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 2037) took 1119 ms. Trace:
I20260709 13:03:22.790580  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:22.789710  8252 rpcz_store.cc:276] 0709 13:03:21.657486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.687779 (+ 30293us) service_pool.cc:224] Handling call
0709 13:03:21.711027 (+ 23248us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:21.721464 (+ 10437us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:22.124924 (+403460us) spinlock_profiling.cc:243] Waited 24.8 ms on lock 0x7b18003b82a0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256ce455 00007f882c1b63c7 00007f88287c89a7 00007f88287d80bf 00007f88287d8038
0709 13:03:22.694649 (+569725us) log.cc:844] Serialized 14 byte log entry
0709 13:03:22.731379 (+ 36730us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:22.732277 (+   898us) log.cc:844] Serialized 26 byte log entry
0709 13:03:22.732469 (+   192us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:22.732508 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.732624 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.776136 (+ 43512us) raft_consensus.cc:1685] finished
0709 13:03:22.776279 (+   143us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.776498 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":372260,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":10385,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":16968,"spinlock_wait_cycles":57119488,"thread_start_us":586911,"threads_started":2,"wal-append.queue_time_us":57922}
I20260709 13:03:22.794642  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:22.795589  8257 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:22.806851  8297 connection.cc:256] RPC call timeout handler was delayed by -2.3083436399999755s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17620000000000002s for a response
I20260709 13:03:22.807973  8326 raft_consensus.cc:2410] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 12 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:03:22.811434  8404 raft_consensus.cc:2368] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 11. Current term is 12.
W20260709 13:03:22.812413  8402 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 2471) took 2003 ms (client timeout 1713 ms). Trace:
W20260709 13:03:22.823773  8402 rpcz_store.cc:269] 0709 13:03:20.809163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.809319 (+   156us) service_pool.cc:224] Handling call
0709 13:03:22.811768 (+2002449us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54003e6054. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:22.812108 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":553904896}
W20260709 13:03:22.821113  8297 connection.cc:256] RPC call timeout handler was delayed by -2.2795436399999858s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1794s for a response
I20260709 13:03:22.855618  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:22.856802  8402 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:22.855733  8257 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:22.879843 12719 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.880868 12719 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:22.614840 12659 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:22.885787  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:22.887317  8257 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:22.887655 12659 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:22.888754  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:22.889101 12718 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.889840 12718 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:22.888839 12642 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:22.894459 12642 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:22.896816 12642 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:22.898139  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:22.899487  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:22.900637 12719 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:22.904202  8257 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:22.882227 12689 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.915956  8332 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:22.917690  8402 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
I20260709 13:03:22.920357  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 652) took 1728 ms. Trace:
I20260709 13:03:22.922578  8402 rpcz_store.cc:276] 0709 13:03:21.189786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.854941 (+1665155us) service_pool.cc:224] Handling call
0709 13:03:22.918451 (+ 63510us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:22.922772  8257 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:22.924491  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:22.925179  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:22.925755  8297 connection.cc:256] RPC call timeout handler was delayed by -1.9789436399999545s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21280000000000002s for a response
W20260709 13:03:22.930178  8297 connection.cc:256] RPC call timeout handler was delayed by -1.7089436399999727s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24280000000000002s for a response
W20260709 13:03:22.933900  8398 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 2477) took 2100 ms (client timeout 2177 ms). Trace:
I20260709 13:03:22.934607  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:22.934446  8398 rpcz_store.cc:269] 0709 13:03:20.858184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.858306 (+   122us) service_pool.cc:224] Handling call
0709 13:03:22.933368 (+2075062us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b54003a1954. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:22.933683 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36544384}
I20260709 13:03:22.936230  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:22.937271  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 2075) took 1694 ms. Trace:
I20260709 13:03:22.937686  8398 rpcz_store.cc:276] 0709 13:03:21.242919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.935780 (+1692861us) service_pool.cc:224] Handling call
0709 13:03:22.937177 (+  1397us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:22.933259  8328 raft_consensus.cc:2393] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 8: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:03:22.943157  8297 connection.cc:256] RPC call timeout handler was delayed by -1.2238436399999841s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2967s for a response
W20260709 13:03:22.943552  8297 connection.cc:256] RPC call timeout handler was delayed by -1.2105107379999822s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1547s for a response
W20260709 13:03:22.943905  8297 connection.cc:256] RPC call timeout handler was delayed by -1.2024107379999691s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15560000000000002s for a response
W20260709 13:03:22.944177  8297 connection.cc:256] RPC call timeout handler was delayed by -1.0602107379999666s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1714s for a response
I20260709 13:03:22.950881  8402 raft_consensus.cc:2410] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:03:22.952597  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 2074) took 1709 ms. Trace:
I20260709 13:03:22.953542  8402 rpcz_store.cc:276] 0709 13:03:21.242006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.934120 (+1692114us) service_pool.cc:224] Handling call
0709 13:03:22.951471 (+ 17351us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:22.955260  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:22.956120 12319 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:22.957165  8402 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:03:22.959800  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:22.957954 12296 raft_consensus.cc:3055] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:22.965687  8402 raft_consensus.cc:2393] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:22.966085  8326 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:22.990574  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:22.951450 12718 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:22.992193  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:22.962986 12296 raft_consensus.cc:740] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:23.007458  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:23.008737  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:23.013146  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.015049 12296 consensus_queue.cc:260] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:22.949889 12689 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:23.030161  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 483468us
I20260709 13:03:23.035460  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:23.036382  8257 raft_consensus.cc:2376] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12. Re-sending same reply.
W20260709 13:03:23.037487  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 46493.9us
I20260709 13:03:23.008998  8326 raft_consensus.cc:2393] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 10: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:23.036804  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 2073) took 1803 ms. Trace:
I20260709 13:03:23.039775  8401 rpcz_store.cc:276] 0709 13:03:21.233187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.122471 (+889284us) service_pool.cc:224] Handling call
0709 13:03:23.036033 (+913562us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b5400277754. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:23.036247 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1750431104}
I20260709 13:03:23.043124  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:23.048831  8331 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:23.044096 12670 consensus_queue.cc:1059] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:03:23.054136 12707 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.054729 12670 raft_consensus.cc:3055] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:23.055245 12670 raft_consensus.cc:740] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:23.055106 12707 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:23.061228 12707 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:23.060964  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:23.062410  8257 raft_consensus.cc:2410] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:03:23.066403  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:23.067227  8405 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:23.068224  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:23.068743  8405 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:03:23.068300 12684 consensus_queue.cc:260] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:23.072552 12693 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:23.080152 12588 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:23.081529  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:23.083642  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:23.084368  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:23.087932 12684 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:23.088282 12758 log.cc:927] Time spent T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:03:23.088881  9003 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:23.090145  9006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 881) took 1266 ms. Trace:
I20260709 13:03:23.090535  9006 rpcz_store.cc:276] 0709 13:03:21.823566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.823721 (+   155us) service_pool.cc:224] Handling call
0709 13:03:21.824094 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:21.824496 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.824528 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:22.937000 (+1112472us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b18002eb1c0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882569f2a6 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a
0709 13:03:22.937839 (+   839us) log.cc:844] Serialized 26 byte log entry
0709 13:03:23.009801 (+ 71962us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:23.009867 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.010037 (+   170us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:23.089624 (+ 79587us) raft_consensus.cc:1685] finished
0709 13:03:23.089646 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:23.089948 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":668,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":8325,"spinlock_wait_cycles":29867264,"thread_start_us":53978,"threads_started":1,"wal-append.queue_time_us":98377}
I20260709 13:03:23.092571  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:23.091506  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:23.095338  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:23.097939  8332 raft_consensus.cc:2393] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:23.100131  8297 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:23.100485  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:23.101326  8257 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
W20260709 13:03:23.101770  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 55696.1us
I20260709 13:03:23.103034  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2442) took 1027 ms. Trace:
I20260709 13:03:23.094336 12296 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:23.056861 12670 consensus_queue.cc:260] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:23.105747  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 59918.3us
I20260709 13:03:23.103442  8332 rpcz_store.cc:276] 0709 13:03:22.076093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.021913 (+945820us) service_pool.cc:224] Handling call
0709 13:03:23.102965 (+ 81052us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:23.111909  8402 raft_consensus.cc:1240] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:23.113708  9008 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:23.114064  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:23.114916 12757 log.cc:927] Time spent T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:03:23.117935  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:23.120867  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 647) took 1934 ms. Trace:
I20260709 13:03:23.123162  8402 rpcz_store.cc:276] 0709 13:03:21.184250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.041305 (+1857055us) service_pool.cc:224] Handling call
0709 13:03:23.096578 (+ 55273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.117950 (+ 21372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.118163 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:23.126278  8331 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:23.127060  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:03:23.130640  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.131269 12670 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:23.131316  8402 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: [10.9-12.10]
I20260709 13:03:23.132109 12580 raft_consensus.cc:3055] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:23.132400  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 650) took 1947 ms. Trace:
I20260709 13:03:23.133033  8402 rpcz_store.cc:276] 0709 13:03:21.185009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.126966 (+1941957us) service_pool.cc:224] Handling call
0709 13:03:23.130766 (+  3800us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:23.132089 (+  1323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.132317 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:23.133973  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 643) took 1954 ms. Trace:
I20260709 13:03:23.134406  8404 rpcz_store.cc:276] 0709 13:03:21.179563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.060839 (+881276us) service_pool.cc:224] Handling call
0709 13:03:23.133720 (+1072881us) spinlock_profiling.cc:243] Waited 737 ms on lock 0x7b54002689d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:23.133916 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1694315904}
I20260709 13:03:23.135116  8402 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:03:23.135867  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 656) took 1932 ms. Trace:
I20260709 13:03:23.132529 12580 raft_consensus.cc:740] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:23.136294  8404 raft_consensus.cc:1240] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:23.137774 12580 consensus_queue.cc:260] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:23.139032 12726 raft_consensus.cc:3055] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:23.139202 12580 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:23.139508 12726 raft_consensus.cc:740] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:23.139786  8330 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 10. Current term is 12. Ops: []
I20260709 13:03:23.140574  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2444) took 1058 ms. Trace:
I20260709 13:03:23.141235  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 650) took 2004 ms. Trace:
I20260709 13:03:23.141749  8254 rpcz_store.cc:276] 0709 13:03:21.136970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.364081 (+227111us) service_pool.cc:224] Handling call
0709 13:03:21.364475 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:21.524902 (+160427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.140733 (+1615831us) spinlock_profiling.cc:243] Waited 97.4 ms on lock 0x7b54003b1d54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:23.141053 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":223910656}
W20260709 13:03:23.145202  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.145270  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2426) took 1073 ms. Trace:
I20260709 13:03:23.145838  8328 rpcz_store.cc:276] 0709 13:03:22.072256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.764624 (+692368us) service_pool.cc:224] Handling call
0709 13:03:23.145046 (+380422us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54002c9f54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:23.145213 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":383410432}
I20260709 13:03:23.140971  8330 rpcz_store.cc:276] 0709 13:03:22.082441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.138934 (+1056493us) service_pool.cc:224] Handling call
0709 13:03:23.139386 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.140272 (+   886us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.140475 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:23.153004  8254 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:03:23.147985  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2445) took 1048 ms. Trace:
I20260709 13:03:23.148494 12659 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:23.137141  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 657) took 1933 ms. Trace:
I20260709 13:03:23.148825  9009 raft_consensus.cc:1240] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:23.155669  8404 rpcz_store.cc:276] 0709 13:03:21.203578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.135496 (+1931918us) service_pool.cc:224] Handling call
0709 13:03:23.135955 (+   459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.136824 (+   869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.137087 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:23.154834 12659 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:23.156579  8328 rpcz_store.cc:276] 0709 13:03:22.099716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.146796 (+1047080us) service_pool.cc:224] Handling call
0709 13:03:23.147231 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.147629 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:23.147663 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:23.147714 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:23.147748 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.147817 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:23.147928 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1896,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":3002}
I20260709 13:03:23.158986  8328 raft_consensus.cc:1240] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 12. Current term is 13. Ops: [12.5-12.5]
I20260709 13:03:23.159669 12659 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:23.136315  8402 rpcz_store.cc:276] 0709 13:03:21.203346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.134054 (+1930708us) service_pool.cc:224] Handling call
0709 13:03:23.134601 (+   547us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.135543 (+   942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.135791 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:23.165802  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:23.169380  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:23.120363 12588 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:23.170938  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:23.171420  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2485) took 2268 ms. Trace:
I20260709 13:03:23.171873  8400 rpcz_store.cc:276] 0709 13:03:20.902506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.068322 (+165816us) service_pool.cc:224] Handling call
0709 13:03:21.068612 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.345102 (+1276490us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.170930 (+825828us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54003160d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:23.171250 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2933445504}
W20260709 13:03:23.172646  8220 leader_election.cc:336] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.203s (SENT)
I20260709 13:03:23.173556  9009 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:23.174276  8220 leader_election.cc:336] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.228s (SENT)
I20260709 13:03:23.174757  8220 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:23.175961  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:23.176761  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.161767  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:23.177846  8401 raft_consensus.cc:2410] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 14 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:03:23.179684  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2496) took 1500 ms. Trace:
I20260709 13:03:23.179853 12678 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:23.180079  8401 rpcz_store.cc:276] 0709 13:03:21.678948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.178738 (+1499790us) service_pool.cc:224] Handling call
0709 13:03:23.179088 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.179305 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:23.179329 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:23.179385 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:23.179420 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.179485 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:23.179611 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:23.128991  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2186) took 1181 ms. Trace:
I20260709 13:03:23.160518  8297 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:23.186920 12534 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:23.183879  8331 rpcz_store.cc:276] 0709 13:03:21.945669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.098611 (+1152942us) service_pool.cc:224] Handling call
0709 13:03:23.118716 (+ 20105us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.126642 (+  7926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.126795 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:23.190937  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.192421 12730 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:23.141000 12726 consensus_queue.cc:260] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:23.195904 12726 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:23.191231  9009 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:23.202100  8326 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 11. Ops: []
W20260709 13:03:23.220762  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.220942  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2440) took 1134 ms. Trace:
I20260709 13:03:23.246237 12689 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:23.247637  8399 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 2466) took 2446 ms (client timeout 1834 ms). Trace:
W20260709 13:03:23.247536  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 87309.5us
W20260709 13:03:23.248276  8399 rpcz_store.cc:269] 0709 13:03:20.800839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.801629 (+   790us) service_pool.cc:224] Handling call
0709 13:03:23.207821 (+2406192us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54002be154. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:23.247435 (+ 39614us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2703059712}
I20260709 13:03:23.249837 12383 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:23.250232  8399 raft_consensus.cc:1275] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:03:23.250854  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:23.251281  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.251915  8399 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:03:23.252264  8399 pending_rounds.cc:85] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 6
I20260709 13:03:23.252481  8399 pending_rounds.cc:107] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:03:23.252740  8399 raft_consensus.cc:2889] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:23.254535  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:23.256883  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 58472.1us
I20260709 13:03:23.259526 12659 consensus_queue.cc:1059] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:23.260780 12659 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:23.261757 12403 raft_consensus.cc:3055] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:23.262394 12403 raft_consensus.cc:740] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:23.262444  8222 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:03:23.263662  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 139153us
I20260709 13:03:23.264022 12403 consensus_queue.cc:260] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:23.265833 12534 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:23.266480 12534 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:23.262010  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.254540  8401 raft_consensus.cc:1240] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:23.248153  8403 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:23.221616  8326 rpcz_store.cc:276] 0709 13:03:22.075603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.120154 (+1044551us) service_pool.cc:224] Handling call
0709 13:03:23.188341 (+ 68187us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.209739 (+ 21398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.210028 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:23.247576  9008 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:23.249038  8297 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:23.288388 12534 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:23.288692  8398 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:23.293864  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 2081) took 2007 ms. Trace:
I20260709 13:03:23.294626  8397 rpcz_store.cc:276] 0709 13:03:21.287044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.077528 (+790484us) service_pool.cc:224] Handling call
0709 13:03:23.267784 (+1190256us) spinlock_profiling.cc:243] Waited 717 ms on lock 0x7b54000fd7d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:23.293596 (+ 25812us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1648321920}
I20260709 13:03:23.297042  8295 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:23.297830  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.299467  8295 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:23.300794 12660 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:23.301399 12660 raft_consensus.cc:697] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:23.302027 12383 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:23.302595 12383 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.303165 12383 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:23.302688 12660 consensus_queue.cc:237] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:23.297859  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:23.307893  8326 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:23.309542  9001 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:23.312114 12730 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:23.313721  8220 connection.cc:256] RPC call timeout handler was delayed by -1.2263637570000014s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1517s for a response
I20260709 13:03:23.313927 12403 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:23.315867  8220 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:23.316992 12275 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:23.313930 12539 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.317638 12275 raft_consensus.cc:697] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:23.318257  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 2042) took 1569 ms. Trace:
I20260709 13:03:23.318841 12275 consensus_queue.cc:237] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:23.318727  8255 rpcz_store.cc:276] 0709 13:03:21.749046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.184147 (+435101us) service_pool.cc:224] Handling call
0709 13:03:22.238104 (+ 53957us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.250616 (+ 12512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.317817 (+1067201us) spinlock_profiling.cc:243] Waited 53.6 ms on lock 0x7b540033ed50. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:23.318053 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123371264}
I20260709 13:03:23.321648  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 600) took 1439 ms. Trace:
W20260709 13:03:23.322108  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 73245.7us
I20260709 13:03:23.322186  8324 rpcz_store.cc:276] 0709 13:03:21.882176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.883065 (+   889us) service_pool.cc:224] Handling call
0709 13:03:21.894733 (+ 11668us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:03:21.900547 (+  5814us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.321173 (+1420626us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b540037c150. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:23.321405 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25942784}
I20260709 13:03:23.323258  8297 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:23.301267  8295 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:23.312764 12730 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:23.326723  8222 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:23.323716  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:23.327494  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 648) took 2140 ms. Trace:
I20260709 13:03:23.330506  8405 rpcz_store.cc:276] 0709 13:03:21.184478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.069376 (+1884898us) service_pool.cc:224] Handling call
0709 13:03:23.069854 (+   478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:23.070353 (+   499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:23.070383 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:23.071259 (+   876us) log.cc:844] Serialized 26 byte log entry
0709 13:03:23.121402 (+ 50143us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:23.121475 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.121548 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:23.324398 (+202850us) raft_consensus.cc:1685] finished
0709 13:03:23.324424 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:23.324563 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3126,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":24358,"thread_start_us":49646,"threads_started":1,"wal-append.queue_time_us":130209}
I20260709 13:03:23.331799 12730 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:23.332168 12665 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:03:23.333051  8406 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 649) took 2148 ms (client timeout 1988 ms). Trace:
I20260709 13:03:23.334219 12659 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:23.334770  8406 rpcz_store.cc:269] 0709 13:03:21.184750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.868224 (+683474us) service_pool.cc:224] Handling call
0709 13:03:23.332667 (+1464443us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b54000fd2d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:23.332849 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":555626752}
I20260709 13:03:23.335843  8297 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:23.336685 12659 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:23.333567  8372 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:23.337991  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 147720us
I20260709 13:03:23.338979 12684 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:23.339397 12488 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:23.341351  8372 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:23.333921  8404 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:23.333851  9002 raft_consensus.cc:1240] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:03:23.318816  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:23.344206 12665 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:23.345021 12716 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:03:23.349161  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 79116.9us
I20260709 13:03:23.208341 12588 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.143s
W20260709 13:03:23.366482  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 223170us
I20260709 13:03:23.347420 12659 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:23.381760  8325 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:23.318248 12539 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:23.353977 12689 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:23.389665 12469 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:23.390585 12469 consensus_queue.cc:1243] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:03:23.393807 12232 consensus_queue.cc:1059] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:23.395043 12319 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:23.396669 12319 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:23.397357 12319 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:23.399071  8151 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45448 (request call id 284) took 20018 ms (client timeout 15000 ms). Trace:
I20260709 13:03:23.400503 12319 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:23.387501  8326 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:23.348462 12716 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:23.385934  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:23.403216  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.408066  8372 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:23.409853  8331 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 12. Current term is 13. Ops: [12.8-12.8]
W20260709 13:03:23.413311  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 77581.4us
I20260709 13:03:23.401345 12718 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:23.416472  8324 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:23.418052  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:23.419992 12716 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:23.350026 12730 consensus_queue.cc:1048] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:03:23.422989  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:23.382748 12707 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:03:23.448964  8325 raft_consensus.cc:2393] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 13: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:23.449556  8403 raft_consensus.cc:2393] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 10: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:23.400710 12665 consensus_queue.cc:1059] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:03:23.416122 12689 raft_consensus.cc:697] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:23.423527 12718 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:23.486018  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:23.485133  8297 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:23.472209  8369 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:23.499188  8248 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:23.488875 12718 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:23.501228  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:23.505693  8324 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
W20260709 13:03:23.506130  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 84503.7us
I20260709 13:03:23.507023  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:23.507404  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:23.507829  8324 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:03:23.508237  8331 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:23.508490  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 60488.5us
I20260709 13:03:23.508958  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:23.509573 12706 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:23.510139 12706 raft_consensus.cc:697] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:23.510082 12383 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:23.511054  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:23.511766  8326 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:23.512403 12383 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:23.512686  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:23.514510 12595 raft_consensus.cc:3055] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:23.514972 12595 raft_consensus.cc:740] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:23.515899 12595 consensus_queue.cc:260] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:23.511358 12706 consensus_queue.cc:237] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:23.518608  8297 leader_election.cc:400] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12.
I20260709 13:03:23.517719  8324 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
W20260709 13:03:23.519958  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 192541us
W20260709 13:03:23.501250  8251 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2028) took 2719 ms (client timeout 1573 ms). Trace:
I20260709 13:03:23.492730 12620 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:03:23.521311  8251 rpcz_store.cc:269] 0709 13:03:20.778308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.802571 (+ 24263us) service_pool.cc:224] Handling call
0709 13:03:23.492466 (+2689895us) spinlock_profiling.cc:243] Waited 912 ms on lock 0x7b5400177d54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:23.497290 (+  4824us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2098434816}
I20260709 13:03:23.507884  8369 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:23.517061 12595 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:23.530269  8220 leader_election.cc:336] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.642s (SENT)
I20260709 13:03:23.530931  8220 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:23.528059  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 226881us
W20260709 13:03:23.551772  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 143765us
I20260709 13:03:23.552076  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 646) took 2345 ms. Trace:
I20260709 13:03:23.524871 12726 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:23.552438  8398 rpcz_store.cc:276] 0709 13:03:21.184001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.938448 (+1754447us) service_pool.cc:224] Handling call
0709 13:03:23.276881 (+338433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.289173 (+ 12292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.529397 (+240224us) spinlock_profiling.cc:243] Waited 338 ms on lock 0x7b54000fd7d0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:23.529552 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":776958848}
I20260709 13:03:23.527920 12665 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:03:23.565642  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 108500us
W20260709 13:03:23.566128  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 58342.9us
I20260709 13:03:23.567132  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:23.569331  9008 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 13 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:23.569919  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:23.567132  8369 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:23.399506  8151 rpcz_store.cc:269] 0709 13:03:03.380242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.381857 (+  1615us) service_pool.cc:224] Handling call
0709 13:03:23.398799 (+20016942us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:11.927294 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:11.928091 (+   797us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:11.928155 (+    64us) write_op.cc:435] Acquired schema lock
0709 13:03:11.928182 (+    27us) tablet.cc:662] Decoding operations
0709 13:03:11.976728 (+ 48546us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:11.976788 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:11.976816 (+    28us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:03:11.978333 (+  1517us) tablet.cc:701] Row locks acquired
0709 13:03:11.978350 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:03:11.978505 (+   155us) write_op.cc:270] Start()
0709 13:03:11.978614 (+   109us) write_op.cc:276] Timestamp: P: 1783602191978454 usec, L: 0
0709 13:03:11.978628 (+    14us) op_driver.cc:348] REPLICATION: starting
0709 13:03:12.170419 (+191791us) log.cc:844] Serialized 7861 byte log entry
0709 13:03:12.585248 (+414829us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54003bf454. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287b9e28 00007f882c1f4581 00007f882c1f3d58 00007f882c1f89f1 00007f882c1f89a9 00007f882c1f8939 00007f882c1f8901 00007f882c1f7b2d 000000000038f354 000000000038f199
0709 13:03:13.025717 (+440469us) op_driver.cc:464] REPLICATION: finished
0709 13:03:13.543018 (+517301us) write_op.cc:301] APPLY: starting
0709 13:03:13.543119 (+   101us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:13.574227 (+ 31108us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:13.574270 (+    43us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:13.580107 (+  5837us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:13.580528 (+   421us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=40
0709 13:03:13.580574 (+    46us) write_op.cc:312] APPLY: finished
0709 13:03:13.592762 (+ 12188us) log.cc:844] Serialized 139 byte log entry
0709 13:03:13.592970 (+   208us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:13.593348 (+   378us) write_op.cc:454] Released schema lock
0709 13:03:13.593440 (+    92us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:13.593934 (+   494us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:13.593963 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4398087,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":516985,"apply.run_cpu_time_us":6857,"apply.run_wall_time_us":50896,"num_ops":20,"prepare.queue_time_us":328933,"prepare.run_cpu_time_us":139352,"prepare.run_wall_time_us":658205,"raft.queue_time_us":361502,"raft.run_cpu_time_us":41704,"raft.run_wall_time_us":334522,"replication_time_us":1046857,"spinlock_wait_cycles":303677312,"thread_start_us":1012474,"threads_started":4,"wal-append.queue_time_us":438938,"wal-append.run_cpu_time_us":26474,"wal-append.run_wall_time_us":2112721}]]}
I20260709 13:03:23.624394 12642 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:23.630534 12761 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:23.638547  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 111841us
I20260709 13:03:23.640681  8331 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
W20260709 13:03:23.645064  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 135759us
I20260709 13:03:23.559604 12665 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:23.625308 12642 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:23.631321 12761 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:23.650089  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 203895us
W20260709 13:03:23.652304  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:23.685014  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.650139 12665 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:23.651140 12642 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:23.703254  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 136202us
I20260709 13:03:23.705920  8297 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:23.688882 12730 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:23.711822  8220 leader_election.cc:336] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.834s (SENT)
I20260709 13:03:23.715936  8220 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:23.705346 12232 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.722810  8250 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 13:03:23.723342  8250 pending_rounds.cc:85] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 10
I20260709 13:03:23.723634  8250 pending_rounds.cc:107] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:03:23.723912  8250 raft_consensus.cc:2889] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:23.725657 12721 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:23.727962  8220 leader_election.cc:336] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.849s (SENT)
I20260709 13:03:23.728508  8220 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:23.726929 12539 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:23.726289 12721 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:23.714814 12659 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:23.744594  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2495) took 2141 ms. Trace:
I20260709 13:03:23.688184 12689 consensus_queue.cc:237] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:23.717101 12774 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:23.725651 12488 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:23.745052  8400 rpcz_store.cc:276] 0709 13:03:21.603424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.172905 (+1569481us) service_pool.cc:224] Handling call
0709 13:03:23.173358 (+   453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:23.198048 (+ 24690us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:23.198072 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:23.244038 (+ 45966us) log.cc:844] Serialized 26 byte log entry
0709 13:03:23.470934 (+226896us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:23.470987 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.471083 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:23.744330 (+273247us) raft_consensus.cc:1685] finished
0709 13:03:23.744355 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:23.744516 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":226475,"threads_started":1,"wal-append.queue_time_us":371676}
I20260709 13:03:23.757086 12718 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:23.738741 12232 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:23.746178 12275 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:23.757375  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 617) took 1509 ms. Trace:
I20260709 13:03:23.762754  8330 rpcz_store.cc:276] 0709 13:03:22.247637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.153449 (+905812us) service_pool.cc:224] Handling call
0709 13:03:23.153889 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:23.154439 (+   550us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:23.385947 (+231508us) log.cc:844] Serialized 14 byte log entry
0709 13:03:23.614797 (+228850us) log.cc:844] Serialized 14 byte log entry
0709 13:03:23.614966 (+   169us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:23.656738 (+ 41772us) log.cc:844] Serialized 26 byte log entry
0709 13:03:23.656935 (+   197us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:23.656986 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.657126 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:23.757149 (+100023us) raft_consensus.cc:1685] finished
0709 13:03:23.757169 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:23.757316 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6038,"prepare.queue_time_us":330010,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":2537,"raft.queue_time_us":53297,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":1724,"spinlock_wait_cycles":15964288,"thread_start_us":339973,"threads_started":2,"wal-append.queue_time_us":275107}
I20260709 13:03:23.764559 12534 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:23.765108  8330 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: []
W20260709 13:03:23.750761  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 55224.3us
I20260709 13:03:23.758584 12761 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:23.768035 12779 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.759372 12718 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:23.761324 12232 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:23.786435  8405 raft_consensus.cc:1240] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:03:23.773597 12548 raft_consensus.cc:3055] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:23.733453  8150 catalog_manager.cc:5697] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 7 to 10, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 10 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:23.785336  8400 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10. Ops: []
W20260709 13:03:23.791141  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 65716.7us
I20260709 13:03:23.791486  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:23.791201  8297 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:23.795194  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.795768  8330 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: []
W20260709 13:03:23.798659  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 78295.2us
I20260709 13:03:23.801651  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:23.802376  8403 raft_consensus.cc:2410] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:03:23.804078  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:23.803963 12779 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:23.805019  8400 raft_consensus.cc:2410] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:03:23.806495 12779 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:23.806368 12784 consensus_queue.cc:1059] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:23.807310 12771 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:23.807437 12779 raft_consensus.cc:3055] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:23.807921 12779 raft_consensus.cc:740] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:23.797150 12718 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.788189 12548 raft_consensus.cc:740] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:23.809193  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:23.819393  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:23.791875  8150 catalog_manager.cc:5697] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 9 to 10, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 10 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:23.825490  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.828400 12779 consensus_queue.cc:260] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:23.829584  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.829883 12779 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:23.835794 12642 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:23.838366  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:23.838831  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:23.839331  8400 raft_consensus.cc:2410] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:03:23.840255  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:23.842453 12534 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:23.829005  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 91186.9us
I20260709 13:03:23.843484  8330 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:03:23.844362  8398 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 14: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:23.840299  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:23.849493  8400 raft_consensus.cc:2393] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 9: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:23.848729 12786 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:23.856446  8369 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:23.853888  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:23.858188  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2087) took 2550 ms. Trace:
I20260709 13:03:23.858652  8402 rpcz_store.cc:276] 0709 13:03:21.308039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.164567 (+1856528us) service_pool.cc:224] Handling call
0709 13:03:23.190750 (+ 26183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.197899 (+  7149us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:23.451211 (+253312us) log.cc:844] Serialized 14 byte log entry
0709 13:03:23.856872 (+405661us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b1800417fa0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88283664bc 00007f88256b448b 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e23a
0709 13:03:23.857148 (+   276us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:23.857228 (+    80us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:23.857270 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.857351 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:23.858128 (+   777us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":552,"prepare.queue_time_us":437394,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":2908,"spinlock_wait_cycles":27665792,"thread_start_us":419066,"threads_started":2,"wal-append.queue_time_us":275711}
I20260709 13:03:23.874928  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:23.818190 12718 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:23.876189  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:23.876873  8330 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:03:23.887696 12548 consensus_queue.cc:260] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:23.889040 12548 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:23.889700  8398 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
W20260709 13:03:23.890064  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.891227  8327 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:23.893756  8297 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:23.891886  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:23.890869  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:23.873742 12643 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:23.901623  8330 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:23.904510 12689 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:23.862047 12786 raft_consensus.cc:697] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:23.873795 12642 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:23.904116 12665 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:23.922245  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:23.934970  8325 raft_consensus.cc:2393] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 8: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:03:23.930938 12403 consensus_meta.cc:287] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.615s	user 0.009s	sys 0.005s
I20260709 13:03:23.938938 12789 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:23.951130  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 599) took 2069 ms. Trace:
I20260709 13:03:23.951910 12637 consensus_queue.cc:1059] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:03:23.940400 12718 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:23.956135 12633 consensus_queue.cc:1059] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:23.949579  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:23.964679 12642 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:23.965122 12642 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:23.965548 12642 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:23.965643  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:23.966532  8405 raft_consensus.cc:2376] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11. Re-sending same reply.
I20260709 13:03:23.967669  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:23.968475 12706 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:23.970160 12633 consensus_queue.cc:1243] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:03:23.969511  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:23.982033 12730 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:23.982560 12730 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.982954 12730 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:23.986701  8398 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:23.989997  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:23.992684  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 616) took 1745 ms. Trace:
I20260709 13:03:23.992987 12689 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:23.993628 12689 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.993940  8369 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:23.958725  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.995389 12706 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:23.958882 12779 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:03:23.960199  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.962877 12785 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:23.992874  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:23.994102 12689 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:23.957553  8972 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:23.950958  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 93011.8us
I20260709 13:03:23.986582  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2071) took 2782 ms. Trace:
I20260709 13:03:24.013477  8404 rpcz_store.cc:276] 0709 13:03:21.204113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.156527 (+1952414us) service_pool.cc:224] Handling call
0709 13:03:23.333617 (+177090us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.343926 (+ 10309us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.986287 (+642361us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54000fd2d0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:23.986496 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":406053632}
I20260709 13:03:23.947999 12786 consensus_queue.cc:237] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:23.945979  8150 catalog_manager.cc:5697] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:23.993459  8332 rpcz_store.cc:276] 0709 13:03:22.247397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.151673 (+904276us) service_pool.cc:224] Handling call
0709 13:03:23.151979 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.161689 (+  9710us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:23.357395 (+195706us) log.cc:844] Serialized 14 byte log entry
0709 13:03:23.483476 (+126081us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:23.483549 (+    73us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:23.483613 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.992354 (+508741us) spinlock_profiling.cc:243] Waited 9.31 ms on lock 0x7b540008a554. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:23.992418 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:23.992603 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":368525,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":110,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":10940,"spinlock_wait_cycles":21402368,"thread_start_us":266203,"threads_started":2,"wal-append.queue_time_us":326510}
I20260709 13:03:24.005719  8400 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
I20260709 13:03:24.028076  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:24.010799  8398 raft_consensus.cc:2393] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 8: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:03:24.011574  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.014451  8403 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:24.019318  9001 raft_consensus.cc:1275] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:23.951555  8329 rpcz_store.cc:276] 0709 13:03:21.881928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.882039 (+   111us) service_pool.cc:224] Handling call
0709 13:03:21.899350 (+ 17311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:21.899702 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.899728 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:23.142697 (+1242969us) spinlock_profiling.cc:243] Waited 8.41 ms on lock 0x7b18003c4d80. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882569f2a6 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a
0709 13:03:23.699322 (+556625us) spinlock_profiling.cc:243] Waited 3.34 ms on lock 0x7b2c000d4468. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:03:23.734855 (+ 35533us) log.cc:844] Serialized 26 byte log entry
0709 13:03:23.858787 (+123932us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:23.858854 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.872650 (+ 13796us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:23.950762 (+ 78112us) raft_consensus.cc:1685] finished
0709 13:03:23.950780 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:23.950961 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3111,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":15756,"spinlock_wait_cycles":27044480,"thread_start_us":93316,"threads_started":1,"wal-append.queue_time_us":128374}
I20260709 13:03:24.015374  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:24.038165  8297 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:03:24.038234  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 159037us
I20260709 13:03:24.042541  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:24.041045  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:24.046322  8251 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:03:24.048547  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:24.050069  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:24.050298  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:24.051352  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:24.051927  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:24.052151  8399 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:03:24.053256  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:24.053761  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:24.054965  8399 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:24.043538  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:23.992759 12730 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:24.014757 12232 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:24.078474 12794 raft_consensus.cc:3055] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:24.068833 12718 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
W20260709 13:03:24.078714  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 63824.7us
I20260709 13:03:24.083825  8150 catalog_manager.cc:5697] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 9 to 11, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 11 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:24.014962 12534 consensus_queue.cc:1059] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:03:24.088793  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:24.078959  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:24.104189  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 146673us
I20260709 13:03:24.104552  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2455) took 1388 ms. Trace:
I20260709 13:03:24.104102  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:24.082049 12730 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:24.104908  8331 rpcz_store.cc:276] 0709 13:03:22.716174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.647861 (+931687us) service_pool.cc:224] Handling call
0709 13:03:23.648220 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.704660 (+ 56440us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:23.713196 (+  8536us) log.cc:844] Serialized 14 byte log entry
0709 13:03:23.996859 (+283663us) spinlock_profiling.cc:243] Waited 2.58 ms on lock 0x7b180037c360. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e23a 00007f88287c8a2a
0709 13:03:24.104220 (+107361us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:24.104276 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:24.104322 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.104395 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:24.104497 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":894,"prepare.queue_time_us":8380,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":2135,"spinlock_wait_cycles":5942016,"thread_start_us":107099,"threads_started":1}
I20260709 13:03:24.108657  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 610) took 2202 ms. Trace:
I20260709 13:03:24.090273 12706 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:24.095846 12620 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:24.085546 12642 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:24.109074  8327 rpcz_store.cc:276] 0709 13:03:21.900431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.989119 (+ 88688us) service_pool.cc:224] Handling call
0709 13:03:22.026219 (+ 37100us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.857678 (+1831459us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b58005c33d0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f882871a0aa 00007f882879c447 00007f88287b4fe7 00007f88287b67de 00007f88287c0b49 00007f88287c1416 00007f88287bdd8f 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4
0709 13:03:24.090708 (+233030us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:24.090739 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:24.090779 (+    40us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:24.090814 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.098125 (+  7311us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:24.102964 (+  4839us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6080,"raft.queue_time_us":6883,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":7519,"spinlock_wait_cycles":2427808000}
I20260709 13:03:24.088255 12794 raft_consensus.cc:740] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:24.106676 12659 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:24.096756 12232 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:24.113879  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 75825.7us
I20260709 13:03:24.089808 12275 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:24.114703 12635 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 13:03:24.090584 12678 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.115252 12706 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.134912  8404 raft_consensus.cc:1240] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:24.135774  8972 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:03:24.133819  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.138690 12243 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:24.118243 12785 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.142570  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:03:24.143559  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.117484 12797 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:24.134578 12637 consensus_queue.cc:1059] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:03:24.150990  8369 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:03:24.151408 12793 log.cc:927] Time spent T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:03:24.141109 12706 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:24.156275  8399 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:24.156713 12785 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:24.159162  8369 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:24.162472  8404 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:03:24.141355 12275 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:24.169283 12804 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:24.169871 12805 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:03:24.170538  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:24.171171  8255 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 7.
W20260709 13:03:24.176473  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.173130 12779 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:24.179668 12779 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:24.180091 12779 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:24.163524  8150 catalog_manager.cc:5697] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 5 to 7, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 7 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:24.183859  8150 catalog_manager.cc:5697] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 6 to 9, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 9 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:24.185879  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.186523 12595 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:24.190840  8222 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:24.164199 12684 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:24.193908 12684 raft_consensus.cc:697] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:24.194794 12488 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:24.194977 12684 consensus_queue.cc:237] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:24.195663 12797 raft_consensus.cc:697] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:24.196564 12797 consensus_queue.cc:237] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:24.197705 12721 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:24.191006 12275 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:24.207358  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:24.209172  8251 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:24.209944  8401 raft_consensus.cc:1240] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:24.210762  8255 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:24.214186 12275 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:24.214908 12785 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:24.214767 12275 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:24.218541  8255 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:24.164976 12730 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:24.141021 12678 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
W20260709 13:03:24.226078  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.164556 12665 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:24.227665  8297 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:24.206225  8972 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:24.245188 12607 raft_consensus.cc:3055] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:24.245654 12607 raft_consensus.cc:740] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:24.246521 12607 consensus_queue.cc:260] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:24.248291 12670 raft_consensus.cc:3055] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:24.248697 12670 raft_consensus.cc:740] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:24.249737 12670 consensus_queue.cc:260] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:24.251066 12670 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:24.251385 12610 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.251945 12610 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:24.254056  8398 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:24.254796 12610 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:24.256495  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:24.256872  8397 raft_consensus.cc:1240] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:24.257382  9000 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:24.252560  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:24.259078  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2048) took 1319 ms. Trace:
I20260709 13:03:24.253361  8405 raft_consensus.cc:1217] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[5.6-15.9]   Dedup: 5.6->[11.7-15.9]
I20260709 13:03:24.259562  8248 rpcz_store.cc:276] 0709 13:03:22.939689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.945770 (+  6081us) service_pool.cc:224] Handling call
0709 13:03:23.497383 (+551613us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.501495 (+  4112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.258818 (+757323us) spinlock_profiling.cc:243] Waited 546 ms on lock 0x7b5400177d50. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:24.259002 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1255762176}
I20260709 13:03:24.261042 12610 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.261603 12610 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:24.262177  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:24.262744  8401 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:24.263845 12610 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:24.263949  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 643) took 3852 ms. Trace:
I20260709 13:03:24.264485  8250 rpcz_store.cc:276] 0709 13:03:20.411221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.512297 (+101076us) service_pool.cc:224] Handling call
0709 13:03:20.530455 (+ 18158us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.724490 (+3194035us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.263585 (+539095us) spinlock_profiling.cc:243] Waited 3.11 s on lock 0x7b540009ecd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:24.263840 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7154959616}
I20260709 13:03:24.265020  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:24.265906  9003 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:03:24.267743  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:24.260457  8405 pending_rounds.cc:85] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 6
I20260709 13:03:24.268945  8328 raft_consensus.cc:1240] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:24.275097  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:24.275774  8402 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:24.276589  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 618) took 1240 ms. Trace:
I20260709 13:03:24.277087  8328 rpcz_store.cc:276] 0709 13:03:23.035837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.708670 (+672833us) service_pool.cc:224] Handling call
0709 13:03:24.268712 (+560042us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:24.275608 (+  6896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.276369 (+   761us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b54003cad50. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:24.276520 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1286368640}
I20260709 13:03:24.283118  8403 raft_consensus.cc:1275] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:03:24.269124  8405 pending_rounds.cc:107] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:03:24.285853  8405 raft_consensus.cc:2889] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:24.286070  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 165223us
W20260709 13:03:24.287715  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.288352  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:24.288921  8252 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:24.289609  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:24.291504  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 58052.7us
I20260709 13:03:24.167575 12718 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.297657  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:24.221946 12730 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.303449  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:24.306888  9001 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:03:24.309048  8402 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:03:24.227320 12383 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:24.311375 12383 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:24.312503  8150 catalog_manager.cc:5697] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 4 to 9, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 9 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:24.314580  8150 catalog_manager.cc:5697] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:24.235633 12275 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:24.231459 12721 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:03:24.327463  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.320925  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2454) took 1605 ms. Trace:
I20260709 13:03:24.320744  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:24.321977  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:24.242537 12488 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:24.323873  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:24.289606 12785 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:03:24.340358  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:24.343030  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.330053  8403 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:24.298482 12620 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.307633 12730 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:24.322276 12777 log.cc:927] Time spent T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.582s	user 0.005s	sys 0.000s
I20260709 13:03:24.318465 12771 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:24.353869  8328 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:03:24.356281  8331 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:24.351248  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:24.359596  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:24.361454  8330 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:24.345678 12785 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.307458 12718 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:24.367353  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.349246 12811 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:24.370623  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:24.371352  8323 raft_consensus.cc:1275] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:03:24.372049  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 223654us
I20260709 13:03:24.318734 12693 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:03:24.382997  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:24.388340  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:24.349340 12721 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:24.377712  8329 raft_consensus.cc:2393] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 10: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:24.419224  8331 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:24.367494 12785 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:24.422150  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:24.374481  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:24.426826  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:24.427569  8328 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:03:24.369915 12642 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:24.316942  8150 catalog_manager.cc:5697] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 8 to 9, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 9 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:03:24.435006  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 60580us
I20260709 13:03:24.435365  8297 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:24.379357 12232 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:24.437402  8323 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:24.438845  8222 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:24.433911 12595 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:24.449159  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:24.428562 12610 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:24.321960 12794 consensus_queue.cc:260] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:24.457929  8331 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:24.366520 12620 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:24.462739  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 80289us
W20260709 13:03:24.488744  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 53551.4us
I20260709 13:03:24.492362  8297 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:24.491470  8974 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:24.498631  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:24.453270 12595 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.423465 12811 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:24.455873 12665 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:24.459223 12232 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:24.437335 12275 consensus_queue.cc:1059] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
W20260709 13:03:24.514048  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 57883.5us
I20260709 13:03:24.552697 12811 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:24.559255  8327 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:24.564342  8329 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:24.567826 12797 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:24.567723 12806 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:03:24.568460 12797 raft_consensus.cc:697] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:03:24.569139 12813 log.cc:927] Time spent T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
W20260709 13:03:24.572927  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.513190 12595 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:24.596468  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 104210us
I20260709 13:03:24.515444 12403 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:24.565210  8295 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:24.464306 12706 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:24.490721 12809 consensus_queue.cc:1059] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.111s
I20260709 13:03:24.515021 12232 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:24.612967  9001 raft_consensus.cc:1240] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:03:24.550333 12383 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:24.625200  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 117059us
I20260709 13:03:24.558701 12678 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:24.599313 12275 consensus_queue.cc:1243] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:03:24.649554  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.597519  8297 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:03:25.090814  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 493333us
W20260709 13:03:24.626318  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:24.830425 12822 log.cc:927] Time spent T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
W20260709 13:03:24.786188  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 322470us
W20260709 13:03:24.777949  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.758293 12232 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:25.190569  8251 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.035s	sys 0.102s
W20260709 13:03:25.587004  8251 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.119s	user 0.035s	sys 0.102s
I20260709 13:03:25.195518  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:25.197814  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.293805  8150 catalog_manager.cc:5697] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 5 to 10, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 10 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:03:25.296351  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:25.303290  8222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.017s	sys 0.046s
W20260709 13:03:25.600517  8222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.004s	user 0.018s	sys 0.046s
W20260709 13:03:25.600867  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 1.05149e+06us
I20260709 13:03:25.307945  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 665) took 1088 ms. Trace:
I20260709 13:03:25.601325  8150 catalog_manager.cc:5697] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 4 to 5, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 5 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:25.602200  8255 rpcz_store.cc:276] 0709 13:03:24.219854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.219963 (+   109us) service_pool.cc:224] Handling call
0709 13:03:24.220450 (+   487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:24.220854 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:24.220892 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:24.221554 (+   662us) log.cc:844] Serialized 26 byte log entry
0709 13:03:24.425721 (+204167us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:24.425776 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.425876 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:25.307659 (+881783us) raft_consensus.cc:1685] finished
0709 13:03:25.307683 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.307878 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":203758,"threads_started":1,"wal-append.queue_time_us":236969}
I20260709 13:03:25.604105  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:25.604725  8255 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:25.321853  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 668) took 1098 ms. Trace:
W20260709 13:03:25.403781  8252 consensus_meta.cc:287] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.102s	user 0.017s	sys 0.005s
I20260709 13:03:25.606165  8252 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:25.606689  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2052) took 1380 ms. Trace:
I20260709 13:03:25.606885 12843 consensus_queue.cc:1048] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:03:25.402845  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 308930us
I20260709 13:03:25.274130  8324 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:03:25.409091  8298 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.4:33825 timed out after 2.058s (SENT) [suppressed 28 similar messages]
W20260709 13:03:25.221947  8330 consensus_meta.cc:287] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.860s	user 0.008s	sys 0.004s
I20260709 13:03:25.773813  8330 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
W20260709 13:03:25.774353  8330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2214) took 1423 ms (client timeout 1706 ms). Trace:
W20260709 13:03:25.774684  8330 rpcz_store.cc:269] 0709 13:03:24.350806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.350943 (+   137us) service_pool.cc:224] Handling call
0709 13:03:25.774268 (+1423325us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:24.657537  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:25.399961  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 889543us
W20260709 13:03:25.320861  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.328774  8326 rpcz_store.cc:276] 0709 13:03:22.715779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.529832 (+814053us) service_pool.cc:224] Handling call
0709 13:03:23.582051 (+ 52219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.646535 (+ 64484us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:23.707774 (+ 61239us) log.cc:844] Serialized 14 byte log entry
0709 13:03:24.223116 (+515342us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b18002a1600. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e23a 00007f88287c8a2a
0709 13:03:24.320514 (+ 97398us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:24.320571 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:24.320631 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.320728 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:24.320872 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30049,"prepare.queue_time_us":55435,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":1626,"raft.queue_time_us":35680,"spinlock_wait_cycles":28134656,"thread_start_us":62349,"threads_started":1}
W20260709 13:03:25.300441  8405 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.982s	user 0.004s	sys 0.012s
W20260709 13:03:25.304745  8403 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.845s	user 0.027s	sys 0.027s
W20260709 13:03:25.176805  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.206045  8974 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:24.774215  9006 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:24.509191 12580 consensus_queue.cc:1059] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 13:03:24.836813 12595 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:25.412134 12607 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:25.199857 12610 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:24.816869 12620 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:25.400643 12635 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:25.842521 12635 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:25.842983 12635 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:24.797035 12670 consensus_meta.cc:287] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.545s	user 0.006s	sys 0.002s
W20260709 13:03:24.822721 12689 consensus_meta.cc:287] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.795s	user 0.009s	sys 0.002s
I20260709 13:03:25.850448 12689 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:25.852710 12689 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:24.788187 12706 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:24.809859 12718 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:24.786108 12721 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:24.796828 12730 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:25.858103 12730 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:25.338135 12771 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:25.859407 12771 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:25.334338 12774 raft_consensus.cc:3055] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:25.862386 12774 raft_consensus.cc:740] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:03:24.786643 12779 consensus_meta.cc:287] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.606s	user 0.012s	sys 0.013s
I20260709 13:03:25.863950 12779 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:24.778869 12780 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.168s
I20260709 13:03:24.830749 12785 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:25.869518 12785 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:25.255692 12794 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:24.569693 12796 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.184s
I20260709 13:03:24.569793 12797 consensus_queue.cc:237] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:25.203920 12383 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.178736 12403 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:25.878557 12383 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:25.203481 12659 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:25.880194 12659 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:25.880757 12383 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:25.882458 12659 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:25.271873 12805 raft_consensus.cc:3055] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:25.271812 12806 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.884500 12806 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:25.886588 12806 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:24.780411 12808 log.cc:927] Time spent T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.221s	user 0.000s	sys 0.000s
I20260709 13:03:25.376766 12810 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.890380 12810 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:25.464427  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:25.892422  8254 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:03:25.194056 12825 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.896111 12825 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:25.419348 12833 log.cc:927] Time spent T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.604s	user 0.004s	sys 0.000s
I20260709 13:03:25.898378 12779 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:25.518806  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:25.899854  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:25.900377  8250 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:03:25.526109  8248 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:03:25.538455 12642 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:25.901798 12642 raft_consensus.cc:697] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:25.902379  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:25.902982  8248 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:25.902711 12642 consensus_queue.cc:237] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:25.552742  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 1.11673e+06us
I20260709 13:03:25.607005  8252 rpcz_store.cc:276] 0709 13:03:24.226072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.226192 (+   120us) service_pool.cc:224] Handling call
0709 13:03:25.606610 (+1380418us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:25.905941  8369 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:25.906281  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:25.907040  8252 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
I20260709 13:03:25.908416  8252 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:03:25.908591 12825 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:25.910086 12706 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:25.420014  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:25.911760  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.607812  8256 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: [10.6-12.7]
I20260709 13:03:25.605943 12830 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.913156 12830 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:25.915338  8256 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:03:25.915499 12830 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:25.605664  8257 rpcz_store.cc:276] 0709 13:03:24.223293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.223428 (+   135us) service_pool.cc:224] Handling call
0709 13:03:24.290770 (+ 67342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:24.300616 (+  9846us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:24.300657 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:24.331590 (+ 30933us) log.cc:844] Serialized 26 byte log entry
0709 13:03:24.429269 (+ 97679us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:24.429298 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.429345 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:25.321588 (+892243us) raft_consensus.cc:1685] finished
0709 13:03:25.321612 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.321789 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6581,"raft.queue_time_us":9580,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":13772,"thread_start_us":60597,"threads_started":1,"wal-append.queue_time_us":145863}
W20260709 13:03:25.625217 12828 log.cc:927] Time spent T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.813s	user 0.000s	sys 0.006s
I20260709 13:03:25.623764  8253 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:25.916615  8256 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 13:03:25.916980  8253 raft_consensus.cc:1240] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 9. Current term is 10. Ops: []
W20260709 13:03:25.642086  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 1.06737e+06us
I20260709 13:03:25.642903  8222 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:25.920059  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 277213us
I20260709 13:03:25.693547 12693 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:25.923036 12693 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:25.925086 12693 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:25.694113 12643 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.925674 12643 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:25.927665 12643 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:25.645668 12844 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.928303 12844 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:25.930040  8254 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:25.698902 12843 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:25.712491 12842 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.720474 12232 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:25.722636 12829 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:25.963429 12829 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:25.724999 12539 consensus_queue.cc:1059] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:03:25.757901 12824 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.941s	user 0.007s	sys 0.003s
W20260709 13:03:25.761646  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:25.767475  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 358343us
I20260709 13:03:25.768731  8297 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:03:25.966360  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 198481us
I20260709 13:03:25.966617 12829 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:25.785295  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2212) took 1451 ms. Trace:
I20260709 13:03:25.967317  8324 rpcz_store.cc:276] 0709 13:03:24.334431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.349545 (+ 15114us) service_pool.cc:224] Handling call
0709 13:03:24.353917 (+  4372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:25.754477 (+1400560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.784053 (+ 29576us) spinlock_profiling.cc:243] Waited 920 ms on lock 0x7b540011d5d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:25.785212 (+  1159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2115052416}
I20260709 13:03:25.967787  8297 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:25.782008 12678 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:25.796362  8405 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.478s	user 0.007s	sys 0.012s
I20260709 13:03:25.795810  8295 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:03:25.803907  8403 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.344s	user 0.029s	sys 0.028s
I20260709 13:03:25.806277 12789 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:25.808382  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 602310us
W20260709 13:03:25.809769  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.809805 12784 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:25.814155 12798 raft_consensus.cc:3055] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:25.821691 12595 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:25.833781  8255 raft_consensus.cc:1275] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:25.834525  9006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 901) took 1257 ms. Trace:
I20260709 13:03:25.837996 12610 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:25.861778  8427 debug-util.cc:398] Leaking SignalData structure 0x7b08000e0c60 after lost signal to thread 7167
I20260709 13:03:25.863382 12721 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:25.883848 12805 raft_consensus.cc:740] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:26.006152 12721 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:25.893554 12718 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.893926 12810 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:25.488576  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:26.011955  8249 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:03:25.920096  8372 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:03:26.012804  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 92784.8us
I20260709 13:03:25.996516 12383 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:25.920841  8222 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:03:26.026887  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 106096us
I20260709 13:03:26.027832  8222 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:26.029026  8222 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:25.691021 12839 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.922132  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:25.934931  8369 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:25.935521 12771 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:25.944551  8250 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:03:25.945406  8248 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
W20260709 13:03:25.966919  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:26.036919  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 114839us
I20260709 13:03:25.788770 12849 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.957163 12843 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:25.984346  8256 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:26.001081 12670 consensus_queue.cc:1059] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:03:26.050614  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:25.989965  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 77539.9us
I20260709 13:03:25.997690  9006 rpcz_store.cc:276] 0709 13:03:24.577370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.577590 (+   220us) service_pool.cc:224] Handling call
0709 13:03:24.760992 (+183402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:25.808838 (+1047846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.833912 (+ 25074us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54006e23d0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:25.834465 (+   553us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":297320192}
W20260709 13:03:25.998256  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 202490us
I20260709 13:03:25.998754  8974 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:03:26.003436  8427 debug-util.cc:398] Leaking SignalData structure 0x7b08000b12a0 after lost signal to thread 8189
I20260709 13:03:26.004138  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:26.055404  8427 debug-util.cc:398] Leaking SignalData structure 0x7b0800382a60 after lost signal to thread 8281
W20260709 13:03:26.055904  8427 debug-util.cc:398] Leaking SignalData structure 0x7b080014ad20 after lost signal to thread 8282
I20260709 13:03:25.991505 12643 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.036535 12539 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:25.992311 12659 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:25.992053 12829 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.013857  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2526) took 1580 ms. Trace:
I20260709 13:03:25.652205  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 666) took 1430 ms. Trace:
I20260709 13:03:25.996806 12798 raft_consensus.cc:740] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:25.930373 12844 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:26.034406  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:26.035121  9000 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:03:25.984236 12403 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:25.931380 12825 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:26.010314 12693 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:26.044158 12849 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:26.076431  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:26.045663 12806 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:26.091288 12774 consensus_queue.cc:260] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:26.007850 12784 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:26.046083 12620 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:26.093807 12620 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:25.997030 12678 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:26.046981  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 112113us
I20260709 13:03:26.096120 12678 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:26.096649 12843 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:26.096695 12678 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:26.096148 12620 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:26.048666  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.958477 12842 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:26.098399  9002 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:03:26.041930 12839 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:26.057379  8427 debug-util.cc:398] Leaking SignalData structure 0x7b080035fa00 after lost signal to thread 8356
I20260709 13:03:26.099444 12774 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:26.099779 12842 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:26.029894 12810 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:26.025810 12383 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:03:26.100396  8427 debug-util.cc:398] Leaking SignalData structure 0x7b08003ce9e0 after lost signal to thread 8430
I20260709 13:03:26.100489 12839 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:26.100410 12810 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.008500  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:26.101261 12810 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:26.026441 12730 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:26.102046 12730 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:26.104043 12730 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:26.064040  9004 raft_consensus.cc:1240] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:26.100605 12383 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:26.104518  8972 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:26.065060  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:26.064536 12643 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:26.043272 12851 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:26.106186 12851 raft_consensus.cc:697] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:26.107203 12643 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:26.107137 12851 consensus_queue.cc:237] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:26.042821 12789 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:26.109666 12794 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:26.007328 12805 consensus_queue.cc:260] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:26.110227 12794 raft_consensus.cc:697] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:26.110446  8397 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:03:26.110929  8397 pending_rounds.cc:85] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 5
I20260709 13:03:26.110460 12805 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:26.112042 12771 raft_consensus.cc:3055] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:26.112442 12771 raft_consensus.cc:740] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:26.098435 12804 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.040033 12689 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:26.047899 12830 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:26.064246  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:26.065245  8251 rpcz_store.cc:276] 0709 13:03:24.222197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.222365 (+   168us) service_pool.cc:224] Handling call
0709 13:03:24.287307 (+ 64942us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:24.360041 (+ 72734us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:24.360075 (+    34us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:24.371686 (+ 11611us) log.cc:844] Serialized 78 byte log entry
0709 13:03:24.467630 (+ 95944us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:24.468000 (+   370us) log.cc:844] Serialized 14 byte log entry
0709 13:03:25.649956 (+1181956us) log.cc:844] Serialized 14 byte log entry
0709 13:03:25.651803 (+  1847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.651887 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:25.651934 (+    47us) raft_consensus.cc:1685] finished
0709 13:03:25.651948 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.652138 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42036,"prepare.queue_time_us":62918,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":74,"raft.queue_time_us":246956,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":42176,"spinlock_wait_cycles":1408,"thread_start_us":1211328,"threads_started":3,"wal-append.queue_time_us":146470}
I20260709 13:03:26.014652 12718 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:03:26.071440  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:26.045697 12840 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:03:26.123389  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 8 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.124517  8330 raft_consensus.cc:2393] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 8: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:26.073524  8324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:37920
W20260709 13:03:26.078763  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 88658.3us
I20260709 13:03:26.125350  8324 raft_consensus.cc:606] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:03:26.125732  8324 raft_consensus.cc:614] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:26.073796 12659 raft_consensus.cc:697] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:26.063673 12539 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:26.082141 12798 consensus_queue.cc:260] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:26.083076 12807 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.042s
W20260709 13:03:26.085279 12855 log.cc:927] Time spent T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:03:26.085523 12721 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:26.086004 12232 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:26.089375 12693 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:26.089781 12825 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.073977 12829 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:26.066764 12785 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:26.065382 12707 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:26.090716  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:26.065119  8403 rpcz_store.cc:276] 0709 13:03:24.430435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.430571 (+   136us) service_pool.cc:224] Handling call
0709 13:03:24.446203 (+ 15632us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:24.446402 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:25.992628 (+1546226us) log.cc:844] Serialized 14 byte log entry
0709 13:03:26.009868 (+ 17240us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:26.009922 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:26.009985 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.010076 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:26.010220 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4482,"prepare.queue_time_us":371889,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":109,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":13908,"thread_start_us":1546780,"threads_started":2,"wal-append.queue_time_us":70042}
I20260709 13:03:26.094348 12784 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:26.098759  9002 pending_rounds.cc:85] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: Aborting all ops after (but not including) 9
I20260709 13:03:26.190757  9002 pending_rounds.cc:107] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:03:26.191143  9002 raft_consensus.cc:2889] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:26.062868  8295 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:26.111126 12794 consensus_queue.cc:237] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:26.073926  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:26.074615  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:26.058885 12595 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:26.077108  8404 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:26.118185  8972 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:03:26.122067  8427 debug-util.cc:398] Leaking SignalData structure 0x7b08003388e0 after lost signal to thread 9034
W20260709 13:03:26.119328  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.111171  8397 pending_rounds.cc:107] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:03:26.122591  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:26.113852 12403 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.126735  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:26.109603 12789 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.081284 12849 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:26.232563  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 178676us
I20260709 13:03:26.128244  8974 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:26.155881 12858 log.cc:927] Time spent T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:03:26.156014 12610 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:26.156479  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.160302  8330 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:26.158985 12706 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:26.164288  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.239351  8326 raft_consensus.cc:2393] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 12: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:03:26.168802  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.115377 12620 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:26.187218  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.151881 12539 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:26.150858 12730 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.193358  8398 raft_consensus.cc:2393] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 8: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:26.113924 12804 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:26.115918 12796 consensus_queue.cc:1059] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 13:03:26.113270 12771 consensus_queue.cc:260] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:26.258956  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.259971  8332 raft_consensus.cc:2410] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:03:26.198169 12693 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:26.227113 12670 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:26.213336  8397 raft_consensus.cc:2889] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:26.232368 12789 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:26.211838 12840 consensus_queue.cc:1059] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:26.211687  9002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 898) took 1989 ms. Trace:
I20260709 13:03:26.127489 12643 consensus_queue.cc:1059] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:03:26.200529 12831 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.219426 12642 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:03:26.213510  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:26.200913  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 82816.8us
W20260709 13:03:26.221925  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:26.274698  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 146511us
I20260709 13:03:26.261215 12785 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:26.237703  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 168237us
W20260709 13:03:26.241015  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 212020us
W20260709 13:03:26.242319  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:26.282032  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.199453 12798 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:26.245863  8369 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:26.243965  8220 leader_election.cc:336] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.577s (SENT)
I20260709 13:03:26.251147  8295 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:26.197063 12635 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:26.193152  8399 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:26.282594  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:26.261126  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.285589  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:26.285748  8326 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:26.127154 12689 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:26.286233  8251 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
I20260709 13:03:26.270987  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.270618  9002 rpcz_store.cc:276] 0709 13:03:24.222017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.230199 (+  8182us) service_pool.cc:224] Handling call
0709 13:03:24.297451 (+ 67252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.191691 (+1894240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.199848 (+  8157us) spinlock_profiling.cc:243] Waited 1.8 s on lock 0x7b540057f8d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:26.211600 (+ 11752us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4140335488}
I20260709 13:03:26.198861 12707 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:26.188369 12825 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:26.276930 12857 log.cc:927] Time spent T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.165s	user 0.000s	sys 0.002s
I20260709 13:03:26.276989 12643 raft_consensus.cc:3055] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:03:26.277482 12863 log.cc:927] Time spent T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.140s	user 0.002s	sys 0.000s
I20260709 13:03:26.277874 12810 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:26.278390 12862 log.cc:927] Time spent T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.199s	user 0.000s	sys 0.002s
I20260709 13:03:26.238766 12403 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:26.283252  8332 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "767eba28679649519167576023555b60"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:03:26.319293  8332 raft_consensus.cc:606] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:26.319736  8332 raft_consensus.cc:614] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:26.284299  8220 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:26.233935 12784 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:26.286936  8330 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:03:26.287712  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:26.266362 12670 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:26.296200 12849 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:26.263381 12730 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:26.278319 12383 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:26.293838  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:26.294629 12610 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:26.258738 12620 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:26.306849  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.368136  8326 raft_consensus.cc:2410] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:03:26.308722 12706 raft_consensus.cc:3055] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:26.269579 12797 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:03:26.311678  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.377300  8327 raft_consensus.cc:2410] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:03:26.283164 12771 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:26.380856  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.331017  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:26.382277  8402 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:03:26.383283  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:26.332549 12839 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:26.290133  8295 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:26.383929  8402 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:26.384759  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:26.340453  8325 raft_consensus.cc:2410] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 12 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:03:26.339417  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:26.280920 12831 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:26.266607 12659 consensus_queue.cc:237] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:26.342327  8406 raft_consensus.cc:1240] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:03:26.342571 12869 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:26.343055  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:26.343998  8400 raft_consensus.cc:1240] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:26.345588 12670 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:26.346138  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:26.290611 12635 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:26.305467 12796 consensus_queue.cc:1059] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:03:26.304265 12643 raft_consensus.cc:740] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:26.295347 12840 consensus_queue.cc:1059] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:03:26.355983  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:26.404829  8399 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:26.404942  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:26.358953  8401 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:03:26.356729  9004 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:26.354489 12784 raft_consensus.cc:697] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:26.349216 12718 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:26.375638  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.328593  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.408460  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:26.408910  8330 raft_consensus.cc:2368] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10.
W20260709 13:03:26.336340  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 90577.8us
I20260709 13:03:26.339383 12810 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:26.386680  8974 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:03:26.396052  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 105977us
I20260709 13:03:26.375814 12706 raft_consensus.cc:740] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:26.402243  8403 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:26.362681 12829 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:26.406463  8404 raft_consensus.cc:2393] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 10: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:26.400789  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:26.407480  8332 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:26.409217  8402 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:26.419917  8398 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:26.421805 12794 raft_consensus.cc:3055] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:26.421859  8295 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:26.422375 12794 raft_consensus.cc:740] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:26.423159 12784 consensus_queue.cc:237] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 7.10, Last appended by leader: 10, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:26.423360 12794 consensus_queue.cc:260] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:26.424511 12794 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:26.424623 12854 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:26.425074 12854 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.425393 12854 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:26.425943  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 181986us
I20260709 13:03:26.426126  8325 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:03:26.426740  8325 raft_consensus.cc:606] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:26.427080  8325 raft_consensus.cc:614] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:26.427119 12718 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.427803 12718 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:26.430095  8325 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:03:26.426280  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:26.430780  8325 raft_consensus.cc:606] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:03:26.408007 12721 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:26.431107  8327 raft_consensus.cc:2410] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:03:26.432075  8222 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:26.432960  8222 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:26.433228 12232 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:26.433705 12232 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.434041 12232 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:26.434442 12842 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:26.434618 12839 raft_consensus.cc:3055] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:26.434881 12842 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:26.435117 12839 raft_consensus.cc:740] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:26.435371 12842 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:26.436053 12839 consensus_queue.cc:260] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:26.437904  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:26.438555  8403 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:26.438768  8325 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:03:26.439392  8325 raft_consensus.cc:606] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:03:26.439844  8325 raft_consensus.cc:614] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:26.429878 12643 consensus_queue.cc:260] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:26.426939  8220 leader_election.cc:336] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.713s (SENT)
I20260709 13:03:26.463783  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.375712 12807 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:03:26.483556  8330 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:26.483371  8402 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:26.484500  8220 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:26.379868 12780 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:03:26.465171  8369 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:26.403097 12849 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:26.489807  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:26.490620  8398 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 12: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:26.437374 12839 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:26.495824 12842 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:26.493825  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.494164 12796 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:26.489807  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:26.466033  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:26.502166  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 78527.8us
I20260709 13:03:26.503366  8295 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:26.508159  8248 raft_consensus.cc:1240] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 9. Current term is 10. Ops: [7.7-9.8]
I20260709 13:03:26.513743  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:26.517979  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:26.518908  8402 raft_consensus.cc:2393] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 9: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:26.530730  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2513) took 2279 ms. Trace:
I20260709 13:03:26.537358  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:26.539014  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.539583  8330 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
W20260709 13:03:26.540410  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 249134us
I20260709 13:03:26.542691  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:26.538120  8256 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:26.546578  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.547381  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:26.546578  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:26.547933  8332 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:26.548154  8251 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:26.547199  8330 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:03:26.492045 12810 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:26.531302  8405 rpcz_store.cc:276] 0709 13:03:24.251506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.252778 (+  1272us) service_pool.cc:224] Handling call
0709 13:03:24.253125 (+   347us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:03:24.304371 (+ 51246us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:24.317771 (+ 13400us) log.cc:844] Serialized 14 byte log entry
0709 13:03:25.969215 (+1651444us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:25.983643 (+ 14428us) log.cc:844] Serialized 78 byte log entry
0709 13:03:26.065192 (+ 81549us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:26.065233 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.065309 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:26.212850 (+147541us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:26.497016 (+284166us) raft_consensus.cc:1685] finished
0709 13:03:26.530447 (+ 33431us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:26.530654 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":177,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":43,"raft.queue_time_us":10838,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":7078,"spinlock_wait_cycles":40733440,"thread_start_us":1727637,"threads_started":2,"wal-append.queue_time_us":638154,"wal-append.run_cpu_time_us":2084,"wal-append.run_wall_time_us":1001166}
I20260709 13:03:26.550009  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:26.550881  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:26.551409  8252 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:26.551676  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:26.552374  8249 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:26.553444  8398 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:26.554558  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:26.546829  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:26.557760  8253 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:26.558840  8326 raft_consensus.cc:1275] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:03:26.561064  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:26.562793  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:26.563345  8330 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:26.563279  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.555289  8255 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:26.567257  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:26.568214  8248 raft_consensus.cc:2393] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 13: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:26.568704  8256 raft_consensus.cc:2393] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 13: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:03:26.569800  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 213636us
I20260709 13:03:26.570465  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:03:26.570914  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.570995  8405 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:26.571215  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.536187 12830 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:26.486187 12635 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 13:03:26.562656  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.569744  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.582561  8327 raft_consensus.cc:2393] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 13: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:26.573534  8328 raft_consensus.cc:2393] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 12: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:26.574193  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.501669 12796 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:26.576818  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:26.588910  8399 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:26.581646  8323 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
W20260709 13:03:26.577819  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 204849us
I20260709 13:03:26.488293 12642 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:03:26.538479 12610 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:26.585402  8326 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:26.495872 12840 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 13:03:26.596387  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:26.597254  8248 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
W20260709 13:03:26.593189  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 166262us
W20260709 13:03:26.593233  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.497253 12232 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:26.562178 12643 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:26.601485  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:26.550535 12810 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:26.638468  8220 leader_election.cc:336] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.961s (SENT)
I20260709 13:03:26.644544  8220 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:26.638829  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:26.645583  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:26.639930  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.537139 12678 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:03:26.641309  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:26.429891 12706 consensus_queue.cc:260] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:26.708244  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.709517  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.649143  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:26.647033  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:26.648553  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.711213  8326 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:26.597203 12796 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:26.714979  8325 raft_consensus.cc:2393] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:26.715840  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:26.719528  8327 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:26.716894  8253 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
W20260709 13:03:26.717042  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:26.721863  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 218566us
W20260709 13:03:26.716892  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.726447  8332 raft_consensus.cc:2410] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:03:26.728791  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.729389  8331 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:26.739507  8252 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:26.742130  8256 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:26.756626  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:26.757707  8332 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:26.752316  8248 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:03:26.639972  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:26.727768 12383 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:26.762895 12383 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:26.580369 12789 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:26.764019  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:26.764209  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:26.769758  8254 raft_consensus.cc:2368] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10.
I20260709 13:03:26.747849  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:26.753275  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:26.765174  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:26.764669  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:26.770995  8255 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:26.762320  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:26.771916  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:26.763592  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:26.767479  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:26.774142  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:26.725512 12810 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:26.780189  9004 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:03:26.709105 12610 raft_consensus.cc:697] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:26.774734  8248 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:26.773243  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:26.741747 12689 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:26.731015 12842 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:26.837527  8278 debug-util.cc:398] Leaking SignalData structure 0x7b08000d2640 after lost signal to thread 7167
I20260709 13:03:26.837352  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.786121  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:26.784555  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 235057us
I20260709 13:03:26.785315  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:26.841562  8330 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:26.860000  8326 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:26.787683  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.792330  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.852213  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:26.743363 12884 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.853566  8325 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
W20260709 13:03:26.854499  8278 debug-util.cc:398] Leaking SignalData structure 0x7b0800086b80 after lost signal to thread 8189
W20260709 13:03:26.866650  8278 debug-util.cc:398] Leaking SignalData structure 0x7b08002b9800 after lost signal to thread 8281
W20260709 13:03:26.866931  8278 debug-util.cc:398] Leaking SignalData structure 0x7b0800310e40 after lost signal to thread 8282
W20260709 13:03:26.867821  8278 debug-util.cc:398] Leaking SignalData structure 0x7b08003a1d60 after lost signal to thread 8356
W20260709 13:03:26.868156  8278 debug-util.cc:398] Leaking SignalData structure 0x7b08002efcc0 after lost signal to thread 8370
W20260709 13:03:26.868376  8278 debug-util.cc:398] Leaking SignalData structure 0x7b080039e1e0 after lost signal to thread 8371
W20260709 13:03:26.868564  8278 debug-util.cc:398] Leaking SignalData structure 0x7b08003d5f80 after lost signal to thread 8372
W20260709 13:03:26.869899  8278 debug-util.cc:398] Leaking SignalData structure 0x7b080031e1e0 after lost signal to thread 8430
W20260709 13:03:26.870736  8278 debug-util.cc:398] Leaking SignalData structure 0x7b0800311680 after lost signal to thread 9006
I20260709 13:03:26.861640  8331 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:26.752952 12780 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:03:26.874599  8278 debug-util.cc:398] Leaking SignalData structure 0x7b08004091e0 after lost signal to thread 9034
I20260709 13:03:26.874816  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:26.874966  8278 debug-util.cc:398] Leaking SignalData structure 0x7b08002f1560 after lost signal to thread 12580
W20260709 13:03:26.875864  8278 debug-util.cc:398] Leaking SignalData structure 0x7b08003f1620 after lost signal to thread 12595
I20260709 13:03:26.876112  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:26.877728  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:26.876111  8278 debug-util.cc:398] Leaking SignalData structure 0x7b08003f1920 after lost signal to thread 12620
I20260709 13:03:26.879669  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.880437  8330 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:26.777472 12706 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:26.760469 12849 raft_consensus.cc:697] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:26.774243 12383 raft_consensus.cc:697] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:26.886718  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:26.887212  8327 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
W20260709 13:03:26.889387  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 289766us
I20260709 13:03:26.890575  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:26.890575  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:26.887181  8278 debug-util.cc:398] Leaking SignalData structure 0x7b08003b2260 after lost signal to thread 12642
W20260709 13:03:26.919101  8278 debug-util.cc:398] Leaking SignalData structure 0x7b08002ebe20 after lost signal to thread 12670
W20260709 13:03:26.919622  8278 debug-util.cc:398] Leaking SignalData structure 0x7b080036b340 after lost signal to thread 12678
W20260709 13:03:26.919909  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.887481  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:03:26.920559  8278 debug-util.cc:398] Leaking SignalData structure 0x7b0800335ae0 after lost signal to thread 12684
I20260709 13:03:26.920524  8326 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:26.921028  8278 debug-util.cc:398] Leaking SignalData structure 0x7b08003d4e00 after lost signal to thread 12689
W20260709 13:03:26.921310  8278 debug-util.cc:398] Leaking SignalData structure 0x7b080031f300 after lost signal to thread 12706
W20260709 13:03:26.921589  8278 debug-util.cc:398] Leaking SignalData structure 0x7b080040d2e0 after lost signal to thread 12718
W20260709 13:03:26.921829  8278 debug-util.cc:398] Leaking SignalData structure 0x7b0800337100 after lost signal to thread 12721
I20260709 13:03:26.839107 12825 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:26.910270  8372 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:26.851712 12620 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:26.925882  8323 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:26.928270  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:26.889523 12829 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:26.786063 12693 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.134s
I20260709 13:03:26.885040 12890 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:26.937407  8220 leader_election.cc:336] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.004s (SENT)
I20260709 13:03:26.942528  8220 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:26.949332  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 236747us
I20260709 13:03:26.938961 12829 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:03:26.995167 12875 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.359s	user 0.000s	sys 0.001s
W20260709 13:03:26.996655 12883 log.cc:927] Time spent T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.253s	user 0.001s	sys 0.000s
I20260709 13:03:27.001669 12730 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:27.002104 12879 log.cc:927] Time spent T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.413s	user 0.001s	sys 0.000s
I20260709 13:03:26.884014 12884 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:26.944747 12693 consensus_queue.cc:1243] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:03:26.910239 12854 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:26.927525 12846 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:26.995558 12885 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.073241 12829 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:26.936468 12642 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:26.927320 12892 consensus_queue.cc:1059] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:03:26.910364 12780 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:26.922044  8278 debug-util.cc:398] Leaking SignalData structure 0x7b0800304f40 after lost signal to thread 12730
I20260709 13:03:27.075546  8326 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
W20260709 13:03:27.114126  8278 debug-util.cc:398] Leaking SignalData structure 0x7b0800308d80 after lost signal to thread 12779
I20260709 13:03:27.084232  8332 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
W20260709 13:03:27.114462  8278 debug-util.cc:398] Leaking SignalData structure 0x7b080015ed40 after lost signal to thread 12780
W20260709 13:03:27.088924  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 372263us
I20260709 13:03:26.940085 12620 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.969231 12890 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:27.071905 12383 consensus_queue.cc:237] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:27.072238 12825 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:27.111088  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 193806us
W20260709 13:03:27.114761  8278 debug-util.cc:398] Leaking SignalData structure 0x7b08002ee3a0 after lost signal to thread 12785
W20260709 13:03:27.130209  8278 debug-util.cc:398] Leaking SignalData structure 0x7b08003904c0 after lost signal to thread 12796
W20260709 13:03:27.130484  8278 debug-util.cc:398] Leaking SignalData structure 0x7b0800192b60 after lost signal to thread 12797
W20260709 13:03:27.130756  8278 debug-util.cc:398] Leaking SignalData structure 0x7b08003fa540 after lost signal to thread 12808
W20260709 13:03:27.131016  8278 debug-util.cc:398] Leaking SignalData structure 0x7b080036d7a0 after lost signal to thread 12812
I20260709 13:03:27.125946  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:27.130046  8369 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:27.110185 12232 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:27.109704 12642 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:27.136593  8326 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:27.137612  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 496880us
I20260709 13:03:27.128727  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:27.057771 12610 consensus_queue.cc:237] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:27.063382 12849 consensus_queue.cc:237] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:27.147365 12780 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:27.145124  8332 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
I20260709 13:03:27.142715 12880 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:27.150190  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:27.150810  8332 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:27.151604 12810 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:27.152064  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:27.152844  8398 raft_consensus.cc:2368] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 14. Current term is 15.
I20260709 13:03:27.153067  8369 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:27.155224  8251 consensus_meta.cc:287] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.607s	user 0.009s	sys 0.001s
I20260709 13:03:27.155630  8251 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:27.158367  8369 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:27.155184  8297 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.006s	sys 0.017s
W20260709 13:03:27.159525  8297 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.006s	sys 0.017s
I20260709 13:03:27.160912  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:27.161094  8297 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:27.165688  8326 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:03:27.166821  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:27.153348 12670 consensus_meta.cc:287] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.711s	user 0.002s	sys 0.004s
I20260709 13:03:27.169183  8326 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 13:03:27.135826 12889 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:27.178009  8326 pending_rounds.cc:85] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 7
I20260709 13:03:27.110371 12885 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:27.156242 12642 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:27.187541  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.133909 12620 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:27.168897 12880 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:27.181011  8297 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:27.181727  8326 pending_rounds.cc:107] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:03:27.177518 12689 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:27.214161  8326 raft_consensus.cc:2889] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:27.216869  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:27.156908 12730 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:03:27.181046 12789 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:27.225564 12891 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:03:27.226020  8326 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
W20260709 13:03:27.227228  8249 consensus_meta.cc:287] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.660s	user 0.007s	sys 0.000s
W20260709 13:03:27.227283 12887 log.cc:927] Time spent T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:03:27.201218  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:27.247884  9009 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:27.230531  8332 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:27.234773  8249 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:27.191817  8252 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:03:27.215992 12689 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:27.250329  8252 pending_rounds.cc:85] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 9
I20260709 13:03:27.253805  8252 pending_rounds.cc:107] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:03:27.254236  8252 raft_consensus.cc:2889] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:27.218897 12895 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:27.257006  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.219800 12831 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:27.181671 12670 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:27.219439 12880 raft_consensus.cc:697] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:03:27.259660  8405 consensus_meta.cc:287] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.688s	user 0.007s	sys 0.001s
I20260709 13:03:27.263401  8298 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:27.265491  8405 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:27.233490 12898 raft_consensus.cc:3055] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:03:27.273837  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 129276us
I20260709 13:03:27.274878  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:27.275475  8252 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:27.279498 12839 consensus_meta.cc:287] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.746s	user 0.008s	sys 0.000s
I20260709 13:03:27.254944 12689 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:27.282940 12539 consensus_meta.cc:287] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.020s	user 0.007s	sys 0.002s
I20260709 13:03:27.283360 12539 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:27.186900 12889 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:27.281147 12895 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:27.285709  8250 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: []
W20260709 13:03:27.327610  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 52938.7us
I20260709 13:03:27.289347  8252 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:27.251434 12403 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:27.331748  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:27.336061  8405 raft_consensus.cc:2393] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:27.319067  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:27.336512  8252 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:27.336860  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.289414 12689 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:27.321722  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:27.317796  9002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 910) took 1021 ms. Trace:
I20260709 13:03:27.337421  8324 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 8.
I20260709 13:03:27.285986 12804 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:27.331748  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:27.314311 12886 log.cc:927] Time spent T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
W20260709 13:03:27.320765 12888 log.cc:927] Time spent T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
W20260709 13:03:27.303917  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.338658  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:27.340052  8326 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:03:27.331975  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:27.337812  9002 rpcz_store.cc:276] 0709 13:03:26.296556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.296710 (+   154us) service_pool.cc:224] Handling call
0709 13:03:26.311054 (+ 14344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:26.319019 (+  7965us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:26.319056 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:26.339650 (+ 20594us) log.cc:844] Serialized 26 byte log entry
0709 13:03:26.487388 (+147738us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b18003d71a0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e0d1 00007f8828779a0f
0709 13:03:26.595937 (+108549us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:26.596460 (+   523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.596864 (+   404us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:27.317364 (+720500us) raft_consensus.cc:1685] finished
0709 13:03:27.317394 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.317716 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7512,"raft.queue_time_us":21077,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":15139,"spinlock_wait_cycles":40298880,"thread_start_us":108387,"threads_started":1,"wal-append.queue_time_us":533977}
I20260709 13:03:27.332758  8406 raft_consensus.cc:1275] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:03:27.374557  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 244546us
W20260709 13:03:27.320405 12794 consensus_meta.cc:287] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.895s	user 0.007s	sys 0.002s
I20260709 13:03:27.372886  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:27.383914  8327 raft_consensus.cc:2376] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9. Re-sending same reply.
I20260709 13:03:27.378540  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:27.384538  8405 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:27.269896 12898 raft_consensus.cc:740] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:27.273420 12884 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:27.381208  8326 pending_rounds.cc:85] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 9
I20260709 13:03:27.290503 12620 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:27.377672 12643 consensus_meta.cc:287] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.728s	user 0.007s	sys 0.003s
I20260709 13:03:27.327812 12895 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:27.385193  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:27.388309  8326 pending_rounds.cc:107] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:03:27.397804  8326 raft_consensus.cc:2889] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:27.285411 12539 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:27.386227  8249 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: []
W20260709 13:03:27.379693 12771 consensus_meta.cc:287] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.985s	user 0.010s	sys 0.001s
I20260709 13:03:27.392800  8397 raft_consensus.cc:2393] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 10: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:27.294575 12889 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:27.292310 12890 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:27.371819 12403 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:27.427049  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 44726.4us
I20260709 13:03:27.430029  8328 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:27.422768  8972 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:27.432093  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:27.384641 12854 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:27.449890  8256 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:03:27.419075 12895 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:27.454998  8249 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:27.435846  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:27.455201  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:27.459941  8251 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:27.389022 12804 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.065s
W20260709 13:03:27.454737 12798 consensus_meta.cc:287] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 1.158s	user 0.011s	sys 0.001s
I20260709 13:03:27.483388  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:27.484189  9006 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:27.394066 12904 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 13:03:27.486415  9007 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:27.488001  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:27.421347 12831 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:03:27.467691 12810 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:27.506116  8251 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:27.455902 12403 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:27.508113  8405 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:27.504631  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:27.470603 12909 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:27.471634 12794 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
W20260709 13:03:27.662695  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 481639us
I20260709 13:03:27.693984  8297 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:27.570019  8222 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:03:27.666944  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.673802  8974 leader_election.cc:400] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 14. Current term is 15.
W20260709 13:03:27.655682 12915 log.cc:927] Time spent T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:03:27.472705 12789 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:03:27.467902 12880 consensus_queue.cc:237] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:27.712352  8297 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:27.714208  8403 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: [12.11-12.11]
W20260709 13:03:27.551280 12911 log.cc:927] Time spent T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:03:27.502223 12900 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:27.483477 12805 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:03:27.547848  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:27.557837 12607 raft_consensus.cc:3055] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:27.510397 12909 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:27.490543 12718 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:27.551854 12885 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:27.727478  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:27.728385  8403 raft_consensus.cc:2368] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9.
I20260709 13:03:27.552249 12893 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:27.683720  8324 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2462) took 1496 ms (client timeout 1738 ms). Trace:
I20260709 13:03:27.729353  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:27.731539  8324 rpcz_store.cc:269] 0709 13:03:26.186830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.186961 (+   131us) service_pool.cc:224] Handling call
0709 13:03:27.683503 (+1496542us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400364554. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:27.683649 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2521389952}
I20260709 13:03:27.732941  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:27.689536 12670 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:27.684665 12707 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:27.686283  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:27.556859 12829 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:27.735189  9002 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:03:27.733642  8324 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:03:27.519148  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:27.736718  8326 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:27.583127  8328 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:27.565824 12642 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:27.575176  9006 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
W20260709 13:03:27.567211 12706 consensus_meta.cc:287] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.645s	user 0.007s	sys 0.003s
W20260709 13:03:27.695053  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 125089us
I20260709 13:03:27.695195  9000 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 8.
I20260709 13:03:27.697294  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:27.698161  8400 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:03:27.762339  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 95396.8us
I20260709 13:03:27.762377  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:27.661607 12232 consensus_queue.cc:1059] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:27.766772 12232 raft_consensus.cc:3055] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:27.767266 12232 raft_consensus.cc:740] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:27.768314 12232 consensus_queue.cc:260] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:27.568540 12796 consensus_meta.cc:287] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.814s	user 0.009s	sys 0.000s
I20260709 13:03:27.735482 12854 consensus_queue.cc:1059] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:03:27.770057 12854 consensus_queue.cc:1243] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:03:27.769768 12796 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:27.763249  8249 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:27.735368 12620 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:27.771366 12620 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:27.771783 12620 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:27.772594 12796 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:27.698853  8401 raft_consensus.cc:1240] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:27.709352  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 199775us
I20260709 13:03:27.774448  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:27.775520 12642 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:27.775584  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:27.776494  8401 raft_consensus.cc:2393] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 12: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:27.710894  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:27.717079  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:27.777128  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:27.704254 12539 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:27.723349  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2229) took 1189 ms. Trace:
I20260709 13:03:27.723465  8323 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:27.726152 12894 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:27.731999  8403 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:27.686172  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 685) took 1358 ms. Trace:
I20260709 13:03:27.782671  8250 rpcz_store.cc:276] 0709 13:03:26.327538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.327703 (+   165us) service_pool.cc:224] Handling call
0709 13:03:26.328071 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:27.328126 (+1000055us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.685918 (+357792us) spinlock_profiling.cc:243] Waited 957 ms on lock 0x7b540009ecd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:27.686108 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2201618176}
I20260709 13:03:27.736864  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:27.784245  8324 raft_consensus.cc:2393] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 10: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:27.711714 12771 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.786423  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:27.723073 12900 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:27.714905 12914 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:27.730933 12403 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:27.714411  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:27.777388  8398 raft_consensus.cc:2410] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:03:27.777837  8249 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:27.778870  8406 raft_consensus.cc:1240] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 9. Current term is 11. Ops: []
I20260709 13:03:27.763182  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:27.734460 12909 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:27.734272 12607 raft_consensus.cc:740] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:27.781977  8331 rpcz_store.cc:276] 0709 13:03:26.534420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.875572 (+341152us) service_pool.cc:224] Handling call
0709 13:03:26.926643 (+ 51071us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.926867 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:27.220163 (+293296us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.722992 (+502829us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:27.723046 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:27.723105 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.723184 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.723296 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4666,"prepare.queue_time_us":252456,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":3169,"raft.queue_time_us":77511,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":11099,"thread_start_us":654626,"threads_started":3,"wal-append.queue_time_us":278166}
I20260709 13:03:27.745748 12689 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:27.759647 12635 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.784686 12789 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:03:27.784525 12831 consensus_queue.cc:1059] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:03:27.794950  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:27.724234 12889 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:03:27.799722  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:27.800915  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:27.792857  8326 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:27.785261  8222 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:27.785262 12539 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:27.785037 12670 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.746234 12893 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:27.810199  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:27.811708  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:27.794446 12900 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:27.801554 12232 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:27.806098  8297 leader_election.cc:400] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9.
I20260709 13:03:27.828565  8297 leader_election.cc:403] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:03:27.820128  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:27.831007  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:27.831816  8400 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:27.827256  8249 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:03:27.842595  8405 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:27.843508  8297 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:27.844822  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:27.844852  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:27.846602  8403 raft_consensus.cc:2368] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 9. Current term is 10.
I20260709 13:03:27.847810  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:27.845191  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:27.818125 12794 raft_consensus.cc:3055] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:27.816520 12539 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:27.798560 12771 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:27.864289  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 139762us
I20260709 13:03:27.864688  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:27.819367 12670 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:27.793747 12894 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:27.866787  8400 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:27.866143  8249 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 13: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:27.814430 12635 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:27.855336 12383 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.872442  8403 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:03:27.849324 12403 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:27.853641 12914 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:03:27.869824  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 90766.5us
I20260709 13:03:27.874843  8323 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:27.855098 12854 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.875108  9002 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:27.869534  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:27.854120 12794 raft_consensus.cc:740] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:27.886554  8398 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:27.887032 12854 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:27.888312  9002 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:27.888423  8323 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:03:27.890358  8399 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:27.893556  8398 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:03:27.899098 12854 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:27.900648  8399 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:27.905503  8249 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
W20260709 13:03:27.926103  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.926896  8256 raft_consensus.cc:1275] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:03:27.927042  8974 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:03:27.891605 12918 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:03:27.874981 12914 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:27.930982  8399 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 6. (term mismatch)
I20260709 13:03:27.931419  8399 pending_rounds.cc:85] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 5
I20260709 13:03:27.931681  8399 pending_rounds.cc:107] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:03:27.931969  8399 raft_consensus.cc:2889] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:27.932389  8399 pending_rounds.cc:107] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:03:27.932714  8399 raft_consensus.cc:2889] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:27.934551  8256 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:27.862900 12607 consensus_queue.cc:260] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:27.925664 12829 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:27.942116  8404 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:03:27.953140  8298 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:27.953281  8403 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:27.876720 12383 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:27.950552  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 169716us
I20260709 13:03:27.931890 12914 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:27.932145 12893 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:27.934089 12831 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:27.965845  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:27.966913  8249 raft_consensus.cc:2393] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 10: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:27.957790  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:27.967949  8399 raft_consensus.cc:2410] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 14 because replica has last-logged OpId of term: 12 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:03:27.968237  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:27.962934  8222 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:27.956634  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:27.963352  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:27.976692  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:27.977645  8399 raft_consensus.cc:2410] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:03:27.958166 12607 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:27.794212 12718 consensus_queue.cc:1175] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ff7a6fe533e94c1bae555b4cae427728" responder_term: 13 status { last_received { term: 7 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 12. Current term is 13. Ops: [12.5-12.5]" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:03:27.971377 12919 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 13:03:27.970587  8404 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:27.982156  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:27.957607 12854 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:27.992705  8402 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 15: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:27.994987 12825 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:27.960882 12885 consensus_queue.cc:1059] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:03:27.997783 12932 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:27.951994 12794 consensus_queue.cc:260] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:28.001670 12938 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:27.961158 12889 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
W20260709 13:03:27.998173 12928 log.cc:927] Time spent T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:03:28.023823 12912 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 13:03:28.009614 12825 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:28.024811 12825 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:28.011879 12924 log.cc:927] Time spent T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
W20260709 13:03:28.029122  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 253148us
I20260709 13:03:27.986985 12620 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:28.010982 12932 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:28.037985  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 75156.2us
I20260709 13:03:28.038861  8222 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:28.039000  8404 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:03:28.040994 12914 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:28.038260 12670 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:28.043036 12936 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:28.043478 12936 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.043824 12936 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:28.046949 12620 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:28.029224 12539 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:28.047920  8324 raft_consensus.cc:2393] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 12: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:28.038383 12932 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:28.049588 12539 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:28.054397 12383 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:28.057055  8222 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:28.055876  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:28.058249  8404 raft_consensus.cc:2410] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:03:28.060801  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:28.061733  8403 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:28.034757 12794 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:28.059276  8972 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:28.067822  8295 leader_election.cc:400] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10.
I20260709 13:03:28.068182  8295 leader_election.cc:403] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:03:28.069166  8295 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:28.070458  8295 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:28.075060  8295 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:28.076121 12944 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:28.076313  8295 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:28.077574 12944 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:28.077821  8295 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:28.078776 12944 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:28.079253  8295 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:28.072165  8220 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:28.081070  8295 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:28.093866  8222 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:28.081770 12670 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:28.080156 12232 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:28.080565 12944 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:28.087181 12885 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:28.064060 12797 consensus_queue.cc:1059] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:03:28.095194 12730 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.026268 12938 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:28.123792 12938 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:28.125046 12944 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:28.125622 12944 raft_consensus.cc:697] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:28.125660 12730 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:28.126616 12944 consensus_queue.cc:237] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:28.128113 12730 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:28.130200 12909 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:28.131364  9005 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:28.138029 12935 log.cc:927] Time spent T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:03:28.112463  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:28.080564 12383 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 13:03:28.140275 12942 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:03:28.147934  8403 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:28.117228  8295 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:28.118059 12796 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:28.152014  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:28.115614 12539 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:28.163955  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 46830.4us
I20260709 13:03:28.146731 12909 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:28.150843 12938 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:28.172514  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:28.174072  8399 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:28.155102 12620 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:28.170024 12930 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:28.142307 12825 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:28.178045  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 84301.7us
I20260709 13:03:28.175185 12539 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:28.181561  8974 leader_election.cc:400] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 9. Current term is 10.
W20260709 13:03:28.176120  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 104020us
I20260709 13:03:28.173115 12889 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:28.181844  8297 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:28.179901  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 69282.4us
I20260709 13:03:28.183099  8220 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:03:28.187155  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.187027  8297 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:28.181888  8974 leader_election.cc:403] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:28.178462 12930 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.197392  8222 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:28.196947  8220 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:28.178416 12938 raft_consensus.cc:697] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:28.202184 12383 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:28.186213  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:28.206154  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 636) took 1100 ms. Trace:
I20260709 13:03:28.182459 12403 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 8. Current term is 9.
I20260709 13:03:28.204438 12774 consensus_queue.cc:1059] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:03:28.208424 12889 consensus_queue.cc:1059] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:03:28.204994 12930 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:28.183529 12232 raft_consensus.cc:697] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:28.214568  8974 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:28.227531  8974 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:28.229070 12829 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:28.198204 12637 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:03:28.212206  8399 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:28.214654  8332 rpcz_store.cc:276] 0709 13:03:27.106532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.151740 (+ 45208us) service_pool.cc:224] Handling call
0709 13:03:28.205930 (+1054190us) spinlock_profiling.cc:243] Waited 77.8 ms on lock 0x7b540011d5d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:28.206082 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":178843520}
I20260709 13:03:28.199455 12949 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:28.185333 12539 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:28.197919 12909 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
W20260709 13:03:28.220216 12895 consensus_meta.cc:287] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.749s	user 0.007s	sys 0.002s
I20260709 13:03:28.204221 12936 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:03:28.214812 12914 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:28.197633 12946 consensus_queue.cc:1059] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:03:28.207744 12952 raft_consensus.cc:2764] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 13:03:28.267086  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 78519.2us
I20260709 13:03:28.232396 12403 raft_consensus.cc:3055] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:28.233879 12383 raft_consensus.cc:697] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:28.221303 12825 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:28.252676 12829 raft_consensus.cc:697] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 14 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:28.272341 12949 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:28.272930 12829 consensus_queue.cc:237] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:28.278025  8297 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:28.222898 12932 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:28.282585 12946 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:28.283149 12946 raft_consensus.cc:697] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 13 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:28.283969 12946 consensus_queue.cc:237] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:28.288199 12900 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:28.290532  8326 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:03:28.291339  8332 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
W20260709 13:03:28.292374  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.223819 12938 consensus_queue.cc:237] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:28.295534 12914 raft_consensus.cc:993] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728: : Instructing follower e64761f43e4240319c41117cf886d22f to start an election
I20260709 13:03:28.278887 12403 raft_consensus.cc:740] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:03:28.363324  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.281134 12774 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:28.293282 12942 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:28.282063 12825 raft_consensus.cc:3055] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:03:28.377099  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 179746us
I20260709 13:03:28.381093  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:28.270071 12895 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:28.382081  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 195907us
I20260709 13:03:28.268865 12950 consensus_queue.cc:1048] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.111s
I20260709 13:03:28.386260  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:28.386406  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 108461us
I20260709 13:03:28.387058  8297 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:28.388746  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:28.373355 12880 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:28.394582  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 197685us
W20260709 13:03:28.396941  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 169478us
I20260709 13:03:28.402670  8403 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:28.380976 12825 raft_consensus.cc:740] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:28.396844  8220 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:28.404465  8399 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:28.378611 12942 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:28.388091 12670 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10.
I20260709 13:03:28.386626 12914 raft_consensus.cc:1081] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Signalling peer e64761f43e4240319c41117cf886d22f to start an election
I20260709 13:03:28.439589  8220 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:28.383309 12774 raft_consensus.cc:697] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:28.425164  8222 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:28.479169  8403 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:28.295428 12232 consensus_queue.cc:237] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:28.394713 12949 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:28.476467  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:28.397073 12721 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.486284  8399 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:03:28.394893 12955 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:03:28.494977  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 55504.6us
I20260709 13:03:28.496830  8220 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:28.478004 12942 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:28.441344 12868 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:03:28.512516  8328 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2231) took 1953 ms (client timeout 1747 ms). Trace:
W20260709 13:03:28.513096  8328 rpcz_store.cc:269] 0709 13:03:26.548503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.595240 (+ 46737us) service_pool.cc:224] Handling call
0709 13:03:28.502202 (+1906962us) spinlock_profiling.cc:243] Waited 719 ms on lock 0x7b540014cb54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:28.502313 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1653961600}
I20260709 13:03:28.405881 12951 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.093s
I20260709 13:03:28.516057  8328 raft_consensus.cc:1275] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:03:28.519748  9005 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:28.488722 12941 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:28.498888 12949 raft_consensus.cc:697] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:28.398949 12880 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:28.397908 12383 consensus_queue.cc:237] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:28.497952 12721 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:28.536168  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:28.530580  8220 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:28.537626  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:28.536952  9004 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 14 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:28.511979 12955 raft_consensus.cc:697] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:28.515442 12958 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:28.526538 12539 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:28.550076 12895 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:28.521238 12936 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:28.555207 12721 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:28.555294  9008 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:28.477155 12403 consensus_queue.cc:260] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:28.561759  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:28.562628  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:28.562958 12637 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 9. Current term is 10.
I20260709 13:03:28.563887  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:28.571388  8220 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:28.572225  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:28.572860  8223 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:28.516551 12868 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:28.580823 12721 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:28.586061 12941 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:28.586503 12941 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.586864 12941 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:28.586102 12957 log.cc:927] Time spent T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:03:28.584956  8220 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:28.561141 12959 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:28.597709 12936 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:28.598291 12959 raft_consensus.cc:697] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:28.598502  8220 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:28.599474 12936 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:28.599910  8220 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:28.600219 12936 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:28.599479 12959 consensus_queue.cc:237] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:28.600648 12936 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:28.600905 12721 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:28.601766 12932 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:28.603121  8220 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:28.603884 12932 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:28.604981  8220 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:28.570354 12900 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:28.569969 12958 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:28.612708 12958 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:28.613230 12958 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:28.549304 12949 consensus_queue.cc:237] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:28.618973 12880 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:28.559904 12962 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:28.557547 12670 consensus_queue.cc:1059] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:03:28.609620 12936 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:28.515970 12825 consensus_queue.cc:260] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:28.622009 12936 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:28.622166 12825 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:28.622582 12539 consensus_queue.cc:1175] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ff7a6fe533e94c1bae555b4cae427728" responder_term: 12 status { last_received { term: 10 index: 6 } last_committed_idx: 6 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:03:28.623637  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:28.561671 12950 consensus_queue.cc:1059] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:03:28.624823 12950 consensus_queue.cc:1243] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:03:28.625286 12950 raft_consensus.cc:3055] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:28.625609 12950 raft_consensus.cc:740] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:28.625936 12955 consensus_queue.cc:237] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:28.626498 12958 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:28.630225  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:28.632200 12868 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:28.611873 12403 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:28.626418 12950 consensus_queue.cc:260] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:28.610836 12941 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:28.635582 12941 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:28.636945 12950 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:28.641784  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.643530  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:28.640728 12930 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:28.700680  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 95757.3us
I20260709 13:03:28.704540  8400 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:03:28.702950  8401 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:39306
I20260709 13:03:28.708236  8401 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:28.708752  8401 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:28.522163 12796 consensus_queue.cc:1059] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.153s
I20260709 13:03:28.713814  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:28.714951  8399 raft_consensus.cc:2393] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 12: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:03:28.716938  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.740726  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:28.745136  8220 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:28.529506 12774 consensus_queue.cc:237] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:28.702236 12958 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:28.708091 12941 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:28.741699  9008 raft_consensus.cc:2393] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 11: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:28.749933  8220 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:28.712984 12895 consensus_queue.cc:1059] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:03:28.752511 12941 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:28.760293  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 219030us
I20260709 13:03:28.754609 12936 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:03:28.774482  8220 leader_election.cc:336] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.002s (SENT)
W20260709 13:03:28.779273  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.779953  8220 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:28.759848 12958 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:28.783483  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:28.769956 12607 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:28.773545 12941 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:28.795945  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 191924us
I20260709 13:03:28.776155 12895 consensus_queue.cc:1243] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:03:28.815397  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 390206us
I20260709 13:03:28.817519  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:28.817871  8222 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:28.818158  8400 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:28.834257  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:28.835548  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:28.836526  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:28.837026  8405 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:28.841495  8401 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:28.849323  8401 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:28.823990 12958 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:03:28.855382  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 81923.7us
I20260709 13:03:28.821617 12938 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:28.858446  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.866201  8400 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
W20260709 13:03:28.908044 12970 log.cc:927] Time spent T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:03:28.908660 12942 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:28.910054  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:28.903797 12930 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:28.843349 12936 consensus_queue.cc:1059] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:03:28.911877 12955 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.912336 12936 raft_consensus.cc:3055] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:28.912446 12955 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:28.914667 12955 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:28.914991 12932 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:28.915503 12962 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.915558 12932 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:28.916124 12962 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:28.912824 12936 raft_consensus.cc:740] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:28.911487  9002 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:28.926354  9004 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
W20260709 13:03:28.925707 12972 log.cc:927] Time spent T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:03:28.935559  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 646) took 1157 ms. Trace:
I20260709 13:03:28.943997  8328 rpcz_store.cc:276] 0709 13:03:27.778146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.550422 (+772276us) service_pool.cc:224] Handling call
0709 13:03:28.550718 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:28.551087 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:28.551120 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:28.551580 (+   460us) log.cc:844] Serialized 26 byte log entry
0709 13:03:28.755766 (+204186us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:28.774234 (+ 18468us) log.cc:844] Serialized 14 byte log entry
0709 13:03:28.774441 (+   207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.774514 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:28.935271 (+160757us) raft_consensus.cc:1685] finished
0709 13:03:28.935300 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.935481 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":18304,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":7334,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":13903,"thread_start_us":203839,"threads_started":1,"wal-append.queue_time_us":274179}
I20260709 13:03:28.947620 12936 consensus_queue.cc:260] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:28.950564 12920 log.cc:927] Time spent T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 1.162s	user 0.005s	sys 0.014s
I20260709 13:03:28.955790  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 2547) took 1634 ms. Trace:
I20260709 13:03:28.956256  8397 rpcz_store.cc:276] 0709 13:03:27.321012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.321410 (+   398us) service_pool.cc:224] Handling call
0709 13:03:28.955554 (+1634144us) spinlock_profiling.cc:243] Waited 49.2 ms on lock 0x7b540019afd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:28.955726 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113161088}
I20260709 13:03:28.957926  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:28.958493  8327 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:28.960944  9001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 912) took 1643 ms. Trace:
I20260709 13:03:28.961392  9001 rpcz_store.cc:276] 0709 13:03:27.317997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.330839 (+ 12842us) service_pool.cc:224] Handling call
0709 13:03:27.331196 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:27.373184 (+ 41988us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:27.373227 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:27.383161 (+  9934us) log.cc:844] Serialized 26 byte log entry
0709 13:03:27.689409 (+306248us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:27.689475 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.689547 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:28.949989 (+1260442us) spinlock_profiling.cc:243] Waited 4.87 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:28.958369 (+  8380us) raft_consensus.cc:1685] finished
0709 13:03:28.958389 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.960882 (+  2493us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7641,"raft.queue_time_us":38428,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":9977,"spinlock_wait_cycles":11200384,"thread_start_us":273411,"threads_started":1,"wal-append.queue_time_us":344152}
I20260709 13:03:28.925395 12958 raft_consensus.cc:697] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:28.965513  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:28.966444  8328 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:28.972285 12898 consensus_queue.cc:260] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:28.976667  8369 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:28.979791  8327 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:28.980672  8220 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:29.010927  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:28.980955  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:28.983505  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 676) took 1286 ms. Trace:
I20260709 13:03:29.012629  8220 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:29.012758  8406 rpcz_store.cc:276] 0709 13:03:27.696617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.697075 (+   458us) service_pool.cc:224] Handling call
0709 13:03:27.711028 (+ 13953us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:27.805834 (+ 94806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.983230 (+1177396us) spinlock_profiling.cc:243] Waited 67.6 ms on lock 0x7b54003e6054. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:28.983437 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155427072}
I20260709 13:03:28.985154  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:29.014307  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:28.985883  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:29.014900  8255 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:29.015427  8249 raft_consensus.cc:2410] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 14 because replica has last-logged OpId of term: 12 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:03:29.017453  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:29.017927  8250 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:29.018060  8248 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:29.017479  8256 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 15: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:29.018589  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:29.013475 12730 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:29.018982  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:29.020823  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.020828  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:29.021895  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.018926  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:29.023757  8327 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:29.025036  8972 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:29.017328 12933 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:29.019971 12825 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:29.038631  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:29.032186  8248 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:29.025954  8369 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:29.040478  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:29.041719  8369 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:29.042737 12869 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:29.043256 12869 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:29.043732 12869 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:29.051198  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:29.033962 12933 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:29.053256 12933 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:29.027269 12880 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:29.056334 12958 consensus_queue.cc:237] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:29.061333 12941 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:29.022498 12730 raft_consensus.cc:697] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:29.081662  8295 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:29.082732 12403 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:29.083158 12403 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.083539 12403 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:29.073827 12891 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:29.085450 12891 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:29.085942 12891 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:29.089993 12892 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.090566 12892 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:29.092864 12892 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:29.076363 12941 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:29.094736  8255 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:03:29.096133  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:29.036813 12637 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:29.096881  8255 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:29.100018 12854 raft_consensus.cc:3055] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:29.066372 12693 raft_consensus.cc:2764] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:03:29.074831 12898 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:29.103729  8328 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:03:29.083685  8295 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:29.105940 12921 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:29.106913  8295 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:29.076656 12825 raft_consensus.cc:697] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:29.107820 12730 consensus_queue.cc:237] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:29.108937  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.109915  8295 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:29.112385  8295 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:29.114138  8295 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:29.102890 12854 raft_consensus.cc:740] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:29.109936  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:29.102429 12637 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.073205  8327 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:29.129714  8295 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:29.175822  8220 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:29.169472 12693 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:29.115100 12921 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:29.114877 12955 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:29.185170  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:29.186910  8220 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:29.208401  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:29.166399 12892 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:29.205461  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:29.167762 12637 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:29.187849  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 58237.8us
I20260709 13:03:29.115037 12895 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:29.209357  8328 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:29.216418  9009 raft_consensus.cc:3055] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:29.216842  9009 raft_consensus.cc:740] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:29.204458 12718 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:29.167307 12670 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
W20260709 13:03:29.233029  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.234973  9001 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:29.208425 12966 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:29.212107 12539 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:29.236461  9009 consensus_queue.cc:260] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:29.216058 12974 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:29.205313 12880 raft_consensus.cc:697] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:29.217765 12896 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.243585  8328 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
W20260709 13:03:29.244762  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 193644us
I20260709 13:03:29.169714 12825 consensus_queue.cc:237] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:29.257551  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:29.287401  8328 raft_consensus.cc:2393] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 12: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:29.288571  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:29.258934  8220 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:29.238185 12966 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.299180 12966 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:29.301908  8220 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:29.295814  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:29.216368 12403 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:29.302831 12958 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:29.230474 12854 consensus_queue.cc:260] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:29.309078  8255 raft_consensus.cc:1275] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:03:29.324390  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 67136.4us
I20260709 13:03:29.309644 12989 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.321944  8372 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:29.304670  8328 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:29.254228 12539 raft_consensus.cc:697] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:29.293646 12718 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:03:29.335932 12880 consensus_queue.cc:237] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:29.341862 12892 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:29.342393 12403 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:29.325599  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:29.342460 12892 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:29.343461 12988 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.343986 12988 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:29.305660 12693 raft_consensus.cc:697] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:29.344919 12637 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:29.328251  8255 raft_consensus.cc:1275] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:29.346078  9002 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14. Ops: [11.9-13.10]
I20260709 13:03:29.346092 12693 consensus_queue.cc:237] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:29.347611 12892 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:29.290984 12896 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:29.297888 12941 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:29.329116  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:29.329346  8220 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:29.349867 12941 raft_consensus.cc:697] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:29.350399 12659 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:29.350862 12718 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:29.351323 12718 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:29.351750 12718 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:29.334987 12539 consensus_queue.cc:237] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:03:29.351045 12941 consensus_queue.cc:237] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:29.331876 12894 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:29.349963 12896 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:29.352897  8328 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:29.359786  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.285924 12974 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:29.346306 12988 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:29.373873 12952 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:29.374532 12952 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:29.375160 12637 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:29.361545 12895 consensus_queue.cc:1175] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8c42259d7c4b463a9f485f42c320c81b" responder_term: 15 status { last_received { term: 15 index: 12 } last_committed_idx: 12 last_received_current_leader { term: 15 index: 12 } }
I20260709 13:03:29.361344 12854 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:29.406250  8328 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:03:29.407572  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:29.431143  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:03:29.437587  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 107327us
I20260709 13:03:29.435920  8220 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:29.449173  9007 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56948 (request call id 905) took 3368 ms (client timeout 1707 ms). Trace:
W20260709 13:03:29.449868  9007 rpcz_store.cc:269] 0709 13:03:26.060072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.060319 (+   247us) service_pool.cc:224] Handling call
0709 13:03:29.428267 (+3367948us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54005983d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:29.428430 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3138737024}
I20260709 13:03:29.434316 12996 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:29.456941 12996 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:29.457340 12996 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:29.458802 12962 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:29.460368 12942 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:29.362010 12989 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:29.385075 12988 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 13:03:29.464222  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:29.377995 12892 consensus_queue.cc:1048] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:03:29.496455  8328 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
W20260709 13:03:29.497241 12992 log.cc:927] Time spent T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:03:29.497714  8222 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:29.497792  8220 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:29.501142 12942 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:29.501528 12942 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.501828 12942 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:29.502292  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:29.502893  8328 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:29.505607 12232 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:29.506119 12232 raft_consensus.cc:697] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:29.506922 12232 consensus_queue.cc:237] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:29.517719  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:03:29.490052  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 51424.5us
W20260709 13:03:29.526906  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 201365us
I20260709 13:03:29.528998  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:29.531839  8328 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:03:29.532052  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:29.533249  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 686) took 3205 ms. Trace:
I20260709 13:03:29.534305  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:29.544665  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:29.547523  8328 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:29.490221 12999 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:29.551514  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:29.558468 12771 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:29.560323  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:29.581946  8328 raft_consensus.cc:2376] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11. Re-sending same reply.
I20260709 13:03:29.582240  8253 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:29.561594  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:29.583505  8256 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:29.562731  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:29.584744  8252 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 16: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:29.529690 12659 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.543193  8257 rpcz_store.cc:276] 0709 13:03:26.327834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.340700 (+ 12866us) service_pool.cc:224] Handling call
0709 13:03:26.341024 (+   324us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:26.380231 (+ 39207us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:26.408141 (+ 27910us) log.cc:844] Serialized 14 byte log entry
0709 13:03:26.725780 (+317639us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:28.395792 (+1670012us) spinlock_profiling.cc:243] Waited 4.96 ms on lock 0x7b18004600c0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d1557 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:03:29.017249 (+621457us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b2c000014c8. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:03:29.017797 (+   548us) log.cc:844] Serialized 52 byte log entry
0709 13:03:29.234394 (+216597us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:29.360754 (+126360us) log.cc:844] Serialized 14 byte log entry
0709 13:03:29.370059 (+  9305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.370266 (+   207us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:29.532892 (+162626us) raft_consensus.cc:1685] finished
0709 13:03:29.532942 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.533145 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48399,"prepare.queue_time_us":141385,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":109,"raft.queue_time_us":108189,"raft.run_cpu_time_us":410,"raft.run_wall_time_us":31999,"spinlock_wait_cycles":16413696,"thread_start_us":531561,"threads_started":3,"wal-append.queue_time_us":928717,"wal-append.run_cpu_time_us":2460,"wal-append.run_wall_time_us":1290122}
W20260709 13:03:29.599570 12869 consensus_meta.cc:287] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.555s	user 0.010s	sys 0.002s
I20260709 13:03:29.599959  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:29.600029 12869 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:29.607366 12916 log.cc:927] Time spent T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 1.819s	user 0.004s	sys 0.000s
I20260709 13:03:29.609120  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2232) took 3048 ms. Trace:
I20260709 13:03:29.610108 12966 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:29.609520  8330 rpcz_store.cc:276] 0709 13:03:26.560306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.881139 (+320833us) service_pool.cc:224] Handling call
0709 13:03:26.881545 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:26.881959 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:27.049553 (+167594us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.481724 (+432171us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:27.498473 (+ 16749us) log.cc:844] Serialized 26 byte log entry
0709 13:03:27.498601 (+   128us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:27.498633 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.498695 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:29.606802 (+2108107us) spinlock_profiling.cc:243] Waited 5.04 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:29.608840 (+  2038us) raft_consensus.cc:1685] finished
0709 13:03:29.608920 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.609050 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2254,"prepare.queue_time_us":255063,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":219309,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":16139,"spinlock_wait_cycles":11597568,"thread_start_us":511527,"threads_started":2,"wal-append.queue_time_us":698727}
I20260709 13:03:29.611963  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:29.613344 12771 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:29.613195  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:29.589171 12659 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:29.614030 12771 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:29.530383 12693 consensus_queue.cc:1059] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.091s
I20260709 13:03:29.621507 12966 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:29.638362 12693 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:29.638870 12693 raft_consensus.cc:697] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:29.639701 12693 consensus_queue.cc:237] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:29.641986 12869 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:29.642975 12869 raft_consensus.cc:3055] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:29.643356 12869 raft_consensus.cc:740] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:29.644263 12869 consensus_queue.cc:260] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:29.585439 12942 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:29.646538  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2238) took 1940 ms. Trace:
I20260709 13:03:29.653995  8332 rpcz_store.cc:276] 0709 13:03:27.706290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.235037 (+528747us) service_pool.cc:224] Handling call
0709 13:03:28.258845 (+ 23808us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:28.291885 (+ 33040us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.646283 (+1354398us) spinlock_profiling.cc:243] Waited 32.1 ms on lock 0x7b54003c9954. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:29.646479 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":73939200}
I20260709 13:03:29.655593  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:29.656472  8332 raft_consensus.cc:2393] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:29.650830 12659 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:29.645407 12869 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:29.658972 12771 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:29.621824 12989 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:29.657351  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:29.647085  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 645) took 2083 ms. Trace:
I20260709 13:03:29.663797  8326 rpcz_store.cc:276] 0709 13:03:27.564147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.819391 (+255244us) service_pool.cc:224] Handling call
0709 13:03:27.860299 (+ 40908us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:28.290997 (+430698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.646870 (+1355873us) spinlock_profiling.cc:243] Waited 430 ms on lock 0x7b540011cbd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:29.647033 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":988562176}
I20260709 13:03:29.665464  8332 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:29.670399  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:29.696911  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:29.698328  8326 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:29.628572 12996 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:29.700961  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:29.652725 12942 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:29.702430  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 203126us
I20260709 13:03:29.706977  8330 raft_consensus.cc:2393] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:29.709204  8222 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:29.721321  8325 raft_consensus.cc:1275] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:03:29.701396 12898 consensus_meta.cc:287] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.588s	user 0.004s	sys 0.005s
I20260709 13:03:29.731827  8332 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
W20260709 13:03:29.707635 13003 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:03:29.735975  8324 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 13:03:29.738866  8324 pending_rounds.cc:85] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 10
I20260709 13:03:29.739215  8324 pending_rounds.cc:107] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:03:29.739521  8324 raft_consensus.cc:2889] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:29.747081  8326 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
W20260709 13:03:29.734674  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 214018us
W20260709 13:03:29.751912  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.720995 12974 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:29.756266  8330 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: [8.6-11.7]
W20260709 13:03:29.757316  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 228356us
W20260709 13:03:29.763832  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:29.775522  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.775666  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2501) took 1009 ms. Trace:
I20260709 13:03:29.776211  8330 rpcz_store.cc:276] 0709 13:03:28.766625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.759120 (+992495us) service_pool.cc:224] Handling call
0709 13:03:29.775151 (+ 16031us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.775333 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.775360 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:29.775406 (+    46us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:29.775439 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.775493 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.775595 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:29.751827 12999 raft_consensus.cc:697] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:29.785367  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:29.786305  9006 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:29.759358 12797 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.762384 12635 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:29.790946  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:29.794889  9004 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:29.762744 12953 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.801901  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:29.793530  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:29.771811 12849 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.802304  8369 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:29.796146  8330 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:29.813607 12849 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:29.816174 12849 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:29.816881  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.818578 12996 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:29.819449 12996 consensus_queue.cc:1059] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:29.820088  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:29.820379 12996 consensus_queue.cc:1243] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer ff7a6fe533e94c1bae555b4cae427728 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:03:29.802459 13012 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:29.825321  8369 connection.cc:256] RPC call timeout handler was delayed by -1.9143809120000697s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1544s for a response
I20260709 13:03:29.796481 12797 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:29.828011 12891 consensus_meta.cc:287] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.742s	user 0.007s	sys 0.005s
I20260709 13:03:29.790652 13011 consensus_queue.cc:1059] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:03:29.828560 12999 consensus_queue.cc:237] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:29.829227 13011 consensus_queue.cc:1243] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:03:29.829169 12797 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:29.830096 13011 raft_consensus.cc:3055] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:29.830806 13011 raft_consensus.cc:740] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:29.831997 13011 consensus_queue.cc:260] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:29.833788 13015 raft_consensus.cc:3055] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:29.834352 13015 raft_consensus.cc:740] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:03:29.834928 12933 consensus_meta.cc:287] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.781s	user 0.007s	sys 0.004s
I20260709 13:03:29.835906 12989 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:29.835477 13015 consensus_queue.cc:260] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:29.836644 12989 raft_consensus.cc:697] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:03:29.837942  8369 connection.cc:256] RPC call timeout handler was delayed by -1.8612809120000975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1603s for a response
I20260709 13:03:29.837950 12989 consensus_queue.cc:237] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:29.839784  8369 connection.cc:256] RPC call timeout handler was delayed by -1.7676809120000598s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17070000000000002s for a response
I20260709 13:03:29.841549  8372 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:03:29.841023  8369 connection.cc:256] RPC call timeout handler was delayed by -1.7199809120000964s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17600000000000002s for a response
I20260709 13:03:29.846371  9006 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:29.870093  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:29.877774  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:29.878538  8330 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:29.840725 12232 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.854774 13015 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:29.885792  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:29.897370  8253 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:29.880890 13011 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:29.900980  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:29.903271  8330 raft_consensus.cc:2393] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 12: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:29.901407  8256 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:29.884263 12730 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:29.912529  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:29.913400  8256 raft_consensus.cc:2393] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 14: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:29.835541 12933 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:29.923353  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:29.924357  8330 raft_consensus.cc:3055] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:29.924649  8330 raft_consensus.cc:740] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:29.809466 12953 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:29.864617 13004 consensus_queue.cc:1048] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:03:29.828589 12891 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:29.929926  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 88474.3us
I20260709 13:03:29.933185  8253 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
W20260709 13:03:29.934406 13019 log.cc:927] Time spent T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
W20260709 13:03:29.935861 13005 log.cc:927] Time spent T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:03:29.912545 12730 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.940857  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:29.959213  8328 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 16.
I20260709 13:03:29.962530  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:29.925294  8330 consensus_queue.cc:260] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:29.963400  8330 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:29.892552 12232 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:29.972899  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: true dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:29.973955  8253 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:29.976495  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2118) took 2255 ms. Trace:
I20260709 13:03:29.952279 12730 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:29.978804  8398 rpcz_store.cc:276] 0709 13:03:27.721161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.894300 (+173139us) service_pool.cc:224] Handling call
0709 13:03:28.055279 (+160979us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:28.055695 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:28.055735 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:28.055787 (+    52us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:28.055828 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.055906 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.976319 (+1920413us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54003e8350. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:29.976436 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2214,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":13394,"spinlock_wait_cycles":369416960}
W20260709 13:03:29.984580 13002 log.cc:927] Time spent T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:03:29.986815 12891 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:29.983003 12849 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:29.991047 12849 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:29.992643  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2497) took 1227 ms. Trace:
I20260709 13:03:29.993572 12849 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:29.995082  8330 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:29.996558  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:29.998158  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:29.998739  8330 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:29.999872  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:30.000454  8330 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 8.
I20260709 13:03:30.003630  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:30.007354  8330 raft_consensus.cc:2393] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 10: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:30.021481  8256 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:03:30.023609 12933 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:30.030339  8253 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:30.034209 12953 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:30.033686  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:30.035504  8253 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:30.022509  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:30.038564  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:30.040051  8256 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:30.040310  8253 raft_consensus.cc:2393] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 11: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
W20260709 13:03:30.031622 12718 consensus_meta.cc:287] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.679s	user 0.010s	sys 0.000s
I20260709 13:03:30.018527 12232 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:30.051576 12854 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:30.062659  9008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56948 (request call id 921) took 1513 ms. Trace:
I20260709 13:03:30.084703  8972 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:30.068883 12829 raft_consensus.cc:3055] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:30.091420  9008 rpcz_store.cc:276] 0709 13:03:28.557898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.561514 (+  3616us) service_pool.cc:224] Handling call
0709 13:03:30.062319 (+1500805us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b54004c2f54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:30.062532 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":411605760}
W20260709 13:03:30.097733  8369 connection.cc:256] RPC call timeout handler was delayed by -1.5939809120000064s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19s for a response
W20260709 13:03:30.098099  8369 connection.cc:256] RPC call timeout handler was delayed by -1.5435809120000386s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1956s for a response
W20260709 13:03:30.098363  8369 connection.cc:256] RPC call timeout handler was delayed by -1.5318809120000196s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19690000000000002s for a response
I20260709 13:03:30.094460 12798 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:30.101451  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:03:30.101929  8369 connection.cc:256] RPC call timeout handler was delayed by -1.4868809120000606s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20190000000000002s for a response
I20260709 13:03:30.102391  9006 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:03:30.103688  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:30.105610  8253 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:30.106143  8369 connection.cc:256] RPC call timeout handler was delayed by -1.4859809120000591s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.202s for a response
W20260709 13:03:30.106742  8369 connection.cc:256] RPC call timeout handler was delayed by -1.4382809120000957s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2073s for a response
I20260709 13:03:30.095268 12936 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:30.119431 12730 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:30.090878 12797 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:03:30.109885  8369 connection.cc:256] RPC call timeout handler was delayed by -1.4283809120000797s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20840000000000003s for a response
I20260709 13:03:30.095175 12829 raft_consensus.cc:740] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:30.108302 12880 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:30.095074 12718 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:30.129451  8369 connection.cc:256] RPC call timeout handler was delayed by -1.4220809120000695s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20910000000000004s for a response
I20260709 13:03:30.132287 12880 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:30.162487 12718 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:30.163458  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:30.164716  8400 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:03:30.165951  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:30.166843  8400 raft_consensus.cc:2410] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:03:30.167750  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:30.168360  8405 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:03:30.170473  8403 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:03:30.161047  8369 connection.cc:256] RPC call timeout handler was delayed by -1.3986809120000316s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2117s for a response
W20260709 13:03:30.141191 13026 log.cc:927] Time spent T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:03:30.160955 12952 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:30.150614 12932 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:30.179145  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:30.179704  8400 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:03:30.179867  8403 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:30.184077  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:30.185333  8369 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:30.190021  8400 raft_consensus.cc:1275] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:30.191407  8369 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:30.153997  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2236) took 2448 ms. Trace:
I20260709 13:03:30.154392 12829 consensus_queue.cc:260] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:30.097155 12854 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:30.171830  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.172279  8397 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:30.172683  8401 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:30.193682  8324 rpcz_store.cc:276] 0709 13:03:27.705800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.151966 (+446166us) service_pool.cc:224] Handling call
0709 13:03:28.152221 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.739968 (+1587747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.153651 (+413683us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54000587d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:30.153920 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3640783104}
I20260709 13:03:30.200488  8324 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:03:30.163873 12880 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:30.209719  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:30.210316  8324 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:30.212198  8253 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:30.212652  8324 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14. Ops: [7.7-13.10]
I20260709 13:03:30.213590  8253 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:30.213948  8324 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:30.214879  8253 raft_consensus.cc:1240] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:30.215036  8324 raft_consensus.cc:1240] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:30.216142  8253 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:30.216238  8324 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: [12.8-12.8]
W20260709 13:03:30.217840  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.218540  8253 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:30.221561  8253 raft_consensus.cc:1240] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:30.175030 12885 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:30.196774 13024 log.cc:927] Time spent T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.191s	user 0.001s	sys 0.002s
I20260709 13:03:30.225031  8369 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:30.195318 13011 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:30.225890  8255 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:03:30.168762 12797 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:30.223444 12829 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:30.230499  8369 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:30.230914 12946 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.944s	user 0.010s	sys 0.006s
I20260709 13:03:30.223004 12880 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:30.234320  8323 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2242) took 2490 ms (client timeout 1742 ms). Trace:
I20260709 13:03:30.236619  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:30.227926 13011 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:30.240338  8323 rpcz_store.cc:269] 0709 13:03:27.738172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.787775 (+ 49603us) service_pool.cc:224] Handling call
0709 13:03:30.228798 (+2441023us) spinlock_profiling.cc:243] Waited 64.3 ms on lock 0x7b540011e4d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:30.228949 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":147795328}
I20260709 13:03:30.244015  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:30.244264  8398 raft_consensus.cc:1240] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:30.247455  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:30.232360 12989 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:30.238143 12946 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.951s	user 0.011s	sys 0.006s
I20260709 13:03:30.232141 12798 raft_consensus.cc:697] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:30.254455 12798 consensus_queue.cc:237] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:29.993052  8332 rpcz_store.cc:276] 0709 13:03:28.765356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.734620 (+969264us) service_pool.cc:224] Handling call
0709 13:03:29.735077 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:29.746387 (+ 11310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.746430 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:29.754484 (+  8054us) log.cc:844] Serialized 26 byte log entry
0709 13:03:29.843997 (+ 89513us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:29.976370 (+132373us) log.cc:844] Serialized 14 byte log entry
0709 13:03:29.976579 (+   209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.976662 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:29.992397 (+ 15735us) raft_consensus.cc:1685] finished
0709 13:03:29.992419 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.992560 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":68,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":3681,"thread_start_us":154526,"threads_started":2,"wal-append.queue_time_us":129570}
I20260709 13:03:30.237844 12996 raft_consensus.cc:2764] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:03:30.249938 13029 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:30.250067 12869 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:30.263900  9000 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56954 (request call id 784) took 3534 ms (client timeout 2031 ms). Trace:
W20260709 13:03:30.264371 12948 log.cc:927] Time spent T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 2.085s	user 0.008s	sys 0.000s
I20260709 13:03:30.264822 12607 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:30.248697 13031 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:03:30.264593  9000 rpcz_store.cc:269] 0709 13:03:26.728938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.753011 (+ 24073us) service_pool.cc:224] Handling call
0709 13:03:30.263624 (+3510613us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b5400527bd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:30.263785 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1426182528}
I20260709 13:03:30.265803 13031 raft_consensus.cc:697] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:30.265903 13027 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:30.266793 13031 consensus_queue.cc:237] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:30.267112 12730 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:30.226366 12885 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:30.269711 12885 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:30.242569 13037 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:03:30.271602 12797 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:30.274717  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 693) took 2266 ms. Trace:
I20260709 13:03:30.275115  8402 rpcz_store.cc:276] 0709 13:03:28.008094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.009762 (+  1668us) service_pool.cc:224] Handling call
0709 13:03:28.018765 (+  9003us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:28.027304 (+  8539us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:28.038745 (+ 11441us) log.cc:844] Serialized 14 byte log entry
0709 13:03:28.110578 (+ 71833us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:28.148807 (+ 38229us) log.cc:844] Serialized 26 byte log entry
0709 13:03:28.149525 (+   718us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:28.149564 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.149627 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:30.263668 (+2114041us) spinlock_profiling.cc:243] Waited 3.63 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:30.274420 (+ 10752us) raft_consensus.cc:1685] finished
0709 13:03:30.274520 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.274660 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":10125,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":204,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":37541,"spinlock_wait_cycles":8343424,"thread_start_us":29394,"threads_started":1,"wal-append.queue_time_us":91354}
I20260709 13:03:30.265518 12607 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:30.262269  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:30.283221  8369 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:30.285902  8324 raft_consensus.cc:1275] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:03:30.287462  8330 raft_consensus.cc:1275] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:30.288131  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:30.288753  8323 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:30.249279 13011 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:30.285948  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.237670  8405 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:30.290709  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:30.269136 12854 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:30.291553  8220 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:30.293093  8324 raft_consensus.cc:1275] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:30.292140 13015 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:30.289902 12989 raft_consensus.cc:2764] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:03:30.297636 13037 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:30.298290 13037 raft_consensus.cc:697] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:30.299204 13037 consensus_queue.cc:237] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:30.292235 12952 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.302829 12952 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:30.305027 12952 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:30.306900  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.306815  8404 raft_consensus.cc:1240] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:30.309180  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:30.311064  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:30.311471  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:30.315953  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2235) took 2610 ms. Trace:
I20260709 13:03:30.318434 13011 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:30.320804 13011 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:30.322597  8972 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:30.322664  8404 raft_consensus.cc:1275] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:30.323216  8369 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:30.323367  8220 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:30.316406  8331 rpcz_store.cc:276] 0709 13:03:27.705540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.140010 (+434470us) service_pool.cc:224] Handling call
0709 13:03:28.206071 (+ 66061us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:28.267175 (+ 61104us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:28.267206 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:28.267651 (+   445us) log.cc:844] Serialized 26 byte log entry
0709 13:03:28.510236 (+242585us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:28.510274 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.645341 (+1135067us) spinlock_profiling.cc:243] Waited 57.9 ms on lock 0x7b540011d5d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:29.645387 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:29.645454 (+    67us) raft_consensus.cc:1685] finished
0709 13:03:29.645471 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.315759 (+670288us) spinlock_profiling.cc:243] Waited 65.7 ms on lock 0x7b540011d5d0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:30.315890 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88108,"raft.queue_time_us":5037,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":8751,"spinlock_wait_cycles":284364928,"thread_start_us":224521,"threads_started":1,"wal-append.queue_time_us":303955,"wal-append.run_cpu_time_us":1725,"wal-append.run_wall_time_us":1028489}
I20260709 13:03:30.324893  8220 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:30.311120  8330 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:03:30.328372  8369 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:30.330957  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:30.331790  8330 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
W20260709 13:03:30.332643  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.332904  8369 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:30.333818  8972 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:30.333855  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:30.335129  8220 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:30.336917  8295 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:30.339138  8330 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14. Ops: [12.8-13.9]
I20260709 13:03:30.339056  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:30.335764  8972 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:30.327931 12936 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:30.340178  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:30.351779  8324 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:30.341835  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:30.352244  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:30.327780 12989 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:30.350835 13031 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:30.352917  8328 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:30.353363  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:30.329458 12798 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:30.353874  8323 raft_consensus.cc:1275] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:03:30.330125 12952 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:30.334947 12854 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.329520 13022 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:30.358754  8220 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:30.337764 12849 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.340626 12797 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:30.365661 13009 consensus_queue.cc:1048] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:30.351814 12932 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:30.365978 12854 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:30.366731 12932 raft_consensus.cc:697] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:30.366925 13023 consensus_queue.cc:1048] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:30.367673 12932 consensus_queue.cc:237] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:30.368168 12854 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:30.351557 12951 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:30.364684 13044 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:30.357455 12933 consensus_queue.cc:1048] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:03:30.340246 13029 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:30.340617 12898 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:30.330606 12869 consensus_queue.cc:1048] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:03:30.333156 12996 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:03:30.341975 12635 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:30.342631 12880 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:30.351545 12232 consensus_queue.cc:1059] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:03:30.372368 12849 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:30.337404 12829 consensus_queue.cc:1059] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:03:30.379565  8328 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:30.388139  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:30.380216  8220 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:30.357012 13031 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.386025  8404 raft_consensus.cc:1275] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:03:30.393486  8404 pending_rounds.cc:85] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 6
I20260709 13:03:30.387893  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:30.393544  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:30.340188 12718 consensus_queue.cc:1048] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:03:30.389477  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:30.386722  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:30.394614  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:30.389001  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:30.395152  8404 pending_rounds.cc:107] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:03:30.391952 12635 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:30.358481 12936 raft_consensus.cc:697] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:30.399080  8404 raft_consensus.cc:2889] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:30.404857  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:30.412533  8398 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:03:30.414489  8399 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:30.372654 13020 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:30.382970 12951 raft_consensus.cc:697] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 14 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:30.394488 12880 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.397300 13031 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:30.396948 13044 raft_consensus.cc:697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:30.428171  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:30.429162  8405 raft_consensus.cc:2393] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 12: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:30.396960 13029 raft_consensus.cc:697] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:30.401250 12635 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:30.412343 12825 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:30.423666 13020 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.394378 13022 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:03:30.447345  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:30.392104 12898 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:30.429665 12880 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:30.448529  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 64407.6us
I20260709 13:03:30.435421 12825 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:30.458259  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 118758us
W20260709 13:03:30.460352  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 121526us
I20260709 13:03:30.453883 13020 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:30.461061  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:30.464120  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 102391us
I20260709 13:03:30.464869  8369 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:30.458945 12825 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:30.462957  8295 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:30.468413  8369 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:30.472986  8297 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:30.475090 12869 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:30.479815  8369 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:30.472782 12946 consensus_queue.cc:1059] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:03:30.504381 12797 raft_consensus.cc:3055] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:30.504769 12797 raft_consensus.cc:740] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:30.473712 12996 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:30.510236  8369 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:30.512311 12989 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:30.513080 12989 consensus_queue.cc:1059] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:03:30.459736 12936 consensus_queue.cc:237] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:30.532804  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 69933us
I20260709 13:03:30.536147  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:30.536967  8330 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
I20260709 13:03:30.537168  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:30.512755 13018 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:03:30.537935  8331 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 8.
I20260709 13:03:30.539829  8972 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:30.540012  8324 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:03:30.466861 12951 consensus_queue.cc:237] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:30.516788 12730 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:30.478482 13044 consensus_queue.cc:237] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:30.482393 13029 consensus_queue.cc:237] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:30.571942 12898 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:30.528468 13027 consensus_queue.cc:1059] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:03:30.574348 13027 raft_consensus.cc:3055] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:30.574702 13027 raft_consensus.cc:740] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:03:30.575595 13053 log.cc:927] Time spent T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.210s	user 0.002s	sys 0.000s
I20260709 13:03:30.575526 13027 consensus_queue.cc:260] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:30.576807 13027 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:30.578686  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2081) took 1364 ms. Trace:
I20260709 13:03:30.579070  8255 rpcz_store.cc:276] 0709 13:03:29.214062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.346680 (+132618us) service_pool.cc:224] Handling call
0709 13:03:29.346950 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.227438 (+880488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.578446 (+351008us) spinlock_profiling.cc:243] Waited 878 ms on lock 0x7b540034a8d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:30.578624 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2020386176}
I20260709 13:03:30.558190  8972 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:30.515787 13063 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:03:30.581933  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 69074.8us
I20260709 13:03:30.587167  8328 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:03:30.526621 12869 consensus_queue.cc:1059] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:03:30.556593 12849 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:30.581075 12789 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:30.579682 12730 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.584529 12831 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:03:30.595759 13054 log.cc:927] Time spent T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:03:30.607640 13068 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:30.606848 12909 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.608758 12849 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:30.608819 12909 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:30.611189 12909 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:30.614075 12718 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:30.614629 12718 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:30.616413 12635 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:30.616858 12718 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:30.562650 12797 consensus_queue.cc:260] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:30.618213 12797 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:30.603683 12730 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:30.597918 13065 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:30.628366  8327 raft_consensus.cc:1240] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 8. Current term is 9. Ops: []
W20260709 13:03:30.623939 13055 log.cc:927] Time spent T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 13:03:30.643872 12996 raft_consensus.cc:3055] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:30.590586 12989 consensus_queue.cc:1059] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:03:30.644328 12996 raft_consensus.cc:740] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:30.645243 12880 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:30.645184 12996 consensus_queue.cc:260] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:30.646826 12996 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:30.647509 12880 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:30.605316 12831 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.648337 12831 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:30.601727 12607 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:30.649868  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:30.651198 12607 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:30.651305 12825 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:30.589799 13063 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:30.652307  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:03:30.652861  8329 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2470) took 4299 ms (client timeout 2002 ms). Trace:
I20260709 13:03:30.653544 13063 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:30.648164  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:30.654572  8403 raft_consensus.cc:2393] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 14: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:30.655422  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:30.656369  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:30.602937 12951 consensus_queue.cc:1059] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:03:30.657089  8403 raft_consensus.cc:2410] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:03:30.597414 12789 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:30.656963  8222 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:30.657616 12789 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:30.660305 13022 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:30.660494 13001 consensus_queue.cc:1048] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:30.661053 13022 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:30.661568 12232 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:30.617285 13020 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:30.662097 12941 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:30.663576  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:30.663841 13020 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:30.664328  8400 raft_consensus.cc:2393] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 16: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:30.664587  8402 raft_consensus.cc:1240] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13. Ops: []
W20260709 13:03:30.653282  8329 rpcz_store.cc:269] 0709 13:03:26.353296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.355312 (+  2016us) service_pool.cc:224] Handling call
0709 13:03:30.652635 (+4297323us) spinlock_profiling.cc:243] Waited 3.34 s on lock 0x7b540008aa54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:30.652767 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7690150784}
I20260709 13:03:30.653015  8405 raft_consensus.cc:2393] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 11: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:30.666741  8400 raft_consensus.cc:1275] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:30.671844  8403 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:03:30.624010 13065 raft_consensus.cc:697] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:30.676527  8222 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:30.676849  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:03:30.643677  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.674342 12718 consensus_queue.cc:1048] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:03:30.685096 12868 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:30.688148 12933 consensus_queue.cc:1059] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:30.621415 13031 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:30.691732  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:30.687675 12831 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:30.643116  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 57630us
I20260709 13:03:30.700436  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:30.715217  9007 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:03:30.715646  8403 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:30.688572 12825 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:30.719422  9007 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:03:30.721166  9007 pending_rounds.cc:85] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: Aborting all ops after (but not including) 9
I20260709 13:03:30.665884 12829 consensus_queue.cc:1059] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:03:30.721565  9007 pending_rounds.cc:107] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:03:30.725966  9007 raft_consensus.cc:2889] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:30.687690 12232 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:30.722877  8329 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:03:30.720044  8298 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.4:33825 timed out after 2.039s (SENT) [suppressed 12 similar messages]
I20260709 13:03:30.728871  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:30.700609 12909 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.710436 13044 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.733968  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:30.735122  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:30.715191 12962 raft_consensus.cc:3055] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:30.675595 12932 consensus_queue.cc:1059] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
W20260709 13:03:30.738349  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 233441us
I20260709 13:03:30.746259 12932 consensus_queue.cc:1243] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:03:30.739979 12962 raft_consensus.cc:740] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:30.738408  8323 raft_consensus.cc:2376] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15. Re-sending same reply.
I20260709 13:03:30.747972 12962 consensus_queue.cc:260] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:30.761540  8323 raft_consensus.cc:1275] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:30.742831 12232 raft_consensus.cc:697] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:30.765566 12232 consensus_queue.cc:237] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:30.768154 12831 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:30.720801 12952 consensus_queue.cc:1048] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:03:30.731741 12989 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.774277 12989 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:30.776988 12989 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:30.779520 12869 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:30.780288 12869 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:30.782979 12869 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:30.688977 13022 consensus_queue.cc:1059] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:03:30.787149  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:30.785148  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:30.791877  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:30.796167  8403 raft_consensus.cc:2393] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:30.801491  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:30.802726  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:30.802835  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 82803.4us
W20260709 13:03:30.804427  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 57049.5us
I20260709 13:03:30.724459 12789 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:30.809365  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:30.812563  8331 raft_consensus.cc:1275] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:30.820653  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:30.821410  8327 raft_consensus.cc:2393] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 16: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:30.821282  8331 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 14. Current term is 15. Ops: [13.9-14.10]
I20260709 13:03:30.822319  8324 raft_consensus.cc:1275] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:30.824098  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:30.825016  8324 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:30.826835  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:30.835441  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:30.838104  8403 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:30.839056  8220 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:30.797703 12952 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.839730  8324 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:30.740271 12909 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:30.839191  8298 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:30.848054  8220 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:30.853826  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:30.854390  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:30.745270 13044 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:30.863662  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:30.864728  9000 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:30.867908  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:30.868707  9003 raft_consensus.cc:2393] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 12: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:30.852931 12718 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:30.852365 12933 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:30.878839  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:30.798100 13065 consensus_queue.cc:237] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:30.882635  8403 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:30.891669  9007 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 15: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:30.892295  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:30.895419  8401 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:30.847597 12868 consensus_queue.cc:1059] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:03:30.897730  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:30.852671 12941 consensus_queue.cc:1048] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:03:30.899371  8324 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:30.843488 13066 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:03:30.851148 12952 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:30.948307  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 2100) took 3221 ms. Trace:
I20260709 13:03:30.951117  8397 raft_consensus.cc:1275] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:30.954818  8400 raft_consensus.cc:3055] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:30.992075  8400 raft_consensus.cc:740] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:30.992991  8401 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:03:30.953641  8974 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:30.994228  8401 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:03:30.970244  9009 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:30.924652  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:31.002873  8398 raft_consensus.cc:1240] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:03:30.991062  8251 rpcz_store.cc:276] 0709 13:03:27.726637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.726816 (+   179us) service_pool.cc:224] Handling call
0709 13:03:27.727106 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:27.727313 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:27.760364 (+ 33051us) log.cc:844] Serialized 14 byte log entry
0709 13:03:29.642742 (+1882378us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b18003e6a40. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88283664bc 00007f88256b448b 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e23a
0709 13:03:30.947800 (+1305058us) spinlock_profiling.cc:243] Waited 4.84 ms on lock 0x7b18003e6a40. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88283664bc 00007f88256b448b 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256ce455 00007f88287b6060 00007f88287c8716 00007f88287d9287
0709 13:03:30.947978 (+   178us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:30.948028 (+    50us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:30.948072 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.948135 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.948250 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23265,"prepare.queue_time_us":69968,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":114804,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":3025,"spinlock_wait_cycles":13929728,"thread_start_us":291740,"threads_started":3,"wal-append.queue_time_us":118118,"wal-append.run_cpu_time_us":1853,"wal-append.run_wall_time_us":1133058}
I20260709 13:03:31.007500  8298 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:31.019338  8324 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 9.
I20260709 13:03:31.019712  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:31.021365  8397 raft_consensus.cc:1275] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:03:30.969276 13047 log.cc:927] Time spent T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.716s	user 0.002s	sys 0.000s
I20260709 13:03:30.992914  8400 consensus_queue.cc:260] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:30.936116 13059 raft_consensus.cc:3055] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:31.036950  8400 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:30.985567 12958 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:30.903995 13079 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:30.963581 13037 raft_consensus.cc:3055] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:31.023262 12831 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:31.049023 12933 raft_consensus.cc:697] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:31.049958  9009 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:03:31.070389  9009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 934) took 1859 ms. Trace:
W20260709 13:03:31.045258 13073 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.319s	user 0.001s	sys 0.000s
I20260709 13:03:31.071079  9009 rpcz_store.cc:276] 0709 13:03:29.210682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.210819 (+   137us) service_pool.cc:224] Handling call
0709 13:03:29.216298 (+  5479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.270641 (+1054343us) spinlock_profiling.cc:243] Waited 822 ms on lock 0x7b18002d35a0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287f9646 00007f882873358e 00007f88287b4f9e 00007f88287b67de 00007f88287c0b49 00007f88287c1416 00007f88287bdd8f 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4
0709 13:03:30.969929 (+699288us) spinlock_profiling.cc:243] Waited 696 ms on lock 0x7b580040d3d0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f882871a0aa 00007f882879c447 00007f88287b4fe7 00007f88287b67de 00007f88287c0b49 00007f88287c1416 00007f88287bdd8f 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4
0709 13:03:31.070045 (+100116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.070316 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3493111424}
I20260709 13:03:31.074306  8400 raft_consensus.cc:1275] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:03:31.074862  8400 pending_rounds.cc:85] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 8
I20260709 13:03:31.075182  8400 pending_rounds.cc:107] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:03:31.075529  8400 raft_consensus.cc:2889] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:31.044786 12885 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:31.040144 13059 raft_consensus.cc:740] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:31.045006 13023 raft_consensus.cc:3055] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:31.027626 13058 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:03:31.046805 13037 raft_consensus.cc:740] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:31.053951 12973 raft_consensus.cc:3055] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:03:31.106385  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.087312 12885 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:31.087635 13088 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.121256  8297 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:31.108665  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:31.118656  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:31.123716  9007 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:31.091986 13023 raft_consensus.cc:740] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:31.095412 12973 raft_consensus.cc:740] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:31.135898  8332 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:03:31.127496  8974 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:31.141223  9002 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:03:31.127808  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:31.105266 13071 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
W20260709 13:03:31.105650 13083 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.288s	user 0.000s	sys 0.002s
I20260709 13:03:31.121987 12831 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:31.124547  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.144402  8369 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:31.123797 12854 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:03:31.125077 12607 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:31.134032  8297 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:31.125077 12885 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:31.145104 13081 log.cc:927] Time spent T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
I20260709 13:03:31.169545  8297 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:31.135068 13015 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.179245  8400 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:31.131155 12797 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:03:31.171958  8369 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:31.181856  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:31.153400  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 111179us
W20260709 13:03:31.155049 13082 log.cc:927] Time spent T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.293s	user 0.001s	sys 0.000s
I20260709 13:03:31.180527  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:31.134207 13068 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:31.146282 12807 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:03:31.167076 12961 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:31.132954 13088 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:31.192624 13044 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:31.190062 12973 consensus_queue.cc:260] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:31.194072 13088 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:31.194494 12973 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:31.195354 12941 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:31.175818 13023 consensus_queue.cc:260] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:31.197453 13023 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:31.197962 13061 raft_consensus.cc:3055] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:31.198380 13061 raft_consensus.cc:740] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:31.199146 13061 consensus_queue.cc:260] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:31.200297 13061 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:31.201594 13076 consensus_queue.cc:1059] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:03:31.202400 13076 consensus_queue.cc:1059] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:31.153108 13037 consensus_queue.cc:260] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:31.203452 13085 log.cc:927] Time spent T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 13:03:31.203729 13037 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:31.204442 12849 raft_consensus.cc:3055] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:31.204856 12849 raft_consensus.cc:740] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:31.207353 12909 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:31.209141 12789 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:31.209298 12952 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:31.190795 13018 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.020s
W20260709 13:03:31.211067 13027 consensus_meta.cc:287] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.634s	user 0.010s	sys 0.000s
W20260709 13:03:31.208690 12996 consensus_meta.cc:287] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.561s	user 0.008s	sys 0.000s
W20260709 13:03:31.212116 12730 consensus_meta.cc:287] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.590s	user 0.005s	sys 0.003s
I20260709 13:03:31.212682  8405 raft_consensus.cc:1275] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:03:31.213075  8405 pending_rounds.cc:85] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 5
I20260709 13:03:31.213333  8405 pending_rounds.cc:107] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:03:31.169107 12831 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:31.213616  8405 raft_consensus.cc:2889] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:31.170872 13079 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:31.147151 13059 consensus_queue.cc:260] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:31.129002 13080 consensus_queue.cc:1048] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
W20260709 13:03:31.216931 13086 log.cc:927] Time spent T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.116s	user 0.002s	sys 0.000s
I20260709 13:03:31.217221 13018 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:31.142549 12933 consensus_queue.cc:237] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:31.189489 13011 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:31.223774  9002 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:31.226657  8332 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:31.175321 13029 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:31.216591  8331 raft_consensus.cc:1240] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:31.212538 12730 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:31.191687  8223 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:31.271653  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 81064.7us
W20260709 13:03:31.272037  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 86022.9us
I20260709 13:03:31.272398  8223 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:31.185456 12670 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:31.189858 12989 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:31.178824 12869 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:03:31.172577 12607 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:31.239289 12996 consensus_queue.cc:1048] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:03:31.263633  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.271653  8326 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14. Ops: [12.8-13.9]
I20260709 13:03:31.273139  8327 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:03:31.237263 12961 raft_consensus.cc:697] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 14 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:31.185374 13015 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:31.228991 12885 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:31.296108  8406 raft_consensus.cc:2393] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 9: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:03:31.265142 13042 consensus_queue.cc:1175] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e64761f43e4240319c41117cf886d22f" responder_term: 12 status { last_received { term: 11 index: 9 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 11. Current term is 12. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:03:31.281474 13059 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:31.279724 12670 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:03:31.313885  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 154629us
I20260709 13:03:31.282442 13011 raft_consensus.cc:697] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:31.340262  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:31.281502 12941 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:31.283150 12952 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:31.359824  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:31.344228 12961 consensus_queue.cc:237] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:31.322985 13001 consensus_queue.cc:1059] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:03:31.352922 12868 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.300577 13029 consensus_queue.cc:1059] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:03:31.355552  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:31.357656  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:31.335474  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:31.358664 12607 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:31.359006  8249 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:31.382882  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:31.383466  8252 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:31.343488 13015 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:31.264704 12849 consensus_queue.cc:260] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:31.397814  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:31.364202  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:31.398810  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.399051  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:31.399705  9007 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:31.400122  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:31.363706  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.400718  8249 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:31.365476  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:31.365490  8253 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:03:31.402098  8248 raft_consensus.cc:2410] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:03:31.361121 12670 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:31.328336  8255 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:03:31.394193  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 116490us
I20260709 13:03:31.301471 12989 raft_consensus.cc:697] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:03:31.403791  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.409380  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:31.409698  8248 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:03:31.405267  8252 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:31.413182  8324 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:03:31.413353  8252 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:03:31.414325  8250 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:03:31.416872  8369 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:31.411270  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:31.420898  9002 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:31.418226  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:31.360386 13100 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.100s
I20260709 13:03:31.436194  8248 raft_consensus.cc:1275] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:03:31.438314  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.440194  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:31.406543 12831 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:31.442663 12936 raft_consensus.cc:3055] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:31.428522 13029 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:31.401345 12868 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:31.450191 12936 raft_consensus.cc:740] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:31.446026 13037 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:03:31.449836 13011 consensus_queue.cc:237] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:31.468107  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:31.468976  8328 raft_consensus.cc:2393] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 9: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:31.471565  8328 raft_consensus.cc:1275] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:03:31.489094  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.497102  8972 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:31.502812 13031 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:31.503765  8253 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:03:31.505978  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:31.506971  9003 raft_consensus.cc:2368] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13.
I20260709 13:03:31.461184 12730 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:31.460026 12989 consensus_queue.cc:237] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:31.466552 12936 consensus_queue.cc:260] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:31.500528  8248 raft_consensus.cc:1275] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:31.501022 12635 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:31.553108  8298 leader_election.cc:400] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13.
I20260709 13:03:31.584978  8298 leader_election.cc:403] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:03:31.585352  8372 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:31.551090  8255 raft_consensus.cc:1275] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:31.552482  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:31.563221  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:31.554690  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 59808.4us
I20260709 13:03:31.587962  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:31.571022  8402 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:03:31.589712  8402 pending_rounds.cc:85] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 7
I20260709 13:03:31.590016  8402 pending_rounds.cc:107] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:03:31.590293  8402 raft_consensus.cc:2889] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:31.590853  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 71430.4us
I20260709 13:03:31.570354  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:31.575598  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:31.575649  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 54209.4us
I20260709 13:03:31.577811  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:31.571846  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:31.585613  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.583477  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:31.595304  8400 raft_consensus.cc:1240] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:31.591883  8402 raft_consensus.cc:1240] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:31.581964  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:31.565205  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.600968  8254 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:31.604175  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:31.604954  8332 raft_consensus.cc:2393] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 14: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:31.552847 12829 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:03:31.613775  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.617590  8255 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:03:31.583796 13031 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.620370  9002 raft_consensus.cc:1240] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 14 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:31.619470  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:31.622881  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:31.551350 13023 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:03:31.602589 13037 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:31.605309 12936 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:31.601761 13066 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13.
I20260709 13:03:31.628516  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:31.638559  8323 raft_consensus.cc:2393] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 11: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:31.587455 13063 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:03:31.639706  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:31.643756  8328 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:03:31.615988 12973 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:03:31.646576  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 50775.8us
I20260709 13:03:31.648242  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:31.650841  8323 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:03:31.627489  8332 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:31.654253  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:31.655004  8249 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:31.655208  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.629398 12829 raft_consensus.cc:697] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:31.633496 13037 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:31.614820 12909 consensus_queue.cc:1059] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:03:31.630595 12885 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:31.640012 13088 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.630941 12831 consensus_queue.cc:1059] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:03:31.626981 13031 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
W20260709 13:03:31.679922  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:31.673038  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.688853  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:31.689837  8248 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:31.660092 13037 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:31.659981 12807 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:03:31.660450 12825 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.653378 12941 consensus_queue.cc:1048] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:03:31.704403  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:31.703033  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 63544.7us
I20260709 13:03:31.656438 12953 consensus_queue.cc:1048] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:03:31.658161 12961 consensus_queue.cc:1059] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 13:03:31.694391 13122 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.713825  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:31.714452  8248 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:31.711701  8249 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:31.657083 12973 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:31.678171 13088 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:31.702792 13101 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:31.753593  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:31.754292  8248 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:31.326210  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:31.757400  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:31.758273  8249 raft_consensus.cc:2393] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 11: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:31.707119 12825 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:31.762987 13101 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:31.776350 13101 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:31.737563 12829 consensus_queue.cc:237] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:31.736620 13122 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:31.783138 12825 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:31.783229  8398 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:03:31.783866  8398 pending_rounds.cc:85] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 9
I20260709 13:03:31.784113  8398 pending_rounds.cc:107] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:03:31.784408  8398 raft_consensus.cc:2889] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:31.784904 13129 log.cc:927] Time spent T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:03:31.785395  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:31.786020  8328 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:31.786901  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:31.771559 12670 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:31.792716  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:03:31.793236  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.793550  9003 raft_consensus.cc:2393] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 12: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
W20260709 13:03:31.794392  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:31.794715 13127 log.cc:927] Time spent T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:03:31.796618 12670 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:31.780169 13088 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:31.798449 13128 log.cc:927] Time spent T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:03:31.804420  9003 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:31.806002 12941 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:31.806066  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:31.806736 12941 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:31.814663  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:31.815894  8249 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:31.819983 12962 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:31.820595 13011 consensus_queue.cc:1059] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:31.831723  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:31.764303 12868 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:31.839108  8369 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:31.840853  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:31.846539  8249 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:31.841868  8972 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:31.842621 12730 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:31.870110  8972 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:31.880009  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:31.900334  9007 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:03:31.809283 12941 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:31.907488 13140 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:31.910651  9006 raft_consensus.cc:1275] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:31.870601  8369 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:31.784209 13122 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:31.922853  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 215512us
W20260709 13:03:31.928992  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 58473.6us
I20260709 13:03:31.902778 12798 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:31.884791 13011 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:31.903013 12670 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.951545  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:31.867904 13071 consensus_queue.cc:1059] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 13:03:31.959323 13029 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:32.010764 13015 raft_consensus.cc:3055] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:32.011281 13015 raft_consensus.cc:740] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:32.011832 12607 raft_consensus.cc:3055] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:03:32.025933  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.974727 13001 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:32.046437 13001 raft_consensus.cc:697] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:32.009999 13029 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:31.976321 12670 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:32.053165  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2581) took 1499 ms. Trace:
I20260709 13:03:32.054913  8405 rpcz_store.cc:276] 0709 13:03:30.553526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.666508 (+112982us) service_pool.cc:224] Handling call
0709 13:03:30.676326 (+  9818us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.215226 (+538900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.052785 (+837559us) spinlock_profiling.cc:243] Waited 536 ms on lock 0x7b54000fcdd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:32.053085 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1232472960}
I20260709 13:03:32.012327 12607 raft_consensus.cc:740] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:32.044203 13031 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:32.039131 13011 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:03:32.057519 13029 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:32.059514 13122 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.012256 13015 consensus_queue.cc:260] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:32.073474  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 109582us
I20260709 13:03:32.093672 13015 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:32.094403 12670 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:32.094700 12898 raft_consensus.cc:3055] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:32.095153 12898 raft_consensus.cc:740] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:32.095918 12898 consensus_queue.cc:260] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:32.096998 12898 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:32.076212 13122 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:32.097934 12973 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:32.098938 13031 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.099530 13122 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:32.099469 13031 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:32.100225 13122 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:32.102572 13122 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.106237 13122 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:32.108587 13122 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:32.109688 13122 consensus_queue.cc:1048] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:32.122383 13001 consensus_queue.cc:237] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:32.133549 12607 consensus_queue.cc:260] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:32.135125 12607 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:32.070194 13125 log.cc:927] Time spent T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.305s	user 0.000s	sys 0.001s
I20260709 13:03:32.145308 12798 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:32.169232 13071 consensus_queue.cc:1243] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer 8c42259d7c4b463a9f485f42c320c81b log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:03:32.146175 13015 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:03:32.183147  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 107302us
I20260709 13:03:32.186605 13161 consensus_queue.cc:1048] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:03:32.190747  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.192731  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:32.193400  8327 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:32.195000  8331 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:32.182010 13031 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:32.204794  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:32.205962  8326 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:32.211577  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:32.212709  8325 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:32.226091  8329 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:03:32.228859  8257 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:32.187912 13015 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:32.236929 13071 consensus_queue.cc:1048] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:32.202276 13157 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:03:32.219363 13031 consensus_queue.cc:1059] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
W20260709 13:03:32.246081  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 50490.1us
I20260709 13:03:32.263047 13015 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:32.247255 13037 consensus_meta.cc:287] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.003s
I20260709 13:03:32.264112 13037 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:32.265542 13163 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:03:32.266212 13037 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:32.274024  8369 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:32.278573 13029 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:32.282135 13031 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:03:32.296391 13145 log.cc:927] Time spent T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.370s	user 0.002s	sys 0.001s
I20260709 13:03:32.289347 13122 consensus_queue.cc:1059] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:03:32.327176 13061 raft_consensus.cc:3055] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:32.327888 13061 raft_consensus.cc:740] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:32.329116 13061 consensus_queue.cc:260] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:32.331120 13147 log.cc:927] Time spent T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.407s	user 0.003s	sys 0.000s
I20260709 13:03:32.345269  8405 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:32.346645  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:03:32.349812  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 426104us
I20260709 13:03:32.350176  8401 raft_consensus.cc:1240] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:03:32.352808  8402 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:32.356092  8397 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
W20260709 13:03:32.358888  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:32.359617  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.359758  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:32.360445  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.363667  8331 raft_consensus.cc:1275] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:03:32.365134  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:32.368410  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:32.372133  8331 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 14. Current term is 15. Ops: [13.9-14.10]
I20260709 13:03:32.375161  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:32.376243  8295 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:32.377101  8331 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:32.353443 12933 consensus_queue.cc:1048] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:32.379814  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:32.381973  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:32.385090  8295 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:32.386822  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:32.390038  8327 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:32.396517  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:32.401299  8400 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:32.404838  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:32.367470 13078 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.411211  8295 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:32.374394 13065 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:32.353296 13009 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:03:32.378477 12898 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.407646 13058 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:32.395090 13063 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:32.416252 13078 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:32.422621  8223 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:32.426242  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.416374  8295 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:32.440822  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:32.447778  8295 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:32.446825  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:32.448531  8223 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:32.412458 12854 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:32.437640 12670 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:03:32.451689  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 176284us
I20260709 13:03:32.458756  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:32.459626  8324 raft_consensus.cc:2410] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 10 because replica has last-logged OpId of term: 9 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 10.
I20260709 13:03:32.459332  8369 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:32.453802  8295 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:32.451368  8223 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:32.441764 13063 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:32.497263  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 2118) took 1141 ms. Trace:
I20260709 13:03:32.497721  8254 rpcz_store.cc:276] 0709 13:03:31.355452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.356220 (+   768us) service_pool.cc:224] Handling call
0709 13:03:31.356493 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.601379 (+244886us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.497040 (+895661us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b540026cd54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:32.497208 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":561471744}
W20260709 13:03:32.498842  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 47578.1us
I20260709 13:03:32.500161  8223 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:32.452165 13138 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:32.497740  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:32.502595  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:32.503166  8324 raft_consensus.cc:1275] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 13:03:32.446372 13029 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:32.452857 13076 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:03:32.493490 13149 log.cc:927] Time spent T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:03:32.513795  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:32.450476 13146 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:32.515651  8251 raft_consensus.cc:1275] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:32.518970  8332 raft_consensus.cc:1275] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:32.442010 12898 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:32.457598 12670 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:32.438823 13065 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:32.453969 12996 raft_consensus.cc:3055] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:03:32.528647  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 74956.7us
I20260709 13:03:32.551782  8295 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:32.552433  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:32.494521 13160 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:32.526786  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:32.447435 13058 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:32.555277 13078 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:32.503880 13063 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:32.527961  8220 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:32.519305 13031 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:32.557469  8405 raft_consensus.cc:1275] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:32.558701  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:32.566907  9005 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:03:32.562155  8248 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
I20260709 13:03:32.562866  8254 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:32.569464  8298 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:32.502110 13139 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:32.573385  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 45555.1us
I20260709 13:03:32.519547 13122 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:03:32.516805 13100 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:32.552443 12670 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:32.553089 12996 raft_consensus.cc:740] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:32.590087  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:32.563220 13160 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:32.592901  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:32.595711  8220 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:32.573902 13031 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:32.597143  8220 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:32.599711  8220 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:32.577443 13088 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:32.550872 13142 consensus_queue.cc:1048] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:03:32.578516 13078 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:32.648136  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:32.662410  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 150806us
I20260709 13:03:32.605063 13012 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:32.595824 13160 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:32.598626 13031 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:32.605051 13088 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:32.608271 13023 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:32.608855 13146 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:32.647078 12932 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.646234 13078 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:03:32.684338 13168 log.cc:927] Time spent T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
W20260709 13:03:32.698683  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 146989us
I20260709 13:03:32.685220 13088 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:32.708262  8220 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:32.704873  8295 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:03:32.690670 13087 log.cc:927] Time spent T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 1.535s	user 0.006s	sys 0.000s
I20260709 13:03:32.680521 13180 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:32.687794 13023 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:32.715520  8295 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:03:32.695070 13173 log.cc:927] Time spent T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:03:32.690899 13078 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:32.704149 13137 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.707532 13012 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.732842 13042 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:32.714702 13076 raft_consensus.cc:697] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:32.717779 13146 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:32.725764 13023 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:32.683560 12996 consensus_queue.cc:260] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:32.710419 13015 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:03:32.695245 12932 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:32.721194 13141 consensus_queue.cc:1048] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:03:32.721561 13139 consensus_queue.cc:1048] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:03:32.766618  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2267) took 2030 ms. Trace:
I20260709 13:03:32.768023  8330 rpcz_store.cc:276] 0709 13:03:30.736614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.736791 (+   177us) service_pool.cc:224] Handling call
0709 13:03:30.737189 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:30.788418 (+ 51229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:30.788465 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:30.794508 (+  6043us) log.cc:844] Serialized 26 byte log entry
0709 13:03:31.018210 (+223702us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:31.018961 (+   751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.019041 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:32.561027 (+1541986us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:32.757629 (+196602us) raft_consensus.cc:1685] finished
0709 13:03:32.757651 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:32.766547 (+  8896us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30413,"raft.queue_time_us":59881,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":16792,"spinlock_wait_cycles":30494592,"thread_start_us":150658,"threads_started":1,"wal-append.queue_time_us":288721}
I20260709 13:03:32.745141 13176 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:03:32.766328 13175 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 13:03:32.754616 12898 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:32.761101 13065 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:32.764647 12936 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.764597 13058 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:32.753098 13042 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:32.751334 13137 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:32.781394 12996 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:32.802453  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 233075us
I20260709 13:03:32.753587 13012 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:32.821462  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 133951us
I20260709 13:03:32.823876  8369 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:32.799546 12898 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:32.799870 13065 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:32.797128 13184 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:32.778101 13176 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:32.844475 13185 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:32.838534  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:32.846988 13177 log.cc:927] Time spent T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:03:32.844993 13157 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:32.848536 13042 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:32.846029 13137 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:32.843814 13012 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:32.845607  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 721) took 1066 ms. Trace:
I20260709 13:03:32.794301 12936 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:32.908471  8398 rpcz_store.cc:276] 0709 13:03:31.779364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.780300 (+   936us) service_pool.cc:224] Handling call
0709 13:03:31.780661 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.784978 (+  4317us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.841575 (+1056597us) spinlock_profiling.cc:243] Waited 2.15 ms on lock 0x7b5400267fd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:32.845530 (+  3955us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4942720}
I20260709 13:03:32.831135 13058 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:03:32.874557  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:32.846038 13174 log.cc:927] Time spent T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:03:32.839354 12898 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:32.898625  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:32.919602  8325 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 16: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:32.900002  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:32.839643 13065 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:32.900938 13171 log.cc:927] Time spent T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.143s	user 0.001s	sys 0.001s
I20260709 13:03:32.895922 13157 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:32.896063 13139 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.902964 12807 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:32.905607 13189 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.952986  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:32.914912 13012 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:32.923467 12898 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:32.830981 13076 consensus_queue.cc:237] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:32.985229  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:32.985334  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:32.999684  8252 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 16: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:32.932958 13065 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:33.006700  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:32.940465 13157 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:33.015065 13203 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:32.895922 13185 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:33.041944  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:33.043635  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:32.982326 13193 log.cc:927] Time spent T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 13:03:33.048372  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:33.053865 13191 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:03:33.056185 13078 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:33.058131 13176 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:33.058705 13078 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:33.053956 13180 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:33.060384 13194 log.cc:927] Time spent T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 13:03:33.062191 13188 log.cc:927] Time spent T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:03:33.001783 13196 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:33.082300 13196 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:33.002964 13012 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:33.088068 13012 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:33.012611 13171 log.cc:927] Time spent T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:03:33.106024 13202 log.cc:927] Time spent T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:03:33.119211 12936 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:32.986912 13020 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.974076 13138 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:33.148857  8254 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:03:33.152773  8254 pending_rounds.cc:85] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 9
I20260709 13:03:33.153132  8254 pending_rounds.cc:107] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:03:33.153580  8254 raft_consensus.cc:2889] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:33.155139  8254 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:03:33.181370  8402 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:03:33.181979  8402 pending_rounds.cc:85] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 5
I20260709 13:03:33.182297  8402 pending_rounds.cc:107] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:03:33.182850  8402 raft_consensus.cc:2889] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:33.187986  8250 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:03:33.052156 12854 consensus_meta.cc:287] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.599s	user 0.011s	sys 0.001s
I20260709 13:03:33.129129 13185 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:33.160902 13196 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:33.149587 13184 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:32.985584 13189 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:32.963856 13139 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:33.265990  8295 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.020s	sys 0.057s
I20260709 13:03:33.139524 13203 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:33.268740  8295 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.021s	sys 0.058s
I20260709 13:03:33.179195 13020 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:33.271278  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 555733us
I20260709 13:03:33.202708 12854 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:33.284428  8254 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:03:33.287339  8295 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:33.288981  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.295953  8400 raft_consensus.cc:1275] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 13:03:33.281492 13185 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.303182  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:33.337301  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:33.222932 13184 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:33.338012  8332 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:03:33.348830  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:33.299582 13180 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:33.332299 13031 consensus_meta.cc:287] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.646s	user 0.009s	sys 0.004s
I20260709 13:03:33.362617  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:33.319120 12958 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:33.297209 13187 consensus_queue.cc:1059] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.083s
W20260709 13:03:33.367468 13088 consensus_meta.cc:287] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.645s	user 0.009s	sys 0.002s
I20260709 13:03:33.368067 13079 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:33.371822  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:33.374361  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:33.374926  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:33.363248 13201 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:03:33.376145  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.377532  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2273) took 2493 ms. Trace:
I20260709 13:03:33.364377 13078 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:33.380435  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:33.384716  8253 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.829s	user 0.041s	sys 0.056s
I20260709 13:03:33.341255 13012 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:03:33.396056  8253 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.841s	user 0.041s	sys 0.057s
I20260709 13:03:33.396102  8327 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:33.395812  8329 rpcz_store.cc:276] 0709 13:03:30.883942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.884077 (+   135us) service_pool.cc:224] Handling call
0709 13:03:30.985126 (+101049us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:32.230628 (+1245502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.377186 (+1146558us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b5400058cd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:33.377456 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2843139840}
W20260709 13:03:33.394295  8401 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.898s	user 0.030s	sys 0.069s
I20260709 13:03:33.397356  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:33.398340  8327 raft_consensus.cc:2393] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:03:33.398905  8401 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.904s	user 0.030s	sys 0.069s
I20260709 13:03:33.398838  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:33.399905  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:33.400362  8249 raft_consensus.cc:2393] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 14: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:33.400539  8254 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
W20260709 13:03:33.397472  8397 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.805s	user 0.022s	sys 0.077s
W20260709 13:03:33.401372  8397 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.022s	sys 0.077s
I20260709 13:03:33.405606  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:33.319767 13185 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:33.390535 13160 consensus_meta.cc:287] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.703s	user 0.006s	sys 0.006s
W20260709 13:03:33.390103 13063 consensus_meta.cc:287] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.808s	user 0.009s	sys 0.004s
I20260709 13:03:33.402072  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:33.407944  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2603) took 1043 ms. Trace:
I20260709 13:03:33.419317  8249 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:03:33.419201  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:33.420131  8248 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:33.415110  8220 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:33.384366 13031 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:33.417567  8397 rpcz_store.cc:276] 0709 13:03:32.364336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.369965 (+  5629us) service_pool.cc:224] Handling call
0709 13:03:32.370311 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:32.370553 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:32.591826 (+221273us) log.cc:844] Serialized 14 byte log entry
0709 13:03:33.407445 (+815619us) log.cc:844] Serialized 14 byte log entry
0709 13:03:33.407627 (+   182us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:33.407671 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:33.407727 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.407790 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:33.407891 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18033,"prepare.queue_time_us":313842,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"thread_start_us":967625,"threads_started":2}
I20260709 13:03:33.431684  8332 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:03:33.435555 13180 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:33.436338 13180 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:33.435643 13187 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:33.440902 13187 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:33.443663 13211 log.cc:927] Time spent T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.101s	user 0.001s	sys 0.001s
I20260709 13:03:33.443929 13180 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:33.447954  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2543) took 1029 ms. Trace:
I20260709 13:03:33.448529  8331 rpcz_store.cc:276] 0709 13:03:32.419024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.419191 (+   167us) service_pool.cc:224] Handling call
0709 13:03:32.419587 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:32.646272 (+226685us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:32.646308 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:32.675542 (+ 29234us) log.cc:844] Serialized 26 byte log entry
0709 13:03:32.903437 (+227895us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:32.903498 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.903709 (+   211us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:33.442910 (+539201us) spinlock_profiling.cc:243] Waited 5.66 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:33.447531 (+  4621us) raft_consensus.cc:1685] finished
0709 13:03:33.447572 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:33.447873 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16373,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"spinlock_wait_cycles":13026688,"thread_start_us":386324,"threads_started":2,"wal-append.queue_time_us":586558}
I20260709 13:03:33.451529 13180 consensus_queue.cc:1059] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:33.453533 13213 raft_consensus.cc:3055] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:33.453933 13213 raft_consensus.cc:740] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:33.454026 13180 consensus_queue.cc:1059] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:33.454777 13213 consensus_queue.cc:260] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:33.455591  8297 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:33.456148 13213 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:33.456342 12730 raft_consensus.cc:3055] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:33.456931 12730 raft_consensus.cc:740] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:33.457824 12730 consensus_queue.cc:260] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:33.458834 12730 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:33.468127  8297 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:03:33.475555  8369 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.095s	sys 0.272s
W20260709 13:03:33.475919  8369 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.095s	sys 0.272s
W20260709 13:03:33.476847  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 652936us
I20260709 13:03:33.396669 12958 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:33.398108 13088 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:33.478113  8406 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.018s	sys 0.083s
W20260709 13:03:33.479974  8406 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.019s	sys 0.083s
I20260709 13:03:33.402750 13012 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:33.407817 13078 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:33.472647 13136 log.cc:927] Time spent T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 1.454s	user 0.004s	sys 0.000s
I20260709 13:03:33.402850 13201 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:33.484160 12807 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:33.485416 12989 raft_consensus.cc:3055] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:33.485819 13139 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:33.486267 13189 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:33.474326 12670 consensus_meta.cc:287] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.835s	user 0.009s	sys 0.004s
I20260709 13:03:33.431728 13137 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:03:33.464020 12996 consensus_meta.cc:287] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.649s	user 0.009s	sys 0.000s
W20260709 13:03:33.470707 13023 consensus_meta.cc:287] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.714s	user 0.010s	sys 0.000s
I20260709 13:03:33.469799 13133 raft_consensus.cc:3055] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:33.419054 13160 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:33.473647 13213 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:33.403355 13079 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:33.424459 13063 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:33.509290 13158 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 1.279s	user 0.003s	sys 0.000s
I20260709 13:03:33.494316 12989 raft_consensus.cc:740] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:33.498519 13020 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:33.467271 12898 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:33.500691 13137 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.504832 13133 raft_consensus.cc:740] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:33.507347 13203 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:33.533802 13170 log.cc:927] Time spent T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.928s	user 0.000s	sys 0.003s
W20260709 13:03:33.535383 13148 log.cc:927] Time spent T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 1.611s	user 0.003s	sys 0.002s
I20260709 13:03:33.525066 13137 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:33.548923 13065 consensus_meta.cc:287] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.509s	user 0.010s	sys 0.000s
W20260709 13:03:33.609802  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 257880us
I20260709 13:03:33.702806 13065 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:33.610441  8252 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:03:33.705943  8325 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:03:33.706993  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 2146) took 1110 ms. Trace:
W20260709 13:03:33.708102  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:33.625149  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 157004us
I20260709 13:03:33.621335  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 2122) took 2259 ms. Trace:
I20260709 13:03:33.709373  8297 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:33.710068  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:33.709821  8256 rpcz_store.cc:276] 0709 13:03:31.361374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.361823 (+   449us) service_pool.cc:224] Handling call
0709 13:03:31.398294 (+ 36471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:31.409888 (+ 11594us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:31.409918 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:31.508260 (+ 98342us) spinlock_profiling.cc:243] Waited 6.31 ms on lock 0x7b180037c600. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882569f2a6 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a
0709 13:03:31.509404 (+  1144us) spinlock_profiling.cc:243] Waited 3.46 ms on lock 0x7b2c000014c8. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:03:31.580705 (+ 71301us) log.cc:844] Serialized 26 byte log entry
0709 13:03:31.858197 (+277492us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:31.909985 (+ 51788us) log.cc:844] Serialized 14 byte log entry
0709 13:03:32.346376 (+436391us) spinlock_profiling.cc:243] Waited 1.63 ms on lock 0x7b180037c600. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882564bc0a 00007f882ed8b219 00007f88256d3a40 00007f88256ce4e4 00007f88256ce455 00007f88287b6060 00007f88287c8716 00007f88287d9287 00007f88287d9208
0709 13:03:32.346577 (+   201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.346651 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:33.471788 (+1125137us) spinlock_profiling.cc:243] Waited 9.93 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:33.620750 (+148962us) raft_consensus.cc:1685] finished
0709 13:03:33.620804 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:33.621116 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":96197,"prepare.queue_time_us":53029,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":51,"raft.queue_time_us":451752,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":7359,"spinlock_wait_cycles":49050368,"thread_start_us":235907,"threads_started":1,"wal-append.queue_time_us":407001}
I20260709 13:03:33.630071  8324 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:33.478112  8369 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:33.715056  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 236905us
I20260709 13:03:33.497452 12807 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:33.632453  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2604) took 1267 ms. Trace:
I20260709 13:03:33.500070 12670 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:33.554654 13079 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:33.560025 13160 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:33.750773 13012 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:33.751856 13031 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:33.561583 13201 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:33.560806 13157 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:33.755767 13157 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:33.756472 13058 raft_consensus.cc:3055] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:33.756982 13058 raft_consensus.cc:740] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:33.758255  8330 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:33.665294 13063 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:33.760133 13203 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:33.665910  8323 raft_consensus.cc:1240] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:03:33.667006 12849 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:33.717296  8406 rpcz_store.cc:276] 0709 13:03:32.364633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.370704 (+  6071us) service_pool.cc:224] Handling call
0709 13:03:32.376479 (+  5775us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:32.376763 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:32.654625 (+277862us) log.cc:844] Serialized 14 byte log entry
0709 13:03:32.820902 (+166277us) log.cc:844] Serialized 14 byte log entry
0709 13:03:33.488231 (+667329us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:33.488292 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:33.489148 (+   856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.489243 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:33.632130 (+142887us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77981,"prepare.queue_time_us":362507,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":95,"raft.queue_time_us":23641,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":53,"thread_start_us":1005549,"threads_started":3,"wal-append.queue_time_us":179154}
I20260709 13:03:33.769234 12807 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:33.772558 13187 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:33.672259 13133 consensus_queue.cc:260] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:33.773854 13133 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:33.778136 13065 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:33.555721 13078 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:33.680581 13221 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.780551 13221 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:33.781186  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.559142 13185 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:33.689543  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 721) took 1145 ms. Trace:
I20260709 13:03:33.817840  8253 rpcz_store.cc:276] 0709 13:03:32.543759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.552105 (+  8346us) service_pool.cc:224] Handling call
0709 13:03:32.554990 (+  2885us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:32.555239 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:33.396683 (+841444us) log.cc:844] Serialized 14 byte log entry
0709 13:03:33.688753 (+292070us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:33.688874 (+   121us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:33.688980 (+   106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.689116 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:33.689343 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32082,"prepare.queue_time_us":7760,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":33515,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":151,"thread_start_us":1100397,"threads_started":2,"wal-append.queue_time_us":264184}
I20260709 13:03:33.690045  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 719) took 1156 ms. Trace:
I20260709 13:03:33.701443 12958 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:33.819896  8250 rpcz_store.cc:276] 0709 13:03:32.516911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.517049 (+   138us) service_pool.cc:224] Handling call
0709 13:03:32.518528 (+  1479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.188567 (+670039us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.672894 (+484327us) spinlock_profiling.cc:243] Waited 669 ms on lock 0x7b5400177d54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:33.673312 (+   418us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1538326912}
I20260709 13:03:33.821218 13058 consensus_queue.cc:260] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:33.822732 13058 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:33.825577 13160 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:33.826380 13160 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:33.707458  8252 rpcz_store.cc:276] 0709 13:03:32.596113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.051337 (+455224us) service_pool.cc:224] Handling call
0709 13:03:33.051865 (+   528us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:33.705802 (+653937us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.706603 (+   801us) spinlock_profiling.cc:243] Waited 558 ms on lock 0x7b540034c6d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:33.706912 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1282683648}
W20260709 13:03:33.708289  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.713784 13176 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:33.715104 13088 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:33.635521  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2605) took 1270 ms. Trace:
I20260709 13:03:33.636814 12989 consensus_queue.cc:260] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:33.834236  8329 raft_consensus.cc:1275] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:33.834995 12989 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:33.665320 12898 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:33.709682 13213 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:33.838021 13213 raft_consensus.cc:697] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:33.833947  8401 rpcz_store.cc:276] 0709 13:03:32.364935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.379630 (+ 14695us) service_pool.cc:224] Handling call
0709 13:03:32.379928 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:32.380120 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:33.402203 (+1022083us) log.cc:844] Serialized 14 byte log entry
0709 13:03:33.634486 (+232283us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:33.634606 (+   120us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:33.634803 (+   197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.634965 (+   162us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:33.635278 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71246,"prepare.queue_time_us":358988,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":2036,"raft.queue_time_us":32280,"raft.run_cpu_time_us":355,"raft.run_wall_time_us":358,"thread_start_us":1138916,"threads_started":2,"wal-append.queue_time_us":159058}
I20260709 13:03:33.840615 13204 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:33.840425 13213 consensus_queue.cc:237] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:33.841630  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 101154us
I20260709 13:03:33.841506 13204 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:33.844904  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:33.733790 12909 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.736333 13161 consensus_queue.cc:1059] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:33.736265 12932 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:33.737462 13071 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:33.740193 12670 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:33.783247 13221 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:33.787330 13210 log.cc:927] Time spent T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.523s	user 0.003s	sys 0.000s
I20260709 13:03:33.505868 13023 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:33.791899  8331 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:33.794867  8295 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:03:33.801638  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 374766us
I20260709 13:03:33.803442  8972 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:33.804186 12974 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:33.809700 13061 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:33.733237 13196 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:33.846727 13001 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.847225  8401 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:03:33.848138 13206 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.848556  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:33.850323  8327 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:33.851756  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:33.853102  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:33.854103  9009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56948 (request call id 944) took 2251 ms (client timeout 2279 ms). Trace:
W20260709 13:03:33.935801  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 91034.8us
I20260709 13:03:33.856666 13009 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.858593 12868 consensus_queue.cc:1048] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:03:33.859175  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.862475 13141 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:33.864144  8257 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 2119) took 2507 ms (client timeout 1890 ms). Trace:
I20260709 13:03:33.865556 13160 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:33.868067 13184 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:33.847225 13031 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.873050 13157 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:33.874328 13018 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:33.875211 13169 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:33.844230 13204 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:33.970748 13031 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:33.974433  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 171329us
I20260709 13:03:33.975327 13141 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:33.977749 13018 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:33.980024 13018 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:33.877868  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:33.886161  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:33.888991  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:33.855605  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:33.940963  9009 rpcz_store.cc:269] 0709 13:03:31.602953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.603940 (+   987us) service_pool.cc:224] Handling call
0709 13:03:33.853640 (+2249700us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b5400527bd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:33.853838 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":771456896}
W20260709 13:03:33.981825  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.964430 13156 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:33.944037  8257 rpcz_store.cc:269] 0709 13:03:31.356882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.357394 (+   512us) service_pool.cc:224] Handling call
0709 13:03:33.863715 (+2506321us) spinlock_profiling.cc:243] Waited 810 ms on lock 0x7b54003471d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:33.863903 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1863524608}
I20260709 13:03:33.965219 13157 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:33.965632 13200 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.988013  8369 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:33.965974 13206 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:33.967947 12909 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:33.969655 13169 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:33.939549 13196 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:33.992511 12789 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:33.997241 12807 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:34.001608 13001 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:34.004978 13001 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:34.005796  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:34.006453  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.006803 12974 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:34.007802  8973 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.006s	sys 0.027s
I20260709 13:03:34.007925  8405 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:34.008932  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:34.010840  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:34.013388  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:34.016774  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:34.018579  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:34.019905  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:34.020530  8401 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
W20260709 13:03:34.021240  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.022965 13206 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:34.023052  8974 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:34.014297 13196 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:34.024073  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 229386us
I20260709 13:03:34.025087  8295 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:34.026309 12849 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:34.026885 12849 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:34.027362 12849 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:34.028417  8250 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 10. Current term is 12. Ops: []
I20260709 13:03:34.028491 12974 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:34.029120 12974 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:34.030170  8398 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:34.069062  8222 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:34.036664 12849 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:34.042331  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:34.043783  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:34.042029  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:34.044149 13023 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:34.032341  8251 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
W20260709 13:03:34.031714  8973 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.006s	sys 0.028s
I20260709 13:03:34.044200 13163 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:34.033938  8399 raft_consensus.cc:2393] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:34.000548 13137 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:34.095649 13137 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:34.096071 13001 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:33.989101 13031 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:34.030454 13009 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:34.054128  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 70428.5us
I20260709 13:03:34.098079  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:34.099296 13009 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:34.100276 13031 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:34.055025  8972 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:34.059757  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:34.059988  8150 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55754 (request call id 200) took 30634 ms (client timeout 15000 ms). Trace:
I20260709 13:03:34.061518 13206 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:34.065511 12898 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:34.065653 12958 consensus_queue.cc:1059] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:34.065922 13141 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:34.066376 13184 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:34.067204 13012 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:34.040735  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:34.111495  8295 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:03:34.119339  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 180199us
I20260709 13:03:34.085981  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:34.089759  8254 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:34.090052 12670 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.091266  8154 catalog_manager.cc:5697] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 5 to 8, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 8 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:34.092098 13163 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:34.092789  8257 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:03:34.093645  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:34.087496  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:34.102931  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:34.091297  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:34.107370  8253 raft_consensus.cc:2393] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 16: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:34.104717  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:34.101523 12807 consensus_queue.cc:1048] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
W20260709 13:03:34.112365  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 47280.3us
I20260709 13:03:34.113217  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:34.115093  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:34.116954  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.134971 13157 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:34.134914  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:34.138191  8251 raft_consensus.cc:2393] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 11: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:34.116747 13137 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:34.014868 13200 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:34.109813 12974 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:03:34.143203  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:34.155484  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.156332  9004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 949) took 2574 ms. Trace:
I20260709 13:03:34.156378  9006 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:34.156999  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:34.158320  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:03:34.159089 13232 log.cc:927] Time spent T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.220s	user 0.002s	sys 0.000s
I20260709 13:03:34.155519 12973 consensus_queue.cc:1048] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:03:34.159466  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:34.161353  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 106484us
I20260709 13:03:34.161947 13012 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:34.162307  8972 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:34.163470  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.164161  9000 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:34.164265 13012 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:34.155700 12974 consensus_queue.cc:1243] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:03:34.151727  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:34.168495  8405 raft_consensus.cc:2410] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:03:34.164916  8397 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:34.117017 13031 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:03:34.173728  8401 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:03:34.142241 13137 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:34.180675  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:34.178926  8256 raft_consensus.cc:2393] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 13: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:34.178036  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:34.143678 13133 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:34.149421  8403 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:03:34.165596  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:34.187281  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 47997.2us
I20260709 13:03:34.192278  8331 raft_consensus.cc:2410] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:03:34.189790  8402 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:34.194154  8248 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:03:34.193106  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:34.135015 12946 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.101s
I20260709 13:03:34.204702  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:34.210634  8323 raft_consensus.cc:2393] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:34.159482  8154 catalog_manager.cc:5697] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 4 to 8, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 8 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:34.183677 13137 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:34.195118  8220 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:34.146377 13196 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:03:34.205379  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:34.216750 13157 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:34.216924  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:34.218075 13236 log.cc:927] Time spent T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:03:34.203052 13233 log.cc:927] Time spent T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.070s	user 0.002s	sys 0.000s
I20260709 13:03:34.148600 13206 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:34.225719  9009 raft_consensus.cc:2393] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 13: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:34.223480  8222 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:34.199082 13207 raft_consensus.cc:3055] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:34.221894  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.183094 13088 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.156913  9004 rpcz_store.cc:276] 0709 13:03:31.581491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.582679 (+  1188us) service_pool.cc:224] Handling call
0709 13:03:31.583081 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:31.583446 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:31.583486 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:31.592646 (+  9160us) log.cc:844] Serialized 26 byte log entry
0709 13:03:32.580279 (+987633us) spinlock_profiling.cc:243] Waited 6.74 ms on lock 0x7b18003f5ce0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e0d1 00007f8828779a0f
0709 13:03:32.901557 (+321278us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:32.901626 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.901789 (+   163us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:33.786347 (+884558us) spinlock_profiling.cc:243] Waited 6.04 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:34.132340 (+345993us) raft_consensus.cc:1685] finished
0709 13:03:34.132383 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:34.156117 (+ 23734us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2351,"raft.queue_time_us":5320,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":15568,"spinlock_wait_cycles":29405312,"thread_start_us":321012,"threads_started":1,"wal-append.queue_time_us":1594267}
I20260709 13:03:34.177377 12670 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:34.233534  8401 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:03:34.235021  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 147291us
I20260709 13:03:34.227720  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.238974  8330 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:34.225462 13184 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:34.250509  9000 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:03:34.251052  8404 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:34.251503  9005 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:03:34.254263  8330 raft_consensus.cc:1240] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:34.228535 12868 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.250572  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:03:34.269279  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 98466.3us
I20260709 13:03:34.238178 13207 raft_consensus.cc:740] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:34.265870  8398 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:03:34.256179  8154 catalog_manager.cc:5697] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 6 to 11, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 11 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:34.256055 13184 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.276446  8398 pending_rounds.cc:85] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 6
I20260709 13:03:34.239673 13226 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:03:34.283502  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:34.284219  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 122024us
I20260709 13:03:34.262176 12996 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:34.270325 13023 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:34.319099  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2087) took 2542 ms. Trace:
I20260709 13:03:34.282401  8398 pending_rounds.cc:107] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:03:34.328030  8398 raft_consensus.cc:2889] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:34.323558  8154 catalog_manager.cc:5697] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 3 to 8, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 8 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:34.329610  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:34.329818  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:34.330927 12868 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:34.331876 13231 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:34.331624  8154 catalog_manager.cc:5697] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 5 to 6, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 6 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:34.317024  9006 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:34.333597 13234 log.cc:927] Time spent T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:03:34.334070 13020 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:34.334682 13020 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:34.336288 12868 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:34.336818 13020 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:34.288915 13088 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:34.340677 13088 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:34.317858 13012 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:34.341588 13185 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:34.342113 13012 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.342164 12898 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:34.342447 13185 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:34.342456  8972 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:34.345291  8399 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:03:34.268553 13142 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:34.355446  8254 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:03:34.289852 13184 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:34.363523  9006 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:34.313396 13169 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:34.325492 13058 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:34.358433 12607 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.406062 13058 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:34.406136 12607 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:34.371541  8330 raft_consensus.cc:1240] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:34.370997  8154 catalog_manager.cc:5697] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:34.378562  9004 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:34.408638 13058 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:34.344628 13023 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.342849 13012 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:34.409663 13023 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:34.409765  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:34.410976  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.411932  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:34.379595  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.414425 13206 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:34.414650  9006 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:03:34.416013  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2552) took 1090 ms. Trace:
I20260709 13:03:34.416638 13009 raft_consensus.cc:3055] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:34.413004  8154 catalog_manager.cc:5697] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:03:34.381767 12989 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.382804 13196 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:34.383292  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:34.383944  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 171968us
I20260709 13:03:34.327363  8255 rpcz_store.cc:276] 0709 13:03:31.634568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.634725 (+   157us) service_pool.cc:224] Handling call
0709 13:03:31.635161 (+   436us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:31.656843 (+ 21682us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:31.656888 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:31.660517 (+  3629us) log.cc:844] Serialized 52 byte log entry
0709 13:03:33.687093 (+2026576us) spinlock_profiling.cc:243] Waited 939 us on lock 0x7b180022b340. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e0d1 00007f8828779a0f
0709 13:03:33.945879 (+258786us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:33.945987 (+   108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.961498 (+ 15511us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:34.158275 (+196777us) spinlock_profiling.cc:243] Waited 44.2 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:34.165236 (+  6961us) raft_consensus.cc:1685] finished
0709 13:03:34.165257 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:34.177049 (+ 11792us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24507,"raft.queue_time_us":25158,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":17920,"spinlock_wait_cycles":103844352,"thread_start_us":207253,"threads_started":1,"wal-append.queue_time_us":2260188}
I20260709 13:03:34.383646 13142 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:34.386915  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.381786  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:34.393806 13230 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.429955 13230 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:34.394176 13207 consensus_queue.cc:260] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:34.395489 13213 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.396370 12946 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.440306 12946 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:34.441390  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.442122  9008 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:34.442977 12946 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:34.398638 12789 consensus_queue.cc:1059] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:34.444347 12789 raft_consensus.cc:3055] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:34.444672 12789 raft_consensus.cc:740] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:34.398713 13061 consensus_queue.cc:1048] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:34.400642 13088 consensus_queue.cc:1059] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:34.446480 13088 raft_consensus.cc:3055] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:34.446873 13088 raft_consensus.cc:740] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:34.446986  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:34.447347 13185 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:34.447630  8330 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:34.355062 13231 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:03:34.447969 13088 consensus_queue.cc:260] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:34.366132 12974 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:34.450035 12670 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:34.450407  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:34.451097  8332 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:34.451210 13163 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:34.343005 12898 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:34.452935 12789 consensus_queue.cc:260] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:34.453657 12898 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:34.454468 12789 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:34.455137  8325 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:03:34.455972 13142 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:34.380714  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.320842 12909 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:34.316422 13157 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:34.417145 13009 raft_consensus.cc:740] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:34.432477  9001 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:34.416402  8326 rpcz_store.cc:276] 0709 13:03:33.325934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.326082 (+   148us) service_pool.cc:224] Handling call
0709 13:03:33.326448 (+   366us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:33.362235 (+ 35787us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:33.362296 (+    61us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:33.362893 (+   597us) log.cc:844] Serialized 52 byte log entry
0709 13:03:33.691331 (+328438us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b180049f360. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e0d1 00007f8828779a0f
0709 13:03:33.980667 (+289336us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:33.983562 (+  2895us) log.cc:844] Serialized 14 byte log entry
0709 13:03:33.987463 (+  3901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.010987 (+ 23524us) spinlock_profiling.cc:243] Waited 40.4 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:34.012812 (+  1825us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:34.415638 (+402826us) raft_consensus.cc:1685] finished
0709 13:03:34.415658 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:34.415839 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36124,"prepare.queue_time_us":3524,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":30303,"raft.queue_time_us":43829,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":61338,"spinlock_wait_cycles":125744384,"thread_start_us":249296,"threads_started":1,"wal-append.queue_time_us":571252}
I20260709 13:03:34.396962 12807 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:34.443482  9004 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:34.445151  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:34.438333  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.458701  9007 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:03:34.470490  9000 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:34.449353 13088 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:34.439796 13156 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:34.519088 13009 consensus_queue.cc:260] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:34.520630 13009 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:34.521606  8406 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
W20260709 13:03:34.107690  8150 rpcz_store.cc:269] 0709 13:03:03.425662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.425903 (+   241us) service_pool.cc:224] Handling call
0709 13:03:34.059655 (+30633752us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:25.730896 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:25.733698 (+  2802us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:25.735843 (+  2145us) write_op.cc:435] Acquired schema lock
0709 13:03:25.739744 (+  3901us) tablet.cc:662] Decoding operations
0709 13:03:25.805320 (+ 65576us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:25.805467 (+   147us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:25.806009 (+   542us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:03:25.807151 (+  1142us) tablet.cc:701] Row locks acquired
0709 13:03:25.807201 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:03:25.807360 (+   159us) write_op.cc:270] Start()
0709 13:03:25.807524 (+   164us) write_op.cc:276] Timestamp: P: 1783602205807303 usec, L: 0
0709 13:03:25.807546 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:03:25.808224 (+   678us) log.cc:844] Serialized 4301 byte log entry
0709 13:03:26.181555 (+373331us) op_driver.cc:464] REPLICATION: finished
0709 13:03:26.352114 (+170559us) write_op.cc:301] APPLY: starting
0709 13:03:26.352225 (+   111us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:27.886640 (+1534415us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:27.886754 (+   114us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:28.004816 (+118062us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:28.015556 (+ 10740us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=22,mrs_lookups=0
0709 13:03:28.021989 (+  6433us) write_op.cc:312] APPLY: finished
0709 13:03:28.116451 (+ 94462us) log.cc:844] Serialized 105 byte log entry
0709 13:03:28.277712 (+161261us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:28.291245 (+ 13533us) write_op.cc:454] Released schema lock
0709 13:03:28.364458 (+ 73213us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:28.407979 (+ 43521us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:28.438393 (+ 30414us) write_op.cc:454] Released schema lock
0709 13:03:30.066906 (+1628513us) spinlock_profiling.cc:243] Waited 6.05 ms on lock 0x7b1800475b40. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
Metrics: {"dns_us":75510,"mutex_wait_us":601778,"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":124214,"apply.run_cpu_time_us":70786,"apply.run_wall_time_us":2031508,"cfile_cache_hit":42,"cfile_cache_hit_bytes":43911,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6218,"cfile_init":2,"lbm_read_time_us":130154,"lbm_reads_10-100_ms":2,"lbm_reads_lt_1ms":10,"num_ops":11,"prepare.queue_time_us":27724,"prepare.run_cpu_time_us":6934,"prepare.run_wall_time_us":82316,"raft.queue_time_us":22837,"raft.run_cpu_time_us":51444,"raft.run_wall_time_us":102196,"replication_time_us":373748,"spinlock_wait_cycles":13903616,"thread_start_us":282717,"threads_started":5,"wal-append.queue_time_us":401766,"wal-append.run_cpu_time_us":22773,"wal-append.run_wall_time_us":2250515}]]}
I20260709 13:03:34.525290 13196 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:34.526100 13196 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:34.532429  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:34.456653 13200 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:34.535768  8330 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
W20260709 13:03:34.537258  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.446864  8154 catalog_manager.cc:5697] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 7 to 9, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 9 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:34.431947 12996 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:34.553541  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 157611us
I20260709 13:03:34.554379  8972 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:34.555965 12946 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:34.556401 12946 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:34.556764 12946 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:34.558386 12909 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:34.568800 12946 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:34.573839 12946 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:34.599309  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 147311us
I20260709 13:03:34.647630  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.502540 12989 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:34.655232  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 132249us
I20260709 13:03:34.648371  9000 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
W20260709 13:03:34.656476  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:34.658593  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 113430us
I20260709 13:03:34.529641 13212 consensus_queue.cc:1059] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:03:34.662976  8220 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:34.435322 13226 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:34.666846  8222 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:34.665853  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 114374us
I20260709 13:03:34.674535  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:34.675340  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:34.674535  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.655772 13207 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:34.678742  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:34.534873 12958 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.678614  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:34.662094 13141 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.675918  8398 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
W20260709 13:03:34.779311  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 99686.9us
I20260709 13:03:34.511091 13213 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:34.776340  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:34.798235  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:34.665417 13058 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:34.819361  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.656587 13012 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:34.676262 12973 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:34.826725  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:34.775861 13156 consensus_queue.cc:1059] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:03:34.832513  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.832758 12958 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:34.833619 13141 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:34.835183 12958 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:34.837116 12958 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:34.882771  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:34.835202  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:34.849802  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.885347  9004 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:34.826601  8331 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:34.682441 13206 raft_consensus.cc:3055] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:34.673580 12909 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.161s
I20260709 13:03:34.841496  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:34.810792 12932 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:34.846920  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.848026  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:34.910144  8248 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:34.849802  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:34.850811  8255 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:34.813021  8154 catalog_manager.cc:5697] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 6 to 10, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 10 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:03:34.832778  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:34.911772  8330 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:34.899281  8397 raft_consensus.cc:2410] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:03:34.843883 13157 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:34.906317  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2619) took 1030 ms. Trace:
W20260709 13:03:34.915050  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 138570us
I20260709 13:03:34.902216  8399 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:34.916105  8220 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:34.920807  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.924309  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:34.924424  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.801944 13169 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.100s
I20260709 13:03:34.915227  8404 rpcz_store.cc:276] 0709 13:03:33.856316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.893601 (+ 37285us) service_pool.cc:224] Handling call
0709 13:03:33.893929 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:34.257488 (+363559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.886493 (+629005us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b54002be154. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:34.886799 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":793166720}
I20260709 13:03:34.832518 13156 raft_consensus.cc:3055] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:34.894135 13206 raft_consensus.cc:740] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:34.931556  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.932092  8297 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:34.932719  8252 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:34.935761  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:34.939754  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.922230 13163 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:34.944552  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 289128us
I20260709 13:03:34.926631 12868 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.946816  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:34.947769  8254 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:03:34.948977  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.950356  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:34.951493  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:34.952215  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.949574  9000 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:03:34.954130  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.950599  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:34.933844 13156 raft_consensus.cc:740] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:03:34.964267  8332 consensus_meta.cc:287] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.513s	user 0.004s	sys 0.004s
I20260709 13:03:34.964715  8332 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:03:34.965747  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:34.966632  8332 raft_consensus.cc:2393] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 13: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:34.967482  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:34.968452  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:34.968930  8332 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:34.970775  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:34.971382  9003 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:34.975729  8154 catalog_manager.cc:5697] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 2 to 7, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 7 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:34.976290 13156 consensus_queue.cc:260] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:34.977790 13156 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:34.978613  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:34.979468  8326 raft_consensus.cc:2410] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:03:34.957338  8250 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
I20260709 13:03:34.982131  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.983486  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:34.983824  9008 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:34.984454  8331 raft_consensus.cc:2410] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:03:34.983621  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:34.985724  8329 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:34.987156 13141 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:34.988188 13256 log.cc:927] Time spent T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:03:34.988950 13141 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:34.989570  9007 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:03:34.941941 12973 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:34.990793 13012 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:34.979074  8404 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 9.
I20260709 13:03:35.012266 12996 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:35.013482  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:35.014290  8404 raft_consensus.cc:2393] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 13: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:34.979382 13163 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:35.017580 13137 consensus_meta.cc:287] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.793s	user 0.010s	sys 0.000s
I20260709 13:03:35.018158 13137 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:35.019748 13141 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:35.020738 13137 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:34.992147  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.997617 12868 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:35.022349  8369 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:34.994513 13226 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:34.995399 12989 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:34.930636 13058 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:34.992892 13206 consensus_queue.cc:260] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:34.974673 13230 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:35.009513 12958 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:35.013507  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:35.000087  8222 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:35.028491  8369 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:34.985415  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:35.055269 13206 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:35.056393 13157 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:35.057138 12958 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:35.081208 13157 raft_consensus.cc:697] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:35.082051  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 2614) took 1234 ms. Trace:
I20260709 13:03:35.082502 13157 consensus_queue.cc:237] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:35.082675  8406 rpcz_store.cc:276] 0709 13:03:33.847997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.848218 (+   221us) service_pool.cc:224] Handling call
0709 13:03:35.081717 (+1233499us) spinlock_profiling.cc:243] Waited 574 ms on lock 0x7b54001cbe54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:35.081918 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1320178944}
I20260709 13:03:35.067389 13213 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:35.066460 12996 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:35.093652 12996 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.094167 12996 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:35.094636  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2186) took 1060 ms. Trace:
I20260709 13:03:35.081355 12958 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:35.095755 12607 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:35.066128 13235 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.931s	user 0.022s	sys 0.038s
I20260709 13:03:35.069528  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:35.095050  8401 rpcz_store.cc:276] 0709 13:03:34.034478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.254568 (+220090us) service_pool.cc:224] Handling call
0709 13:03:34.269287 (+ 14719us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:34.324026 (+ 54739us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:34.343555 (+ 19529us) log.cc:844] Serialized 14 byte log entry
0709 13:03:34.415639 (+ 72084us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b480002cc58. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882d6e7aa1 00007f882d6deb37 00007f882d6de9ca 00007f882d7d626f 00007f882d7d5f37 00007f882d7ea55c 00007f882d7ea508 00007f882d7ea488 00007f882d7ea443 00007f882d7e96ec 00007f882c713a96
0709 13:03:34.654613 (+238974us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:34.831614 (+177001us) log.cc:844] Serialized 26 byte log entry
0709 13:03:34.855593 (+ 23979us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:34.855651 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.855761 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:35.094319 (+238558us) raft_consensus.cc:1685] finished
0709 13:03:35.094421 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:35.094576 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46331,"prepare.queue_time_us":19750,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":27578,"raft.run_cpu_time_us":1103,"raft.run_wall_time_us":97284,"spinlock_wait_cycles":46812928,"thread_start_us":310146,"threads_started":1,"wal-append.queue_time_us":748666}
I20260709 13:03:35.109397  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:35.111091  8326 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:35.111997 13200 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.112500 13200 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:35.113394  8326 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 14. Current term is 15. Ops: [7.7-14.10]
W20260709 13:03:35.113823  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 168467us
W20260709 13:03:35.113672  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 311399us
W20260709 13:03:35.123528  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 191486us
I20260709 13:03:35.115525  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2301) took 1807 ms. Trace:
I20260709 13:03:35.125867  8297 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:35.123584 12670 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:35.126839 12670 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:35.127226 12670 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:35.127131  8297 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:35.129024  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.994345  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:35.143054  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:34.978029  8154 catalog_manager.cc:5697] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:03:35.158025  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:35.129899  9005 raft_consensus.cc:2376] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13. Re-sending same reply.
I20260709 13:03:35.160441  9008 raft_consensus.cc:1240] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:03:35.144830  9000 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:03:35.143394  8397 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
W20260709 13:03:35.165791  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 46022.5us
W20260709 13:03:35.173259 12974 consensus_meta.cc:287] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.723s	user 0.012s	sys 0.002s
I20260709 13:03:35.174017 12974 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:35.175078  9006 raft_consensus.cc:1240] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:35.178085  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 250518us
I20260709 13:03:35.177023  8222 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:35.185611  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:35.185302  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:35.186587  9004 raft_consensus.cc:2393] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 13: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:35.180505  8295 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:35.114174 13262 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:35.195827  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:35.192829  8154 catalog_manager.cc:5697] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 1 to 9, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 9 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:35.204162  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:35.124794  8328 rpcz_store.cc:276] 0709 13:03:33.308412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.309013 (+   601us) service_pool.cc:224] Handling call
0709 13:03:33.309393 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:33.361631 (+ 52238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:33.361666 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:33.377711 (+ 16045us) log.cc:844] Serialized 26 byte log entry
0709 13:03:33.818577 (+440866us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b18003ef4a0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e0d1 00007f8828779a0f
0709 13:03:34.128090 (+309513us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:34.128184 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.128284 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:35.065317 (+937033us) spinlock_profiling.cc:243] Waited 6.66 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:35.115211 (+ 49894us) raft_consensus.cc:1685] finished
0709 13:03:35.115233 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:35.115352 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43326,"raft.queue_time_us":54438,"raft.run_cpu_time_us":64,"raft.run_wall_time_us":64,"spinlock_wait_cycles":21520000,"thread_start_us":260363,"threads_started":1,"wal-append.queue_time_us":756577}
I20260709 13:03:35.195873  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:35.206868  8369 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:35.165095 13213 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:03:35.210831  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.213039  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:35.160681 13246 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:35.221803  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:35.217831  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:35.224896 13261 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.040s
W20260709 13:03:35.227303 12789 consensus_meta.cc:287] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.772s	user 0.006s	sys 0.010s
I20260709 13:03:35.228348  8154 catalog_manager.cc:5697] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:35.205186  9000 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:03:35.230298  8154 catalog_manager.cc:5697] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:35.230746  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 47839.3us
I20260709 13:03:35.225258  8325 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:03:35.231580  8325 pending_rounds.cc:85] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 6
I20260709 13:03:35.196344 13262 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.238269  8325 pending_rounds.cc:107] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:03:35.240629  8325 raft_consensus.cc:2889] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:35.212711 13185 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:35.220747 13246 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.216171 13213 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:35.234987  8154 catalog_manager.cc:5697] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 5 to 6, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 6 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:03:35.241129  8329 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:35.259742  8331 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 10. Current term is 12. Ops: []
I20260709 13:03:35.258082  9009 raft_consensus.cc:1275] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:03:35.234362 13001 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:35.267661  8325 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:35.241286 13240 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.275240  9003 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:35.275363  8327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2563) took 1151 ms. Trace:
I20260709 13:03:35.272601  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:35.270605 13058 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:35.274545  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2111) took 1308 ms. Trace:
I20260709 13:03:35.243299 13262 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:35.284219  8253 rpcz_store.cc:276] 0709 13:03:33.967892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.987997 (+ 20105us) service_pool.cc:224] Handling call
0709 13:03:35.261366 (+1273369us) spinlock_profiling.cc:243] Waited 84.7 ms on lock 0x7b5400346cd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:35.261532 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":194775424}
W20260709 13:03:35.283424  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 106516us
I20260709 13:03:35.285907  8332 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:03:35.254626 13261 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:35.276093  8327 rpcz_store.cc:276] 0709 13:03:34.123935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.125519 (+  1584us) service_pool.cc:224] Handling call
0709 13:03:34.183824 (+ 58305us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:34.230968 (+ 47144us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:34.230999 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:34.278999 (+ 48000us) log.cc:844] Serialized 52 byte log entry
0709 13:03:34.779185 (+500186us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:34.926128 (+146943us) log.cc:844] Serialized 14 byte log entry
0709 13:03:35.116169 (+190041us) log.cc:844] Serialized 14 byte log entry
0709 13:03:35.125948 (+  9779us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:35.126032 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:35.271751 (+145719us) raft_consensus.cc:1685] finished
0709 13:03:35.271773 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:35.275261 (+  3488us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":199141,"prepare.queue_time_us":302037,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":76,"raft.queue_time_us":213379,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":11270,"thread_start_us":542550,"threads_started":2,"wal-append.queue_time_us":863317}
I20260709 13:03:35.247594 13163 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:35.293274  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:35.296878  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 78064.5us
I20260709 13:03:35.300253  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:35.260643 13213 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:35.305511  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:35.306533  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:35.308230  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.272974 12973 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:35.261754 13246 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:35.311049  8329 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 15. Current term is 16. Ops: [13.9-15.10]
I20260709 13:03:35.292048 13001 raft_consensus.cc:697] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:35.314687 13001 consensus_queue.cc:237] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:35.291882 13240 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:35.316980 13058 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:35.301700 13261 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:35.318065 13264 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:35.318857  8327 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:35.318826 13264 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:35.320163 13261 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:35.320360  8297 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:35.321025 13264 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:35.305397  8249 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
W20260709 13:03:35.331439  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 151019us
I20260709 13:03:35.331143  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:35.332612  8295 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:03:35.335057  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 352963us
I20260709 13:03:35.336452  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:35.346684  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.337105  8397 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
W20260709 13:03:35.355481  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 71218.5us
I20260709 13:03:35.355806 13012 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.356600 13012 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:35.358186  8222 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:03:35.359762  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.364002  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:35.282418 13212 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:03:35.370559  8249 raft_consensus.cc:2393] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 12: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:35.333748 13141 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:35.335800 13058 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:35.374614  8403 raft_consensus.cc:1275] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:03:35.361090 13248 log.cc:927] Time spent T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.245s	user 0.000s	sys 0.001s
I20260709 13:03:35.383133 13200 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:35.390262  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:35.395227  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:35.380061 13023 consensus_meta.cc:287] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.968s	user 0.005s	sys 0.014s
I20260709 13:03:35.395509  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:35.399971  8404 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
W20260709 13:03:35.387256 13184 consensus_meta.cc:287] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.996s	user 0.012s	sys 0.006s
I20260709 13:03:35.377214 13058 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.404469  8332 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:35.398300 12898 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.363505 13031 consensus_queue.cc:1059] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:03:35.386847 13141 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:35.439122  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:35.443110  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:35.447116  8254 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:03:35.446357  8323 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:35.489739  8220 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:35.408346 13058 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:35.496731  8220 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:35.497628  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 139585us
I20260709 13:03:35.502112  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:35.503412  8249 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:03:35.505573  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:35.505723  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 173154us
I20260709 13:03:35.499172  8222 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:35.444005 13200 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:35.506743  8295 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:35.505290  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:35.506843  8220 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:35.509867  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:03:35.511685  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.512483 13223 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:35.513489  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:35.443990 13141 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:35.444703 12898 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:35.524657  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:35.509270 12932 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:35.527794  8222 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:35.410245 13184 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:35.402751 13023 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:35.539281  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:35.537806  8220 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:35.543385  8295 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:35.533214 13223 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:35.576606  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:35.536180 13273 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:35.584322  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.586663  8402 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:03:35.592288  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2629) took 1547 ms. Trace:
I20260709 13:03:35.593335 13184 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:35.548506 13012 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:35.592731  8400 rpcz_store.cc:276] 0709 13:03:34.043330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.286116 (+242786us) service_pool.cc:224] Handling call
0709 13:03:34.330028 (+ 43912us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:34.330537 (+   509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:34.330569 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:34.331105 (+   536us) log.cc:844] Serialized 26 byte log entry
0709 13:03:34.661750 (+330645us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:34.661795 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.661875 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:34.989923 (+328048us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b480002cc58. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882d6e7aa1 00007f882d6deb37 00007f882d6de9ca 00007f882d7d626f 00007f882d7d5f37 00007f882d7ea55c 00007f882d7ea508 00007f882d7ea488 00007f882d7ea443 00007f882d7e96ec 00007f882c713a96
0709 13:03:35.590067 (+600144us) raft_consensus.cc:1685] finished
0709 13:03:35.590089 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:35.590349 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1242,"raft.run_cpu_time_us":1466,"raft.run_wall_time_us":658346,"spinlock_wait_cycles":30713728,"thread_start_us":330249,"threads_started":1,"wal-append.queue_time_us":784008}
I20260709 13:03:35.594548 13012 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:35.595052 13012 raft_consensus.cc:697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:35.596235 13012 consensus_queue.cc:237] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:35.597811 12932 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:35.598251 12932 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.598559 12932 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:35.600281 13206 consensus_meta.cc:287] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.502s	user 0.003s	sys 0.005s
I20260709 13:03:35.601722  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:35.602900  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:35.603557  8254 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:35.604358  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:35.546293 13141 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:35.611316  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.607955  8324 raft_consensus.cc:1240] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:03:35.552296 13157 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:03:35.585203 13271 log.cc:927] Time spent T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:03:35.622821  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:35.623365  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 80064.6us
W20260709 13:03:35.604869 13156 consensus_meta.cc:287] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.627s	user 0.014s	sys 0.000s
I20260709 13:03:35.540002 13163 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:35.635354  8332 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 13: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:35.633946  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:35.555209 13200 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:35.641040  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:35.605762 13279 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:35.646119  8295 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:35.654358  8325 raft_consensus.cc:2393] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 12: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:35.632218 13264 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:03:35.655638 13088 consensus_meta.cc:287] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.119s	user 0.009s	sys 0.012s
I20260709 13:03:35.687578  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:03:35.697333  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.654917 13142 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:35.700294  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 172576us
I20260709 13:03:35.701121  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:35.719907  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:35.720130  9009 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:03:35.720520  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:35.685164 12607 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.683239 13200 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:35.638633 13157 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:35.722586  8403 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 14. Current term is 15. Ops: [10.9-14.10]
I20260709 13:03:35.685407 13264 raft_consensus.cc:697] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:35.724850  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:35.644984 13163 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:35.724617  8325 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:03:35.733392  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:35.735702 13280 log.cc:927] Time spent T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 13:03:35.737668 13276 log.cc:927] Time spent T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:03:35.680167 13278 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:03:35.741376  8222 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:35.757148  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 219393us
I20260709 13:03:35.759150 13185 consensus_queue.cc:1059] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:35.759912 13269 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.760528 13269 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:35.758103 12974 consensus_queue.cc:1059] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:03:35.765487 13246 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:35.768102 13246 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:35.768857  8326 raft_consensus.cc:1240] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:03:35.770215  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:35.770987  8401 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:35.759434  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:35.773289  9009 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:35.775367  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:35.775967  8332 raft_consensus.cc:2393] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 12: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:35.778645  8295 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:35.780915  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:35.783425  8298 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:35.743904 13200 raft_consensus.cc:697] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:03:35.784173  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 82919.5us
I20260709 13:03:35.788136  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:35.790681  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:35.800854  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:35.801259  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.749763 13012 consensus_queue.cc:1059] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:03:35.728184 12607 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:35.809643 13246 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:35.810184 13246 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.810524 13246 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:35.811126 12670 consensus_meta.cc:287] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.683s	user 0.011s	sys 0.002s
I20260709 13:03:35.811280  8255 raft_consensus.cc:1240] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:03:35.811504 12670 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:35.812362  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:35.812835 13207 consensus_meta.cc:287] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 1.015s	user 0.011s	sys 0.014s
I20260709 13:03:35.802804  8220 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:35.782128 13088 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:35.809798 13059 raft_consensus.cc:3055] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:35.814881  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:35.816684  8295 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:35.805680  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:35.820186  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:35.817282  8220 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:35.820851  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:35.821313  8404 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:35.821738  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:35.821908  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:35.823314  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:35.791396 13278 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:35.825129  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:35.791121 12932 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:35.827131  9009 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:35.827162  8400 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
W20260709 13:03:35.816208 13282 log.cc:927] Time spent T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
W20260709 13:03:35.832266  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 48915.3us
W20260709 13:03:35.832551  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.834220 13200 consensus_queue.cc:237] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:35.837530 13277 raft_consensus.cc:3055] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:35.837978 13277 raft_consensus.cc:740] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:35.838263 12932 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:35.838657 13163 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:35.838939 13277 consensus_queue.cc:260] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:03:35.839526  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 695) took 1182 ms. Trace:
I20260709 13:03:35.823419 13279 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:35.840322 13277 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:35.839991  8326 rpcz_store.cc:276] 0709 13:03:34.657211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.168513 (+511302us) service_pool.cc:224] Handling call
0709 13:03:35.193478 (+ 24965us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:35.769269 (+575791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:35.839259 (+ 69990us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b540029a9d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:35.839448 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1322544512}
I20260709 13:03:35.842576 12607 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:35.839543  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:35.816265 13088 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.844869  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:35.816606 13059 raft_consensus.cc:740] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:35.833280  8298 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:35.824188 13023 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:35.830551 12996 consensus_meta.cc:287] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.736s	user 0.016s	sys 0.008s
I20260709 13:03:35.857509  8332 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:35.859411  8255 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 14: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:35.829959 13012 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:03:35.858934 13009 consensus_meta.cc:287] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.338s	user 0.014s	sys 0.009s
I20260709 13:03:35.825064 12670 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:35.844686 12898 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:35.846103 13088 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:35.859833 13290 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:35.862391  8223 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:35.865000  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:35.866158  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:03:35.863566 13213 consensus_meta.cc:287] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.504s	user 0.012s	sys 0.013s
I20260709 13:03:35.871508 13278 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:35.861207 13226 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:35.873561 13059 consensus_queue.cc:260] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:35.876072  8252 raft_consensus.cc:2410] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:03:35.867787 13223 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:35.898468 13059 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:35.894037 13226 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.899363 13226 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:35.880291 13164 raft_consensus.cc:3055] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:03:35.881953  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 48705.9us
I20260709 13:03:35.840266 13279 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:35.883311  8401 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:35.896873 13274 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:35.898319 13278 raft_consensus.cc:697] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:35.864018 13012 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.905215 13290 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:35.905401 13020 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:35.907526 13278 consensus_queue.cc:237] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:35.864555 12996 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:35.929504 12996 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:35.908372  8298 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:35.902827 13164 raft_consensus.cc:740] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:03:35.921890  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 93834.3us
I20260709 13:03:35.905793 13279 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:35.927497 13290 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:35.830207 13264 consensus_queue.cc:237] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:35.932883  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:35.951668  8401 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:35.933986  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 144844us
I20260709 13:03:35.939107  8298 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:35.914345 13012 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:35.941419  8220 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:35.900457 13213 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:35.957250  8220 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:35.962132  8298 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:35.962354  8972 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:35.940434 12898 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:35.962383 12670 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:35.962532 13290 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:03:35.964346  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 102013us
I20260709 13:03:35.970448  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:03:35.975136  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.964473  8298 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:35.971788  8401 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:35.980958  8222 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:35.982347  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:35.983175  8972 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:35.984306  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:35.963686 12973 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:35.983960 13296 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:35.963863 13185 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:35.921984 13274 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:35.970888 12670 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:35.969626 12607 consensus_queue.cc:1059] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:03:35.971221 13290 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.013087 13269 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:35.996505 13020 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.985858 13157 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:36.018797  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:35.964629 13184 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:03:36.002198  8298 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:03:36.022686  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 48392.9us
I20260709 13:03:36.027920  8223 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:36.016726 12670 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:36.024730  8401 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:35.971944 13299 consensus_queue.cc:1059] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:03:35.997579 12973 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.012405 12996 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.992491 13223 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:35.992233 12898 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:36.030161 12607 raft_consensus.cc:3055] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:35.997722 13141 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:36.033547 13213 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:36.090884  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:03:36.039736  8372 connection.cc:256] RPC call timeout handler was delayed by -1.0879442299999482s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29710000000000003s for a response
I20260709 13:03:36.091826  9003 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:36.033996 13302 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.021111 13290 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:36.034736  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:35.954571 13164 consensus_queue.cc:260] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:36.037935  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:36.040733  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:36.102546  8398 raft_consensus.cc:2410] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:03:35.996641 13296 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:03:36.095281  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 93175us
I20260709 13:03:36.027036 13020 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:36.044299 12607 raft_consensus.cc:740] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:36.106212 12607 consensus_queue.cc:260] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:36.107123 13020 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:36.045692 13223 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:36.090503 13185 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:36.042562 13009 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:36.102427  9005 raft_consensus.cc:2376] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13. Re-sending same reply.
I20260709 13:03:36.043687 12973 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:36.104452  8249 raft_consensus.cc:2393] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 11: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:36.102773  9009 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:36.110487  9001 raft_consensus.cc:3055] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:36.107564 12607 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:36.109977  8298 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:36.113037  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 132158us
I20260709 13:03:36.094540  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:36.119210  8330 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:36.118227  9001 raft_consensus.cc:740] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:36.119635  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:36.115739 13185 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:36.125717 13185 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:36.126014  9001 consensus_queue.cc:260] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:36.127303  9001 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:36.129937  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:36.115154 13223 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:36.119217  8295 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.510s (SENT) [suppressed 13 similar messages]
I20260709 13:03:36.154386  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:36.158021  8401 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:36.159950  9003 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:36.162293  8372 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:36.109970 13296 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:36.167857  9006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 961) took 1780 ms. Trace:
I20260709 13:03:36.107766 12996 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:36.169370  9006 rpcz_store.cc:276] 0709 13:03:34.377037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.416274 (+ 39237us) service_pool.cc:224] Handling call
0709 13:03:34.416583 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:35.175533 (+758950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.157031 (+981498us) spinlock_profiling.cc:243] Waited 758 ms on lock 0x7b540069ebd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:36.157219 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1743756160}
I20260709 13:03:36.132016 13164 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:36.101975 13302 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:36.172729  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:03:36.173549  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 63662.4us
I20260709 13:03:36.188406  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:36.188954  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:36.189085  9003 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
W20260709 13:03:36.189903  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 162040us
I20260709 13:03:36.190905  8223 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:36.191188  8398 raft_consensus.cc:2368] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14.
I20260709 13:03:36.162485 13196 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:36.123512 13157 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:03:36.193939  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:36.194305  8404 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:36.168922 13163 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:36.171921 13213 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.197456 13213 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:36.170691 13184 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:36.201938  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:36.202801  8298 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:36.196049  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:36.204873  9005 raft_consensus.cc:2410] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:03:36.205638  8372 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:36.208460  9007 raft_consensus.cc:2368] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14.
I20260709 13:03:36.170652 13309 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:36.209295  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:36.181581 13311 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:36.213841 13311 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:36.215636  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 690) took 1618 ms. Trace:
W20260709 13:03:36.216017 13262 consensus_meta.cc:287] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.867s	user 0.010s	sys 0.010s
I20260709 13:03:36.211935 13023 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 13:03:36.216483 13262 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:36.220738  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:36.216055  8331 rpcz_store.cc:276] 0709 13:03:34.597190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.098553 (+501363us) service_pool.cc:224] Handling call
0709 13:03:35.259570 (+161017us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:35.260146 (+   576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.215416 (+955270us) spinlock_profiling.cc:243] Waited 97.3 ms on lock 0x7b540037ad50. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:36.215572 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":223791872}
I20260709 13:03:36.220279  9009 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:36.192674 13020 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:36.230647  8398 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:36.225289  8372 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:36.170095 13001 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
W20260709 13:03:36.238888  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.239352  8372 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:36.236665 13157 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:36.236764 13274 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:36.228539 13298 log.cc:927] Time spent T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 13:03:36.247313 13157 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:36.247936 13029 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:36.201285 12670 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:36.254464  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:03:36.254623 13301 log.cc:927] Time spent T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc: Append to log took a long time: real 0.235s	user 0.002s	sys 0.000s
I20260709 13:03:36.254118 13223 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:36.254976  8223 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:36.253119 13009 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:36.255744  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 270023us
W20260709 13:03:36.230355 12958 consensus_meta.cc:287] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.135s	user 0.012s	sys 0.007s
I20260709 13:03:36.221935 13184 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:36.230509 13207 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:36.267120  8298 leader_election.cc:400] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14.
I20260709 13:03:36.275094  8298 leader_election.cc:403] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:03:36.234915 13269 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:36.277966  8372 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:36.266144 13157 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:36.282104  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:36.266175 12898 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:36.286201  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:36.303949  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 60486.2us
I20260709 13:03:36.274065 13309 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:36.307336  8398 raft_consensus.cc:2393] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 13: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:36.275117 13184 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:36.303702 13001 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:03:36.327339  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 53979.1us
I20260709 13:03:36.328269  8372 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:36.332568 13278 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:36.277230 12958 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:36.340193  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:36.341216  8398 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:36.273769 13246 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:36.343856  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.317898 13184 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:36.339263  8372 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:36.346567 13207 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:36.347215 13207 raft_consensus.cc:697] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:36.351922  9001 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:03:36.352393  9001 pending_rounds.cc:85] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: Aborting all ops after (but not including) 9
I20260709 13:03:36.352665  9001 pending_rounds.cc:107] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:03:36.352873  9001 raft_consensus.cc:2889] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:36.302989 13269 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:36.329741 13262 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:36.361456  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:36.363354  8398 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:36.362221  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 56849.9us
I20260709 13:03:36.313830 12898 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:36.364226  8223 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:36.350690 13277 consensus_meta.cc:287] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.510s	user 0.011s	sys 0.003s
I20260709 13:03:36.369289  9000 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:36.352228 13163 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:03:36.365758 13299 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:03:36.370473 12868 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:36.369925 12996 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:36.398686 12958 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:36.374089 13163 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.374519 13207 consensus_queue.cc:237] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:36.393316 12898 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:36.377121 13309 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:36.349224 13296 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:36.428776 13296 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:36.407995  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 80066.1us
W20260709 13:03:36.410636  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 46437.9us
I20260709 13:03:36.379124 13029 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.036s
W20260709 13:03:36.413908 13059 consensus_meta.cc:287] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.515s	user 0.011s	sys 0.001s
I20260709 13:03:36.405510 13319 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:36.388372 13274 consensus_queue.cc:1059] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 13:03:36.417655 13223 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:36.418296 12670 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:36.408331 13163 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:36.438582 13312 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.482996 13312 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:36.484879 13279 consensus_meta.cc:287] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.526s	user 0.006s	sys 0.004s
I20260709 13:03:36.485258 13312 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:36.436533 13029 consensus_queue.cc:1243] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:03:36.439670 13261 consensus_meta.cc:287] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.119s	user 0.013s	sys 0.003s
I20260709 13:03:36.444322 13185 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:36.445559  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.445984 13012 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:36.446892 13246 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:36.521149  8222 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:36.464756  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:36.464857  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:36.522125  9001 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:36.465222  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:03:36.470458 13058 consensus_meta.cc:287] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.945s	user 0.015s	sys 0.004s
I20260709 13:03:36.474185 13196 raft_consensus.cc:3055] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:36.541589 13196 raft_consensus.cc:740] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:36.495999 13157 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:36.541960 13137 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:36.543026  8404 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:36.475217 13184 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:36.543895 13157 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:36.545397 13184 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:36.476130  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:36.546409  8251 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 14: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:36.546469  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:36.547636  8251 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:36.477157 12973 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:36.478241 13088 consensus_meta.cc:287] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.575s	user 0.012s	sys 0.000s
I20260709 13:03:36.549832 13088 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:36.478892  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:36.480581 13226 consensus_meta.cc:287] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.581s	user 0.003s	sys 0.007s
I20260709 13:03:36.486207  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:36.486928  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:36.554299  8406 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:36.487970  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:36.506340  9004 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:36.485827  9006 raft_consensus.cc:2410] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:03:36.489665  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 732) took 1104 ms. Trace:
I20260709 13:03:36.516675  8398 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:36.555224  9001 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
W20260709 13:03:36.499673 13303 log.cc:927] Time spent T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.398s	user 0.001s	sys 0.000s
I20260709 13:03:36.447376 13269 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:36.561291 12973 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:36.561781 13012 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:36.523247  8222 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:36.521215 13310 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:36.538672  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:36.540232 12849 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:36.485400 13279 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:36.576642 13279 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:36.577083 13213 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:36.474223 13319 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:36.552800  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:36.558460  8397 raft_consensus.cc:2393] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:36.555258  8403 rpcz_store.cc:276] 0709 13:03:35.385203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.385371 (+   168us) service_pool.cc:224] Handling call
0709 13:03:35.401967 (+ 16596us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:35.731705 (+329738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.489386 (+757681us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b5400267fd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:36.489609 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":736520960}
I20260709 13:03:36.557340 13226 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:36.582616  8330 raft_consensus.cc:1240] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:36.583467  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:36.583938  8326 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 15. Current term is 16. Ops: []
W20260709 13:03:36.584223  8248 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2127) took 1598 ms (client timeout 1907 ms). Trace:
I20260709 13:03:36.542564 13196 consensus_queue.cc:260] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:36.521667 13261 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:36.568171 13185 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:36.554013 13058 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:36.569566 13310 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:36.571658 13302 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:36.572819  8406 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:03:36.573825  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:36.495244 13290 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:36.578409  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:36.579138  9001 pending_rounds.cc:85] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: Aborting all ops after (but not including) 8
I20260709 13:03:36.595438  9001 pending_rounds.cc:107] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:03:36.595834  9001 raft_consensus.cc:2889] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:36.581115 13319 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:36.563730  8222 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:36.578439  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2307) took 2135 ms. Trace:
W20260709 13:03:36.598059  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.598408  8222 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:36.583506  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2581) took 1146 ms. Trace:
W20260709 13:03:36.584649  8248 rpcz_store.cc:269] 0709 13:03:34.985773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.025202 (+ 39429us) service_pool.cc:224] Handling call
0709 13:03:36.583313 (+1558111us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b5800002a98. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:03:36.584170 (+   857us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":304144128}
I20260709 13:03:36.589107  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:36.599777  8222 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:36.585153 13012 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:36.603953  8257 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:36.598269  8324 rpcz_store.cc:276] 0709 13:03:34.443492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.449536 (+  6044us) service_pool.cc:224] Handling call
0709 13:03:34.449871 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:35.616781 (+1166910us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.578161 (+961380us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b540020ec54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:36.578366 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2662394752}
I20260709 13:03:36.593619  8251 raft_consensus.cc:2393] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 14: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:36.594988  8250 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
W20260709 13:03:36.604831  8250 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2128) took 1618 ms (client timeout 1510 ms). Trace:
I20260709 13:03:36.602308  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2577) took 1791 ms. Trace:
I20260709 13:03:36.599357  8323 rpcz_store.cc:276] 0709 13:03:35.447734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.491166 (+ 43432us) service_pool.cc:224] Handling call
0709 13:03:35.510491 (+ 19325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:35.578136 (+ 67645us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:35.578181 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:35.578238 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:35.578280 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.583302 (+1005022us) spinlock_profiling.cc:243] Waited 36 ms on lock 0x7b5400364554. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:36.583348 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:36.583456 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":82881792}
I20260709 13:03:36.606019  8325 raft_consensus.cc:1240] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:03:36.602798  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:36.597659  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:36.606307  8222 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:36.601207  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:36.605196  8250 rpcz_store.cc:269] 0709 13:03:34.986189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.993543 (+  7354us) service_pool.cc:224] Handling call
0709 13:03:36.578265 (+1584722us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b5800002a98. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:03:36.604767 (+ 26502us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":377269120}
I20260709 13:03:36.607693 13309 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:36.608354 13309 raft_consensus.cc:697] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:36.608520 13137 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:36.609499 13309 consensus_queue.cc:237] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:36.607257  8332 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:03:36.614678  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:36.616178  8222 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:36.621956  8253 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:03:36.622776 13302 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:36.625370  8222 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:36.626799  8222 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:36.617194  8328 rpcz_store.cc:276] 0709 13:03:34.811405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.229528 (+418123us) service_pool.cc:224] Handling call
0709 13:03:35.229899 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:35.230260 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:35.230560 (+   300us) log.cc:844] Serialized 14 byte log entry
0709 13:03:36.404947 (+1174387us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b1800446c80. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e23a 00007f88287c8a2a
0709 13:03:36.594425 (+189478us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:36.594485 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:36.602016 (+  7531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.602105 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:36.602246 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":294,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":4558,"raft.queue_time_us":16799,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":11699,"spinlock_wait_cycles":6235904,"thread_start_us":179709,"threads_started":1}
I20260709 13:03:36.628084  8222 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:36.635041  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:36.606730 13012 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:36.605006 12973 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:36.638259  8324 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:36.633184 12868 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:36.622694 13290 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:36.646086  8251 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:03:36.629844 13302 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:36.634123 13196 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:36.622699 13260 consensus_queue.cc:1059] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:03:36.637199 13262 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:36.655213  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:36.636749 13278 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14.
W20260709 13:03:36.644661 13289 log.cc:927] Time spent T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.845s	user 0.001s	sys 0.001s
I20260709 13:03:36.651496  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:36.640149 13012 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:36.657219  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 65662.3us
I20260709 13:03:36.658180  8251 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:36.670614  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:36.641400 12868 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.642534 13261 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:36.675971  8251 raft_consensus.cc:2368] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14.
I20260709 13:03:36.687951  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:36.692407  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:36.725816  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:36.675791 12868 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:36.726599  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:36.674398 13088 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:36.691406 13278 consensus_queue.cc:1059] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:03:36.691192 13157 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:36.733047 13268 log.cc:927] Time spent T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 1.398s	user 0.004s	sys 0.001s
I20260709 13:03:36.733407 13304 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:36.737474  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:36.699765  8295 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:36.734195 13310 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:36.677191 13262 raft_consensus.cc:697] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:36.693691 13320 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:36.768107 13226 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:36.768472 13328 consensus_queue.cc:1059] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:03:36.785408  8251 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 17: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:36.785414  8254 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 15. Current term is 16. Ops: [15.10-15.10]
W20260709 13:03:36.786634  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 86959.8us
I20260709 13:03:36.789636  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:36.795276  9001 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 17: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:36.792536  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:36.753085 13260 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.796731  8295 leader_election.cc:400] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14.
I20260709 13:03:36.801115  8295 leader_election.cc:403] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:03:36.799417  8330 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:36.802129 13029 consensus_queue.cc:1059] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:36.803493 13207 raft_consensus.cc:3055] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:36.803867 13207 raft_consensus.cc:740] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:36.804724 13207 consensus_queue.cc:260] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:36.800292 13278 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:36.806824 13278 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:36.808766 13278 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:36.808570 13163 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:36.790047 13319 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:36.787109 13157 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:36.809589 13319 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:36.810024 13319 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:36.800493  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:36.814400  8295 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:36.816296  9000 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:36.816605  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:36.816821  8256 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:03:36.817219  9001 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:36.741415 13156 consensus_queue.cc:1059] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:03:36.820914 13311 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:36.824262  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 2179) took 1064 ms. Trace:
I20260709 13:03:36.824776  8255 rpcz_store.cc:276] 0709 13:03:35.749976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.813230 (+ 63254us) service_pool.cc:224] Handling call
0709 13:03:36.813975 (+1000745us) spinlock_profiling.cc:243] Waited 28.3 ms on lock 0x7b540026be54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:36.814103 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65188608}
I20260709 13:03:36.793233 13264 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:36.823346  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:36.820942 13316 log.cc:927] Time spent T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.513s	user 0.003s	sys 0.000s
I20260709 13:03:36.834227  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:36.822516 13278 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:36.837199  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:36.822603 13328 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:36.826377 13058 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:36.843027  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:36.813031 13260 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:36.832156 13156 raft_consensus.cc:3055] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:36.856550  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:36.857174  9002 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:36.857983  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:36.858446  9002 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:36.859571  9001 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:03:36.860394  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:36.860864  9001 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:36.866044  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:36.861611  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 75280.6us
W20260709 13:03:36.876812  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:36.877794  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.917332  9001 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
W20260709 13:03:36.945185  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.836618 13264 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:37.083477  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.089624  9001 raft_consensus.cc:1240] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:37.107152  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2649) took 1302 ms. Trace:
I20260709 13:03:37.108542  9002 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:37.107611  8401 rpcz_store.cc:276] 0709 13:03:35.804690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.169003 (+364313us) service_pool.cc:224] Handling call
0709 13:03:36.185535 (+ 16532us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.206990 (+ 21455us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:36.207036 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:36.228892 (+ 21856us) log.cc:844] Serialized 26 byte log entry
0709 13:03:36.301581 (+ 72689us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:36.301622 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.301704 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:36.820298 (+518594us) spinlock_profiling.cc:243] Waited 2.49 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:37.092680 (+272382us) raft_consensus.cc:1685] finished
0709 13:03:37.092711 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:37.107080 (+ 14369us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3713,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"spinlock_wait_cycles":5720320,"thread_start_us":47516,"threads_started":1,"wal-append.queue_time_us":53225}
I20260709 13:03:37.078845 13306 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.086439 13299 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.854002 13156 raft_consensus.cc:740] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:36.841125 13262 consensus_queue.cc:237] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:37.093863 12670 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:37.080327 13157 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:37.137905  8252 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 2157) took 1958 ms (client timeout 1885 ms). Trace:
I20260709 13:03:37.140020  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:37.143340  8252 rpcz_store.cc:269] 0709 13:03:35.179736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.192172 (+ 12436us) service_pool.cc:224] Handling call
0709 13:03:37.123001 (+1930829us) spinlock_profiling.cc:243] Waited 651 ms on lock 0x7b540034b7d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:37.137830 (+ 14829us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1498263296}
I20260709 13:03:37.144295  8248 raft_consensus.cc:2368] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15.
I20260709 13:03:37.151609  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:37.151875  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:37.152482  8252 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:37.141322 12868 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:37.142371 13299 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:37.146404 13157 consensus_queue.cc:1059] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 13:03:37.162153  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:37.162832  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:37.153421 13240 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:37.147717 13306 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:37.168074 13260 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:37.172003 13302 raft_consensus.cc:3055] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:37.173774 13207 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:37.148638 12670 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:37.178061  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 697) took 1479 ms. Trace:
I20260709 13:03:37.148540 13156 consensus_queue.cc:260] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:37.182358  8325 rpcz_store.cc:276] 0709 13:03:35.698161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.734262 (+ 36101us) service_pool.cc:224] Handling call
0709 13:03:35.734570 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.606425 (+871855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.177754 (+571329us) spinlock_profiling.cc:243] Waited 871 ms on lock 0x7b54003cad54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:37.177993 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2003446016}
I20260709 13:03:37.179962  8248 raft_consensus.cc:1275] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:03:37.185235  8248 pending_rounds.cc:85] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 8
I20260709 13:03:37.185787  8248 pending_rounds.cc:107] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:03:37.186290  8248 raft_consensus.cc:2889] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:37.188077 13196 consensus_meta.cc:287] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.530s	user 0.011s	sys 0.000s
I20260709 13:03:37.188944  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:37.189791  8252 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:37.179745 13302 raft_consensus.cc:740] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:37.182641 13260 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:37.184692 13157 raft_consensus.cc:3055] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:37.182991 13274 consensus_queue.cc:1059] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:03:37.194427 13299 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:37.197764 13156 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:37.204068 12670 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:37.198144 13157 raft_consensus.cc:740] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:37.193948 13240 consensus_queue.cc:1059] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
W20260709 13:03:37.203898 13338 log.cc:927] Time spent T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:03:37.202467 13260 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:37.231529 13305 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 1.069s	user 0.012s	sys 0.028s
I20260709 13:03:37.233070 13260 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:37.234799  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:37.235142  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:37.235759  8328 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:37.215762 13302 consensus_queue.cc:260] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:37.237776 13302 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:37.227080 13157 consensus_queue.cc:260] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:37.223629 13001 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:37.238422 13001 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:37.239440  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:37.242156 13157 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:37.242283  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:37.243069  8328 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:37.244130  8406 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:37.245112  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:37.245798  8250 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:37.245754  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:03:37.246956  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.245167  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:03:37.247473  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.251426  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:37.251507  8972 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:37.252393 13314 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:37.252697  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:37.253158 12868 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:37.253990 13306 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:37.254653  8222 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:37.255931 13226 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:37.255955  8298 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:37.256559 13226 raft_consensus.cc:697] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:37.257056 13251 raft_consensus.cc:3055] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:37.257480 13226 consensus_queue.cc:237] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:37.260635  8972 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:37.250720  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:37.253301  8400 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:37.217993  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 132087us
I20260709 13:03:37.252784 13314 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:37.268112  8298 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:37.268286 13314 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:37.268316 12670 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:37.269594 13270 raft_consensus.cc:3055] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:37.270097 13270 raft_consensus.cc:740] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:37.270985 13270 consensus_queue.cc:260] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:37.271939 13270 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:37.276140 13343 log.cc:927] Time spent T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.444s	user 0.001s	sys 0.000s
I20260709 13:03:37.282248  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:37.284718  8401 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:37.262558  8369 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:37.257498 13251 raft_consensus.cc:740] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:37.267714  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:37.292341  8402 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
W20260709 13:03:37.267918  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.286971  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:37.292869  8250 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:03:37.293273  8401 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:37.295421  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:37.296969  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:37.299283  8250 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:03:37.300797  8297 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:37.299327  8222 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:37.301218  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:37.293880  8369 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:37.297168  8400 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:03:37.303121  8401 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:03:37.269546 13185 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:03:37.284785 13338 log.cc:927] Time spent T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:03:37.310797  8222 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:37.312533  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:37.304328  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:37.313177  8248 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 10.
I20260709 13:03:37.313752  8400 raft_consensus.cc:2410] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:03:37.306068  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:37.298352 13012 consensus_meta.cc:287] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.614s	user 0.005s	sys 0.005s
I20260709 13:03:37.285065 13240 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:37.300266 12996 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:37.322265  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:37.291908 12670 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.311554 12849 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.312220 13185 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.309906 13196 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:37.324126  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:37.354144  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:37.340058  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:37.333846  8402 raft_consensus.cc:2368] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15.
I20260709 13:03:37.311833 13262 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:37.351538  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:37.320708 13240 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.292517 13001 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:37.319696 12973 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:37.365031  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:37.374881  9001 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:37.374794  8974 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:37.377609  8405 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 17: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:37.378417  8974 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:37.361981 13346 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:37.379654 13346 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:37.379760 13275 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:37.380169 13346 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:37.380286 13275 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:37.380811 13275 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:37.327428 12996 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:37.382084 12996 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:37.394037  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:37.394948  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:37.395627  9003 raft_consensus.cc:2368] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15.
I20260709 13:03:37.320878 13012 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:37.361218 13196 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.401825  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:37.370396 13185 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:37.363190 13299 raft_consensus.cc:3055] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:37.455538  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:37.459057  8401 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:37.377019 12974 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:37.409754  9008 raft_consensus.cc:2368] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14.
I20260709 13:03:37.456033  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:37.378525 13240 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:37.349818 13133 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.384838 13156 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:37.463898  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:37.526785  8405 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:37.464077 13141 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:37.355063 13251 consensus_queue.cc:260] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:37.354907 12849 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:37.465287 13344 raft_consensus.cc:3055] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:03:37.467723 13350 log.cc:927] Time spent T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:03:37.477571 12974 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:37.559810  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:37.400882 12973 raft_consensus.cc:697] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:37.480374 13012 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:37.392904 13262 raft_consensus.cc:697] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:37.490669  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:37.492399  8251 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:37.494988  8256 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
W20260709 13:03:37.495869  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.403240 13264 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:37.500615 13275 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:37.504707 12996 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:37.507730  8401 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
W20260709 13:03:37.508527  9006 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56954 (request call id 843) took 1320 ms (client timeout 1725 ms). Trace:
I20260709 13:03:37.511186  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2130) took 1058 ms. Trace:
I20260709 13:03:37.334182 12670 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:37.521226  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:37.605144  9007 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:37.519850  8972 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:03:37.606355  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 86627.6us
W20260709 13:03:37.573726 13278 consensus_meta.cc:287] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.732s	user 0.008s	sys 0.003s
I20260709 13:03:37.560468  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:37.602236 13307 log.cc:927] Time spent T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 1.415s	user 0.002s	sys 0.004s
I20260709 13:03:37.563038  9000 raft_consensus.cc:2393] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 13: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
W20260709 13:03:37.566157  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 187774us
I20260709 13:03:37.568619  8295 leader_election.cc:400] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15.
I20260709 13:03:37.616130  8974 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:37.616541  8295 leader_election.cc:403] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 13:03:37.570338  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 269569us
W20260709 13:03:37.571204  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.572889  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:37.573957  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2332) took 1218 ms. Trace:
I20260709 13:03:37.570438  8406 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
W20260709 13:03:37.577802  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.619287  8222 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:37.582012 12973 consensus_queue.cc:237] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:37.553299 13133 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:37.588270 13262 consensus_queue.cc:237] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:37.589740 13315 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:37.590217 13001 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:37.581341 12974 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:37.552358 13344 raft_consensus.cc:740] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:03:37.592175 13319 consensus_meta.cc:287] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.782s	user 0.009s	sys 0.002s
I20260709 13:03:37.462879 13296 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:37.598768  8398 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:37.637523  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:37.638096  8398 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:37.601263  8405 raft_consensus.cc:2410] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:03:37.551994 13270 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:37.602620  8403 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
W20260709 13:03:37.603295  9006 rpcz_store.cc:269] 0709 13:03:36.187612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.192593 (+  4981us) service_pool.cc:224] Handling call
0709 13:03:37.508257 (+1315664us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b54005695d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:37.508434 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":629263104}
I20260709 13:03:37.604218  8257 rpcz_store.cc:276] 0709 13:03:36.452738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.538403 (+ 85665us) service_pool.cc:224] Handling call
0709 13:03:37.510985 (+972582us) spinlock_profiling.cc:243] Waited 38.6 ms on lock 0x7b5400309654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:37.511112 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88748416}
W20260709 13:03:37.607389  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.555959 13356 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:37.572528 13251 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:37.621189 13361 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:37.459995 13196 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:37.460357 13299 raft_consensus.cc:740] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:03:37.621701  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.624406  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:37.657500  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:37.620455  8297 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:37.618330  8329 rpcz_store.cc:276] 0709 13:03:36.355132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.355266 (+   134us) service_pool.cc:224] Handling call
0709 13:03:36.361120 (+  5854us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:36.361327 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:36.374522 (+ 13195us) log.cc:844] Serialized 14 byte log entry
0709 13:03:37.176178 (+801656us) spinlock_profiling.cc:243] Waited 1.1 ms on lock 0x7b1800062260. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e23a 00007f88287c8a2a
0709 13:03:37.573613 (+397435us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:37.573678 (+    65us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:37.573729 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.573796 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:37.573902 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11976,"prepare.queue_time_us":9464,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":3444,"raft.queue_time_us":169957,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":18928,"spinlock_wait_cycles":2954752,"thread_start_us":381100,"threads_started":2,"wal-append.queue_time_us":979715}
W20260709 13:03:37.608688 13347 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 13:03:37.621658  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:37.658253  8222 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:03:37.636018  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 296019us
I20260709 13:03:37.635426 13260 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:37.639233  8398 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:37.643848  8405 raft_consensus.cc:1275] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:37.644323  8402 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:03:37.672649  8402 pending_rounds.cc:85] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 9
I20260709 13:03:37.672920  8402 pending_rounds.cc:107] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:03:37.673143  8402 raft_consensus.cc:2889] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:37.615648 13278 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14.
W20260709 13:03:37.573545 13164 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.006s	sys 0.025s
I20260709 13:03:37.613722 13362 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:37.657594  8401 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 14. Current term is 15. Ops: []
W20260709 13:03:37.625322  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 56897.9us
I20260709 13:03:37.647665 13061 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:37.675091  8405 raft_consensus.cc:1240] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:37.627950 13355 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:37.674070  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:37.677402  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.628930 13364 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:37.684203  8295 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:37.680097 13302 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:37.687916  8972 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:37.679827 13164 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.007s	sys 0.025s
W20260709 13:03:37.693650  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 76413.2us
I20260709 13:03:37.627082 13328 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:03:37.675284 13360 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:37.639321 13319 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:37.694603  8974 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:37.685405 13185 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:37.701386  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.693051 12989 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:37.687222 13260 raft_consensus.cc:697] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:37.655285 13270 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:37.640534 13315 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:37.690948 13200 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:37.689860 13001 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:37.737362  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.696846 13302 raft_consensus.cc:697] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:37.710510 13328 consensus_queue.cc:1048] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:37.695844 13360 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:37.749040  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 61171.2us
I20260709 13:03:37.697803 13362 raft_consensus.cc:697] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 14 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:37.712388 13319 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:37.712723 13369 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:03:37.713689 13339 log.cc:927] Time spent T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.664s	user 0.009s	sys 0.012s
I20260709 13:03:37.714789 13185 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:37.700482 13012 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:03:37.719170 13061 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:37.719769 12989 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:37.720351  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 2171) took 2217 ms. Trace:
W20260709 13:03:37.722450  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:37.730798 13336 log.cc:927] Time spent T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.920s	user 0.000s	sys 0.002s
I20260709 13:03:37.732627  9009 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:37.735035  9002 raft_consensus.cc:1240] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:37.707254  8369 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:37.697958 13344 consensus_queue.cc:260] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:37.734850  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:37.653405 13356 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:37.747334  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 52850.4us
I20260709 13:03:37.711534 13133 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:37.753257  9008 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:03:37.756723  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 97981.3us
I20260709 13:03:37.699898 13157 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:03:37.738440 13200 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:03:37.761809  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 72272.7us
I20260709 13:03:37.759277 13369 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.779753 13365 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.751650 12974 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:37.771219 12670 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:37.868937  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 69878.4us
I20260709 13:03:37.896662  8250 raft_consensus.cc:1240] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:37.899209  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 739) took 1447 ms. Trace:
W20260709 13:03:37.881873 13359 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 13:03:37.777710  8254 rpcz_store.cc:276] 0709 13:03:35.502616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.605237 (+102621us) service_pool.cc:224] Handling call
0709 13:03:35.618786 (+ 13549us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.794823 (+1176037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.720110 (+925287us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400346cd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:37.720292 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2682163584}
I20260709 13:03:37.871093  8297 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:03:37.871093  9004 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56954 (request call id 847) took 1682 ms (client timeout 2025 ms). Trace:
I20260709 13:03:37.881307 13200 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:37.887420 13339 log.cc:927] Time spent T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:03:37.699056 13355 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:37.747500 13273 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:03:37.747745 13001 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:37.758628 12849 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:37.767122  8405 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:37.756389 13261 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.118s
W20260709 13:03:37.776324 13314 consensus_meta.cc:287] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.507s	user 0.007s	sys 0.000s
W20260709 13:03:37.767726 13358 log.cc:927] Time spent T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:03:37.762434 13009 consensus_queue.cc:1059] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.120s
W20260709 13:03:37.892557  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 110957us
W20260709 13:03:37.893386  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 129851us
I20260709 13:03:37.934141  9005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 977) took 1580 ms. Trace:
I20260709 13:03:37.763368 13061 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:37.899827  8253 rpcz_store.cc:276] 0709 13:03:36.451553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.462928 (+ 11375us) service_pool.cc:224] Handling call
0709 13:03:36.463402 (+   474us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.622386 (+158984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.896137 (+1273751us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b5400177d54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:37.899127 (+  2990us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":364067584}
I20260709 13:03:37.753769 13360 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:37.885814 13346 consensus_meta.cc:287] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.002s
W20260709 13:03:37.904757  9004 rpcz_store.cc:269] 0709 13:03:36.188874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.192132 (+  3258us) service_pool.cc:224] Handling call
0709 13:03:37.870915 (+1678783us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b540054c9d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:37.871022 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":596509952}
I20260709 13:03:37.880824 13344 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:37.768011 13260 consensus_queue.cc:237] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:37.910087 13371 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:37.777247 13270 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:37.912441  8297 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:37.763502 13367 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:37.896739 12974 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.755781 13299 consensus_queue.cc:260] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:37.870599 13156 consensus_queue.cc:1059] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.105s
W20260709 13:03:37.928432  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.895769 13365 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:37.896821 13369 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:37.895676  8249 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 2175) took 2286 ms (client timeout 2137 ms). Trace:
I20260709 13:03:37.772773 13240 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:37.773126 13275 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:37.769892 13302 consensus_queue.cc:237] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:37.916049 13164 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:37.770392 12996 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:37.948660  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2657) took 1715 ms. Trace:
I20260709 13:03:37.770922 13362 consensus_queue.cc:237] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:37.911115 12670 consensus_queue.cc:1059] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:03:37.936558 13314 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:37.934581  9005 rpcz_store.cc:276] 0709 13:03:36.353311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.353654 (+   343us) service_pool.cc:224] Handling call
0709 13:03:36.366539 (+ 12885us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.616598 (+250059us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:36.616646 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:36.634948 (+ 18302us) log.cc:844] Serialized 26 byte log entry
0709 13:03:36.693981 (+ 59033us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:36.694032 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.694106 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:37.730202 (+1036096us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:37.923617 (+193415us) raft_consensus.cc:1685] finished
0709 13:03:37.923652 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:37.934075 (+ 10423us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11636,"raft.queue_time_us":48575,"raft.run_cpu_time_us":47,"raft.run_wall_time_us":47,"spinlock_wait_cycles":5259648,"thread_start_us":269443,"threads_started":2,"wal-append.queue_time_us":171893}
I20260709 13:03:37.967975  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2331) took 1613 ms. Trace:
W20260709 13:03:37.969992  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 262770us
I20260709 13:03:37.876227 12989 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
W20260709 13:03:37.978930  8249 rpcz_store.cc:269] 0709 13:03:35.609185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.610713 (+  1528us) service_pool.cc:224] Handling call
0709 13:03:37.895432 (+2284719us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b540033ed54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:37.895606 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1070710528}
I20260709 13:03:37.983306  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:38.001847  8253 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:37.971868 13367 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:37.985998  8223 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:37.946920 13346 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:38.007403  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:37.991803  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:37.992472  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 48907.8us
W20260709 13:03:37.965263  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 53279.5us
I20260709 13:03:37.970468 13270 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:37.990615  8404 rpcz_store.cc:276] 0709 13:03:36.233072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.283311 (+ 50239us) service_pool.cc:224] Handling call
0709 13:03:36.308994 (+ 25683us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:36.543397 (+234403us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.948307 (+1404910us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54001cc354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:37.948579 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":537729408}
I20260709 13:03:37.997874  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:38.010551  8297 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:37.987339  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:38.001557  8369 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:38.006400  8223 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:37.999692  8331 rpcz_store.cc:276] 0709 13:03:36.354755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.358238 (+  3483us) service_pool.cc:224] Handling call
0709 13:03:36.368216 (+  9978us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.636399 (+268183us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:36.649634 (+ 13235us) log.cc:844] Serialized 14 byte log entry
0709 13:03:36.750238 (+100604us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:36.750987 (+   749us) log.cc:844] Serialized 26 byte log entry
0709 13:03:36.751125 (+   138us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:36.751156 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.751222 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:37.713360 (+962138us) spinlock_profiling.cc:243] Waited 4.68 ms on lock 0x7b240012d6e8. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255579c4 00007f8825555acb 00007f8825710dc9 00007f8825710a55 00007f882571009d 00007f882834210e 00007f882835988c 00007f8828358bb6 00007f88283587d8 00007f8828362221
0709 13:03:37.964411 (+251051us) raft_consensus.cc:1685] finished
0709 13:03:37.967755 (+  3344us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:37.967917 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13808,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":62522,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":7869,"spinlock_wait_cycles":10767232,"thread_start_us":312202,"threads_started":2,"wal-append.queue_time_us":167220}
I20260709 13:03:37.975448 12974 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:37.991343 13308 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.013569  8297 leader_election.cc:400] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14.
I20260709 13:03:38.013715  8254 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 14: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:37.991036 13306 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.990217 12932 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:38.041390  8369 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:38.045847  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:38.012398 13317 log.cc:927] Time spent T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 1.701s	user 0.012s	sys 0.005s
I20260709 13:03:38.072255  8297 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:38.074151  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:38.012141 13261 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:37.993788 13157 consensus_queue.cc:1059] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
W20260709 13:03:38.039985 13373 log.cc:927] Time spent T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:03:38.007054 13367 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:38.062206 13366 log.cc:927] Time spent T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.261s	user 0.000s	sys 0.001s
I20260709 13:03:38.090844 13365 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:38.092532 13346 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:38.068239 13306 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:03:38.097586 13379 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.081s
I20260709 13:03:38.098407 13315 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:38.100076 13164 raft_consensus.cc:3055] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:38.100483 13164 raft_consensus.cc:740] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:38.101294 13164 consensus_queue.cc:260] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:38.070396 13274 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:38.102404 13240 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:38.068041 13156 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:38.103427 13356 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:38.062577 13270 raft_consensus.cc:697] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:38.106074 13001 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:38.106374 13270 consensus_queue.cc:237] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:38.107391  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2199) took 2773 ms. Trace:
I20260709 13:03:38.040457 13184 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:38.107754  8399 rpcz_store.cc:276] 0709 13:03:35.334213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.334476 (+   263us) service_pool.cc:224] Handling call
0709 13:03:35.352343 (+ 17867us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:35.352815 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:35.352864 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:36.266259 (+913395us) spinlock_profiling.cc:243] Waited 6.85 ms on lock 0x7b2c000e1c48. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:03:36.270858 (+  4599us) log.cc:844] Serialized 26 byte log entry
0709 13:03:36.306954 (+ 36096us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:36.306996 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.307063 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:37.895313 (+1588250us) spinlock_profiling.cc:243] Waited 5.02 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:38.107167 (+211854us) raft_consensus.cc:1685] finished
0709 13:03:38.107187 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.107313 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6727,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":3943,"spinlock_wait_cycles":27301120,"thread_start_us":34170,"threads_started":1,"wal-append.queue_time_us":34530}
I20260709 13:03:38.108619 13184 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:38.062630 13275 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:38.103451 13156 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:38.111109 13156 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:38.112314  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.038781 13273 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:03:38.082660 13375 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:03:38.082075 13314 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:38.167320  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 93088.3us
I20260709 13:03:38.173532  8369 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:38.170360  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 98202.6us
I20260709 13:03:38.096833 12974 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:38.176885  8297 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:03:38.178244  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 281481us
I20260709 13:03:38.195917  8974 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:38.197614  8974 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:38.076030 13163 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:38.076314 13261 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.206087  8297 leader_election.cc:400] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15.
I20260709 13:03:38.206528  8297 leader_election.cc:403] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:38.169417 13262 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.208178  8297 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:38.209811  8369 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:38.210340 13029 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:38.210431 13200 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:38.211280 13275 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:03:38.211783 12670 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:38.068688 12932 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:38.230036  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 215694us
I20260709 13:03:38.198764 13273 raft_consensus.cc:697] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:38.230762  8223 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:38.231798  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:38.232951  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:38.192724  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 288529us
I20260709 13:03:38.219640  8297 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:38.197407 13371 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:38.223490 13157 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:38.225234 12974 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:38.069245 13308 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:38.208242 13261 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:38.243285  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.241573  8223 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:38.263494  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:38.270200  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 72654.7us
I20260709 13:03:38.228977 13251 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:38.275745  8369 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:38.276178  8974 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:38.240666 12670 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.243969 13012 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:38.241386 12989 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:38.277905  8369 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:38.287106  8253 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:38.290058  8297 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:38.267431 13356 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:38.291447  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:38.267552 13029 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:38.296521  8297 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:03:38.297829  8369 connection.cc:256] RPC call timeout handler was delayed by -1.5648172539999905s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2137s for a response
I20260709 13:03:38.299459  8331 raft_consensus.cc:1240] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:03:38.221597 13262 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:38.274577 13364 consensus_meta.cc:287] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.577s	user 0.009s	sys 0.000s
I20260709 13:03:38.305342  8327 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:38.309808  8298 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:38.310817  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:38.312837  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:38.315865  8298 leader_election.cc:400] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15.
I20260709 13:03:38.315882  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:38.316862  8298 leader_election.cc:400] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14.
I20260709 13:03:38.318399  8332 raft_consensus.cc:2368] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 13. Current term is 14.
I20260709 13:03:38.260885 13009 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:03:38.325448  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:38.333536  8323 raft_consensus.cc:1240] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:03:38.333957  8331 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:38.278863 13163 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:38.278851 13001 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:38.284363 12670 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:38.339529  8369 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:38.343969  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:38.344614  9008 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:38.302101 13251 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:38.332919  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:38.333647  8329 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:38.290067 13240 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:38.335577  8328 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:03:38.326117  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:38.276855 13388 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:38.289266 13274 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:38.339965  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:38.303540 13260 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:38.343230  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:38.345847  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:38.332020  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:38.359524  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:38.347074  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:38.347538  9006 raft_consensus.cc:1240] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:38.347721  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:38.349546  8403 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:38.303843 13346 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.300232 13012 raft_consensus.cc:697] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:03:38.351120  8371 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.030s	sys 0.049s
I20260709 13:03:38.353458  8330 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
I20260709 13:03:38.332898 13315 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:38.324977 13364 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15.
W20260709 13:03:38.369982  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 52224.7us
I20260709 13:03:38.360031  8406 raft_consensus.cc:2393] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 13: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:38.360142  8327 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:38.361341  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:03:38.365681  8371 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.030s	sys 0.049s
I20260709 13:03:38.365500  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:38.359802  8326 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:38.366358  8972 leader_election.cc:400] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 13. Current term is 14.
I20260709 13:03:38.368522  9002 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
W20260709 13:03:38.356489 13196 consensus_meta.cc:287] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.676s	user 0.010s	sys 0.001s
I20260709 13:03:38.372591  9004 raft_consensus.cc:2410] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:03:38.372967  8972 leader_election.cc:403] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:03:38.358451 13029 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:38.390545  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:38.391224  9006 raft_consensus.cc:2376] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12. Re-sending same reply.
I20260709 13:03:38.364871 13260 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.370500 13314 consensus_queue.cc:1059] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:03:38.363763  9009 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "038cd0877a674c2996410c9f327df911"
dest_uuid: "f865c4d761f5446eb12b3a029234772e"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:45628
I20260709 13:03:38.422641  9009 raft_consensus.cc:606] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:03:38.423012  9009 raft_consensus.cc:614] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:38.372087 13251 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:38.338742 13270 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:03:38.366204 13365 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:38.394851  8403 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
I20260709 13:03:38.394177  8974 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:38.338299 13390 consensus_queue.cc:1059] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:03:38.379400 13367 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:38.430878  8404 raft_consensus.cc:1275] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:03:38.435685  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 96268.8us
W20260709 13:03:38.438359  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 48771.9us
I20260709 13:03:38.440774  8327 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
W20260709 13:03:38.441552  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 328090us
I20260709 13:03:38.441300  8974 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:38.371201 13389 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:38.443295  8406 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:38.431368 13207 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.445034  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:38.461054  8400 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:38.379690 13273 consensus_queue.cc:237] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:38.422537 13260 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:38.420323 13275 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15.
I20260709 13:03:38.370707 13346 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:38.449194 12974 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:38.449915  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:38.447378  8972 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:38.379935 13196 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:38.375639  8142 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 81804f59e3124d27998ea80fc1c36663 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica f865c4d761f5446eb12b3a029234772e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:03:38.468498  8142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 96304.9us
I20260709 13:03:38.430779 13184 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:38.456918 13389 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:03:38.459643  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 218116us
I20260709 13:03:38.437129 13251 raft_consensus.cc:697] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:38.445218  8326 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:38.420929 13029 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:38.449137  8404 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:38.462368 13365 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:03:38.512933 13389 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:38.513526 13251 consensus_queue.cc:237] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:38.514112 13403 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:38.515889 13403 consensus_queue.cc:1059] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:38.517896 13403 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:38.518918  8223 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:38.519802 13142 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:38.520102  8223 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:38.521464 13196 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:38.522718 13029 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:38.459352 13362 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:38.453383 13200 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:38.500159 13308 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:38.478451 12932 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:38.485714 13163 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:38.503264  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:38.524914 13163 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.491155  8298 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:03:38.455821  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 159321us
I20260709 13:03:38.462584 13012 consensus_queue.cc:237] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:38.525377 13163 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:38.527072  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:38.527289 13390 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.512741  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:38.529754 13390 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:38.519901 13262 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:38.531360  9009 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22"
dest_uuid: "f865c4d761f5446eb12b3a029234772e"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:45628
I20260709 13:03:38.531941  9009 raft_consensus.cc:606] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:38.532269  9009 raft_consensus.cc:614] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:38.532531 13390 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:38.511111 13346 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:38.399220 13400 consensus_queue.cc:1059] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:03:38.530330  9005 raft_consensus.cc:2393] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 14: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:38.534005 13185 raft_consensus.cc:3055] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:38.534490 13185 raft_consensus.cc:740] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:03:38.534778  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 43703.4us
I20260709 13:03:38.535316 13185 consensus_queue.cc:260] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:38.536208 13200 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:38.536537 13185 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:38.517870  8399 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
W20260709 13:03:38.536954 13369 consensus_meta.cc:287] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.545s	user 0.008s	sys 0.000s
I20260709 13:03:38.546836 13369 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:38.553882  9009 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:38.523813 13302 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:38.504125 13261 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:38.537918 13360 consensus_meta.cc:287] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.575s	user 0.008s	sys 0.000s
I20260709 13:03:38.535620  8298 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:38.543730 13156 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:38.527462  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:38.545970  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:38.546243  8331 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:38.534114 13400 raft_consensus.cc:3055] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:38.565707  8298 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:38.546983  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:38.536787 13200 raft_consensus.cc:697] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:38.548470 13164 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:38.649708  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:38.650256  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:38.548358  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:03:38.541043 13357 log.cc:927] Time spent T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.870s	user 0.000s	sys 0.004s
I20260709 13:03:38.652081  8328 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 17: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:38.559315  9001 raft_consensus.cc:1275] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:03:38.652381  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:38.561906  9008 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:03:38.652832 13378 consensus_queue.cc:1059] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:03:38.565755  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:38.653609 13378 consensus_queue.cc:1243] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer 8c42259d7c4b463a9f485f42c320c81b log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:03:38.653841  8253 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
W20260709 13:03:38.485718 13372 log.cc:927] Time spent T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.540s	user 0.003s	sys 0.009s
I20260709 13:03:38.566416  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:38.567194  8295 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:38.634626 13356 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:38.657622 13388 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 13. Current term is 14.
W20260709 13:03:38.657825  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 90691.6us
I20260709 13:03:38.658157 13356 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:38.658711 13356 raft_consensus.cc:697] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:38.659511 13356 consensus_queue.cc:237] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:38.660254 12996 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:03:38.660547  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 134402us
W20260709 13:03:38.503225 13344 consensus_meta.cc:287] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.538s	user 0.006s	sys 0.003s
I20260709 13:03:38.634840 13400 raft_consensus.cc:740] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:38.662576 13200 consensus_queue.cc:237] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:38.663264 13400 consensus_queue.cc:260] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:38.664446 13400 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:38.665477  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:38.635699 13360 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:38.666323  8403 raft_consensus.cc:2393] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 15: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:38.667014  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:38.667579 13360 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:38.668642 13369 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:38.669130  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:38.636197  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:38.639739  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:38.641705  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:38.638247  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 66200.6us
I20260709 13:03:38.518092 13385 raft_consensus.cc:3055] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:38.562657 13009 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.647130  8401 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:38.648712 13156 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:38.495076 13240 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:03:38.683988  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.649549 13278 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:38.671437  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:38.672122  8400 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
W20260709 13:03:38.643657 13061 consensus_meta.cc:287] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.679s	user 0.009s	sys 0.000s
I20260709 13:03:38.674039  8330 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:38.652143  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:38.674836  8298 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:38.673637  8329 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
W20260709 13:03:38.675662  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 160350us
I20260709 13:03:38.538177 13020 raft_consensus.cc:3055] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:38.527431 13362 raft_consensus.cc:697] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:38.677757  8325 raft_consensus.cc:2368] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12.
I20260709 13:03:38.463946 13207 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:38.679976  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:38.680480  9009 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:03:38.680965  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:38.659838  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 191397us
I20260709 13:03:38.722184  8248 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: [10.6-12.7]
I20260709 13:03:38.725828  8251 raft_consensus.cc:2393] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 11: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:38.726516  8257 raft_consensus.cc:1240] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:38.725929  8369 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:38.652196 13367 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:38.729637  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:38.729780  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:38.666796 13260 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:38.731014  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:38.732251  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:38.732626  8256 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:38.743170  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:38.694819  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:38.743758  8251 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:38.730162  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 55410.3us
I20260709 13:03:38.736564  8250 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:03:38.737864  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:38.738107  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:38.738955  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:38.745941  8325 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:38.739787  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:38.746706  8329 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 16: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:38.740424  8222 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:38.745365  8254 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:38.677393 13385 raft_consensus.cc:740] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:38.753140  8250 raft_consensus.cc:1240] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:03:38.752238  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:38.758836  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:38.576965 13314 consensus_queue.cc:1059] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.177s
I20260709 13:03:38.760702  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:38.772704  8327 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:38.775087  8328 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:03:38.684158 13240 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.756558  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:38.727837 13360 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.788115  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:38.788758  8403 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
W20260709 13:03:38.792479  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 43872.1us
I20260709 13:03:38.753906 13308 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:38.725399 13278 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:38.728186 13020 raft_consensus.cc:740] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:38.775583  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:38.777945  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:38.779218  8253 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:38.782958  9009 raft_consensus.cc:1240] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:38.772250  8251 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:38.752954 13384 consensus_queue.cc:1059] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:03:38.795162  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:38.796846  8329 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:03:38.720717 13009 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:38.824806  8369 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:38.825065  8400 raft_consensus.cc:1275] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:38.823454  8329 raft_consensus.cc:606] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:03:38.823748  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:38.784695 13240 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:38.829945  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:38.831804  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:38.840611  8330 raft_consensus.cc:2393] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 12: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:38.834519 13407 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:38.761713 13156 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
W20260709 13:03:38.842885  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 57784.6us
I20260709 13:03:38.836421 13370 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:38.782308 12974 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.850920  8369 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:38.854949  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:38.856194  8328 raft_consensus.cc:2410] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:03:38.856362  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:38.857074  8248 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:38.795969 13308 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.742630 13061 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:38.794572 13251 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.858989  8295 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:38.860425  8369 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:38.819778 13278 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:38.848457  8327 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:03:38.789675 13141 consensus_queue.cc:1059] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:03:38.792425 13360 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:38.866391  8251 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:38.841555 13407 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.864560  8222 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:38.884603  8327 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:38.897110  9009 raft_consensus.cc:3055] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:38.898219  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:38.899253  8328 raft_consensus.cc:2393] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 15: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:38.862021 13387 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:38.875267  8369 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:38.875492  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:38.874931  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:38.907773  8330 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:38.895309  8400 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:38.897537  9009 raft_consensus.cc:740] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:38.865059 13308 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:38.863785  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 70568.8us
I20260709 13:03:38.875604 13328 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:38.911820  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:38.873322 13157 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:38.874655  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:38.867959 13001 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:38.864733 13302 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:38.853859 12932 consensus_queue.cc:1059] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:03:38.938244  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:38.939062  8328 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:38.911082  9009 consensus_queue.cc:260] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:38.925972  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:38.894177 13356 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.866619 13370 raft_consensus.cc:697] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:38.943423  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:38.944046  8402 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:38.915048 13346 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:38.905992 13407 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:38.863544 13264 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:03:38.843478 13389 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:38.928018 13270 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.865480 12974 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:38.877575 13251 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:38.951028  9009 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:38.941568  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:38.948511 12932 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:38.931448 13142 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:38.939267 13261 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:38.872684 13362 consensus_queue.cc:237] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:38.938565 13328 raft_consensus.cc:697] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:38.949720  8330 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:03:38.954071  8400 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
W20260709 13:03:38.952906 13421 log.cc:927] Time spent T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:03:38.967228  9005 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:38.950187 13156 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:03:38.970120  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 50435us
I20260709 13:03:38.864233 13385 consensus_queue.cc:260] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:38.968197  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:38.931885 13365 consensus_queue.cc:1059] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:03:38.969352  8369 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:38.975416  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:38.958200 12932 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.981501  8325 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:38.996354  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:38.982422  8972 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:38.982393  8328 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 17: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:38.945727 13356 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:38.965169 13184 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:39.008530 13420 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:03:39.010082 13412 log.cc:927] Time spent T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:03:39.010290  8972 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:38.944062 13302 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:39.006807 12932 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:39.019797 13240 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:39.020730 13001 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:39.022014 13382 raft_consensus.cc:3055] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:03:39.033175 12989 consensus_meta.cc:287] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.739s	user 0.011s	sys 0.000s
I20260709 13:03:39.024219 12974 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:39.028191  8402 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:03:38.973388 13156 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.979069 13260 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:38.970603 13200 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:39.033579  8972 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:39.094898  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 61393.2us
I20260709 13:03:39.095631  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:38.956804 13270 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:38.983359 13207 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:39.035476  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:39.031008 13185 consensus_queue.cc:1059] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:03:39.031004 13184 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
W20260709 13:03:39.042523 13415 log.cc:927] Time spent T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:03:39.023635 13251 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:39.056458  8328 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:03:39.058081 13163 consensus_meta.cc:287] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.529s	user 0.006s	sys 0.004s
I20260709 13:03:39.054294 13382 raft_consensus.cc:740] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:39.058956 13240 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:39.059122 13410 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 13:03:39.029541 13370 consensus_queue.cc:237] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:39.035374 13419 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:39.166661 13061 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:39.060091  8326 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:39.061205 13302 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:39.061589 13365 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.167793 13365 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:39.169178 13400 consensus_meta.cc:287] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.504s	user 0.012s	sys 0.000s
I20260709 13:03:39.170060 13365 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:39.062227  9006 raft_consensus.cc:2368] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 15. Current term is 16.
I20260709 13:03:39.062608 13001 raft_consensus.cc:697] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:39.065330 13387 raft_consensus.cc:3055] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:39.171060 13387 raft_consensus.cc:740] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:39.171479 13001 consensus_queue.cc:237] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:39.172077 13387 consensus_queue.cc:260] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:39.173071 13387 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:39.179832  8400 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:03:39.074072 13315 consensus_queue.cc:1059] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:38.984777 13328 consensus_queue.cc:237] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:39.079062  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:39.091580  9009 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:03:39.092176  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 227654us
I20260709 13:03:39.091638  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:39.197518 12670 consensus_meta.cc:287] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.842s	user 0.009s	sys 0.010s
I20260709 13:03:39.198032 12670 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:39.131325  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:39.199105  9008 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:03:39.134675  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:39.200150  8323 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
I20260709 13:03:39.200459 13365 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.200982 13365 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:39.093253 12989 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:39.202636  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:39.203183 13365 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:39.197113  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:39.203770  8402 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:39.205188 13361 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:03:39.205272  8327 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:39.205724 13388 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:39.206162 13388 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.206537 13388 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:39.093717 13296 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.215559 13296 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:39.154615 13163 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:39.223394 13163 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:39.132588 12974 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:03:39.158236 13423 log.cc:927] Time spent T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:03:39.224645  8405 raft_consensus.cc:2393] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 12: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:39.158811 13360 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:39.226130  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:39.227643  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:39.228941  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:39.160574 13356 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:39.161125 13376 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:39.089776 13346 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:39.093971 13156 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:39.213111  8402 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:39.234980  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:39.064982 12932 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:39.174373 13382 consensus_queue.cc:260] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:39.175508  8330 raft_consensus.cc:2410] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:03:39.177175  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:39.250578  8329 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:39.179955  8256 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
W20260709 13:03:39.073570  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.252080  8252 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:39.198379  8297 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:39.046751 13009 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:39.201192  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:39.254380  8369 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:39.135605  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:39.089468 13385 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:39.242187 13315 raft_consensus.cc:3055] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:39.260401  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:39.260659  8256 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:03:39.244190  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:39.195367  9000 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:39.247323  8248 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:03:39.261495  9000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56948 (request call id 988) took 1147 ms. Trace:
I20260709 13:03:39.262158  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:39.262637  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:39.263039  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:39.263229  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:39.263656  8402 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:03:39.263760  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:39.252972  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:39.264624  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:39.255966  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:39.229808 13376 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.256702  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:39.265272  8402 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:39.265774  9009 raft_consensus.cc:2393] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:39.265614  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:39.254565  8323 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:39.259488 13388 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:39.257246  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 58962.3us
I20260709 13:03:39.259294  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:39.267036  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:39.267572  8323 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:39.268079  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:39.268504  8297 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:39.269168  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:39.241904 13296 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:39.195462 13164 consensus_meta.cc:287] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.526s	user 0.009s	sys 0.001s
I20260709 13:03:39.269797  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:39.249958 13184 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:39.279543  8325 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:03:39.280820  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:39.282210  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:39.269946  8297 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:39.283275  8404 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:03:39.266309  8248 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:03:39.269799  8253 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:39.284510  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:39.284689  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:39.271122  8295 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:39.261950  9000 rpcz_store.cc:276] 0709 13:03:38.114134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.114918 (+   784us) service_pool.cc:224] Handling call
0709 13:03:39.078896 (+963978us) spinlock_profiling.cc:243] Waited 4.34 ms on lock 0x7b5800175898. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:03:39.261411 (+182515us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9973120}
I20260709 13:03:39.147470 13200 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:39.278352  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:39.259706 13365 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:39.285668  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:39.272930  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:39.283128  8325 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:39.287153  8248 raft_consensus.cc:1240] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:39.287551  8249 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:03:39.285642  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:39.289054  8253 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 14: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:39.290035  8250 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:39.258412 13360 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:39.290777  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:39.319402  9009 raft_consensus.cc:2393] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 15: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:39.292691  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:39.285090  8257 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:39.290139 13389 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:39.293007 13424 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.314845 13270 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:39.315086 13200 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:39.316499 13278 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:39.229975 13346 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:39.317976  8323 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:39.274412 13382 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:39.271649 13376 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:39.260710 13315 raft_consensus.cc:740] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:39.318629  8402 raft_consensus.cc:2393] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 12: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:39.295166  8329 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 16.
I20260709 13:03:39.294446  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:39.303345  8251 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: [10.6-12.7]
W20260709 13:03:39.386890  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 92503.9us
I20260709 13:03:39.387686  8369 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:39.388000  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:39.273886 13356 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:39.282670 13240 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.093s
I20260709 13:03:39.402274  8369 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:39.405789  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:39.282217 12989 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:39.302481 13427 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:39.363698  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 746) took 1048 ms. Trace:
I20260709 13:03:39.411762  8399 rpcz_store.cc:276] 0709 13:03:38.315400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.315964 (+   564us) service_pool.cc:224] Handling call
0709 13:03:39.363491 (+1047527us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b54003e8354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:39.363631 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":363832192}
I20260709 13:03:39.356287 13296 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:03:39.368394  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 287171us
I20260709 13:03:39.302678 13388 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:39.376748  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 54297.2us
I20260709 13:03:39.408284 13404 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:39.258104 13163 consensus_queue.cc:1059] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:03:39.301697 13164 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:39.388676  8323 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:39.301970 13001 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.261349 13251 consensus_queue.cc:1059] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:03:39.313164 13400 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:39.390784  8329 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:03:39.393041  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:39.459704  8402 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:39.398370 13308 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:03:39.410534  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:39.312722 12932 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:39.415014  8298 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:39.415552 13419 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:39.470530  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 55567.5us
I20260709 13:03:39.423528  9007 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:39.382515  8257 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:39.435096 13404 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.426501  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:39.445101  8323 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 16.
I20260709 13:03:39.387205 13424 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:39.413249 13296 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:39.458151 13377 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.100s	user 0.015s	sys 0.005s
W20260709 13:03:39.457062  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 187164us
W20260709 13:03:39.457835  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.391753 13407 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:39.461799  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:39.417629 12989 raft_consensus.cc:697] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:03:39.464751  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 54285.6us
I20260709 13:03:39.467478  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:39.403128 13156 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:39.441717 13164 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.403796 13207 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:39.425812  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:39.490186  8250 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:39.469081 13029 raft_consensus.cc:3055] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:39.478703  8297 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:39.479988  8323 raft_consensus.cc:1240] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:39.476892 13400 raft_consensus.cc:697] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:39.485476  8369 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:39.441500 13061 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:03:39.471721 13163 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:03:39.476650 13377 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.124s	user 0.016s	sys 0.005s
I20260709 13:03:39.476598 13296 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:39.474228  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.493091  8402 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:03:39.478410 13404 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:39.493918  8404 raft_consensus.cc:2368] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15.
I20260709 13:03:39.462505 13386 consensus_queue.cc:1059] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 13:03:39.476770 13431 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:39.481853  8255 raft_consensus.cc:2368] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13.
I20260709 13:03:39.463269 13001 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:39.503496  8974 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:39.505225 13356 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:39.490135 13164 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:39.510582  8297 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:39.477309 13419 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:39.520546 13389 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:39.494807 13029 raft_consensus.cc:740] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:39.502758 13434 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:39.526206  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:39.488456 13200 consensus_queue.cc:1059] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
W20260709 13:03:39.553175  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:39.568176  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 69695.7us
I20260709 13:03:39.529367  9007 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:03:39.510231 13431 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.522495 13356 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.578310  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:39.540429  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:39.581355  8324 raft_consensus.cc:1275] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:39.567169 13427 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:39.567556 13434 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.585510  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:39.509189 13207 consensus_queue.cc:1059] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 13:03:39.585244 13434 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:39.575693 13431 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:39.529641  8399 raft_consensus.cc:2393] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 13: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:39.573557 13346 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:39.618160  8406 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:39.580085  8402 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:39.606065 13424 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:39.606698 13029 consensus_queue.cc:260] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:39.610013 13431 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:39.612046 13156 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:39.612392 13434 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:39.613618 13419 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:39.579282 13356 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:39.613941 13433 log.cc:927] Time spent T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:03:39.614351 13308 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:39.614938 13207 raft_consensus.cc:3055] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:39.639459 13207 raft_consensus.cc:740] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:39.640362 13207 consensus_queue.cc:260] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:39.621958 13061 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:39.641517 13207 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:39.641784 13061 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:39.615119 13001 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:39.577459 13278 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:39.642851 13434 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:39.620728  8297 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:39.622296  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:39.623603  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:39.644873  8257 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:39.631107  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:39.636008  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:39.645563  8248 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:39.615835 13407 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:39.644944  9007 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:39.625367  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:39.646194  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:39.617465 13427 raft_consensus.cc:697] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:39.648154  8323 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:03:39.577163 12989 consensus_queue.cc:237] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:39.650702 13278 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:39.653241  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:39.653749 13427 consensus_queue.cc:237] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:39.655476 13431 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:39.632942 13400 consensus_queue.cc:237] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:39.653388 13278 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.652845  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:39.662452  8326 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:39.630548 13163 raft_consensus.cc:697] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:39.663477  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:39.668567  8323 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:03:39.669066  8248 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:39.670245  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:39.669579  8369 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:39.681515 13407 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:39.682070 13407 raft_consensus.cc:697] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:39.682561  8323 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:39.688016  9007 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
W20260709 13:03:39.691975  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.682863 13407 consensus_queue.cc:237] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:39.696980  8372 leader_election.cc:400] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 15. Current term is 16.
I20260709 13:03:39.697404  8372 leader_election.cc:403] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:39.683396  8328 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:39.700186  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:39.700443 13061 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:39.700809  8403 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:39.663422 13278 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:39.663825 13029 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:39.709601  8403 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:03:39.722504 13306 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:39.724921 13296 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:39.712179 13156 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:03:39.731724 13020 consensus_queue.cc:260] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:39.732766 13020 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:39.711542 13141 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:03:39.737375 13369 raft_consensus.cc:3055] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:39.737815 13369 raft_consensus.cc:740] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:39.738636 13369 consensus_queue.cc:260] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:39.739962 13369 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:39.742105 13355 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:39.751025 12670 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:39.754748  8325 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:39.755424  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2350) took 1006 ms. Trace:
I20260709 13:03:39.755923  8329 raft_consensus.cc:606] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:03:39.757761  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:39.761716  8329 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "767eba28679649519167576023555b60"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:03:39.762310  8329 raft_consensus.cc:606] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:39.762128 13315 consensus_queue.cc:260] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:39.763341 13315 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:39.766270  8404 raft_consensus.cc:1275] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:39.767917 13164 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:39.770848  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:39.772033  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:39.773031 13164 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:39.773947  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.775161  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:39.777021  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2164) took 1081 ms. Trace:
I20260709 13:03:39.777264  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:39.784807  8252 rpcz_store.cc:276] 0709 13:03:38.695747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.743130 (+ 47383us) service_pool.cc:224] Handling call
0709 13:03:39.251900 (+508770us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.253664 (+  1764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.776785 (+523121us) spinlock_profiling.cc:243] Waited 477 ms on lock 0x7b540034a8d0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:39.776950 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1098175360}
I20260709 13:03:39.768376  8404 raft_consensus.cc:1240] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:39.755735  8325 rpcz_store.cc:276] 0709 13:03:38.749479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.293810 (+544331us) service_pool.cc:224] Handling call
0709 13:03:39.294081 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.754601 (+460520us) spinlock_profiling.cc:243] Waited 23.1 ms on lock 0x7b54002d7920. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f8828735dc0 00007f88287c01c0 00007f88287c1231 00007f88287bdd8f 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c
0709 13:03:39.755156 (+   555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.755370 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53067520}
I20260709 13:03:39.781978  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 2699) took 1279 ms. Trace:
I20260709 13:03:39.793380  8401 rpcz_store.cc:276] 0709 13:03:38.502815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.502962 (+   147us) service_pool.cc:224] Handling call
0709 13:03:39.781750 (+1278788us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54003e6f54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:39.781909 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":279742720}
I20260709 13:03:39.793973  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:39.794795  8325 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:39.807554  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:39.810846  8297 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:39.815182  9001 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:39.819628  8399 raft_consensus.cc:2393] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 13: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:39.820546  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:39.820788  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:39.777720 13296 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:39.822474  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.759239  8398 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:39.826135  8403 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:03:39.826328  8398 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:39.833487  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:39.835634  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.837414  8297 leader_election.cc:400] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15.
I20260709 13:03:39.837919  8297 leader_election.cc:403] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:39.839159  8401 raft_consensus.cc:1240] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:39.840330  8220 connection.cc:256] RPC call timeout handler was delayed by -1.1002784789999396s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.242s for a response
I20260709 13:03:39.846809  8325 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:03:39.848045  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:39.849242 13355 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15.
I20260709 13:03:39.791811 13356 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:39.857918 13163 consensus_queue.cc:237] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:39.859452 13356 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:39.862461 13156 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 15. Current term is 16.
I20260709 13:03:39.838728 13275 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:39.867794  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:39.868429  8329 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:39.870062  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:39.870631  8252 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:39.871528  8326 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:39.873592 13001 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.874122 13001 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:39.876240 13001 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:39.876037 13404 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:39.876797 13001 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:39.871778 13437 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:39.878196 13437 raft_consensus.cc:697] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:39.879150 13437 consensus_queue.cc:237] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:39.879786  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:39.807114 13164 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:39.881954 13164 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.877349 13001 raft_consensus.cc:697] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:39.882515 13164 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:39.884707 13164 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:39.885967  8249 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:03:39.886039  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:39.883828 13001 consensus_queue.cc:237] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:39.882207 13404 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:39.890635  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:39.891484  8325 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:03:39.819931 13299 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:39.894418  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:39.893153  8369 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:03:39.895031  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 49198us
W20260709 13:03:39.898491  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.900771  8297 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:39.903577  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:39.904172  8256 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:39.845468 13196 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:39.907524  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:39.845145 13262 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.910032  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:39.871524 13365 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:39.914391  8325 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:39.922497  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:39.928648  8326 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:39.922138  8256 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:03:39.930472  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:39.985373  8399 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:39.988613  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 65094.1us
I20260709 13:03:39.989135  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 2628) took 1061 ms. Trace:
I20260709 13:03:39.911314 13344 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:03:39.965092 13433 log.cc:927] Time spent T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:03:39.982509  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:40.013062  8251 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:39.913498 13355 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:39.913249 13196 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.930109  8405 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:39.994275  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:39.989534  8330 rpcz_store.cc:276] 0709 13:03:38.927760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.967818 (+ 40058us) service_pool.cc:224] Handling call
0709 13:03:39.988944 (+1021126us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b540008a554. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:39.989080 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":460080000}
I20260709 13:03:39.997305 13438 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:39.993278 13387 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:39.900620 12670 consensus_queue.cc:1048] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:03:40.007923 13436 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.008899 13407 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:40.009852 13278 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:40.010706 13384 raft_consensus.cc:3055] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:39.907835 13275 consensus_queue.cc:1059] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 13:03:40.015503  8252 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:03:40.018958  8326 raft_consensus.cc:2410] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:03:40.019966 13020 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:39.908017 13200 consensus_queue.cc:1059] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 13:03:39.925215 13164 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:39.986932 13434 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:40.024915  8328 raft_consensus.cc:2376] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13. Re-sending same reply.
I20260709 13:03:40.031849  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:40.049824 13436 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:40.072746 13376 consensus_meta.cc:287] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.669s	user 0.010s	sys 0.002s
I20260709 13:03:40.043103 13387 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:40.048062  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 61217.1us
I20260709 13:03:40.075837 13387 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:40.051997  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 147494us
I20260709 13:03:40.054203  8402 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:03:40.026564 13196 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:39.925136 13262 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:40.063495  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:40.072348  8330 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:40.085491  8326 raft_consensus.cc:2376] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14. Re-sending same reply.
I20260709 13:03:39.914189 13356 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 13:03:40.076809  8222 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:40.050957 13384 raft_consensus.cc:740] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:40.049826 13344 raft_consensus.cc:697] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:40.083706  8327 raft_consensus.cc:2410] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:03:40.086715  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:40.097587  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:40.098604  8327 raft_consensus.cc:2410] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 13:03:40.098556  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 108954us
I20260709 13:03:40.110308  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 2222) took 1176 ms. Trace:
I20260709 13:03:40.110881  8406 rpcz_store.cc:276] 0709 13:03:38.934040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.252723 (+318683us) service_pool.cc:224] Handling call
0709 13:03:40.110078 (+857355us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b54002be654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:40.110214 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":604282752}
I20260709 13:03:40.055490 13407 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:40.090636  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:40.134371  8330 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:40.134830  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:40.136471  8327 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:40.073210 13376 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:40.115578  8404 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:03:40.141376  8327 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:40.081641 13306 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:03:40.145682  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:40.092078 13278 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:40.077469 13434 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:40.096553 13369 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.070124 13164 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:40.153220  8328 raft_consensus.cc:1275] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:03:40.144320 13061 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:40.159660 13061 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:40.161861 13061 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:40.172289 13369 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:40.154893 13452 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:40.153906  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:40.112319 13438 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.155799  8220 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:40.175024 13369 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:40.181653  8406 raft_consensus.cc:2393] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 15: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:40.198412  9005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 991) took 1646 ms. Trace:
I20260709 13:03:40.215615  9005 rpcz_store.cc:276] 0709 13:03:38.552067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.552245 (+   178us) service_pool.cc:224] Handling call
0709 13:03:38.552572 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:38.969412 (+416840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.198035 (+1228623us) spinlock_profiling.cc:243] Waited 414 ms on lock 0x7b540067acd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:40.198318 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":953097984}
I20260709 13:03:40.200383  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2231) took 1257 ms. Trace:
W20260709 13:03:40.201157  8331 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2345) took 1893 ms (client timeout 1658 ms). Trace:
W20260709 13:03:40.218142  8331 rpcz_store.cc:269] 0709 13:03:38.308057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.310595 (+  2538us) service_pool.cc:224] Handling call
0709 13:03:40.200916 (+1890321us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b540011e9d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:40.201052 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51885952}
I20260709 13:03:40.205703  8332 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
W20260709 13:03:40.211242 13417 log.cc:927] Time spent T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 1.255s	user 0.004s	sys 0.000s
I20260709 13:03:40.217166  8405 rpcz_store.cc:276] 0709 13:03:38.943196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.729402 (+786206us) service_pool.cc:224] Handling call
0709 13:03:39.729717 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.025957 (+296240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.200056 (+174099us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54003e7954. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:40.200315 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":460349312}
I20260709 13:03:40.233259  8220 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:40.235903  8220 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:40.237663  8220 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:40.241163  8330 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
W20260709 13:03:40.241982  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 165217us
I20260709 13:03:40.241344  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:40.179775 13436 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:40.246214  8406 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:40.235075 13364 raft_consensus.cc:3055] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:40.214794 13141 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:40.232813 13443 log.cc:927] Time spent T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 13:03:40.242161 12670 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:40.273865  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:40.280920  8405 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:40.233498 13387 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.235354 13356 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.246702 12868 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:40.258921 13364 raft_consensus.cc:740] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:40.214520 13438 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:40.280702 13445 log.cc:927] Time spent T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.168s	user 0.003s	sys 0.001s
I20260709 13:03:40.242038 13200 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:40.293628  9004 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:40.289166 13387 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:40.302119  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:40.311296  8405 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:40.289299 13141 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:40.304317 13438 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:40.241240 13384 consensus_queue.cc:260] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:40.269872 13344 consensus_queue.cc:237] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:40.306174  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2359) took 1356 ms. Trace:
I20260709 13:03:40.323704  8323 rpcz_store.cc:276] 0709 13:03:38.949723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.684071 (+734348us) service_pool.cc:224] Handling call
0709 13:03:39.688574 (+  4503us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:39.690599 (+  2025us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:39.690628 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:39.691026 (+   398us) log.cc:844] Serialized 26 byte log entry
0709 13:03:39.901536 (+210510us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:39.929686 (+ 28150us) log.cc:844] Serialized 14 byte log entry
0709 13:03:39.929884 (+   198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.929959 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.305969 (+376010us) raft_consensus.cc:1685] finished
0709 13:03:40.305991 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.306118 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":28004,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":9053,"raft.queue_time_us":35323,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":18610,"thread_start_us":111611,"threads_started":1,"wal-append.queue_time_us":444903}
I20260709 13:03:40.278928 13455 raft_consensus.cc:2764] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:03:40.310281  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2692) took 1917 ms. Trace:
I20260709 13:03:40.328580  8406 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
W20260709 13:03:40.331714  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 92066.9us
I20260709 13:03:40.310362 13381 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.331199  8405 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:40.292562 13275 consensus_queue.cc:1059] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:03:40.295815 12932 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:03:40.326855  8397 rpcz_store.cc:276] 0709 13:03:38.392351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.392696 (+   345us) service_pool.cc:224] Handling call
0709 13:03:38.433935 (+ 41239us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:38.773382 (+339447us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:38.773412 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:38.787339 (+ 13927us) log.cc:844] Serialized 26 byte log entry
0709 13:03:39.850881 (+1063542us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b18003bae80. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88283664bc 00007f88256b448b 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e0d1
0709 13:03:39.851050 (+   169us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:39.851087 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.852271 (+  1184us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b540019a5d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:39.852317 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.210437 (+358120us) spinlock_profiling.cc:243] Waited 3.89 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:40.309962 (+ 99525us) raft_consensus.cc:1685] finished
0709 13:03:40.309998 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.310198 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41885,"raft.queue_time_us":14575,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":10901,"spinlock_wait_cycles":768328704,"thread_start_us":52149,"threads_started":1,"wal-append.queue_time_us":124258}
I20260709 13:03:40.334437  8297 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:40.332465  8222 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:03:40.325897 13446 log.cc:927] Time spent T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 13:03:40.338094  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:40.339021  8405 raft_consensus.cc:2410] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:03:40.332216  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:40.340101  8406 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:40.341063  8220 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:40.338968 13001 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:40.341888 13001 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:40.342527  8399 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:40.294582 13356 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:40.297842 12670 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:40.318099 13308 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:03:40.318632 13200 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:40.354339 13454 log.cc:927] Time spent T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:03:40.337584 13141 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:03:40.355299 13200 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:40.337347 13384 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:40.357271  8220 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:40.346477 13275 consensus_queue.cc:1243] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:03:40.365370 13356 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:40.367458  8399 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:40.368067 13434 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:40.337394 13381 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:40.379099  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:40.379171  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:40.379980  8326 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:40.380057  8323 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:40.368824 13376 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:40.375305  8974 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:40.375916 13240 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:40.381110  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:40.382211  8331 raft_consensus.cc:2376] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10. Re-sending same reply.
I20260709 13:03:40.352736 13156 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:40.351080 13458 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:40.360271 13200 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:40.396404  9001 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:40.382800  8330 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:40.383917  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:40.396400  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:40.396618  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:40.435223  9005 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:03:40.333925 13364 consensus_queue.cc:260] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:40.359678 13456 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:40.395841  8220 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:40.422914 12670 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:40.391723 13299 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:40.381124 13381 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:40.442225  9007 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:40.396278 13156 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:40.395053 13376 consensus_queue.cc:1059] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:03:40.429069 13452 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:40.429561 13458 raft_consensus.cc:697] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:40.429622  9001 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:03:40.431777  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:40.445248 13387 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:40.445597  9008 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:03:40.446082 13275 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:40.447175 13458 consensus_queue.cc:237] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:03:40.448998 13156 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:40.451052  9006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:45618 (request call id 2) took 1035 ms. Trace:
I20260709 13:03:40.451460  9006 rpcz_store.cc:276] 0709 13:03:39.416037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.416217 (+   180us) service_pool.cc:224] Handling call
0709 13:03:40.441751 (+1025534us) spinlock_profiling.cc:243] Waited 742 ms on lock 0x7b54007198d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c4735 00007f88287bc9fa 00007f882d7844c9 00007f8826d00c54 00007f8826d00be2 00007f8826d00b11 00007f8826d00a8c 00007f8826cffd42 00007f8826af948c 00007f8826af88c6
0709 13:03:40.450972 (+  9221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1705799936}
I20260709 13:03:40.451875 13278 raft_consensus.cc:3055] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:40.461726 13275 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:40.461747 13278 raft_consensus.cc:740] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:40.462446 13275 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:40.463012 13278 consensus_queue.cc:260] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:40.463740  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:40.464409 13278 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:40.464754 13275 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:40.465811 12670 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.466353 12670 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:40.381852 13240 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:40.473181 13240 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:40.473809 13407 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:40.476142  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:40.479733 13196 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:40.480765  8222 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:40.481103  8401 raft_consensus.cc:2368] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16.
I20260709 13:03:40.481993 13452 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:40.482381 13452 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.482707 13452 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:40.483353  8401 raft_consensus.cc:1275] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:03:40.486248  8406 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
W20260709 13:03:40.487294  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.487604 13456 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:40.488102 13456 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:40.489984 13424 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:40.490587 13424 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:40.394085  8369 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:40.493880 13164 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:40.494076 13407 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:40.494643 13407 raft_consensus.cc:697] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:40.495124  9004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56948 (request call id 992) took 1968 ms. Trace:
I20260709 13:03:40.495528  9004 rpcz_store.cc:276] 0709 13:03:38.526698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.526828 (+   130us) service_pool.cc:224] Handling call
0709 13:03:40.494945 (+1968117us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b5400527bd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:40.495065 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4058114432}
I20260709 13:03:40.435668  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:40.438681  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:40.497188  8399 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:40.439564 13364 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:40.503763  8397 raft_consensus.cc:1240] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
W20260709 13:03:40.443555  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 47795us
W20260709 13:03:40.443948  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 109591us
I20260709 13:03:40.490089 13456 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:40.433673  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:40.495440 13407 consensus_queue.cc:237] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:40.498756  8222 leader_election.cc:400] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16.
I20260709 13:03:40.393709 13306 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:40.502965  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:40.512581  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:40.440119 13020 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:40.505472  8220 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:40.505702  8398 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:40.506107  8297 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:40.496628  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 102612us
W20260709 13:03:40.510898  8142 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 81804f59e3124d27998ea80fc1c36663 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica f865c4d761f5446eb12b3a029234772e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 18 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:40.520622 13356 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:40.435285 13200 raft_consensus.cc:697] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:40.511430  8222 leader_election.cc:403] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:40.511982  8298 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:40.526600  8369 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:40.523312  8297 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:40.514319  9002 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:40.523312  8295 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:40.524386  8220 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:40.525890 13020 raft_consensus.cc:2764] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:03:40.537672  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:40.507807 13424 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:40.534390  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:40.531349 13356 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:40.537608  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:40.546964  8403 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:40.541126 13157 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:40.548420  8369 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:40.541057 13020 consensus_queue.cc:1048] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:40.531213 12670 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:40.531163 13370 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:40.537295 12973 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:03:40.542650  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:40.568610  8406 raft_consensus.cc:2393] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 14: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:40.542888 13200 consensus_queue.cc:237] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:40.543519  8400 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 2226) took 1608 ms (client timeout 1585 ms). Trace:
W20260709 13:03:40.577633  8400 rpcz_store.cc:269] 0709 13:03:38.934935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.985521 (+ 50586us) service_pool.cc:224] Handling call
0709 13:03:40.543295 (+1557774us) spinlock_profiling.cc:243] Waited 39.7 ms on lock 0x7b5400276354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:40.543414 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":91205504}
I20260709 13:03:40.544210 12996 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.525892 13275 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.549139  8403 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:03:40.551687  8220 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:40.548244 13157 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:03:40.533473 13400 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.834s	user 0.008s	sys 0.002s
I20260709 13:03:40.553947 13164 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:40.550575 13466 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:40.557691 13274 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:40.559618  8397 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:03:40.563592  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:40.597738  8329 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:40.581684 13299 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.599789  8369 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:40.592555 13400 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.893s	user 0.009s	sys 0.002s
I20260709 13:03:40.571238  9002 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:03:40.573676  8398 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
W20260709 13:03:40.603817  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 74447.5us
I20260709 13:03:40.585903  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:40.587702  8220 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:40.587913  8401 raft_consensus.cc:1240] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:40.604604  8331 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:40.560405 13452 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:40.592028 13157 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:40.568063 13385 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:40.621703  8324 raft_consensus.cc:1240] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:40.569294 13370 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.594188 13356 raft_consensus.cc:697] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:40.578193 13376 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:40.566716  8974 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:03:40.625646  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 92003.4us
I20260709 13:03:40.586521 12670 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:40.569267  8972 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:40.607478  9005 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:03:40.608856 13470 consensus_queue.cc:1059] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:40.593817 13474 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.588866 13306 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:40.591398 13164 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:40.618810  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 95565.6us
I20260709 13:03:40.581099 13438 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 13:03:40.624112  8369 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:40.629988  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.601847 13466 raft_consensus.cc:697] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:40.611882 13456 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:40.589123 12996 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:40.591055 13275 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:40.619752 13156 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:40.635358 13367 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:40.610071 13299 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:40.649008  8295 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:40.651296 13164 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:40.624279 13370 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:40.623160 13457 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.623220 13274 raft_consensus.cc:697] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:40.649935 13474 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:40.657819 13382 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:40.666538  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:40.624151 13384 consensus_queue.cc:1059] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:03:40.645958  8297 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:40.631771 13463 consensus_queue.cc:1048] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:03:40.657867 13240 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:40.772172 13240 raft_consensus.cc:697] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:40.771749 13466 consensus_queue.cc:237] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:40.773058 13240 consensus_queue.cc:237] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:40.617051 13455 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:03:40.625195 13376 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:40.777576  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.777920 13274 consensus_queue.cc:237] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:40.778975 13376 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:40.663111 13156 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.780074 13156 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:40.767966 13382 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:40.783128 13382 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:40.784160  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 135224us
I20260709 13:03:40.767936 13457 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:40.787267  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 141426us
I20260709 13:03:40.787504 13457 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:40.663123 13367 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.788177 13367 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:40.789582 13400 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:40.790220 13400 raft_consensus.cc:2764] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 13:03:40.790772 13400 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:40.768410 13061 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.791939 13400 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.792244 13457 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:40.792419 13061 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:40.792750 13457 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:40.793919 13465 consensus_queue.cc:1059] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:40.792573 13400 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:40.771575 13384 raft_consensus.cc:3055] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:40.796525 13384 raft_consensus.cc:740] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:40.797297 13384 consensus_queue.cc:260] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:40.797865 13468 log.cc:927] Time spent T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 13:03:40.798516 13384 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:40.798768 12996 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:40.800357 13469 log.cc:927] Time spent T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 13:03:40.801525 13356 consensus_queue.cc:237] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:40.802007  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:40.802618  8402 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:03:40.802846 13452 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:40.794755 13465 raft_consensus.cc:3055] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:40.803417 13465 raft_consensus.cc:740] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:40.803568  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:40.804149  8402 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:40.806521  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:40.794713 13061 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:40.794936 13457 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:40.768174 13407 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.666002 13196 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:40.768697  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2603) took 2224 ms. Trace:
I20260709 13:03:40.814765  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:40.816241 13407 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:40.828197  8326 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 9.
I20260709 13:03:40.831738  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:40.837572  8331 raft_consensus.cc:2376] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13. Re-sending same reply.
I20260709 13:03:40.827147  8297 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:40.841662  8330 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:40.828392  8332 rpcz_store.cc:276] 0709 13:03:38.544758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.544877 (+   119us) service_pool.cc:224] Handling call
0709 13:03:40.205512 (+1660635us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.224959 (+ 19447us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.768492 (+543533us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b540011e9d0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:40.768632 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3818135808}
I20260709 13:03:40.842962  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:40.843601  8332 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:40.843829  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:40.845016  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:40.867030  8220 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:40.868520  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:40.874181  8402 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:03:40.885351  8401 raft_consensus.cc:1240] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:40.914561  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2243) took 1410 ms. Trace:
I20260709 13:03:40.915043  8404 rpcz_store.cc:276] 0709 13:03:39.504038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.809203 (+305165us) service_pool.cc:224] Handling call
0709 13:03:40.115149 (+305946us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.141628 (+ 26479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.914363 (+772735us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b54002be650. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:40.914498 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":702597504}
I20260709 13:03:40.881459 13457 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:40.929806  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:40.925127  8332 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
I20260709 13:03:40.930457  8331 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:40.870975 13278 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:03:40.901022 13437 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.006s	sys 0.032s
I20260709 13:03:40.938848 13306 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:40.939999 13306 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:40.938691 13437 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.010s	sys 0.032s
W20260709 13:03:40.942454  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.943121 13306 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:40.944242  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:40.878448 13355 raft_consensus.cc:3055] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:03:40.951112  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 381879us
I20260709 13:03:40.953517  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:40.956956  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:40.957201  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:40.958907  8332 raft_consensus.cc:2393] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 11: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:40.831820 13465 consensus_queue.cc:260] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:40.996773  8332 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 14. Current term is 15. Ops: [14.12-14.12]
W20260709 13:03:40.996917  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 78886us
I20260709 13:03:40.999150  8331 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:03:40.950867 13278 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:41.003140  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:41.005169  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.001288  8972 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:41.006203 13369 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:40.930096 13457 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:40.954810 13355 raft_consensus.cc:740] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:41.007373 13278 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:40.925354 13061 consensus_queue.cc:1059] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 13:03:40.996341 12670 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:41.003654 13240 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:40.954449 13164 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:41.032131  8327 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:03:41.027657  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:41.033690  8972 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:41.014475 13457 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:41.061755 13474 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:41.062559 13369 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:41.064052 13369 raft_consensus.cc:697] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 15 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:41.064375 13503 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:41.065102 13369 consensus_queue.cc:237] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:41.069365  8972 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:41.080416  8400 raft_consensus.cc:1240] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:03:41.076653  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:41.083612  8972 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:41.023285 13400 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:41.084523  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:41.085215  8331 raft_consensus.cc:1275] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:03:41.086645  8327 raft_consensus.cc:2410] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:03:41.035863 13299 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:41.035509 13275 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:41.095559  8295 leader_election.cc:400] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13.
I20260709 13:03:41.096174  8295 leader_election.cc:403] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:41.032423 13470 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
W20260709 13:03:41.104961 13482 log.cc:927] Time spent T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:03:41.106622 13494 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.107147 13494 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:41.101006  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:41.102130 13485 log.cc:927] Time spent T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:03:41.087677 13355 consensus_queue.cc:260] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:41.073594 13240 raft_consensus.cc:697] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:41.036773 12670 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:41.083992 13503 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:41.131053  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 53143.4us
I20260709 13:03:41.135509  9005 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 15 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:41.136725  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:41.140302  8298 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:41.144454 13369 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:41.145255  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:41.145591 13376 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:03:41.146622 13489 log.cc:927] Time spent T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:03:41.147334  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:41.147760 13369 raft_consensus.cc:697] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:41.148460  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:41.149055  9008 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:41.146577  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:41.150066  9005 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 14: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:41.114252 13196 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.156639  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:41.157807  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:41.162772  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:41.163709  8255 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:41.164953  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:41.122388 13299 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:41.198779  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:41.199343  8250 raft_consensus.cc:1240] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:41.200192  8253 raft_consensus.cc:1240] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:03:41.124558 13407 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:41.205654  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:41.170022  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:41.212229  9005 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:03:41.212818  8254 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:41.213038  8250 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:03:41.169046  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:41.178989  8255 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:41.168108  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:41.124711 13470 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13.
I20260709 13:03:41.209571  8256 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:41.214011  8257 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:41.217067  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:41.148710 13369 consensus_queue.cc:237] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:41.220772 13483 log.cc:927] Time spent T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
W20260709 13:03:41.222405  8282 heartbeater.cc:646] Failed to heartbeat to 127.6.255.62:40337 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.6.255.62:40337 timed out after 15.000s (SENT)
I20260709 13:03:41.222208 13370 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:41.223505  9007 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:41.224586  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:41.154428 13471 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:41.230031  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:41.229259  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:41.229741  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:41.268675  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:41.197479 13456 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:41.205538 13452 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:41.224612 13479 log.cc:927] Time spent T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.210s	user 0.000s	sys 0.000s
I20260709 13:03:41.270733  8257 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:03:41.271370 13404 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:41.271938  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:41.324357  8248 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:41.272130 12670 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:41.273661  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:41.327144  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:41.275894 13484 log.cc:927] Time spent T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:03:41.329512 13164 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:41.330128 12973 consensus_meta.cc:287] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.741s	user 0.014s	sys 0.000s
I20260709 13:03:41.330520 12973 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16.
I20260709 13:03:41.159536 13196 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:41.331533 13262 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:41.332507 13196 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:41.279737 13488 log.cc:927] Time spent T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 13:03:41.333534 13404 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:41.335655 13404 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:41.335775  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:41.336391  9008 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:03:41.211956 13299 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:41.270066 13471 raft_consensus.cc:697] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:41.304155 13274 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:41.209923 13275 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:41.311851  8254 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:03:41.312389  8255 raft_consensus.cc:3055] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:41.376190  8255 raft_consensus.cc:740] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:03:41.313478  8298 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.4:33825 timed out after 1.553s (SENT) [suppressed 17 similar messages]
I20260709 13:03:41.317857  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:41.378921  8250 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:03:41.317981 13061 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:41.318616 13456 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:41.320024 13494 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:41.322029  8400 raft_consensus.cc:1275] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:03:41.338609  8222 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:41.342267  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:41.224989 12868 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.344187 13486 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:41.331547 13157 consensus_meta.cc:287] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.683s	user 0.006s	sys 0.003s
I20260709 13:03:41.346993 13370 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:41.347362 13001 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:41.350241 13477 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.351218  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:41.351904 13156 consensus_meta.cc:287] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.571s	user 0.008s	sys 0.003s
W20260709 13:03:41.353880 13367 consensus_meta.cc:287] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.565s	user 0.010s	sys 0.000s
I20260709 13:03:41.355594 13355 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:41.323338 13381 raft_consensus.cc:3055] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:41.360001 13315 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:41.361496  8249 raft_consensus.cc:1240] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:41.375041  8248 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:41.377161  8255 consensus_queue.cc:260] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:41.316044 13452 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:41.380882  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:41.383479  8254 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:41.281291 13382 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:41.384924  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:41.385563  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 47091.9us
I20260709 13:03:41.434167  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:41.434191  8222 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:41.436592  8222 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:41.434873  8250 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:03:41.438354  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:41.440240  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.440791  8254 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:03:41.442384  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:41.383388 13275 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:41.388115  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2727) took 1058 ms. Trace:
W20260709 13:03:41.391108  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.391789  8253 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:03:41.398943  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:41.451769  8251 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:03:41.452212  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:41.383925 13061 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:41.401612 13477 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:41.386612 12868 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:41.393471 13157 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:41.460593 13157 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:41.402655 13156 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:41.462718 13156 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:41.462886 13061 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:41.463891  8397 raft_consensus.cc:1240] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:41.465165  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 762) took 1091 ms. Trace:
I20260709 13:03:41.465534  8397 rpcz_store.cc:276] 0709 13:03:40.373856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.613844 (+239988us) service_pool.cc:224] Handling call
0709 13:03:40.635725 (+ 21881us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.464321 (+828596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.464933 (+   612us) spinlock_profiling.cc:243] Waited 828 ms on lock 0x7b5400277254. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:41.465104 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1903948544}
I20260709 13:03:41.467186  8401 raft_consensus.cc:1240] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:41.270205 13240 consensus_queue.cc:237] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:41.468528 13477 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:41.387959 13455 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:41.428443  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:41.470896  8248 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:03:41.471853  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:41.472396 13457 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:41.472930  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:41.429621  8255 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:41.448105  8372 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:41.444352  8254 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:41.385797  8257 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:03:41.443958 13463 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:41.481710  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:41.416576 13381 raft_consensus.cc:740] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:41.450433  8404 raft_consensus.cc:1240] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:41.442701 13308 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:41.450214  8406 rpcz_store.cc:276] 0709 13:03:40.330144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.582850 (+252706us) service_pool.cc:224] Handling call
0709 13:03:40.583200 (+   350us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:40.596034 (+ 12834us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:40.596068 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:40.596797 (+   729us) log.cc:844] Serialized 52 byte log entry
0709 13:03:40.808347 (+211550us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:40.808400 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.808499 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:41.387883 (+579384us) raft_consensus.cc:1685] finished
0709 13:03:41.387907 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.388057 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9125,"raft.queue_time_us":11226,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":4017,"thread_start_us":182708,"threads_started":1,"wal-append.queue_time_us":319875}
I20260709 13:03:41.456382  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:41.488729  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:41.489841  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:41.490311  8253 raft_consensus.cc:2393] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 15: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:41.458772 13164 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.476511  8252 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:41.477821  8257 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:03:41.438935  8250 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:41.390426 13486 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:41.479912  8248 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
W20260709 13:03:41.484130  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 170670us
I20260709 13:03:41.484225  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:41.426148 13367 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:41.454376 13299 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:41.498126 13376 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.499326 13382 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:41.514122  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:41.515110  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.499644 13367 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:41.499732 12868 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:41.500260  9005 raft_consensus.cc:2393] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:41.501143  8251 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:41.520480 13141 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.496949 13164 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:41.520988 13141 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:41.523131 13164 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:41.484298 13061 consensus_queue.cc:1059] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:03:41.524577 13061 raft_consensus.cc:3055] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:41.524946 13061 raft_consensus.cc:740] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:41.525749 13061 consensus_queue.cc:260] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:41.526187  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:41.526559 13364 raft_consensus.cc:3055] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:41.485244 13463 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.526826  8332 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:41.527284 13463 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:41.502187 13457 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:41.507107  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:41.507316  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:03:41.508292 13473 log.cc:927] Time spent T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.726s	user 0.004s	sys 0.000s
I20260709 13:03:41.529567 13141 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:41.530390 13452 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:41.533267  9002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56954 (request call id 867) took 1150 ms. Trace:
I20260709 13:03:41.533660 13376 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:41.534842  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:41.535916  8331 raft_consensus.cc:2410] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:03:41.537077  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:41.538209 13456 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:41.538406  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:41.527118 13364 raft_consensus.cc:740] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:41.539204  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:41.539783 13364 consensus_queue.cc:260] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:41.540608  8330 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:03:41.541095 13364 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:41.510834  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 67155.1us
I20260709 13:03:41.504093 13486 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:41.509518  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2622) took 2648 ms. Trace:
I20260709 13:03:41.544010  8324 rpcz_store.cc:276] 0709 13:03:38.860926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.594207 (+733281us) service_pool.cc:224] Handling call
0709 13:03:39.594499 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.624634 (+1030135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.509309 (+884675us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540011d0d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:41.509470 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2361618688}
I20260709 13:03:41.538638  8327 raft_consensus.cc:1275] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:03:41.535866 13376 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:41.546093  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:41.551913  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:41.552253  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:41.432942 13471 consensus_queue.cc:237] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:41.553496  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:41.489497 13308 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:41.554212  8331 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:41.554656 13308 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:41.555356  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:41.486603 13453 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:41.557639 13453 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.558046 13453 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:41.558223  8328 raft_consensus.cc:1275] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:41.533674  9002 rpcz_store.cc:276] 0709 13:03:40.383378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.383558 (+   180us) service_pool.cc:224] Handling call
0709 13:03:41.533109 (+1149551us) spinlock_profiling.cc:243] Waited 77.6 ms on lock 0x7b54004c2554. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:41.533208 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":178495232}
I20260709 13:03:41.548962  8369 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:41.561645 12989 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:41.562454  8251 raft_consensus.cc:1275] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:41.563794  8369 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:41.568174  8254 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:03:41.571233  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:41.526995 13061 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:41.512871 13499 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.580013 13499 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:41.573820 13457 consensus_queue.cc:1048] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:03:41.573199 12989 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:41.583515  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:41.572742 13141 consensus_queue.cc:1059] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:03:41.585000  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:41.585193 13455 consensus_queue.cc:1059] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:41.585906  8405 raft_consensus.cc:2410] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:03:41.586587 12973 consensus_queue.cc:1059] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 14.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:03:41.587234 13278 consensus_meta.cc:287] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.563s	user 0.007s	sys 0.003s
I20260709 13:03:41.587558 12973 raft_consensus.cc:3055] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:41.587621 13278 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:41.588078 12973 raft_consensus.cc:740] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:41.588773 13384 raft_consensus.cc:3055] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:41.589255 13384 raft_consensus.cc:740] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:41.588079  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:41.590586 13141 raft_consensus.cc:3055] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:41.540678 13275 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:41.578751  8257 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:03:41.578658  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:41.575241  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:41.590190 13384 consensus_queue.cc:260] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:41.583349  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:41.603596  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:41.604187  8255 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:41.604554  8253 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:03:41.605084  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:41.576148  8251 raft_consensus.cc:1275] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:03:41.605491  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:41.587692  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:41.588958 12973 consensus_queue.cc:260] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:41.606988 13278 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:41.591002 13141 raft_consensus.cc:740] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:41.561915 13381 consensus_queue.cc:260] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:41.606781 13364 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:41.607393 12973 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:41.607965 13381 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:41.609010 13141 consensus_queue.cc:260] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:41.610277 13141 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:41.604454  8295 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:41.612507  8295 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:41.613101 13207 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:41.614042  8295 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:41.603761 13275 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.615106  8295 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:41.614964 13275 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:41.616017 13487 log.cc:927] Time spent T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.593s	user 0.002s	sys 0.001s
I20260709 13:03:41.616397  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:41.616614 13364 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:41.617082 13275 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:41.618597  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:41.625741  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:41.624213  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:41.619499  9003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 1004) took 1802 ms. Trace:
I20260709 13:03:41.631772  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:41.630383  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:41.607306  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:41.638746  8251 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:41.613360  8252 raft_consensus.cc:1275] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:41.626329  8402 raft_consensus.cc:1240] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
W20260709 13:03:41.641721  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.643433  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:41.644032  9002 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:03:41.630851  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:41.646400  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:41.583457 12989 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.647344  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:41.656675  8398 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:41.657941  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:41.658675  8406 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:41.648471  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:41.640792  9003 rpcz_store.cc:276] 0709 13:03:39.816784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.816933 (+   149us) service_pool.cc:224] Handling call
0709 13:03:39.818622 (+  1689us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:39.841796 (+ 23174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:39.841841 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:39.842403 (+   562us) log.cc:844] Serialized 26 byte log entry
0709 13:03:40.818621 (+976218us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b18004123c0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e0d1 00007f8828779a0f
0709 13:03:40.885667 (+ 67046us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:40.885701 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.885758 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:41.615466 (+729708us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:41.619232 (+  3766us) raft_consensus.cc:1685] finished
0709 13:03:41.619262 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.619433 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3223,"raft.queue_time_us":5573,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":20909,"spinlock_wait_cycles":73394816,"thread_start_us":66873,"threads_started":1,"wal-append.queue_time_us":1097174}
W20260709 13:03:41.649266  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:41.652558  8142 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 81804f59e3124d27998ea80fc1c36663 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica f865c4d761f5446eb12b3a029234772e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 18 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:41.605134  8369 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:41.655138  8295 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:41.661795  8298 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:41.671172  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:41.670002  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:41.665958  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:41.679888  8972 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:41.680475  9000 raft_consensus.cc:2376] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14. Re-sending same reply.
I20260709 13:03:41.681260  9002 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:41.616436 13200 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:03:41.705089  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 43553.4us
W20260709 13:03:41.704654  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 49621.7us
I20260709 13:03:41.721302  8295 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:41.703985  8401 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 11: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:41.705910  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:41.706278  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 103270us
W20260709 13:03:41.713793  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.622506 13499 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:41.618125 13355 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:41.617336 13364 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:41.660809 12989 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:41.650754 13486 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:03:41.725710  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.733824  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:41.761010  9007 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:41.760890  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:41.625648 13455 consensus_queue.cc:1059] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.091s
I20260709 13:03:41.704404 13163 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.762781  8295 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:41.704254 13275 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:41.650514 13400 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.085s
I20260709 13:03:41.765766  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:41.718185 13438 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:41.772619  8406 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:03:41.777369  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:41.777735  8330 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:03:41.778126  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 51333.5us
I20260709 13:03:41.778371  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:41.799223  8369 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:41.647604 13424 consensus_queue.cc:1048] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.084s
I20260709 13:03:41.706146 13384 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:41.727913 13477 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:03:41.748064  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.731037 13355 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:41.767858 13469 log.cc:927] Time spent T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:03:41.704604 13471 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 13:03:41.804522  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:41.765281 13200 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:41.789242 13274 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.760188 13486 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:41.808153  9002 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:03:41.835937  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:41.816154  8331 raft_consensus.cc:1275] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:03:41.845510  8331 pending_rounds.cc:85] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 8
I20260709 13:03:41.845932  8331 pending_rounds.cc:107] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:03:41.846242  8331 raft_consensus.cc:2889] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:41.816610  8328 raft_consensus.cc:1275] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:41.786208 13275 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.729158 13356 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 13:03:41.734145 13302 consensus_queue.cc:1059] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.070s
I20260709 13:03:41.797367 13499 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.865497 13499 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:41.818781  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 138957us
I20260709 13:03:41.843336  8327 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:03:41.845392  8399 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:41.867465  8972 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:41.789441 13370 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:41.870092 13370 raft_consensus.cc:697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:41.871025 13370 consensus_queue.cc:237] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:41.871758 13376 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:41.872414 13376 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:41.873221  8369 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:41.874648  8369 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:41.848194  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.789579 13207 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:03:41.880035 13207 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:41.880841 13207 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:41.812449 13518 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:41.881647 13518 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.882133 13518 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:41.884313  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 228695us
I20260709 13:03:41.781771 13369 consensus_queue.cc:1059] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
I20260709 13:03:41.889946 13376 raft_consensus.cc:3055] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:41.890341 13376 raft_consensus.cc:740] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:41.891191 13376 consensus_queue.cc:260] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:41.892498 13376 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:41.833788 13157 consensus_queue.cc:1048] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.060s
W20260709 13:03:41.891733 13513 log.cc:927] Time spent T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.445s	user 0.000s	sys 0.002s
I20260709 13:03:41.897349 13465 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:41.898502  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 759) took 1525 ms. Trace:
I20260709 13:03:41.898869  8403 rpcz_store.cc:276] 0709 13:03:40.373042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.585750 (+212708us) service_pool.cc:224] Handling call
0709 13:03:40.586118 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:40.600330 (+ 14212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:40.600362 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:41.336982 (+736620us) spinlock_profiling.cc:243] Waited 1.58 ms on lock 0x7b2c000e1c48. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:03:41.345899 (+  8917us) log.cc:844] Serialized 26 byte log entry
0709 13:03:41.444004 (+ 98105us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:41.444039 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.444109 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:41.891163 (+447054us) spinlock_profiling.cc:243] Waited 1.69 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:41.898286 (+  7123us) raft_consensus.cc:1685] finished
0709 13:03:41.898309 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.898444 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37884,"raft.queue_time_us":14799,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":8434,"spinlock_wait_cycles":7531648,"thread_start_us":97661,"threads_started":1,"wal-append.queue_time_us":100691}
I20260709 13:03:41.900767  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:41.811960 13196 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:41.901858  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:41.791625 13163 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:41.904135 13163 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:41.905063 13517 log.cc:927] Time spent T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
W20260709 13:03:41.905862  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.907500  8399 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:03:41.908625  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:41.909071  8403 raft_consensus.cc:1275] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:03:41.910175 13520 log.cc:927] Time spent T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:03:41.815078 13355 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:41.888908  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:41.814411 13477 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:41.938243  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 60983.9us
W20260709 13:03:41.942046  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 168280us
W20260709 13:03:41.849401 13503 consensus_meta.cc:287] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.699s	user 0.009s	sys 0.001s
I20260709 13:03:41.958297  8398 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:41.959550  8297 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:41.811210 13404 consensus_queue.cc:1048] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.110s
I20260709 13:03:41.849143 13486 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:41.849867 13200 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:41.961043  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 298242us
I20260709 13:03:41.961371  8369 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:41.859618 13275 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:41.972703  8220 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:41.972805  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:41.943648 13477 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:41.992681  8369 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:41.856015 13274 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:41.962950 13503 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:41.971714 13200 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:42.016988 13157 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:03:41.994693 13196 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:42.040915  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2648) took 1236 ms. Trace:
I20260709 13:03:42.051587  8325 rpcz_store.cc:276] 0709 13:03:40.804469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.804847 (+   378us) service_pool.cc:224] Handling call
0709 13:03:40.805095 (+   248us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:40.805407 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:40.806946 (+  1539us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:40.807350 (+   404us) log.cc:844] Serialized 26 byte log entry
0709 13:03:41.654431 (+847081us) spinlock_profiling.cc:243] Waited 4.94 ms on lock 0x7b18003b2d80. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e0d1 00007f8828779a0f
0709 13:03:41.819638 (+165207us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:41.819681 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.819772 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:42.040605 (+220833us) raft_consensus.cc:1685] finished
0709 13:03:42.040625 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:42.040846 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":234,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":17512,"spinlock_wait_cycles":11358976,"thread_start_us":164997,"threads_started":1,"wal-append.queue_time_us":1177078}
I20260709 13:03:42.042158  8403 raft_consensus.cc:1240] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:41.984288 13438 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:42.037500 13207 consensus_queue.cc:1048] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:03:41.995594 13516 log.cc:927] Time spent T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 13:03:42.092900  9004 raft_consensus.cc:1275] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:42.028076 13164 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:42.056866 13521 log.cc:927] Time spent T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:03:42.089262  8255 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:42.030247 13387 consensus_queue.cc:1048] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 13:03:42.018349 13308 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:42.111048 13519 log.cc:927] Time spent T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.250s	user 0.001s	sys 0.000s
W20260709 13:03:42.138899  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 409394us
I20260709 13:03:42.159050  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2380) took 1594 ms. Trace:
I20260709 13:03:42.160343  8223 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:42.159601  8323 rpcz_store.cc:276] 0709 13:03:40.565092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.565365 (+   273us) service_pool.cc:224] Handling call
0709 13:03:40.565830 (+   465us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:03:40.599057 (+ 33227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:40.599091 (+    34us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:03:40.601276 (+  2185us) log.cc:844] Serialized 104 byte log entry
0709 13:03:40.833377 (+232101us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:40.962962 (+129585us) log.cc:844] Serialized 14 byte log entry
0709 13:03:40.998357 (+ 35395us) log.cc:844] Serialized 14 byte log entry
0709 13:03:40.998703 (+   346us) log.cc:844] Serialized 14 byte log entry
0709 13:03:41.612756 (+614053us) spinlock_profiling.cc:243] Waited 4.11 ms on lock 0x7b1800270460. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:03:41.910948 (+298192us) log.cc:844] Serialized 14 byte log entry
0709 13:03:42.158446 (+247498us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b1800270460. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882564bc0a 00007f882ed8b219 00007f88256d3a40 00007f88256ce4e4 00007f88256ce455 00007f88287b6060 00007f88287c8716 00007f88287d9287 00007f88287d9208
0709 13:03:42.158670 (+   224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.158741 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:42.158788 (+    47us) raft_consensus.cc:1685] finished
0709 13:03:42.158804 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:42.158977 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1132603,"prepare.queue_time_us":3095550,"prepare.run_cpu_time_us":148,"prepare.run_wall_time_us":4181,"raft.queue_time_us":330074,"raft.run_cpu_time_us":517,"raft.run_wall_time_us":26038,"spinlock_wait_cycles":15964800,"thread_start_us":476879,"threads_started":4,"wal-append.queue_time_us":310441}
W20260709 13:03:42.114018 13381 consensus_meta.cc:287] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.504s	user 0.001s	sys 0.010s
I20260709 13:03:42.164181  8223 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:42.118413 13463 consensus_meta.cc:287] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.591s	user 0.010s	sys 0.000s
I20260709 13:03:42.095009 13518 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:42.119164 13196 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.191471  9006 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:42.191960  8223 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:42.194037  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:42.198009  8404 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.005s	sys 0.018s
W20260709 13:03:42.198346  8404 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.005s	sys 0.018s
W20260709 13:03:42.198367  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.113981 13001 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:42.113866 13528 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:03:42.195711  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 223065us
I20260709 13:03:42.118216 13207 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:42.204107  8298 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:42.190845 13475 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:42.215163  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 198793us
I20260709 13:03:42.230018  8399 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:03:42.205909  8220 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:42.158052 13163 consensus_queue.cc:1048] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:03:42.245944  8220 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:42.246661  8974 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:42.206041 13528 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.255390  9001 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:42.217641 13262 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:42.208232 13315 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:42.264101  8974 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:42.229087 13499 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:42.252106 13537 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:42.272351 13262 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:42.273039 13262 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.273509 13262 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:42.274674 13536 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:42.276684  8974 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:42.277627 13536 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:42.278090 13536 raft_consensus.cc:697] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 14 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:42.278939 13536 consensus_queue.cc:237] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:42.279912 12989 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:42.281286  8400 raft_consensus.cc:2368] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 13. Current term is 14.
I20260709 13:03:42.281700  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:42.282673  8974 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:42.283679 13503 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:42.283820 13452 raft_consensus.cc:3055] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:42.284224 13452 raft_consensus.cc:740] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:03:42.284929 13453 consensus_meta.cc:287] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.726s	user 0.009s	sys 0.000s
I20260709 13:03:42.226845 13207 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:42.234694 13306 consensus_queue.cc:1059] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 13:03:42.205371 13196 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:42.238051 13157 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:03:42.238705 13370 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:42.195199 13463 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:42.285126 13262 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:42.286666 13503 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:03:42.324990 13262 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:42.328069 13355 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:42.228116 13001 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:42.331570 13001 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:42.333668  8256 raft_consensus.cc:2410] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 13:03:42.335772 13530 log.cc:927] Time spent T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
W20260709 13:03:42.288352 13141 consensus_meta.cc:287] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.678s	user 0.008s	sys 0.000s
I20260709 13:03:42.285389 13453 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:42.346418  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 423956us
I20260709 13:03:42.261976 13528 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:42.347998  8297 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:42.288796 12973 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.302067 12868 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:42.303783 13308 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:42.319077 13306 consensus_queue.cc:1243] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:03:42.317684 13278 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:42.313519 13370 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.285044 13452 consensus_queue.cc:260] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:42.295758 13424 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:42.343613 13400 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:42.352855 13367 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.315150 13315 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:42.393754  8397 raft_consensus.cc:2393] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 16: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:03:42.361625  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 163448us
I20260709 13:03:42.417021  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:42.417750  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:42.418450  8255 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:42.420735 13471 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.421347 13471 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:42.423520 13471 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:42.386281 13356 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 13:03:42.401878  9004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 1013) took 1176 ms. Trace:
I20260709 13:03:42.435385  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:03:42.430131  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 82240us
I20260709 13:03:42.426994  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:42.436836  8255 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:42.427734  8250 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:03:42.435150  9004 rpcz_store.cc:276] 0709 13:03:41.225783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.226572 (+   789us) service_pool.cc:224] Handling call
0709 13:03:41.226804 (+   232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.093582 (+866778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.401522 (+307940us) spinlock_profiling.cc:243] Waited 830 ms on lock 0x7b5400527bd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:42.401793 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1909827968}
I20260709 13:03:42.438987  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:42.416632  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:42.440004  8972 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:42.442787  8250 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:42.443145  9009 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:42.444547  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:42.444744  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:42.445557  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:42.447614  8249 raft_consensus.cc:2393] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 14: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:42.449707  8255 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:42.450762  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:42.451318  8255 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:42.398628 13370 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:42.386142 13475 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:42.474727  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 228839us
W20260709 13:03:42.343879 13020 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.008s	sys 0.026s
W20260709 13:03:42.457659  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 258634us
I20260709 13:03:42.456821  9006 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:03:42.465320  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:42.484119  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:42.392288 13382 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.425416 13438 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:42.486145  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 49431.9us
I20260709 13:03:42.579469  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:42.579963  8297 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:42.391705 13274 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:42.580755  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:42.553985  9005 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:42.551534  8220 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:42.448580 12989 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:42.453107 13455 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.397469 13278 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:42.452417 13361 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:03:42.640154  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 88709.6us
I20260709 13:03:42.453992 13001 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:42.580790  8297 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:42.487238 13020 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.009s	sys 0.027s
I20260709 13:03:42.559636  8248 pending_rounds.cc:85] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 6
I20260709 13:03:42.388998 12973 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:42.485386 13452 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:42.415691 13400 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:42.398587 13275 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:42.448419 13376 consensus_queue.cc:1059] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 9.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.136s
I20260709 13:03:42.774781  8372 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:42.427196 13367 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:42.731506  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 150762us
I20260709 13:03:42.706135 13361 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:42.732308  8248 pending_rounds.cc:107] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:03:42.788935  8248 raft_consensus.cc:2889] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:42.581141 13551 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:42.719125 13503 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:42.637053 13544 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.740036  8222 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:42.741704  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:42.642267 13518 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:42.549203 13549 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.478579 13540 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:42.735945  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 270660us
I20260709 13:03:42.807404  8250 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
W20260709 13:03:42.737681 13384 consensus_meta.cc:287] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.923s	user 0.009s	sys 0.004s
I20260709 13:03:42.809056  8404 raft_consensus.cc:2393] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:42.735141 13356 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:42.815161  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 75194.7us
I20260709 13:03:42.842084  9007 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:03:42.641131 13499 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.147s
I20260709 13:03:42.838722  9005 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:03:42.851115  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 772) took 1146 ms. Trace:
I20260709 13:03:42.838099  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:42.636313 13382 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:42.851766  8250 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:42.851500  8400 rpcz_store.cc:276] 0709 13:03:41.704551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.711925 (+  7374us) service_pool.cc:224] Handling call
0709 13:03:42.840681 (+1128756us) spinlock_profiling.cc:243] Waited 484 ms on lock 0x7b54001cc354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:42.851060 (+ 10379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1112985472}
W20260709 13:03:42.830685 13164 consensus_meta.cc:287] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.726s	user 0.005s	sys 0.005s
I20260709 13:03:42.884773 13164 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:42.885787 13164 consensus_queue.cc:1048] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:42.827907 13549 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:42.889818 13549 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:42.793684 13361 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:42.789157 13543 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:42.886010 13157 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:42.705461 13455 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:42.789268 13355 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:42.885926 13384 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:42.738955 13001 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:42.923570 13547 log.cc:927] Time spent T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.205s	user 0.000s	sys 0.000s
I20260709 13:03:42.925143 13001 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:42.925802 13545 log.cc:927] Time spent T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.528s	user 0.000s	sys 0.001s
I20260709 13:03:42.926630 13275 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:42.927814 12973 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:42.928947 13367 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:42.930481 13278 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:42.841836 13207 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:42.844213 13518 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:42.931417 13455 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:42.931602 13518 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:42.936064  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:42.920743  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:42.936836  8406 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:42.937263  9005 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:42.937911  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:42.933501  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 747) took 1031 ms. Trace:
I20260709 13:03:42.944327  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:42.945545  9004 raft_consensus.cc:2393] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 12: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:42.940068  8222 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:03:42.947193  8369 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.015s	sys 0.011s
W20260709 13:03:42.947868  8369 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.015s	sys 0.011s
I20260709 13:03:42.878284 13463 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:42.898157 13551 raft_consensus.cc:697] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:42.965996  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:42.969148  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.953745 13550 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:42.943516  8324 rpcz_store.cc:276] 0709 13:03:41.902253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.917347 (+ 15094us) service_pool.cc:224] Handling call
0709 13:03:41.917820 (+   473us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:41.980929 (+ 63109us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:41.980978 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:41.984526 (+  3548us) log.cc:844] Serialized 52 byte log entry
0709 13:03:42.234290 (+249764us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:42.234341 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.235248 (+   907us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:42.933220 (+697972us) raft_consensus.cc:1685] finished
0709 13:03:42.933245 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:42.933410 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":204178,"raft.queue_time_us":18280,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":12634,"thread_start_us":203247,"threads_started":1,"wal-append.queue_time_us":297099}
I20260709 13:03:42.919641 13196 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:42.893021 13471 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:03:42.815762 13544 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:42.960176 13564 log.cc:927] Time spent T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 13:03:42.952757 13486 consensus_meta.cc:287] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.968s	user 0.007s	sys 0.005s
I20260709 13:03:42.991029  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:42.994189  9005 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:03:42.994979  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:42.989414 13240 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:43.000204  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:43.017764  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:43.024148  8298 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:43.019606  8257 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:03:42.965262 13384 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:42.920416 13157 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:43.029945  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:43.022590  8328 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 9.
I20260709 13:03:43.048203  8250 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
I20260709 13:03:43.053884  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2263) took 1431 ms. Trace:
I20260709 13:03:43.054433  8405 rpcz_store.cc:276] 0709 13:03:41.622339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.637522 (+ 15183us) service_pool.cc:224] Handling call
0709 13:03:41.701357 (+ 63835us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:41.912746 (+211389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:41.912775 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:41.969108 (+ 56333us) log.cc:844] Serialized 26 byte log entry
0709 13:03:42.259761 (+290653us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:42.260034 (+   273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.017007 (+756973us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54002be654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:43.017061 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:43.017103 (+    42us) raft_consensus.cc:1685] finished
0709 13:03:43.017120 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.053810 (+ 36690us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":634,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":4880,"spinlock_wait_cycles":486221312,"thread_start_us":268486,"threads_started":1,"wal-append.queue_time_us":921338}
I20260709 13:03:43.061578 13453 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:43.029294 13207 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:03:43.028965 13384 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.064711 13384 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:43.065936 13551 consensus_queue.cc:237] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:43.028789 13486 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:43.070056 13400 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:43.070834  9002 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:43.071831 13486 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:43.072296 13382 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:42.828262 13540 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:42.995105 13315 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:43.075795  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 75647.9us
I20260709 13:03:43.098271  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:43.126732  8369 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:42.986970 13550 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.028854 13275 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.079890 13499 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:43.088743 13299 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:43.095369 13486 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.149325  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:43.150060  9006 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:43.149008 13299 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:43.151011 13571 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.151458 13571 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:43.153329  8372 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:43.133553  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:43.154371  9004 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:43.089143 13570 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:43.135282  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:03:43.134658  8370 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.793s	user 0.005s	sys 0.013s
I20260709 13:03:43.136759  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:43.155344  8370 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.005s	sys 0.013s
I20260709 13:03:43.144730  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:43.149768 13486 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:43.156703  8405 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:03:43.007233 13558 log.cc:927] Time spent T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:03:43.006268 13361 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:43.161456  8406 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:03:42.998704 13200 consensus_meta.cc:287] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.938s	user 0.012s	sys 0.000s
I20260709 13:03:43.130230 13207 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:43.158152  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:43.159554  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:43.169127  8404 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:03:43.163000  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:43.132481 13550 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:43.164089  8974 leader_election.cc:400] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 13. Current term is 14.
I20260709 13:03:43.166565  8372 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:03:43.174194  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 364020us
I20260709 13:03:43.180099  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:43.180893  8406 raft_consensus.cc:2410] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 17 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260709 13:03:43.181150  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.182047  8404 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:03:43.182425 13400 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:43.182950 13400 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:43.183537 13471 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:43.184033 13471 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:43.184659  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:43.185002 13400 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:43.185567 13315 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:43.185786  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:43.186231 13315 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:43.187143  8331 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:03:43.188220 13315 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:43.191990  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:43.193873 13471 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:43.201973  9009 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:43.158799 13518 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:43.156042 13499 raft_consensus.cc:697] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 16 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:03:43.294857  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 98452.9us
I20260709 13:03:43.344336  8220 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:43.170310 13561 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:43.148379 13275 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:43.172751 13207 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:43.196030 13196 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:43.185606 13400 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:43.162583 13557 consensus_queue.cc:1048] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.193s
I20260709 13:03:43.163275 13560 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:43.366171 13455 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:43.371321 13574 log.cc:927] Time spent T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.239s	user 0.000s	sys 0.000s
I20260709 13:03:43.374822 13463 consensus_queue.cc:1059] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:43.424830  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:43.307354 13299 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:43.376292 13527 raft_consensus.cc:3055] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:03:43.378404 13477 consensus_meta.cc:287] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.384s	user 0.010s	sys 0.002s
I20260709 13:03:43.378690 13361 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:43.355561 13578 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.184s
I20260709 13:03:43.366657 13001 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:43.380120 13499 consensus_queue.cc:237] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:43.382493 13571 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:43.384737 13384 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:43.386710 13486 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:43.387125 13544 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:43.389899  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 777) took 1188 ms. Trace:
I20260709 13:03:43.518688  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:43.519470  8328 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
W20260709 13:03:43.390205 13370 consensus_meta.cc:287] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.901s	user 0.010s	sys 0.000s
I20260709 13:03:43.391309 13457 raft_consensus.cc:3055] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:43.525585 13457 raft_consensus.cc:740] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:43.526535 13457 consensus_queue.cc:260] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:43.527822 13457 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:43.396272 13315 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:43.528996 13315 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:43.531283 13315 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:43.367106 13376 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.532226 13376 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:43.400985 13275 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:43.511173 13001 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:43.402592 13157 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:43.404838  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2749) took 1670 ms. Trace:
I20260709 13:03:43.536770 13001 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:43.535825  8401 rpcz_store.cc:276] 0709 13:03:41.734274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.911911 (+177637us) service_pool.cc:224] Handling call
0709 13:03:42.039035 (+127124us) spinlock_profiling.cc:243] Waited 4.31 ms on lock 0x7b5800170798. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:03:42.039318 (+   283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:42.233458 (+194140us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:42.254774 (+ 21316us) log.cc:844] Serialized 14 byte log entry
0709 13:03:42.485801 (+231027us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:42.577370 (+ 91569us) log.cc:844] Serialized 26 byte log entry
0709 13:03:42.577544 (+   174us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:42.577585 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.577650 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:43.404564 (+826914us) raft_consensus.cc:1685] finished
0709 13:03:43.404658 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.404783 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10431,"prepare.queue_time_us":25005,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":178808,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":29544,"spinlock_wait_cycles":9915264,"thread_start_us":344146,"threads_started":2,"wal-append.queue_time_us":1034062}
I20260709 13:03:43.538079  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:43.404851 13543 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.405511 12973 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:43.540336 13543 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:43.540547 12973 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:43.541695  8256 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2181) took 2282 ms (client timeout 1528 ms). Trace:
W20260709 13:03:43.542094  8256 rpcz_store.cc:269] 0709 13:03:41.259138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.268390 (+  9252us) service_pool.cc:224] Handling call
0709 13:03:43.541244 (+2272854us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400308c54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:43.541398 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2446827392}
I20260709 13:03:43.542644 12973 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:43.543407 12973 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.543938 12973 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:43.546128  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2206) took 1116 ms. Trace:
I20260709 13:03:43.546598  8248 rpcz_store.cc:276] 0709 13:03:42.430127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.455487 (+ 25360us) service_pool.cc:224] Handling call
0709 13:03:42.559460 (+103973us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:42.789675 (+230215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:42.789720 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:42.790237 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 13:03:42.972881 (+182644us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:42.972934 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.973054 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:43.545861 (+572807us) raft_consensus.cc:1685] finished
0709 13:03:43.545888 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.546045 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":182303,"threads_started":1,"wal-append.queue_time_us":304053}
I20260709 13:03:43.173671 13382 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:43.550015 13382 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.550454 13382 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:43.552217  8402 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:03:43.499027 13477 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:43.173729 13554 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:03:43.440580  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.478497  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:43.365126 13207 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:43.353358 13274 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:43.602056 13274 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:43.604305 13274 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:43.606288  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:43.607427  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:43.608013  8404 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:03:43.608188  8253 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:43.610162  8297 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:43.498005  9009 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:43.171911 13200 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:43.506350  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2205) took 1076 ms. Trace:
I20260709 13:03:43.613726  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:43.613202  8255 rpcz_store.cc:276] 0709 13:03:42.429886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.451855 (+ 21969us) service_pool.cc:224] Handling call
0709 13:03:42.452067 (+   212us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:42.458453 (+  6386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:42.458488 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:42.484997 (+ 26509us) log.cc:844] Serialized 26 byte log entry
0709 13:03:42.988656 (+503659us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:43.502259 (+513603us) log.cc:844] Serialized 14 byte log entry
0709 13:03:43.505742 (+  3483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.505932 (+   190us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:43.506014 (+    82us) raft_consensus.cc:1685] finished
0709 13:03:43.506063 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.506281 (+   218us) inbound_call.cc:177] Queueing success response
0709 13:03:43.599683 (+ 93402us) spinlock_profiling.cc:243] Waited 2.26 ms on lock 0x7b4800002dd8. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882d6e7aa1 00007f882d6deb37 00007f882d6de9ca 00007f882d7d626f 00007f882d7d5f37 00007f882d7ea55c 00007f882d7ea508 00007f882d7ea488 00007f882d7ea443 00007f882d7e96ec 00007f882c713a96
Metrics: {"mutex_wait_us":8444,"raft.queue_time_us":8732,"raft.run_cpu_time_us":2720,"raft.run_wall_time_us":1125191,"spinlock_wait_cycles":5204864,"thread_start_us":629991,"threads_started":2,"wal-append.queue_time_us":599727}
I20260709 13:03:43.368113 13196 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:43.415197 13557 consensus_queue.cc:1243] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer 8c42259d7c4b463a9f485f42c320c81b log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:03:43.361164 13518 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.143s
I20260709 13:03:43.439711 13527 raft_consensus.cc:740] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:43.518379  8403 rpcz_store.cc:276] 0709 13:03:42.201410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.201967 (+   557us) service_pool.cc:224] Handling call
0709 13:03:42.202320 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.346894 (+144574us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:42.346933 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:42.346986 (+    53us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:42.347022 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.389166 (+1042144us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54001e7654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:43.389214 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.389364 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":332011776}
W20260709 13:03:43.511705 13452 consensus_meta.cc:287] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.731s	user 0.009s	sys 0.002s
I20260709 13:03:43.542554 13543 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:43.425938  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 258443us
I20260709 13:03:43.639932  8254 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
W20260709 13:03:43.556686 13562 log.cc:927] Time spent T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:03:43.627517  8297 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:43.433475 13400 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:43.639885  8369 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:43.642189  8223 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:43.591910  8332 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 13:03:43.653666  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:43.657264  8323 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:43.608009 13384 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:43.631035 13196 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:43.623903 13356 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:03:43.712015  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 69943.1us
I20260709 13:03:43.629484 13494 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:43.713264  8223 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:43.718788  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:43.721670  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:43.722376  8255 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:43.723536  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:43.725695  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:43.726423  8256 raft_consensus.cc:2393] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 15: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:43.696487 13576 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.609055 13370 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:43.708824 13453 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:43.707723 13556 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:43.637317 13554 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:43.767015  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:43.710374 13540 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:43.711155 13551 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:43.639422 13315 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.093s
I20260709 13:03:43.791417  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2405) took 1557 ms. Trace:
I20260709 13:03:43.791965  8329 rpcz_store.cc:276] 0709 13:03:42.233650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.234756 (+  1106us) service_pool.cc:224] Handling call
0709 13:03:42.235074 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:42.239849 (+  4775us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:42.239879 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:42.252238 (+ 12359us) log.cc:844] Serialized 26 byte log entry
0709 13:03:42.472814 (+220576us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:42.472876 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.472975 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:43.791184 (+1318209us) raft_consensus.cc:1685] finished
0709 13:03:43.791206 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.791355 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1037,"raft.queue_time_us":1193,"spinlock_wait_cycles":6764928,"thread_start_us":167969,"threads_started":1,"wal-append.queue_time_us":893650}
I20260709 13:03:43.766593  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:43.658443 13387 consensus_queue.cc:1048] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.098s
I20260709 13:03:43.726380 13452 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:43.751386 13543 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:43.750725 13376 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:43.726598 13356 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:43.769279 13471 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:43.780586 13515 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.806831  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:43.807574  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:43.828670  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:43.803653  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:43.799860 13477 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:43.823232  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:43.786350 13453 raft_consensus.cc:697] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:03:43.811790 13528 consensus_meta.cc:287] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.421s	user 0.011s	sys 0.001s
W20260709 13:03:43.870013  8397 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 2748) took 2280 ms (client timeout 2103 ms). Trace:
W20260709 13:03:43.870572  8397 rpcz_store.cc:269] 0709 13:03:41.582901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.587826 (+  4925us) service_pool.cc:224] Handling call
0709 13:03:43.862600 (+2274774us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b5400276d54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:43.862836 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1847260416}
I20260709 13:03:43.769523 13576 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:43.818912 13452 raft_consensus.cc:697] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:43.826772 13570 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:43.779438 13556 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:43.764488 13527 consensus_queue.cc:260] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.11, Last appended by leader: 11, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:43.832554 13471 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:43.995076 13376 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:44.024933 13579 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:44.032821 13370 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:44.038436 13566 log.cc:927] Time spent T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.931s	user 0.009s	sys 0.020s
I20260709 13:03:43.831869 13515 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:43.869710 13528 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:44.046736 13356 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:44.048331 13586 log.cc:927] Time spent T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.182s	user 0.000s	sys 0.000s
W20260709 13:03:44.051134 13457 consensus_meta.cc:287] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.523s	user 0.011s	sys 0.000s
W20260709 13:03:44.127295  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 499568us
I20260709 13:03:44.136868 13274 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:43.839365  8256 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:03:44.073048 13452 consensus_queue.cc:237] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:44.072567 13453 consensus_queue.cc:237] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:44.029469 13200 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:44.089550 13567 log.cc:927] Time spent T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 1.092s	user 0.000s	sys 0.003s
I20260709 13:03:44.072816 13576 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:44.167824  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2204) took 1738 ms. Trace:
I20260709 13:03:44.017287 13570 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:44.056080 13579 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:44.170753 13579 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:44.171912 13607 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:44.172644  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:44.052590 13207 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:44.176322  8297 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:44.176831 13207 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:44.084060 13471 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:44.178551  8402 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14. Ops: []
W20260709 13:03:44.068939 13475 consensus_meta.cc:287] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.584s	user 0.006s	sys 0.007s
I20260709 13:03:44.179862 13538 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:44.074935 13556 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:44.072033 13515 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:44.080935 13528 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:44.086320 13550 consensus_meta.cc:287] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.791s	user 0.010s	sys 0.000s
I20260709 13:03:44.188808 13550 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:44.192144 13550 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:44.042358 13196 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:44.058302 13376 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:43.815387  8255 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:44.143824 13367 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.042460 13400 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:44.128633  8249 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:03:44.179975 13475 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:44.205690 13376 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:44.077975 13554 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:44.145175 13274 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:44.212589 13367 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:44.211544  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:44.180646 13538 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.168315  8252 rpcz_store.cc:276] 0709 13:03:42.429631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.447030 (+ 17399us) service_pool.cc:224] Handling call
0709 13:03:42.447305 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:42.447635 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:42.447674 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:42.448113 (+   439us) log.cc:844] Serialized 26 byte log entry
0709 13:03:42.731658 (+283545us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:42.731693 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.731757 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:44.088797 (+1357040us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:44.167105 (+ 78308us) raft_consensus.cc:1685] finished
0709 13:03:44.167158 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:44.167566 (+   408us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":869,"raft.queue_time_us":10375,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":6072,"spinlock_wait_cycles":25691520,"thread_start_us":283228,"threads_started":1,"wal-append.queue_time_us":488306}
I20260709 13:03:44.180655 13001 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:44.199519 13196 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:44.204164  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:44.215679 13367 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:44.217918  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:44.218623 13274 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:44.220672  8255 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:44.222236  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:44.092940 13546 log.cc:927] Time spent T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 1.142s	user 0.000s	sys 0.021s
I20260709 13:03:44.223361  8252 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:03:44.228081  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:44.229542  8252 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:44.224510 13538 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:44.239423 13382 consensus_meta.cc:287] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.688s	user 0.011s	sys 0.008s
I20260709 13:03:44.242106 13001 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:44.246362  9001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 1008) took 3126 ms. Trace:
I20260709 13:03:44.246754  9001 rpcz_store.cc:276] 0709 13:03:41.120240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.120374 (+   134us) service_pool.cc:224] Handling call
0709 13:03:41.120740 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.263557 (+1142817us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.245818 (+1982261us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54005cf8d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:44.246128 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2608692608}
I20260709 13:03:44.256160 12973 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:44.271476  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:44.273911 13494 consensus_queue.cc:1059] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:44.275861  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2401) took 2084 ms. Trace:
I20260709 13:03:44.276432 13400 raft_consensus.cc:3055] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:44.277026 13400 raft_consensus.cc:740] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:44.278662 13400 consensus_queue.cc:260] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:44.281244  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2400) took 2467 ms. Trace:
I20260709 13:03:44.288282  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:44.288857  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:44.289961  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:44.290820  8405 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:44.291256  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:44.276398  8323 rpcz_store.cc:276] 0709 13:03:42.191718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.196159 (+  4441us) service_pool.cc:224] Handling call
0709 13:03:42.196461 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:43.657809 (+1461348us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.275256 (+617447us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b540011e9d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:44.275623 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3358748928}
I20260709 13:03:44.281735  8330 rpcz_store.cc:276] 0709 13:03:41.814179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.815855 (+  1676us) service_pool.cc:224] Handling call
0709 13:03:41.851310 (+ 35455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:41.851721 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:41.851761 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:41.852309 (+   548us) log.cc:844] Serialized 26 byte log entry
0709 13:03:42.311661 (+459352us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:42.311703 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.311773 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:44.091908 (+1780135us) spinlock_profiling.cc:243] Waited 17.2 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:44.280623 (+188715us) raft_consensus.cc:1685] finished
0709 13:03:44.280689 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:44.281041 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51498,"raft.queue_time_us":866,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":47651,"spinlock_wait_cycles":39564544,"thread_start_us":324942,"threads_started":1,"wal-append.queue_time_us":840920}
I20260709 13:03:44.245371 13001 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:44.295213 13274 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:44.297870 13274 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:44.300799 13274 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:44.301491 13196 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:44.260923 13382 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:44.302727 13369 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.303936 13369 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:44.305964 13164 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.307269 13369 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:44.307024 13164 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:44.310477 13594 log.cc:927] Time spent T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.471s	user 0.002s	sys 0.000s
I20260709 13:03:44.316566  9009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56948 (request call id 1025) took 1179 ms. Trace:
I20260709 13:03:44.280211 13400 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:44.317129  9009 rpcz_store.cc:276] 0709 13:03:43.133745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.135065 (+  1320us) service_pool.cc:224] Handling call
0709 13:03:44.313083 (+1178018us) spinlock_profiling.cc:243] Waited 46.6 ms on lock 0x7b5400533254. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:44.313269 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":107079040}
I20260709 13:03:44.317881 13306 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.318660 13306 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:44.319653  8399 raft_consensus.cc:1240] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:03:44.321022 13306 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:44.321231 13367 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:03:44.322074  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.322153 13367 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:44.324252  8372 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:44.324939 13367 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:44.302449 13196 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:44.325587  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:44.327944 13196 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:44.321022 13382 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:44.333940  8249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 2240) took 1886 ms. Trace:
I20260709 13:03:44.334527  8249 rpcz_store.cc:276] 0709 13:03:42.447787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.461260 (+ 13473us) service_pool.cc:224] Handling call
0709 13:03:42.461676 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:44.205331 (+1743655us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.333446 (+128115us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b540034a8d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:44.333834 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3831296768}
I20260709 13:03:44.336302  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:44.337091  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:44.337982  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:44.338817  8397 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
I20260709 13:03:44.338997  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:44.296033  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:44.329099  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:44.350946  9001 raft_consensus.cc:2393] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 16: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:44.357597  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:44.365819  8406 raft_consensus.cc:2368] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 13. Current term is 14.
I20260709 13:03:44.376042  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:44.377602 13367 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:44.378351 13367 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:44.380781 13367 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:44.382332  8249 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:44.384809  8249 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:03:44.391294 13572 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.369870  8255 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:44.400883  8255 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:44.414834  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:44.415941  8220 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:44.416749  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:44.417821  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:44.371935 13001 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:44.367843 13369 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:44.418066 13164 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:44.420271  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:44.421545  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2280) took 1241 ms. Trace:
W20260709 13:03:44.434545  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.434922 13306 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:44.435189  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.435660 13306 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:44.438426 13306 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:44.439301 13306 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.439980 13306 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:44.427404  8399 rpcz_store.cc:276] 0709 13:03:43.180338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.180737 (+   399us) service_pool.cc:224] Handling call
0709 13:03:43.181035 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:44.320263 (+1139228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.417043 (+ 96780us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54003e6554. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:44.421461 (+  4418us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2617084160}
I20260709 13:03:44.444325  8403 raft_consensus.cc:2393] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 13: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:44.447165  8252 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:44.449827  8404 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:44.454999  8406 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:44.446534 13001 raft_consensus.cc:697] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:03:44.459517  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 937428us
I20260709 13:03:44.478680 13552 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:44.479460 13552 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.480156 13552 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:44.474858  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:44.504208  9005 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:44.504483  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2753) took 2328 ms. Trace:
I20260709 13:03:44.505230  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:44.505401 12973 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:44.506122 12973 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:44.506641  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:44.508038  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:44.506709  9007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 886) took 2169 ms. Trace:
I20260709 13:03:44.509390  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:44.504904  8398 rpcz_store.cc:276] 0709 13:03:42.176206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.178412 (+  2206us) service_pool.cc:224] Handling call
0709 13:03:42.196965 (+ 18553us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.342012 (+145047us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:42.342060 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:42.342119 (+    59us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:42.342168 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.504008 (+2161840us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54003e7454. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:44.504087 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:44.504300 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":332959616}
I20260709 13:03:44.516700  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:44.517516  8406 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:44.521108  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:44.521948  9004 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:44.524850  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:44.511968  9007 rpcz_store.cc:276] 0709 13:03:42.337683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.347254 (+  9571us) service_pool.cc:224] Handling call
0709 13:03:42.391500 (+ 44246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.842502 (+451002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.506318 (+1663816us) spinlock_profiling.cc:243] Waited 450 ms on lock 0x7b5400718ed4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:44.506561 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1035368320}
I20260709 13:03:44.525990  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:44.526356  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:44.527117  9007 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:44.527426  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:44.527990  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:44.529125 13579 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:44.530102  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:44.531283  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:44.531405 13527 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:44.531592 13579 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:44.532068  8398 raft_consensus.cc:2410] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 16 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:03:44.532891  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:44.533759 13538 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:44.536549 13001 consensus_queue.cc:237] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:44.541800  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:44.515852  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 59302.1us
W20260709 13:03:44.545598  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.451923 13572 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:44.549921  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:44.550657  8397 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:03:44.552201  9006 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:44.551601  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:44.556180  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:44.557109  9002 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:44.563248 12973 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:44.564127  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2402) took 2372 ms. Trace:
I20260709 13:03:44.564811  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:44.565919  9001 raft_consensus.cc:2410] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 16 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:03:44.566871  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:44.567878  9001 raft_consensus.cc:2393] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 13: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:44.470849 13369 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:44.564545  8332 rpcz_store.cc:276] 0709 13:03:42.192025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.192139 (+   114us) service_pool.cc:224] Handling call
0709 13:03:42.205003 (+ 12864us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:43.652030 (+1447027us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.563594 (+911564us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400364a54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:44.563960 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3104445824}
I20260709 13:03:44.518358 13306 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:44.596144  8404 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
W20260709 13:03:44.621935  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 77682us
I20260709 13:03:44.628052 13538 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:44.629505  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:44.632694  8254 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49220 (request call id 782) took 2216 ms (client timeout 2558 ms). Trace:
W20260709 13:03:44.633194  8254 rpcz_store.cc:269] 0709 13:03:42.416566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.416780 (+   214us) service_pool.cc:224] Handling call
0709 13:03:44.632288 (+2215508us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b540009ecd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:44.632508 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2810667776}
W20260709 13:03:44.635046  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 85237.1us
I20260709 13:03:44.639933  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2200) took 2211 ms. Trace:
I20260709 13:03:44.640492  8253 rpcz_store.cc:276] 0709 13:03:42.428051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.428189 (+   138us) service_pool.cc:224] Handling call
0709 13:03:43.541510 (+1113321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:43.608638 (+ 67128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.638425 (+1029787us) spinlock_profiling.cc:243] Waited 66.5 ms on lock 0x7b5400308c54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:44.639445 (+  1020us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400308c50. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:44.639710 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2712486656}
I20260709 13:03:44.643234 13369 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:44.644059 13619 log.cc:927] Time spent T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:03:44.644452  8257 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 2237) took 3063 ms (client timeout 1581 ms). Trace:
W20260709 13:03:44.644898  8257 rpcz_store.cc:269] 0709 13:03:41.580404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.593973 (+ 13569us) service_pool.cc:224] Handling call
0709 13:03:44.644143 (+3050170us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400407c54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:44.644272 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2787617152}
I20260709 13:03:44.606900 13400 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:03:44.673825  8369 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.969s	user 0.053s	sys 0.111s
I20260709 13:03:44.675776 13200 consensus_queue.cc:1059] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:03:44.676586  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 67249.3us
I20260709 13:03:44.689316  9006 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:03:44.692045  9004 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:44.694408  8297 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:44.697667 13465 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.698398 13465 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:44.700491 13465 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:44.756470 12973 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.757305 12973 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:44.759866 13471 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:44.760599 13471 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
W20260709 13:03:44.763228  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 68943.9us
I20260709 13:03:44.771260  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:44.706087 13200 raft_consensus.cc:3055] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:44.773072  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:44.782043  9002 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:03:44.712565 13572 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:44.792279  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 43521.5us
I20260709 13:03:44.786064  8222 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:03:44.797775  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.794752 13364 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:44.773510 13579 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:44.784948 13200 raft_consensus.cc:740] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:44.807497  8372 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:44.798533 13494 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:44.808224 13364 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:44.820010 13570 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:44.820699 13554 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:44.826802  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:44.783659 13400 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:44.840094  8974 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:44.831687 13579 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:03:44.837598 13200 consensus_queue.cc:260] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:44.916491  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:44.917496  8329 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
W20260709 13:03:44.918609  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 303924us
I20260709 13:03:44.919323 13496 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:44.920637 13496 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:44.921542 13544 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:44.922295 13544 raft_consensus.cc:697] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:44.923660 13544 consensus_queue.cc:237] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:44.926501  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 91743us
I20260709 13:03:44.927603  8974 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:44.929206  8974 leader_election.cc:400] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 13. Current term is 14.
I20260709 13:03:44.929525 13515 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:44.930143  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:44.933779 12973 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:44.936367  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:44.938055  8974 leader_election.cc:403] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:03:44.945674 13579 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:44.842733 13554 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:44.958613  8369 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.254s	user 0.054s	sys 0.112s
I20260709 13:03:44.939028 13515 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.963122 13240 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:44.941248 13369 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:44.968657  9007 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:44.970518  8330 raft_consensus.cc:2393] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 11: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:44.951665 12973 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.963150 13364 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:44.970458 13369 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.970882 13240 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.968339 13515 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:44.969903 13538 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:03:44.977393  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 1.27245e+06us
I20260709 13:03:44.970342 13471 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:44.988724  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 59731.4us
W20260709 13:03:44.986919 13552 consensus_meta.cc:287] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.504s	user 0.006s	sys 0.007s
W20260709 13:03:44.994360  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 198125us
I20260709 13:03:44.984740 13369 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:44.985929 13240 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:44.997721 13475 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:44.993386 13200 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:44.993892 13527 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 13. Current term is 14.
W20260709 13:03:45.026544  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 275949us
I20260709 13:03:45.028074 13631 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.028735 13494 raft_consensus.cc:697] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:45.031883  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:45.026782 13630 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.986464 12973 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:45.038691  8332 raft_consensus.cc:2393] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 17: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:45.021445  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:45.051052 13630 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:45.051301  8142 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 81804f59e3124d27998ea80fc1c36663 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:03:44.996836 13552 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:45.034003 13400 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:45.058941  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 63678.1us
I20260709 13:03:45.057981  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:45.015604 13538 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:45.064530  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:45.071015  8369 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:45.072731 13570 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:45.073077  8369 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:45.075510 13452 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.076265 13452 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:45.077353 13604 log.cc:927] Time spent T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.913s	user 0.000s	sys 0.005s
I20260709 13:03:45.078228 13554 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:45.078366 13496 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:45.055008 13475 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:45.080428  8298 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:45.081815  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:45.082002  8298 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:45.082937 13364 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:03:45.084208 13615 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.473s	user 0.001s	sys 0.000s
I20260709 13:03:45.084862 13364 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:45.085711 13400 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:45.095430  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.102710 13552 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:45.103088 13623 log.cc:927] Time spent T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:03:45.080466 13538 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:45.104954 13625 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:03:45.060776 13499 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:03:45.112706  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 84309.9us
I20260709 13:03:45.118181 13570 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:45.118644 13570 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:45.119005 13570 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:45.120177 13240 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:45.085332 13364 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:45.028736 13631 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:45.135154  9003 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:45.146952  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:45.125684 13364 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:45.198375  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2418) took 1448 ms. Trace:
I20260709 13:03:45.198206  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:45.198688  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:45.198945  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:45.199280  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:45.210246  8325 raft_consensus.cc:2410] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:03:45.199349  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:45.199476  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:45.200114  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:45.200843  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:45.212083  8397 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 12: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:03:45.208832  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 45796.7us
I20260709 13:03:45.157161  8222 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:45.215780  8398 raft_consensus.cc:2393] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 14: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:45.218274 13240 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:45.219058  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:45.210289  8331 rpcz_store.cc:276] 0709 13:03:43.703014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.867662 (+164648us) service_pool.cc:224] Handling call
0709 13:03:43.868012 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:44.132796 (+264784us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:44.133405 (+   609us) log.cc:844] Serialized 14 byte log entry
0709 13:03:44.194514 (+ 61109us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:44.194580 (+    66us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:44.194634 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.132309 (+937675us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b54003f11d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:45.132377 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.151432 (+ 19055us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23138,"prepare.queue_time_us":5843,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":54,"raft.queue_time_us":34703,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":16741,"spinlock_wait_cycles":607084416,"thread_start_us":22680,"threads_started":1,"wal-append.queue_time_us":74037}
I20260709 13:03:45.220899  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:45.219753  8397 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:45.215574 13639 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:45.221616  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 64530us
I20260709 13:03:45.221623  8398 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:45.161402 13544 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.231758  8330 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:03:45.233244  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:45.234228  8328 raft_consensus.cc:2393] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 12: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:45.237457 13275 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:03:45.239024 13452 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:45.246155 12973 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:45.246273 13620 log.cc:927] Time spent T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.351s	user 0.002s	sys 0.000s
I20260709 13:03:45.247082  8327 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:43.604995  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:45.158039 13494 consensus_queue.cc:237] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:45.268486  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:45.269964  8398 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:03:45.161948 13630 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:45.267417 13631 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:45.315245 13642 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:45.315645  8295 connection.cc:256] RPC call timeout handler was delayed by -2.3772893580000982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15360000000000001s for a response
I20260709 13:03:45.317868  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:45.318838  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:45.319069 13475 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:45.319590  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:45.320999  8222 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:45.322372  8369 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:45.323629 13275 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.324775  8250 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:03:45.325232  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:45.325805  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:45.326607 13544 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:45.327174  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:45.291851  8974 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:45.151014 13499 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:45.315322  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:45.354843  8295 connection.cc:256] RPC call timeout handler was delayed by -2.1920525780000162s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16620000000000001s for a response
W20260709 13:03:45.364382  8295 connection.cc:256] RPC call timeout handler was delayed by -2.1604844570000523s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16410000000000002s for a response
W20260709 13:03:45.370785  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 48535.1us
W20260709 13:03:45.376497  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 55617.6us
I20260709 13:03:45.377704  8326 raft_consensus.cc:2410] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:03:45.377967  8329 raft_consensus.cc:2410] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:03:45.379984  8257 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:03:45.388166  8254 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:03:45.381373  8332 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:45.391961  8253 raft_consensus.cc:2368] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 13. Current term is 14.
I20260709 13:03:45.392102  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:45.392638  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:45.392992  8329 raft_consensus.cc:2410] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 13 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:03:45.393528  8250 raft_consensus.cc:2393] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 17: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:45.394112  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:45.401487 13629 log.cc:927] Time spent T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.358s	user 0.002s	sys 0.005s
W20260709 13:03:45.404649 13636 log.cc:927] Time spent T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.375s	user 0.000s	sys 0.006s
W20260709 13:03:45.406934 13628 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.345s	user 0.000s	sys 0.007s
I20260709 13:03:45.410581  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2203) took 2981 ms. Trace:
I20260709 13:03:45.410965  8251 rpcz_store.cc:276] 0709 13:03:42.429346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.446003 (+ 16657us) service_pool.cc:224] Handling call
0709 13:03:42.483807 (+ 37804us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:42.547875 (+ 64068us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:42.550810 (+  2935us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:44.549243 (+1998433us) spinlock_profiling.cc:243] Waited 8.41 ms on lock 0x7b2c000014c8. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:03:44.705621 (+156378us) log.cc:844] Serialized 78 byte log entry
0709 13:03:44.990503 (+284882us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:45.017947 (+ 27444us) log.cc:844] Serialized 14 byte log entry
0709 13:03:45.409950 (+392003us) spinlock_profiling.cc:243] Waited 7.79 ms on lock 0x7b18003e0620. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88283664bc 00007f88256b448b 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256ce455 00007f88287b6060 00007f88287c8716 00007f88287d9287
0709 13:03:45.410070 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.410191 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:45.410232 (+    41us) raft_consensus.cc:1685] finished
0709 13:03:45.410243 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.410416 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3951,"prepare.queue_time_us":27931,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":1780,"raft.queue_time_us":103299,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":75058,"spinlock_wait_cycles":55464960,"thread_start_us":323599,"threads_started":2,"wal-append.queue_time_us":327910}
I20260709 13:03:45.418875  8295 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:45.421764 13475 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:45.418718 13494 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:45.422302 13475 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:45.422739 13475 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:45.362342 13538 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:45.445529  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2291) took 1842 ms. Trace:
I20260709 13:03:45.446136  8402 rpcz_store.cc:276] 0709 13:03:43.603553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.618563 (+ 15010us) service_pool.cc:224] Handling call
0709 13:03:43.618926 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:44.179059 (+560133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.431815 (+1252756us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b54000febd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:45.445408 (+ 13593us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1285530368}
I20260709 13:03:45.448307  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:45.449393  8251 raft_consensus.cc:2393] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 12: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:45.461650 13424 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:45.462419 13424 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:45.465087  8972 connection.cc:256] RPC call timeout handler was delayed by -1.685480392000045s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25579999999999997s for a response
I20260709 13:03:45.481453 13544 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:45.482859 13544 consensus_queue.cc:1059] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:45.490929 13499 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:45.498261  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:45.499154  8329 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:03:45.500522  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:45.503131  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:45.503930  8330 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18.
I20260709 13:03:45.504748  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:45.230273 13639 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:45.386507 13630 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:03:45.506353  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.384491 13579 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:03:45.512122  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 120707us
I20260709 13:03:45.370193 13275 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:45.387830 13200 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:45.516618  8329 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:03:45.437260  8331 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:45.531584  8222 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:45.541044  8402 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:45.550127  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:45.552915  8257 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:45.553388  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:45.559458  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 46549.5us
I20260709 13:03:45.559679  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:45.511121 13630 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:45.560617  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:45.564746  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:45.563110  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.566385  8332 raft_consensus.cc:2393] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 16: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:03:45.564834  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 143458us
W20260709 13:03:45.569096  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 98178.3us
I20260709 13:03:45.560343  8400 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:45.566860  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:45.548941 13515 consensus_meta.cc:287] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.553s	user 0.007s	sys 0.004s
W20260709 13:03:45.577256  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 67399.6us
I20260709 13:03:45.580662  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:45.581849  9001 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:45.584751  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:45.585949 13644 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:45.586544 13644 raft_consensus.cc:697] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 14 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:45.580875  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:45.590560  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:45.591662  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.571600 13630 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:45.594939  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:45.595510  8397 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:03:45.593662  9005 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:45.596807  9003 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 12: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:45.497402 13364 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:45.579167 13640 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:45.594480  9000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 1030) took 1089 ms. Trace:
I20260709 13:03:45.604839  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:45.610080  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:45.614085  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:45.615052  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 323321us
I20260709 13:03:45.567256 13538 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:45.614578  8251 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:03:45.577003 13200 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:45.634083  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:45.605986  9000 rpcz_store.cc:276] 0709 13:03:44.328162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.353525 (+ 25363us) service_pool.cc:224] Handling call
0709 13:03:44.391865 (+ 38340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:44.570231 (+178366us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:44.570616 (+   385us) log.cc:844] Serialized 14 byte log entry
0709 13:03:44.852164 (+281548us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:44.852241 (+    77us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:44.907806 (+ 55565us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.417138 (+509332us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b54004b87d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:45.417187 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.417309 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6294,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":107,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":8729,"spinlock_wait_cycles":409668992,"thread_start_us":281039,"threads_started":1,"wal-append.queue_time_us":411853}
I20260709 13:03:45.624974  8220 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:45.631399  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:45.635848  8254 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:45.637917  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:45.639447  9006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56954 (request call id 899) took 1114 ms. Trace:
I20260709 13:03:45.643378  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:45.636802  8399 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:03:45.639954  9006 rpcz_store.cc:276] 0709 13:03:44.525412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.527126 (+  1714us) service_pool.cc:224] Handling call
0709 13:03:45.639239 (+1112113us) spinlock_profiling.cc:243] Waited 24.2 ms on lock 0x7b54005a8cd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:45.639374 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55605632}
I20260709 13:03:45.646314  8250 raft_consensus.cc:2393] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 10: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:45.618211 13607 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:45.650339 13607 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:45.652710 13607 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:45.653985 13644 consensus_queue.cc:237] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:45.655066 13200 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:45.658448  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:45.659164  8253 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:45.661798  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:45.662115 13463 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.662479  8248 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:45.662739 13463 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:45.663244 13653 raft_consensus.cc:3055] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:45.663694 13653 raft_consensus.cc:740] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:45.664067  8254 raft_consensus.cc:2410] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 16 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
I20260709 13:03:45.664986 13639 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:45.665849  8298 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:45.650367  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:45.667557  8256 raft_consensus.cc:2393] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 12: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:03:45.646453 13369 consensus_meta.cc:287] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.631s	user 0.011s	sys 0.000s
I20260709 13:03:45.668229 13369 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:45.668824  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 43947.4us
I20260709 13:03:45.671368  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:45.625794 13640 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:45.679056  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 212369us
I20260709 13:03:45.679153 13640 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:45.685024  9006 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
W20260709 13:03:45.688591  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 74835.6us
I20260709 13:03:45.707001  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:45.709438  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:45.682554 13570 consensus_meta.cc:287] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.563s	user 0.010s	sys 0.003s
I20260709 13:03:45.680330 13200 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:45.728981  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:45.731745  8256 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:03:45.732815  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:45.706991 13554 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.688283 13369 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:45.749545  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 61072.9us
I20260709 13:03:45.749817  8399 raft_consensus.cc:2393] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 16: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:03:45.737480  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 149780us
I20260709 13:03:45.752413  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:45.753731  8398 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 12: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:45.583388 13515 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:45.730556 13200 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:45.735445  8257 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:03:45.788686  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:45.798548  8399 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:45.805480  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:45.809195  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:45.806064  8972 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:03:45.812911  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 190587us
I20260709 13:03:45.788676 13200 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:45.730383 13570 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:45.750268 13475 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:45.821404  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:45.827395  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:45.830034  8398 raft_consensus.cc:2410] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:03:45.759127 13554 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:45.810382 13275 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:45.818172 13659 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:45.835002 13644 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:03:45.843279  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 173270us
W20260709 13:03:45.832449  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 123274us
I20260709 13:03:45.849925  8254 raft_consensus.cc:2368] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14.
I20260709 13:03:45.844228  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:45.848296  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:45.849015  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:45.849910  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:45.850677 13494 consensus_queue.cc:1059] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:45.851089  8399 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:45.859549 13554 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:45.861750 13475 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:45.872299 13622 log.cc:927] Time spent T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.921s	user 0.005s	sys 0.000s
I20260709 13:03:45.875370 13666 consensus_queue.cc:1059] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:45.875910 13424 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:45.876394 13659 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:45.877501  8220 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:45.878368 13424 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:45.878613 13659 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:45.880119  8220 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:45.844779  8972 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:45.866657  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:45.885258  8253 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
W20260709 13:03:45.863270 13646 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
I20260709 13:03:45.862468 13364 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:45.892473  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 56725.5us
I20260709 13:03:45.896003  8398 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:45.898916  8399 raft_consensus.cc:1240] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:45.868695 13538 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:45.887786 13452 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:45.914832  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 76622.8us
I20260709 13:03:45.881024 13644 raft_consensus.cc:697] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:45.917461 13424 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.921568 13577 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:03:45.928158 13643 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.606s	user 0.001s	sys 0.005s
W20260709 13:03:45.928659 13645 log.cc:927] Time spent T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.692s	user 0.003s	sys 0.000s
I20260709 13:03:45.930730  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:45.915980  8972 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:03:45.932461 13634 log.cc:927] Time spent T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.902s	user 0.000s	sys 0.003s
I20260709 13:03:45.935019 13630 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:45.935859 13554 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:45.936034 13642 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:45.936705 12973 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:45.938645  8223 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:45.940905 13515 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:45.942219 13653 consensus_queue.cc:260] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:45.948546  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:45.991994  8402 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:45.992276 13653 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:45.913745 13658 raft_consensus.cc:3055] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:45.993489 13658 raft_consensus.cc:740] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:03:45.953275  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 86060.2us
I20260709 13:03:45.994555 13656 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:45.994658  8297 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:45.956041 13570 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:45.963296 13463 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:45.964300 13644 consensus_queue.cc:237] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:45.965140 13632 log.cc:927] Time spent T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.910s	user 0.008s	sys 0.000s
W20260709 13:03:45.973994  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 138491us
I20260709 13:03:45.980587 13475 consensus_queue.cc:1059] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.003s
W20260709 13:03:45.993947  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 86352.2us
I20260709 13:03:45.975740 13577 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:03:46.003261  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 123210us
I20260709 13:03:46.007026  8220 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:46.005595  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 66999.9us
I20260709 13:03:46.014285  8402 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 16.
I20260709 13:03:46.014905  8297 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:46.013226 13424 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:46.019322  8297 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:46.013006 13577 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:46.006245 13642 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:46.030593  8252 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:46.030941  8253 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:03:46.032073  8253 pending_rounds.cc:85] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 11
I20260709 13:03:46.033789  8972 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:46.018420 13275 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:46.045960  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:46.046619  9005 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:46.048306 13369 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:46.049064 13369 consensus_queue.cc:1059] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:46.049345  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:46.049953 13650 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:46.050478  8972 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:46.050730 13369 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:46.051474 13650 raft_consensus.cc:3055] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:46.051770  8972 leader_election.cc:400] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 13. Current term is 14.
I20260709 13:03:46.051929 13650 raft_consensus.cc:740] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:46.052269 13369 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:46.052848 13369 raft_consensus.cc:697] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 13 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:46.052899 13650 consensus_queue.cc:260] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:46.053905 13369 consensus_queue.cc:237] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:46.054529 13650 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:46.058624  8154 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45456 (request call id 284) took 42677 ms (client timeout 15000 ms). Trace:
W20260709 13:03:46.062107  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.063066 13527 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:46.063534 13527 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.063879 13527 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:46.038578  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:46.039065  8248 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:46.002491 12973 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:46.041498  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2302) took 1606 ms. Trace:
I20260709 13:03:46.006776 13554 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:46.041667  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:46.065697  8372 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:46.042289  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:46.039328  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:46.032416  8253 pending_rounds.cc:107] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:03:46.073761  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:46.074249  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:46.075733  8253 raft_consensus.cc:2889] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:46.075059  8254 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
I20260709 13:03:46.054211 13200 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:46.077420  8220 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:46.075605  8256 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 11.
I20260709 13:03:46.076282  8255 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:03:46.091807  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:46.092569  8248 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:46.094858  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:46.095852  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:46.098548  8249 raft_consensus.cc:2393] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 12: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:46.099938 13641 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:46.100930 13424 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:46.101042 13675 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:46.102892 13658 consensus_queue.cc:260] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:46.104233 13658 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:46.078853 13645 log.cc:927] Time spent T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:03:46.107939 13640 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:46.066234  8403 rpcz_store.cc:276] 0709 13:03:44.434871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.636667 (+201796us) service_pool.cc:224] Handling call
0709 13:03:44.637128 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:44.681382 (+ 44254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:44.681445 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:44.749693 (+ 68248us) log.cc:844] Serialized 26 byte log entry
0709 13:03:44.991730 (+242037us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:44.991787 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.991877 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:45.964569 (+972692us) spinlock_profiling.cc:243] Waited 5.24 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:46.041185 (+ 76616us) raft_consensus.cc:1685] finished
0709 13:03:46.041212 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.041423 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30960,"raft.queue_time_us":12097,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":13521,"spinlock_wait_cycles":12050176,"thread_start_us":188001,"threads_started":1,"wal-append.queue_time_us":237799}
I20260709 13:03:46.074599 13275 consensus_queue.cc:1059] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:03:46.110281 13275 raft_consensus.cc:3055] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:46.110641 13275 raft_consensus.cc:740] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:46.111455 13275 consensus_queue.cc:260] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:46.034610 13642 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.112668 13275 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:46.117128 13670 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:46.025516 13552 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:03:46.117619 13670 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:46.115011  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 49378.1us
I20260709 13:03:46.121450  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:46.030508 13494 consensus_queue.cc:1243] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer 8c42259d7c4b463a9f485f42c320c81b log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:03:46.123626  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:46.124888  8250 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:46.127897  8254 raft_consensus.cc:2393] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 15: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:46.126091 13639 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:46.132642  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:46.133689  8402 raft_consensus.cc:2393] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:46.078223 13306 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:46.113020 13642 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:46.137051  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 55668.3us
W20260709 13:03:46.137452  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 60111.3us
I20260709 13:03:46.138288  8220 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:46.141294  8142 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 81804f59e3124d27998ea80fc1c36663 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:03:46.123207 13670 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:46.144842  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:46.146157  9002 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:03:46.147951  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:46.148528  8404 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:46.146288 13306 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:46.153136 13306 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:46.157853  8256 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:46.159523  8254 pending_rounds.cc:85] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 6
I20260709 13:03:46.159854  8254 pending_rounds.cc:107] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:03:46.160107  8254 raft_consensus.cc:2889] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:46.161289  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:46.163524  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:46.164695  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2409) took 3080 ms. Trace:
I20260709 13:03:46.165089  8252 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:03:46.165292 13638 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:46.059198  8154 rpcz_store.cc:269] 0709 13:03:03.380784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.415355 (+ 34571us) service_pool.cc:224] Handling call
0709 13:03:46.058173 (+42642818us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:35.769894 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:35.770595 (+   701us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:35.770673 (+    78us) write_op.cc:435] Acquired schema lock
0709 13:03:35.770699 (+    26us) tablet.cc:662] Decoding operations
0709 13:03:35.797811 (+ 27112us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:35.797898 (+    87us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:35.797924 (+    26us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:03:35.799225 (+  1301us) tablet.cc:701] Row locks acquired
0709 13:03:35.799287 (+    62us) write_op.cc:260] PREPARE: finished
0709 13:03:35.799479 (+   192us) write_op.cc:270] Start()
0709 13:03:35.799636 (+   157us) write_op.cc:276] Timestamp: P: 1783602215799401 usec, L: 0
0709 13:03:35.799673 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 13:03:35.863769 (+ 64096us) log.cc:844] Serialized 5860 byte log entry
0709 13:03:36.620214 (+756445us) op_driver.cc:464] REPLICATION: finished
0709 13:03:36.777920 (+157706us) write_op.cc:301] APPLY: starting
0709 13:03:36.789502 (+ 11582us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:37.212457 (+422955us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:37.212579 (+   122us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:38.227143 (+1014564us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:38.227498 (+   355us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=30,mrs_lookups=0
0709 13:03:38.227558 (+    60us) write_op.cc:312] APPLY: finished
0709 13:03:38.227973 (+   415us) log.cc:844] Serialized 139 byte log entry
0709 13:03:38.297756 (+ 69783us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:38.297910 (+   154us) write_op.cc:454] Released schema lock
0709 13:03:38.297980 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:38.336741 (+ 38761us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:38.336792 (+    51us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":20119,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":157332,"apply.run_cpu_time_us":37493,"apply.run_wall_time_us":1534308,"cfile_cache_hit":62,"cfile_cache_hit_bytes":66857,"num_ops":15,"prepare.queue_time_us":150892,"prepare.run_cpu_time_us":58866,"prepare.run_wall_time_us":213877,"raft.queue_time_us":286413,"raft.run_cpu_time_us":23214,"raft.run_wall_time_us":56483,"replication_time_us":820347,"thread_start_us":523477,"threads_started":5,"wal-append.queue_time_us":277574,"wal-append.run_cpu_time_us":25632,"wal-append.run_wall_time_us":2723292}]]}
I20260709 13:03:46.166249  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:46.166519 13657 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:46.166828  8252 raft_consensus.cc:2376] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15. Re-sending same reply.
I20260709 13:03:46.167124 13657 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:46.167672  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:46.168268  8252 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:46.175140  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 145344us
I20260709 13:03:46.176192  8297 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:46.181834  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:46.182492  8297 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:46.182762  8222 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:46.185973  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:46.187943  8326 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:46.188194  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:46.188757 13200 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:46.189602  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:46.193529  8222 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:46.194975  8250 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:46.195739  8326 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
W20260709 13:03:46.197660  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 59489.9us
I20260709 13:03:46.165207 13676 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:46.201617  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:46.199222  8222 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:46.202432  8220 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:46.203994  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:46.208333 13639 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:46.178990 13630 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:46.192508 13364 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:46.210136  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:46.210407 13452 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:46.210844 13452 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.211210 13452 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:46.211246 13463 consensus_queue.cc:1059] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:46.212394 13463 raft_consensus.cc:3055] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:46.212752 13463 raft_consensus.cc:740] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:46.213709 13465 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:46.216210  8327 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:46.216681  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:46.217855  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:46.217914 13465 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:46.218840  8220 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:46.219805 13631 consensus_queue.cc:1059] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:46.220911  8220 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:46.221038 13424 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:46.221920 13424 raft_consensus.cc:3055] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:46.222271 13424 raft_consensus.cc:740] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:46.224174 13463 consensus_queue.cc:260] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:46.225937 13463 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:46.226441  8332 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:03:46.165122  8324 rpcz_store.cc:276] 0709 13:03:42.986391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.986556 (+   165us) service_pool.cc:224] Handling call
0709 13:03:42.986904 (+   348us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:42.993777 (+  6873us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:42.993806 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:43.545753 (+551947us) spinlock_profiling.cc:243] Waited 1.06 ms on lock 0x7b2c000d4468. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:03:44.676415 (+1130662us) spinlock_profiling.cc:243] Waited 65.7 ms on lock 0x7b2c000d4468. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:03:44.677064 (+   649us) log.cc:844] Serialized 52 byte log entry
0709 13:03:44.988616 (+311552us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:44.988653 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.000725 (+ 12072us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:45.929777 (+929052us) spinlock_profiling.cc:243] Waited 37.5 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:46.066497 (+136720us) raft_consensus.cc:1685] finished
0709 13:03:46.066518 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.066650 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":201,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":7871,"spinlock_wait_cycles":239731200,"thread_start_us":311114,"threads_started":1,"wal-append.queue_time_us":352706}
I20260709 13:03:46.226125  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:46.227200  8220 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:46.236413  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:46.237457  8326 raft_consensus.cc:2410] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
W20260709 13:03:46.241675  8369 connection.cc:256] RPC call timeout handler was delayed by -1.7035574279999537s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
W20260709 13:03:46.242182  8369 connection.cc:256] RPC call timeout handler was delayed by -1.5307574280000154s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1706s for a response
W20260709 13:03:46.242589  8369 connection.cc:256] RPC call timeout handler was delayed by -1.4389574279999806s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18080000000000002s for a response
W20260709 13:03:46.242934  8369 connection.cc:256] RPC call timeout handler was delayed by -1.4002574280000317s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18510000000000001s for a response
W20260709 13:03:46.244167  8369 connection.cc:256] RPC call timeout handler was delayed by -1.3984574280000288s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18530000000000002s for a response
W20260709 13:03:46.244761  8369 connection.cc:256] RPC call timeout handler was delayed by -1.1779574280000134s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2098s for a response
I20260709 13:03:46.245549 13669 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:03:46.246073 13627 log.cc:927] Time spent T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 1.220s	user 0.007s	sys 0.013s
I20260709 13:03:46.246357 12973 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:46.240823 13369 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
I20260709 13:03:46.252858  8323 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:03:46.265362  8404 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:03:46.247184  8248 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 16.
I20260709 13:03:46.249840  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:46.231712 13200 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:46.250828  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:46.208359 13657 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:46.267805  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:46.262902  8972 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:46.267843  8256 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:46.276260  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:46.279354  8252 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:46.288956  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:46.247619 13572 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:03:46.268146 12973 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:46.269021 13577 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:46.280825 13364 raft_consensus.cc:697] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:46.312136  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:46.299278 12973 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:46.313084  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 44234.4us
W20260709 13:03:46.313871  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 89805us
I20260709 13:03:46.316593  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:46.324808  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 97697.8us
W20260709 13:03:46.316720  8972 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.334s (SENT) [suppressed 12 similar messages]
I20260709 13:03:46.274082 13661 consensus_queue.cc:1059] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:03:46.253691 13669 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:46.326982  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:46.327668  8257 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
W20260709 13:03:46.339859  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 136721us
I20260709 13:03:46.340809  8297 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:46.342022  8972 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:46.343031  8222 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:46.323585 13306 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:46.326511 12973 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:46.373093 13687 log.cc:927] Time spent T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:03:46.301362 13527 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:46.374395 13364 consensus_queue.cc:237] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:46.366048 13651 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:46.346534 13661 consensus_queue.cc:1243] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer 8c42259d7c4b463a9f485f42c320c81b log is divergent from this leader: its last log entry 15.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:03:46.379863  8256 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:46.382041  8369 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:46.313690 13644 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:03:46.390177 13692 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:46.369066  8152 catalog_manager.cc:5697] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 9 to 10, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 10 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:46.423821  9008 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 15: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:46.397478 13538 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:46.397759 13650 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:46.435067  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:46.441283  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 98335.6us
I20260709 13:03:46.423735 13692 raft_consensus.cc:697] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:46.441730  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:46.444024  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 88891.1us
I20260709 13:03:46.437225  8152 catalog_manager.cc:5697] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:46.449982  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:46.451601  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:46.452951  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:46.456807  8369 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:46.458300  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:46.436476 13538 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:46.462785  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:46.458372  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:46.464533  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:46.464555  8325 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:03:46.459282  8324 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:46.387832 13642 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:46.463804  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:46.446489 13650 raft_consensus.cc:697] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:46.470451  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:46.462050 13653 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:46.480706  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:46.468946 13538 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:46.483268  8152 catalog_manager.cc:5697] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:46.486734  8404 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:03:46.500982  8329 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:46.478875 13672 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:46.506829 13669 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:46.511626 13527 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:46.514206  8152 catalog_manager.cc:5697] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:46.515628  8152 catalog_manager.cc:5697] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 7 to 8, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 8 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:46.516604 13577 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:46.517180  8152 catalog_manager.cc:5697] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 4 to 12, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 12 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:03:46.518889  8152 catalog_manager.cc:5697] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 7 to 9, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 9 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:03:46.520584 13642 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:46.524348  8404 raft_consensus.cc:1240] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:46.529680 13552 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:46.544809 13275 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:46.545758 13275 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:46.548782 13275 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:46.551165  8152 catalog_manager.cc:5697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 7 to 15, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 15 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:46.563575  8152 catalog_manager.cc:5697] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 9 to 11, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 11 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:46.566812 12973 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:46.565042  8152 catalog_manager.cc:5697] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 9 to 10, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 10 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:03:46.575356 13676 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:46.577863 13676 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:46.580891  9002 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:03:46.586189  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.590713 13696 raft_consensus.cc:3055] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:46.591197 13696 raft_consensus.cc:740] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:46.592041 13696 consensus_queue.cc:260] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:46.594336  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:46.518257 13692 consensus_queue.cc:237] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:46.598579  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:46.610203  8325 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:03:46.615700  8222 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:03:46.617082  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.593286 13696 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:46.624977 13630 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:46.625640  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:46.626271  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:46.627429  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:46.631893  8400 raft_consensus.cc:2368] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14.
I20260709 13:03:46.619554 13570 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.633564 13570 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:46.637888  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 103137us
I20260709 13:03:46.650344  8248 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:46.654358  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:46.632364 13630 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.665939  8220 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:46.620949 13452 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:46.701843  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:46.667743 12973 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:46.666862 13463 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.611241 13650 consensus_queue.cc:237] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:46.640848 13699 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:46.632228 13701 consensus_queue.cc:1059] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:03:46.711531  8401 raft_consensus.cc:2393] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 15: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:46.752661  8253 raft_consensus.cc:1240] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:46.750784 13463 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:46.756570 13463 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:46.750748 13577 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:46.757990 13577 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:46.757939  8253 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:03:46.760033  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:46.760650  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.762656 13570 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:46.711251  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 45409.5us
I20260709 13:03:46.749112  8369 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:46.645764 13706 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.767046  8220 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:46.751717  8152 catalog_manager.cc:5697] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 9 to 11, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 11 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:46.665788 13630 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:46.770879  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 52017.6us
I20260709 13:03:46.771627  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:46.772795  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:46.773715 13669 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:46.774407 13669 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:46.774833 13669 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:46.774621 13676 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:03:46.775836 13549 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:46.775360 13669 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:46.778621 13699 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:46.781392 13639 consensus_meta.cc:287] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.573s	user 0.009s	sys 0.010s
I20260709 13:03:46.782413  8369 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:46.783051 13570 consensus_queue.cc:1059] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:46.785347 13570 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:46.785578 13549 raft_consensus.cc:3055] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:46.785982 13549 raft_consensus.cc:740] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:46.786295  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:46.787029 13653 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:46.786870 13549 consensus_queue.cc:260] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:46.787407 13653 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:46.787815 13653 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:46.788125 13549 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:46.789909  8152 catalog_manager.cc:5697] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 5 to 11. New cstate: current_term: 11 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:03:46.791188  8152 catalog_manager.cc:5697] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:03:46.793004 13424 consensus_queue.cc:260] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:46.794263 13424 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:46.790508 13670 consensus_meta.cc:287] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.628s	user 0.009s	sys 0.001s
I20260709 13:03:46.783319  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:46.796343  8332 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:03:46.793313  8152 catalog_manager.cc:5697] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 10 to 13, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 13 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:46.798985  8220 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:46.718628 13515 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:46.830348  8152 catalog_manager.cc:5697] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:46.836033  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:46.836845  8326 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:46.828922  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:46.838968  9008 raft_consensus.cc:2410] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 15 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:03:46.843312  8152 catalog_manager.cc:5697] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:46.798151 13676 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:46.797948 13639 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:46.796018 13699 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:46.865029  8152 catalog_manager.cc:5697] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 6 to 11, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 11 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:46.838586 13709 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:46.869226 13452 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:46.869575  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:46.870203  8331 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:46.877074 13708 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:46.884207 13499 raft_consensus.cc:3055] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:46.880056  8331 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 10.
I20260709 13:03:46.882632 13667 raft_consensus.cc:3055] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:46.799283 13670 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:46.829511 13538 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:46.878628 13658 consensus_meta.cc:287] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.774s	user 0.007s	sys 0.005s
I20260709 13:03:46.880805 13672 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:46.887535 13577 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:46.795306 13669 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:46.794770 13706 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:46.900776  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 106324us
I20260709 13:03:46.902376  8254 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:46.800168 13654 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.098s
I20260709 13:03:46.896665 13708 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.907212  8403 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 15: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:46.907539  8326 raft_consensus.cc:2410] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 13 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:03:46.908123  8328 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:46.898241 13499 raft_consensus.cc:740] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:46.899717 13667 raft_consensus.cc:740] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:46.915612  8401 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:46.921135  8332 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:46.912515 13708 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:46.932857 13579 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:46.925531 13670 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:46.939352 13499 consensus_queue.cc:260] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:46.920718 13653 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:46.946293 13653 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:46.949295 13667 consensus_queue.cc:260] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:46.950555 13667 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:46.959184  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:46.960366  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:46.960577  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:46.961043  8331 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:46.961453  8326 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 12: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:46.961990  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:46.963765  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:46.965646  8326 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:46.966564 13538 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:46.967872  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:46.968389  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:46.969035  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:46.970005 13577 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.970093  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:46.970685  8330 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:46.970577 13577 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:46.977842  8220 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:46.972579  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:46.975204  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:46.976962 13554 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:46.976994  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.997782  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:46.941133 13499 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:46.980916  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:46.982563  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:47.001890 13554 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:46.984176  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:47.003193  9004 raft_consensus.cc:1275] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:46.988698 13706 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:47.008476  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:46.999681  9005 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 15: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:46.999879  8324 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:03:47.001325  8220 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:46.977782  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:47.013566  8220 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:47.016084  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:47.005848  8326 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:03:47.017401  8220 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:47.023183  8328 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:47.029803  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:47.029866  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:47.032011  8297 leader_election.cc:400] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14.
I20260709 13:03:47.032488  8297 leader_election.cc:403] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:03:47.033133 13364 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14.
I20260709 13:03:47.033675  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:47.033849  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:47.035000 13709 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:47.015962  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:47.036798 13577 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:47.037503 13676 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.022s
W20260709 13:03:47.037845  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.038010 13669 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:47.016782 13538 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:03:47.040622  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:47.041215  8295 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:47.044258  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:47.052784  9007 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:47.041400  8220 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:47.068837  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:47.071168  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:03:47.072042  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.070744  9004 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:03:47.017753 13639 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:47.074085  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:47.074290  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:47.042804 13452 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:47.066901 13465 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:47.077265 13671 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:47.084986  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:47.085726 13607 consensus_queue.cc:1059] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:47.086254  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:47.086620 13607 raft_consensus.cc:3055] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:47.087033 13607 raft_consensus.cc:740] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:47.087771  8400 raft_consensus.cc:1275] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:47.087867 13607 consensus_queue.cc:260] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:47.089043  8400 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:47.084286  8297 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:47.092581  8329 raft_consensus.cc:1275] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:03:47.068576 13703 consensus_queue.cc:1059] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:03:47.103264  8400 raft_consensus.cc:1275] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:03:47.104594  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 98850.1us
I20260709 13:03:47.078702 13669 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.081306 13650 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:03:47.107206  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 273222us
I20260709 13:03:47.089401 13607 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:47.078874 13475 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:03:47.113642  8400 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:47.082784 13708 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:47.106940 13465 raft_consensus.cc:697] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:03:47.134261  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.136219  8295 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:47.137840  8295 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:03:47.153836  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.164237 13676 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:47.169798 13675 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:47.170817 13675 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:47.173051 13671 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:47.120795 13650 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:47.116168 13680 consensus_queue.cc:1059] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 13:03:47.196415  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:47.204089  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:47.204895  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:47.192198 13275 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.208632  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:47.209233  9003 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:47.208694 13275 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:47.209885 13650 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:47.202167  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:47.214349 13275 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:47.222584 13689 log.cc:927] Time spent T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.956s	user 0.005s	sys 0.000s
I20260709 13:03:47.175083  8369 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:47.176352  8332 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:47.176873 13708 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:47.178395 13630 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:47.124161 13706 consensus_queue.cc:1048] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:03:47.123412 13667 consensus_queue.cc:1048] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:03:47.180097  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 2261) took 1019 ms. Trace:
I20260709 13:03:47.186209 13640 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:47.114486 13669 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:47.187659 13465 consensus_queue.cc:237] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:47.254647  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:47.259115  8223 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:47.278427  8256 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:47.279137  8223 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:47.260954  8254 rpcz_store.cc:276] 0709 13:03:46.160546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.434807 (+274261us) service_pool.cc:224] Handling call
0709 13:03:47.179958 (+745151us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b5400076cd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:47.180045 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1004893056}
I20260709 13:03:47.264081  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:47.280299  8253 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:47.265645  8248 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:03:47.280840  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:47.280186  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:47.281497  8250 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 16.
I20260709 13:03:47.282294  8257 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:47.287256  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:47.288194  8252 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 12: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:47.287321  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:47.290041  8254 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:03:47.291204  8248 raft_consensus.cc:1240] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:47.281404  8251 raft_consensus.cc:2376] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 16. Re-sending same reply.
W20260709 13:03:47.276470 13728 log.cc:927] Time spent T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:03:47.289369  8223 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:47.266939 13672 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:47.308249  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:47.310760  8972 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:47.314085  8256 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:03:47.315083  9003 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:03:47.315676  8972 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:47.295892 12973 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:47.298203 13631 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:47.285820 13708 consensus_queue.cc:1059] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:03:47.331627  8253 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
I20260709 13:03:47.348665  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:47.482985  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 156550us
W20260709 13:03:47.377678  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 202625us
I20260709 13:03:47.537911  8369 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:47.539048 13579 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:47.541196  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:47.325840 13723 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:47.542313 13710 log.cc:927] Time spent T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.223s	user 0.002s	sys 0.002s
I20260709 13:03:47.315268 13240 consensus_queue.cc:1059] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.079s
W20260709 13:03:47.389268 13728 log.cc:927] Time spent T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:03:47.326779 13527 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:47.324764 13696 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:47.554163 13696 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:47.554652 13696 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:47.325007 13656 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.103s
I20260709 13:03:47.372399 13631 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:47.324842 13672 raft_consensus.cc:697] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:47.559062 13631 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:47.559917 13672 consensus_queue.cc:237] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:47.562800 13669 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:47.530851  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 393052us
I20260709 13:03:47.330605 13538 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:47.533807 13732 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.162s	user 0.001s	sys 0.010s
I20260709 13:03:47.333307 13650 consensus_queue.cc:1059] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.090s
I20260709 13:03:47.533785 13554 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:47.549489  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:47.542524 13723 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:47.553707 13579 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:47.564524  8295 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:47.330317 13651 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:03:47.546492 13724 log.cc:927] Time spent T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.153s	user 0.005s	sys 0.000s
I20260709 13:03:47.585117  8369 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:47.590366  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 53117.8us
I20260709 13:03:47.582187 13651 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:47.591781  8369 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:47.591548  8972 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:47.596326  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:47.601992 13723 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:47.609701 13670 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:47.611310  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:47.597321 13650 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:47.612936 13650 raft_consensus.cc:697] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 15 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:47.613958 13650 consensus_queue.cc:237] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:47.616227 13515 raft_consensus.cc:3055] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:47.616632 13515 raft_consensus.cc:740] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:47.587224 13669 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:47.589152 13654 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:47.594281 13651 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:47.642529  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.643508 13669 raft_consensus.cc:697] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:47.650370 13669 consensus_queue.cc:237] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:47.644132  8399 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 13:03:47.645676 13729 log.cc:927] Time spent T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.092s	user 0.010s	sys 0.001s
I20260709 13:03:47.647219 13570 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:47.648161  8369 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:47.657567  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:47.659092  8369 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:47.662635 13654 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:47.663219 13654 raft_consensus.cc:697] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:47.664131 13654 consensus_queue.cc:237] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:47.641775 13579 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:47.675608  8298 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:47.642037 13630 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:47.641963 13671 raft_consensus.cc:3055] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:47.680166  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:47.682478  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:47.683161  9009 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 18: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:47.656833 13722 consensus_queue.cc:1059] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:03:47.680900 13653 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:47.692595  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:47.680717 13579 raft_consensus.cc:697] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:03:47.695672 13733 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.149s	user 0.005s	sys 0.003s
I20260709 13:03:47.695778 13745 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:47.696420  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:47.697089  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:47.642104 13515 consensus_queue.cc:260] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:47.699900 13702 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:47.724135 13515 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:47.694796 13696 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:47.726725 13696 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:47.727340 13745 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:47.702126  8329 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:03:47.729596  8329 pending_rounds.cc:85] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 11
I20260709 13:03:47.729640 13745 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:47.730017  8329 pending_rounds.cc:107] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:03:47.730334  8329 raft_consensus.cc:2889] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:47.703823  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:47.731213 13579 consensus_queue.cc:237] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:47.731956  8326 raft_consensus.cc:2410] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 15 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:03:47.704149  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:47.732827  8328 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
W20260709 13:03:47.734515  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 58990us
I20260709 13:03:47.744256  8298 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:47.706290 13722 raft_consensus.cc:3055] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:03:47.746687  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 213514us
W20260709 13:03:47.706913  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.707274 13527 raft_consensus.cc:3055] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:47.697259  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:47.713050 13650 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:47.714677 13740 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:03:47.719796  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 665826us
I20260709 13:03:47.682396 13671 raft_consensus.cc:740] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:47.717193 13670 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.735121 13475 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:47.737445 13680 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:47.737912 13659 consensus_queue.cc:1059] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:47.739105 13607 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:47.742697 13653 raft_consensus.cc:697] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:47.743336  8402 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:47.748554  8369 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:47.700525  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 136112us
I20260709 13:03:47.752081 13275 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:47.775040  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:47.749322  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:47.776284  8326 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:03:47.751320  8328 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:47.777019  8295 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:47.754403  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.750908 13527 raft_consensus.cc:740] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:47.704672 13651 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:47.787034 13671 consensus_queue.cc:260] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:47.788040 13671 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:47.788508 13738 log.cc:927] Time spent T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.237s	user 0.004s	sys 0.000s
I20260709 13:03:47.789929 13656 raft_consensus.cc:3055] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:47.790356 13656 raft_consensus.cc:740] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:47.753749 13552 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:47.791157 13656 consensus_queue.cc:260] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:47.791592 13552 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:47.792397 13656 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:47.792202 13653 consensus_queue.cc:237] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:47.793649 13552 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:47.789153 13651 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:47.746867 13722 raft_consensus.cc:740] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:47.751852 12973 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.786593 13665 raft_consensus.cc:3055] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:47.799883  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:47.787464 13570 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:47.802019  8972 leader_election.cc:336] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.510s (SENT)
I20260709 13:03:47.778391 13676 raft_consensus.cc:3055] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:47.757452 13670 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:47.816381  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 44527.7us
I20260709 13:03:47.834347  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:47.785746 13475 raft_consensus.cc:697] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:47.818630  8253 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 15: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
W20260709 13:03:47.761461  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.819700  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:47.821834  8326 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
W20260709 13:03:47.824712  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 52531.3us
I20260709 13:03:47.813284  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:47.810448  8222 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:47.834136  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:47.782979 13001 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:47.843416 13642 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:47.840564  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:47.808365 13709 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:47.778239 13607 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:47.842288  8222 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:47.839665  8255 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:03:47.852836  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:47.855952  8251 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:47.818150 13665 raft_consensus.cc:740] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:47.859002  8222 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:47.798513 13680 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:47.825649 13275 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:47.834527 13676 raft_consensus.cc:740] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:47.872130  8295 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:03:47.873935  8972 leader_election.cc:336] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.554s (SENT)
I20260709 13:03:47.874569  8972 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:47.875706  8255 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: []
W20260709 13:03:47.876354  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 101388us
I20260709 13:03:47.879534  8369 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:47.841313 12973 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:47.863785 13642 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:47.866492 13694 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:47.887120  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:47.887933  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:47.888751  8253 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:47.890508  8403 raft_consensus.cc:1275] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:03:47.890787  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:47.892496  8295 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:47.881177 13740 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:47.864265 13709 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:47.869050 13527 consensus_queue.cc:260] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:47.897118 13549 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:47.900667 13275 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:47.899225  8369 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:47.903872 13680 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:47.905638 13607 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:47.905546  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:47.925504 13549 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:47.935106 13549 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:47.907333  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:47.908149 13675 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:47.935845 12973 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:47.909551  8295 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:47.937314  8295 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:47.939554  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:47.940796 13607 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:47.941538 13607 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:47.941910 13607 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:47.894637 13475 consensus_queue.cc:237] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.11, Last appended by leader: 11, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:47.877419 13722 consensus_queue.cc:260] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:47.921916 13749 log.cc:927] Time spent T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:03:47.882843 13001 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:47.923410  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.924465  8223 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:47.908277 13670 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:47.980334 13630 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:47.951959  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 52809us
I20260709 13:03:47.982287  8369 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:47.946578 13607 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:47.884130 13723 consensus_queue.cc:1059] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:03:47.960770  8250 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:47.971796  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 47415.9us
I20260709 13:03:48.006803  8223 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:47.953131 13275 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:48.008955 13630 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:47.993523  8369 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:47.943616  8295 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:48.021657 13675 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:48.022822 13001 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:48.023450 13001 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.023613  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:48.023983 13001 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:48.024350  8329 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:03:48.009747 13275 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:48.025743  8327 raft_consensus.cc:1240] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: []
W20260709 13:03:48.029584  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 86063.6us
I20260709 13:03:48.030841  8295 leader_election.cc:400] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14.
I20260709 13:03:48.032105  8295 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:48.005787 13670 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:03:48.014257 13674 raft_consensus.cc:3055] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:48.033624 13674 raft_consensus.cc:740] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:48.034821 13674 consensus_queue.cc:260] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:48.036449 13674 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:48.039077 13675 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:48.039824 13675 raft_consensus.cc:697] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:47.991169 13676 consensus_queue.cc:260] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:48.041167 13676 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:48.014115 13723 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:48.043968  8223 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:48.045612 13463 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:48.046051 13722 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:48.046209 13463 raft_consensus.cc:697] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:48.046873  8223 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:48.047266 13463 consensus_queue.cc:237] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:48.047876 13630 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:48.056118  8295 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:48.057862  8250 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
W20260709 13:03:48.085920  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.095243 13680 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.095832 13680 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:48.097927 13680 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:48.097965 12973 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:48.099159  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 56208.4us
I20260709 13:03:48.100167  8295 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:47.896589 13631 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:47.961341 13527 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:48.123446  8255 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 13:03:48.125165  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.130842  8369 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:48.132393  8369 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:48.138799  8254 raft_consensus.cc:1240] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:03:48.142150  8295 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:48.142860  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.145447  8295 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:47.957801 13756 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:48.158620  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:48.159446  8254 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:03:48.163030  8295 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:48.168947 13364 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:48.169063 13755 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:48.170089 13740 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:48.169888 13755 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:48.171702 13667 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:48.168972  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:48.174551  8254 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:48.175189  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:48.129694 13706 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.143033 13577 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.129726 13659 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:03:48.274330  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 99218.4us
I20260709 13:03:48.137779 13694 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.140367 13570 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.956337  8251 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:03:48.161320 13515 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:48.283840  8254 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
I20260709 13:03:48.122851 12973 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:48.041002 13675 consensus_queue.cc:237] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:48.300267 13364 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:48.309334  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.321894  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:48.287369 13667 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.288389 13659 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:48.293016 13639 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.325953  8972 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:48.307593 13670 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:48.322845 13680 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.406977  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:48.286752 13706 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:48.348014  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:48.410997  8251 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:48.339041 13659 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:48.380416 13579 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.105s	sys 0.180s
I20260709 13:03:48.304169 13515 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:48.433504 13734 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:48.434451 13515 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:48.303927 13570 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:48.436787 13570 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:48.341456 13639 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:48.437636 13763 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:48.438074 13763 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:48.438424 13763 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:48.439236 13639 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:48.443178 13752 log.cc:927] Time spent T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc: Append to log took a long time: real 0.520s	user 0.000s	sys 0.002s
I20260709 13:03:48.296206 13577 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:48.308090 13694 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:48.340236 13740 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:03:48.405210 13730 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 1.008s	user 0.000s	sys 0.004s
I20260709 13:03:48.430999 13549 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:48.433140 13757 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:03:48.454140  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 291146us
I20260709 13:03:48.456233  8295 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:03:48.429688 13579 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.105s	sys 0.180s
I20260709 13:03:48.340262 13667 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:48.460297  8251 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:03:48.492859  8254 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:03:48.511366  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:48.513023  8251 raft_consensus.cc:2368] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11.
W20260709 13:03:48.498948  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.490226  9000 consensus_queue.cc:237] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:48.421825 13680 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:48.528431  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:03:48.529153  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 238294us
I20260709 13:03:48.529294  8332 raft_consensus.cc:2376] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13. Re-sending same reply.
I20260709 13:03:48.531317  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:48.532711  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:48.533691  8332 raft_consensus.cc:2368] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13.
I20260709 13:03:48.530534  8369 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:48.539121  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:48.542438  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:48.473688 13750 log.cc:927] Time spent T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.620s	user 0.000s	sys 0.003s
W20260709 13:03:48.504595 13754 log.cc:927] Time spent T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.544s	user 0.001s	sys 0.001s
W20260709 13:03:48.468616 13656 consensus_meta.cc:287] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.676s	user 0.010s	sys 0.000s
I20260709 13:03:48.547880  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:48.567592  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:48.553246  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:48.570252  8323 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 12: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:48.555011  8251 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:48.572570  8251 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:48.573782  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:48.574083  8251 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:03:48.575609  8251 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:03:48.576890  8251 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:48.556309  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:48.558077  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:48.460731 13709 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:48.462625 13740 raft_consensus.cc:697] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 15 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:48.560575 13579 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.561985  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:48.492345 13607 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:48.483690 13644 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.096s
W20260709 13:03:48.558435 13001 consensus_meta.cc:287] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.534s	user 0.009s	sys 0.001s
I20260709 13:03:48.558586 13723 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:48.605336  8330 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:48.491147 13665 consensus_queue.cc:260] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:03:48.503427 13755 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:48.469971 13631 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:48.478246 12973 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:48.559916  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:48.465969 13694 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:48.463338 13764 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:03:48.587080  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 48041us
I20260709 13:03:48.587577 13757 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:48.529615 13499 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:03:48.564813 13737 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 1.013s	user 0.014s	sys 0.002s
I20260709 13:03:48.597996 13723 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.597353 13658 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:03:48.642640 13275 consensus_meta.cc:287] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.617s	user 0.008s	sys 0.003s
I20260709 13:03:48.581135  8332 raft_consensus.cc:1240] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:48.596199 13579 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:48.600656 13001 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:48.635582  8324 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
I20260709 13:03:48.656193  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:48.658213  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2726) took 1284 ms. Trace:
I20260709 13:03:48.668319  8331 rpcz_store.cc:276] 0709 13:03:47.373917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.374061 (+   144us) service_pool.cc:224] Handling call
0709 13:03:47.374382 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:47.670066 (+295684us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:47.670097 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:47.670535 (+   438us) log.cc:844] Serialized 26 byte log entry
0709 13:03:47.801929 (+131394us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:47.801988 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:47.802295 (+   307us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:48.155693 (+353398us) spinlock_profiling.cc:243] Waited 3.64 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:48.657884 (+502191us) raft_consensus.cc:1685] finished
0709 13:03:48.657916 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:48.658143 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":343880,"raft.queue_time_us":295353,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":1558,"spinlock_wait_cycles":8381696,"thread_start_us":112607,"threads_started":1,"wal-append.queue_time_us":141873}
I20260709 13:03:48.670265  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:48.670821  8331 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
I20260709 13:03:48.639971 13665 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:48.661209  8329 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:03:48.656278  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:48.673079  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:48.674129  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:48.678565  8332 raft_consensus.cc:2376] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14. Re-sending same reply.
W20260709 13:03:48.649703 13739 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.087s	user 0.016s	sys 0.043s
I20260709 13:03:48.670856 13658 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:48.676105  8249 raft_consensus.cc:1275] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:03:48.679859  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.686285  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:48.691444  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.655076 13723 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:48.657269 13465 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.693526  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:03:48.680445  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.696888  8220 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:48.698886 12973 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:48.699380 12973 raft_consensus.cc:697] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:48.662072 13757 raft_consensus.cc:697] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:48.654000 13764 raft_consensus.cc:697] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:48.701402  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:48.703497  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:48.715266  8249 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:03:48.715816  8331 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:48.716998  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:48.717561  8972 leader_election.cc:400] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13.
I20260709 13:03:48.667161 13549 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:48.717809  8331 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
W20260709 13:03:48.689857 13739 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.134s	user 0.016s	sys 0.043s
I20260709 13:03:48.710151  8256 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:03:48.691279 13694 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.705294  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:48.651885 13499 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:48.725880  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:48.727188  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:48.719027  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:48.657121 13275 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:48.730023  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:48.700757 13644 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:48.732220  8331 raft_consensus.cc:2410] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:03:48.701678 13577 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:48.734920  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:48.646806 13740 consensus_queue.cc:237] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:48.741823  8402 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:48.735144  8253 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
W20260709 13:03:48.722714 13527 consensus_meta.cc:287] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.579s	user 0.011s	sys 0.000s
I20260709 13:03:48.734733  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:48.731333  8256 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 18: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:48.740029  8295 leader_election.cc:400] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 10 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11.
I20260709 13:03:48.741026  8972 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:48.744308  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:48.744685  8972 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:48.745416  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:48.746640  8326 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:48.731073  8220 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:48.745999  8972 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:48.777148 13527 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:48.777407 13275 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:48.779016 13658 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:48.779417 13475 consensus_queue.cc:1059] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:48.780002  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:48.780505 13675 raft_consensus.cc:3055] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:48.780840 13675 raft_consensus.cc:740] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:48.781162  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:48.781113 13475 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:48.714286 13465 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:48.781629 13675 consensus_queue.cc:260] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:48.782984 13675 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:48.783769 13465 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:48.784576 13001 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:48.786571 13499 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:48.787060 13758 log.cc:927] Time spent T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 13:03:48.788223  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:48.788833  8249 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:48.789673  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:48.790258  8256 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:03:48.700265 12973 consensus_queue.cc:237] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:48.791263  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:48.791955  8256 raft_consensus.cc:2393] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 14: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:48.792810  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:48.793193  8253 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
W20260709 13:03:48.796353  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 65339.2us
I20260709 13:03:48.797873  8220 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:48.793414  8256 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:48.748620  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:48.745831 13760 log.cc:927] Time spent T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:03:48.747396 13577 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:48.812178  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.812862  8295 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:48.813208  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:48.813894  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:48.835278  8295 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:48.815269 13538 consensus_queue.cc:1048] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:48.838768  8399 raft_consensus.cc:2410] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:03:48.815441  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:48.817893  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:48.821763  8369 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:48.824743  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:48.836212  8398 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:48.814538  8972 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:48.837966  8295 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:48.767402 13739 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:48.766566 13607 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:48.824105 13770 raft_consensus.cc:2764] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:03:48.756744 13772 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.735505 13694 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:48.861677  8403 raft_consensus.cc:2393] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:48.900465  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:48.861200  8399 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:48.905166 13694 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:48.828994 13671 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.906194 13671 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:48.909123  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 94638.3us
I20260709 13:03:48.910444  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:48.748296 13644 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:48.891597 13709 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.838792 13275 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:48.856155 13499 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:48.928679  8256 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:48.935494  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:48.913760 13772 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:48.892014 13579 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:48.828320 13781 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:48.973953 13538 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:48.974439 13777 log.cc:927] Time spent T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:03:48.938340 13499 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.856645 13667 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:48.937812  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:48.939277 13671 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:48.923516 13577 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:48.942116 13765 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:48.942677 13767 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:48.943717 13776 log.cc:927] Time spent T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
I20260709 13:03:48.944630  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:03:48.951855 13721 log.cc:927] Time spent T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 1.908s	user 0.004s	sys 0.007s
I20260709 13:03:48.952039  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:48.974798 13538 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:48.955242  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:49.021508  8331 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:48.955591  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:49.023296  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:48.895233 13764 consensus_queue.cc:237] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:48.962599  8251 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:03:49.024530  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:49.025480  8254 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:48.967275  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:48.990677 13499 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:48.838943 13757 consensus_queue.cc:237] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:48.974061 13781 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.829233 13658 consensus_queue.cc:1059] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.095s
I20260709 13:03:48.926077  8972 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:48.992674  8369 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:48.995978 13577 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:48.996711  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:48.964205 13680 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:48.954028  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:48.936136 13709 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:48.914143 13734 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:48.932657 13739 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:49.030637  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:49.036100  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 43528.9us
I20260709 13:03:48.995265  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:49.036947  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:49.038851  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 112842us
I20260709 13:03:49.055267  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:03:49.026623 13676 consensus_meta.cc:287] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.984s	user 0.018s	sys 0.002s
W20260709 13:03:49.029739 13792 log.cc:927] Time spent T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:03:49.026913 13665 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.041777  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:49.001579 13579 consensus_queue.cc:1059] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:03:49.064327  8332 raft_consensus.cc:2410] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
I20260709 13:03:49.047947 13577 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:49.070804  8331 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:49.072506  8330 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:03:49.053156 13763 consensus_meta.cc:287] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.614s	user 0.009s	sys 0.007s
I20260709 13:03:49.059650  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:49.075289  8328 raft_consensus.cc:2410] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:03:49.051239 13651 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:49.045773 13001 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:49.044139 13781 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:49.038223 13538 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:49.073719 13674 consensus_meta.cc:287] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 1.037s	user 0.015s	sys 0.001s
I20260709 13:03:49.049479 12973 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:49.070329 13653 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:49.102815  8397 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:49.090725  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:49.096101  8250 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 2278) took 1265 ms (client timeout 1520 ms). Trace:
I20260709 13:03:49.063088 13739 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:03:49.103824  8250 rpcz_store.cc:269] 0709 13:03:47.818971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.819124 (+   153us) service_pool.cc:224] Handling call
0709 13:03:49.084405 (+1265281us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b540016f654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:49.084524 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":275906944}
I20260709 13:03:49.107309  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:49.108800  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 53387us
W20260709 13:03:49.109338  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:49.109962  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.110687  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:49.114500  8250 raft_consensus.cc:1240] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:49.115020  8331 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:03:49.118414  8330 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:49.105780 13653 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.119493 13653 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:49.115697  8249 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:03:49.107139 13739 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:49.085062 13763 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:49.100030 13001 raft_consensus.cc:697] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:49.079458 13665 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:49.105931 13774 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:49.137353  8331 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:49.143122  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 819) took 1047 ms. Trace:
I20260709 13:03:49.143616  8248 rpcz_store.cc:276] 0709 13:03:48.095712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.102261 (+  6549us) service_pool.cc:224] Handling call
0709 13:03:48.102661 (+   400us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:48.103137 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.103176 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:48.103911 (+   735us) log.cc:844] Serialized 52 byte log entry
0709 13:03:48.904922 (+801011us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b18002b20a0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e0d1 00007f8828779a0f
0709 13:03:48.975698 (+ 70776us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:48.975738 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.975807 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:49.140642 (+164835us) raft_consensus.cc:1685] finished
0709 13:03:49.140669 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.143055 (+  2386us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10983,"raft.queue_time_us":20955,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":15367,"spinlock_wait_cycles":4052224,"thread_start_us":70578,"threads_started":1,"wal-append.queue_time_us":925740}
I20260709 13:03:49.144443 13774 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:49.154006 13653 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:49.156208 13653 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:49.111297 13740 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:49.158596  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2447) took 1136 ms. Trace:
I20260709 13:03:49.159087  8326 rpcz_store.cc:276] 0709 13:03:48.021916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.023654 (+  1738us) service_pool.cc:224] Handling call
0709 13:03:48.023930 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.747112 (+723182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.158319 (+411207us) spinlock_profiling.cc:243] Waited 722 ms on lock 0x7b54003cb254. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:49.158530 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1661409536}
I20260709 13:03:49.117780 13669 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:49.205236 13669 raft_consensus.cc:697] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:49.206195 13669 consensus_queue.cc:237] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:49.179230  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:49.212807  8326 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 10.
I20260709 13:03:49.214082  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:49.214651  8222 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:49.180138  8250 raft_consensus.cc:1240] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 15. Current term is 16. Ops: []
W20260709 13:03:49.215583  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.127154 13656 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:49.219607  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:49.181154 13001 consensus_queue.cc:237] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:49.240617  8405 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: []
W20260709 13:03:49.183404 13722 consensus_meta.cc:287] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 1.137s	user 0.012s	sys 0.000s
I20260709 13:03:49.188078  8255 raft_consensus.cc:1275] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:49.189808  8256 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:49.190215 13709 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:49.192095 13665 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:49.195508 13499 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:49.178303 13644 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:49.248191 13644 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:49.248207 13665 consensus_queue.cc:1048] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:49.249047 13644 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:49.251289  8222 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:49.204699 13659 consensus_meta.cc:287] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.755s	user 0.012s	sys 0.001s
I20260709 13:03:49.218292  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:49.221223  8220 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:49.224050  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:49.254819  8250 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:49.224959  8248 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:49.226727  8399 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:49.215590  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:49.197896 13793 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:03:49.259492  8326 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:49.105466 12973 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:49.266737  8222 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:49.267453  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.269515  8298 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:49.273147  8298 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:49.238694 13674 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:49.239020 13653 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:49.196694 13723 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:49.256872 13773 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:49.265127 13709 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:49.262003 13656 raft_consensus.cc:697] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 17 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:49.300938  8331 raft_consensus.cc:1275] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:03:49.302021  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 185007us
I20260709 13:03:49.298830  8250 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:49.272604 13708 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:49.272990 13275 raft_consensus.cc:697] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:49.270783 13644 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:49.267709 13659 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:49.284575 13463 consensus_queue.cc:1059] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:03:49.316102  8331 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:03:49.278878 13734 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:49.274977 13772 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:49.275729 13640 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:49.318946  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:49.347434  8328 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:49.266743 13579 consensus_queue.cc:1059] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:03:49.242728 13740 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:49.284582 13552 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.278712 13675 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:49.287408 13765 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:49.309468 13658 raft_consensus.cc:3055] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:49.291507 13709 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.371135 13709 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:49.334883  8324 raft_consensus.cc:1275] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 13:03:49.362071 13740 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:49.356272 13764 consensus_queue.cc:1059] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:03:49.366708  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:49.368305  8327 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:49.304593 13674 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:49.299908 13653 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:49.346979 13808 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:49.389108  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 145940us
W20260709 13:03:49.389544  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 112732us
I20260709 13:03:49.355947 13644 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:49.401441  8369 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:49.400218  8324 raft_consensus.cc:1240] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:49.408951  8222 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:49.368439 13658 raft_consensus.cc:740] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:49.378422 13579 consensus_queue.cc:1243] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer 8c42259d7c4b463a9f485f42c320c81b log is divergent from this leader: its last log entry 16.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:03:49.410807  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:49.415438  8251 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:49.373167 13694 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:49.395645 13740 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:49.421203 13821 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:49.424460  8331 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:49.406522 13001 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:49.413476 13644 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:49.442971  8251 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:03:49.372170 13552 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:49.414006 13463 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:49.372987 13765 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:49.459054  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:49.461000  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:49.418460 13773 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:49.461567  9007 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:49.462877  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:49.465377  9005 raft_consensus.cc:2410] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:03:49.438537  8222 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:03:49.464452  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 55789.4us
I20260709 13:03:49.468658  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:49.469686  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:49.472620  9006 raft_consensus.cc:2410] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
I20260709 13:03:49.424091 13821 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:49.471166  8222 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:49.456758 13741 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.407076 13656 consensus_queue.cc:237] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:49.406337 13808 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:49.479861  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:49.425706 13694 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:49.463078 13499 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:49.462153 13527 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.468927 13812 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:49.462800 13463 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:49.427120 13740 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:49.425635 13275 consensus_queue.cc:237] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:49.497473  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:49.456894 13001 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:49.507052 13463 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:49.507654 13463 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:49.498543 13824 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:49.509454 13808 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:49.511047 13808 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:49.511469 13808 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.511816 13808 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:49.513902 13674 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:49.488622 13741 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:49.487622 13723 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:49.516736 13659 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:49.516783  9009 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:49.491510 12973 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:49.466640 13658 consensus_queue.cc:260] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:49.525398 13579 consensus_queue.cc:1059] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:49.526091 13658 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:49.526548 13579 raft_consensus.cc:3055] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:49.526963 13579 raft_consensus.cc:740] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:49.495611 13812 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.530160 13552 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:49.530522 13812 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:49.535209 13694 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:49.510982 13765 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:49.493345 13527 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:49.539388 13771 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.823s	user 0.005s	sys 0.000s
I20260709 13:03:49.542050  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2453) took 1418 ms. Trace:
I20260709 13:03:49.542533 13644 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:49.500525 13803 log.cc:927] Time spent T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:03:49.545650  9005 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:03:49.481642 13821 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:49.553005 13767 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:49.542462  8325 rpcz_store.cc:276] 0709 13:03:48.123979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.124105 (+   126us) service_pool.cc:224] Handling call
0709 13:03:48.158042 (+ 33937us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:48.303526 (+145484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.303571 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:48.307282 (+  3711us) log.cc:844] Serialized 26 byte log entry
0709 13:03:48.506311 (+199029us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:48.676803 (+170492us) log.cc:844] Serialized 14 byte log entry
0709 13:03:48.677008 (+   205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.677085 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:49.538827 (+861742us) spinlock_profiling.cc:243] Waited 25.8 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:49.541703 (+  2876us) raft_consensus.cc:1685] finished
0709 13:03:49.541808 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.541964 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18554,"prepare.queue_time_us":392292,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":18809,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":17577,"spinlock_wait_cycles":59265408,"thread_start_us":372416,"threads_started":3,"wal-append.queue_time_us":317180}
I20260709 13:03:49.540758 13781 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:49.558393  9006 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 15 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:49.563892 13774 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:49.564038 13709 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:49.564919 13708 raft_consensus.cc:3055] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:49.565335 13708 raft_consensus.cc:740] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:49.568063  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 773) took 1040 ms. Trace:
I20260709 13:03:49.566448  8401 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:03:49.569778 13708 consensus_queue.cc:260] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:49.570883 13708 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:49.572192  8223 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:49.572522  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:49.573814  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:49.574589  9004 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 18: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:49.544142 13739 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:49.577227  9008 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:49.568435  8329 rpcz_store.cc:276] 0709 13:03:48.527387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.527505 (+   118us) service_pool.cc:224] Handling call
0709 13:03:49.567911 (+1040406us) spinlock_profiling.cc:243] Waited 2.26 ms on lock 0x7b540011e4d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:49.568011 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5191040}
I20260709 13:03:49.547821 13824 raft_consensus.cc:697] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:49.585258  8254 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15. Ops: []
W20260709 13:03:49.590610  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 90726.2us
I20260709 13:03:49.594606  8328 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:49.593221  9007 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
W20260709 13:03:49.601564  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 93493.3us
I20260709 13:03:49.610442 12973 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:49.615144 13709 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:49.630555  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.631644  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:49.632891  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:49.633576  8399 raft_consensus.cc:3055] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:49.632983  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:49.633958  8399 raft_consensus.cc:740] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:03:49.636299  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 161273us
I20260709 13:03:49.634923  8399 consensus_queue.cc:260] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:49.640344  8399 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:49.647126  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:49.648885  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:49.649690  8402 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:03:49.651228  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:49.654117  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:49.655936 13824 consensus_queue.cc:237] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:49.671406  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 99289.9us
I20260709 13:03:49.674259  8251 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:49.677839  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:49.678449  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.678891  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:49.679096  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.678544  8256 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:49.681034  8399 raft_consensus.cc:1275] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 7. (term mismatch)
I20260709 13:03:49.681492  8399 pending_rounds.cc:85] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 6
I20260709 13:03:49.683254  8399 pending_rounds.cc:107] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 15.7
I20260709 13:03:49.683655  8399 raft_consensus.cc:2889] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:49.643466 13577 consensus_meta.cc:287] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.547s	user 0.010s	sys 0.000s
I20260709 13:03:49.684861  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:49.633944 13740 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:49.687078  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:49.689795 13781 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:49.690918 13739 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:49.698431  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:49.699194  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:49.702817  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.710997  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:49.718561  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.719560  8223 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:49.724669  8223 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:49.702509 13515 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:49.729177  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:49.729499  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:49.769790  8400 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:49.757144 13767 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.723039 13694 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:49.714991 13463 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:03:49.782140  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 60253.5us
I20260709 13:03:49.759123 13814 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:49.711256 13577 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:49.763476 13640 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:49.726907 13772 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:49.719658 13831 consensus_queue.cc:1048] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
W20260709 13:03:49.784766  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 56455.2us
I20260709 13:03:49.722715 13658 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:03:49.774559 13674 consensus_queue.cc:1059] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:03:49.763165 13515 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:49.722868 13826 consensus_queue.cc:1059] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:03:49.813913  9004 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:49.791177  8223 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:49.817982  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:49.828254  8403 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:03:49.830286  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:49.831768  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:49.791819  8222 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:49.817740  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:03:49.789888 13816 log.cc:927] Time spent T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
I20260709 13:03:49.834712  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:49.838861  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 47111.1us
I20260709 13:03:49.843221  8222 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:49.833783 13814 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:49.767915 13827 consensus_queue.cc:1059] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:03:49.849352 13827 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:03:49.849687 13809 log.cc:927] Time spent T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.535s	user 0.004s	sys 0.000s
I20260709 13:03:49.849892 13827 raft_consensus.cc:697] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:49.850312 13515 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:49.831336 13772 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:49.850919 13827 consensus_queue.cc:237] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:49.851554 13527 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:49.852941 13772 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:49.802624 13808 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:49.831100 13669 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:49.813822 13577 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:49.807643 13763 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:49.848186  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:49.790023 13463 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:49.854784 13741 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:49.842554 12973 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:49.839991 13826 consensus_queue.cc:1243] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:03:49.782768 13767 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:49.873813 13808 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:49.868175 13669 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:49.885644  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 169914us
I20260709 13:03:49.887522  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:49.868651 13757 raft_consensus.cc:3055] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:49.869069 13814 raft_consensus.cc:697] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:49.875859 12973 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.894016  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:49.876072 13515 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:49.838825 13812 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:49.896360  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:49.905117  8324 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:49.894551  8328 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:03:49.895251  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:49.900482  8329 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:03:49.906651  8331 raft_consensus.cc:2376] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18. Re-sending same reply.
I20260709 13:03:49.896631  8372 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:49.901608  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:49.902212  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:49.853704 13818 consensus_queue.cc:1048] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:03:49.903676  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:49.875366 13840 consensus_queue.cc:1048] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:03:49.909768  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:49.891510 13838 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:49.892143 13757 raft_consensus.cc:740] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:49.899713 13640 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.902966 12973 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:49.914805  8249 raft_consensus.cc:2368] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13.
I20260709 13:03:49.916002  8406 pending_rounds.cc:85] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 6
I20260709 13:03:49.927915  8406 pending_rounds.cc:107] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:03:49.928223  8406 raft_consensus.cc:2889] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:49.883982 13821 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:49.899459 13808 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:03:49.912210 13832 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:49.950242  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:49.996090  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:49.950142 13838 raft_consensus.cc:697] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:49.998016  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:49.999280  8324 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:49.933631 13640 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:49.999738  8295 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:50.007396  8401 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:50.008028  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:50.009093  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:50.010445  8404 raft_consensus.cc:2393] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 12: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:50.011740  8403 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:03:50.012707 13706 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:50.012974  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:50.014318  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:50.017547 13813 consensus_queue.cc:1048] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.084s
W20260709 13:03:50.019496 13835 log.cc:927] Time spent T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:03:50.023402 13838 consensus_queue.cc:237] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:50.027875 13723 raft_consensus.cc:3055] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:50.028409 13723 raft_consensus.cc:740] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:50.031118 13640 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:50.034067  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:50.040530  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:50.041848  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:50.042510  9007 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:03:50.048631  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:50.052879  9007 raft_consensus.cc:1217] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 15 LEADER]: Deduplicated request from leader. Original: 12.8->[14.9-14.9]   Dedup: 14.9->[]
I20260709 13:03:50.054072  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:50.058575  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:50.053515  9007 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 15 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:03:49.954134 13814 consensus_queue.cc:237] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:50.065111  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:50.069046  8974 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:50.070086  9009 raft_consensus.cc:2410] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:03:50.056964 13806 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.071537  9007 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:49.998337 13757 consensus_queue.cc:260] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:50.076962  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:50.029254 13723 consensus_queue.cc:260] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:50.079833  8405 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18.
I20260709 13:03:50.086033  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:50.086740  8398 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:50.074335 13808 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:50.078740 13756 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:50.084172 13832 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:50.090097 13757 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:50.075887 13806 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:50.104355  8398 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:50.155020  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2733) took 1457 ms. Trace:
I20260709 13:03:50.155474  8330 rpcz_store.cc:276] 0709 13:03:48.697171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.106447 (+409276us) service_pool.cc:224] Handling call
0709 13:03:49.115615 (+  9168us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:49.124319 (+  8704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.154846 (+1030527us) spinlock_profiling.cc:243] Waited 8.86 ms on lock 0x7b5400365950. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:50.154964 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20374400}
I20260709 13:03:50.102979 13812 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:50.158607  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:50.161898 13001 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:50.164816 13833 log.cc:927] Time spent T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.483s	user 0.000s	sys 0.001s
W20260709 13:03:50.127547 13848 log.cc:927] Time spent T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:03:50.093223 13740 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:03:50.094659 13739 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:03:50.129184 13806 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:50.105800 13767 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:50.171783 13755 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:50.172370 13755 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:50.121297 13821 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:50.174389 13755 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:50.174465  9006 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:50.111441 13832 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:50.110973 13463 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:50.179824 12973 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:50.185074 13653 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:50.185536  8324 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:03:50.186839  8404 raft_consensus.cc:3055] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:50.187261  8404 raft_consensus.cc:740] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:50.188043  8404 consensus_queue.cc:260] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:50.189126  8404 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:50.191507 13669 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:50.108510 13756 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:50.199134 13756 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:50.200290  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.200062 13579 consensus_queue.cc:260] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:50.202440  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.207666  8222 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:50.208005 13579 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:50.208377 13840 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:50.208884 13840 raft_consensus.cc:697] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:50.210933 13840 consensus_queue.cc:237] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:50.216902  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:50.216881  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:50.242247  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:50.247036  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:50.254025  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:50.257680  8404 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:03:50.254716  9000 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:03:50.242702 13757 consensus_queue.cc:1059] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
W20260709 13:03:50.265676 13654 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
W20260709 13:03:50.277168  8397 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 2839) took 1462 ms (client timeout 1632 ms). Trace:
W20260709 13:03:50.277592  8397 rpcz_store.cc:269] 0709 13:03:48.814002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.815206 (+  1204us) service_pool.cc:224] Handling call
0709 13:03:50.276442 (+1461236us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b54000221d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:50.276549 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":604150016}
I20260709 13:03:50.264858 13778 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:03:50.275731 12973 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:50.302138 13797 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.017s	sys 0.010s
W20260709 13:03:50.302801 13797 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.018s	sys 0.010s
W20260709 13:03:50.325011 13830 log.cc:927] Time spent T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.406s	user 0.002s	sys 0.000s
I20260709 13:03:50.354754 13772 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.355334 13772 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:50.358496 13772 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:50.363306 13757 raft_consensus.cc:3055] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:50.363756 13757 raft_consensus.cc:740] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:50.364540 13757 consensus_queue.cc:260] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:50.376452 13722 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:50.377170 13722 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
W20260709 13:03:50.379084  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.379004 13831 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:50.379411 13722 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:50.380023 13762 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:03:50.379686 13831 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:50.383219 13831 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:50.395074  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 566099us
I20260709 13:03:50.396463 13772 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:50.399327  8223 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:50.401593 13757 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:50.401889 12973 consensus_queue.cc:1059] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:50.403041 12973 raft_consensus.cc:3055] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:50.403405 12973 raft_consensus.cc:740] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:50.404281 12973 consensus_queue.cc:260] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:50.422210  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:50.422698  8972 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:50.425915  8972 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:50.426977 13739 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:50.427177  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:50.427464 13739 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:50.427906 13739 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:50.428385  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 2291) took 1703 ms. Trace:
I20260709 13:03:50.429684  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:50.422035 13772 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:50.430269  9004 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:03:50.430321 13772 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:03:50.430847  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.433697  8972 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:50.434415 13656 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:50.435055 13656 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:50.436180  8972 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:50.436874 13806 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:50.438234 13499 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.438980  8972 leader_election.cc:400] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13.
I20260709 13:03:50.438726 13499 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:50.439277  8972 leader_election.cc:403] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:50.440779 13499 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:50.442435 13659 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.428743  8252 rpcz_store.cc:276] 0709 13:03:48.725282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.183447 (+458165us) service_pool.cc:224] Handling call
0709 13:03:49.674370 (+490923us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:49.716926 (+ 42556us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:49.716962 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:49.717027 (+    65us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:49.717079 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.717162 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:50.374686 (+657524us) spinlock_profiling.cc:243] Waited 1.7 ms on lock 0x7b180045db40. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882569f2a6 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4
0709 13:03:50.428221 (+ 53535us) spinlock_profiling.cc:243] Waited 491 ms on lock 0x7b5400076cd0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:50.428332 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59203,"raft.queue_time_us":41952,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":5637,"spinlock_wait_cycles":1132172544}
I20260709 13:03:50.430812 13772 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:50.452077  8372 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:50.442976 13736 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:50.457326  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:50.458822  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:50.463116  8250 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: [13.7-15.8]
I20260709 13:03:50.466385  8250 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:50.467206  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.480756  9004 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:50.484879  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:50.485706  8248 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:03:50.486619  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:50.487416  8250 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:50.483460  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:50.521381  8254 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:50.521746  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:50.410099 13832 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:50.526477  8248 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:50.521009  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 521289us
I20260709 13:03:50.523110  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:50.527922  8295 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:50.528436  8249 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:50.498382  8250 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:03:50.523866  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:50.523283  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.500821  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:50.535326  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:50.535394  8298 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:50.536649  9002 raft_consensus.cc:2410] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:03:50.538877  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:50.481545 13672 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:50.443164 13765 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.479154 12973 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:50.495649 13656 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13.
W20260709 13:03:50.570698  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 53759.2us
W20260709 13:03:50.572451  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 202491us
I20260709 13:03:50.443545 13778 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.105s
I20260709 13:03:50.573539  8248 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:03:50.479521 13465 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:03:50.577672  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.495934 13827 consensus_queue.cc:1059] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.087s
I20260709 13:03:50.544932 13669 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:50.541972 13538 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:50.443014 13659 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:50.582147  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 54312.8us
W20260709 13:03:50.583218  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 47907.9us
I20260709 13:03:50.610590  8298 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:50.610782  8295 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:50.611394  8323 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:50.614248  8254 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:03:50.585805  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:50.614975  8250 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:50.586642  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:50.587472  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:50.589673  8328 raft_consensus.cc:3055] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:50.590129  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:50.591735  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:50.616742  8328 raft_consensus.cc:740] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:50.580577 13665 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.456161 13736 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:50.522433 13658 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:03:50.567828 13764 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.606804  8297 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:50.567229 13672 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:50.601872  8253 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:03:50.453632  8405 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:46846
I20260709 13:03:50.628715  8405 raft_consensus.cc:606] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:50.590298 13667 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:50.579195 13800 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.622524  8254 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:03:50.634975  8298 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:50.635928  8253 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:03:50.576519 13765 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:50.618233  8328 consensus_queue.cc:260] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:50.648015  8328 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:50.627270  8295 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:50.629151  8405 raft_consensus.cc:614] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:50.617352  8329 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:03:50.630159 13672 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:50.633589  8248 raft_consensus.cc:1275] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:50.616711 13734 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.638155 13671 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:50.653254 13734 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:50.654332 13475 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:50.654788 13475 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:50.655171 13475 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:50.655437 13734 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:50.625399  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:50.657030  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:50.657606  8330 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:50.658443  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:50.659174  8330 raft_consensus.cc:2393] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 14: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:50.659921  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:50.660995  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:50.651643  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:50.662045  8330 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:03:50.663282  8330 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:03:50.627765 13665 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:50.666528 13665 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:50.668181 13659 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:50.669899 13736 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:50.634148 13764 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:50.672341 13764 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:50.678263 13549 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:50.678853  8369 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:50.679870 13549 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:50.683488  8329 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:50.647370 13800 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:50.685581 13800 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:50.605190 13499 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:50.686343  8332 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:03:50.687110  8326 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:50.616535 13669 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:50.688933 13723 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:50.689743 13669 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:50.689790  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:50.691001  8257 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:50.691156  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:50.691990  8249 raft_consensus.cc:2393] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 14: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:03:50.692118  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 85390.7us
I20260709 13:03:50.692764  8255 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:03:50.693403  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:50.693781  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:50.694089  8251 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:03:50.694582  8254 raft_consensus.cc:2393] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 14: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:50.695111 13774 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:50.695760 13774 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:50.696159 13774 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:50.697121  8254 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:03:50.697782  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:50.701249  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 66353.4us
I20260709 13:03:50.698249  8324 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:50.697813  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:50.693814  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:50.704023  8248 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:03:50.691942  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:50.704756  8257 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:50.697150  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:50.699234 13675 raft_consensus.cc:3055] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:50.695240  8251 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:03:50.706183  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.705799  8405 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:03:50.713999  8255 raft_consensus.cc:2393] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 17: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:50.692951  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:50.707670  8328 raft_consensus.cc:1275] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:50.715586  8249 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 16.
I20260709 13:03:50.716400  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:50.717747  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:03:50.718518  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 91342.5us
I20260709 13:03:50.718781  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:50.715001  8223 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:50.721678  8254 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:03:50.729027  8369 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:50.623373 13846 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:50.738588  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:50.729084 13671 raft_consensus.cc:697] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:50.739435  8399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:46846
I20260709 13:03:50.739944 13680 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:50.742254  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:50.743033 13814 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:50.743614 13800 consensus_queue.cc:1059] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:50.728685 13675 raft_consensus.cc:740] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:50.747687 13765 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:50.748054  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:50.781870  8328 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 16: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:50.748594 13654 raft_consensus.cc:3055] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:50.630754 13676 consensus_queue.cc:1059] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.118s
I20260709 13:03:50.749020 13549 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:50.750842  8222 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:50.794480  8324 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
W20260709 13:03:50.795480  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 44706.1us
I20260709 13:03:50.751829  8295 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:50.608983 13465 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:50.755789  8251 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:50.754030  9006 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:50.773747 13798 consensus_queue.cc:1059] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:03:50.775993 13831 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:03:50.760525  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:50.768803  8369 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:50.770584  8330 raft_consensus.cc:2393] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 13: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:50.771171  8399 raft_consensus.cc:606] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:50.765973  8298 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:50.731367 13499 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:50.775599 13680 raft_consensus.cc:697] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:50.789422  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:50.771437 13846 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:50.906010  8328 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18.
I20260709 13:03:50.776445 13842 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:50.801936  8257 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:03:50.896601  8399 raft_consensus.cc:614] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:03:50.834770  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:50.809545  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.813637  8220 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:50.796000 13764 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:50.763972 13579 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:50.777727 13824 consensus_queue.cc:1048] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.134s
I20260709 13:03:50.901048 13669 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.800508  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:50.906414  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:50.908216  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 139465us
I20260709 13:03:50.860802  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:50.909551  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:50.911022  8404 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:50.938762  8369 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:50.790159 13654 raft_consensus.cc:740] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:50.775993 13554 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:50.928543  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 114981us
W20260709 13:03:50.929513  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 177734us
I20260709 13:03:50.945721  8220 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:50.946350  8295 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:50.801934 12973 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:03:50.908846 13757 consensus_meta.cc:287] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.001s
I20260709 13:03:50.866698  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:50.948992  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.807096 13465 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:50.935038 13770 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:50.935551 13800 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:50.942760  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:50.943624  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:50.912902 13658 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:03:50.950049  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 184166us
I20260709 13:03:50.947886 12973 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:50.800508 13671 consensus_queue.cc:237] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:50.951808  8220 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:50.830765 13672 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:50.786604 13739 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:50.992939  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:50.940364 13824 consensus_queue.cc:1243] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [LEADER]: Peer ff7a6fe533e94c1bae555b4cae427728 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:03:50.935153 13807 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.935652 13579 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:50.955193  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:50.953320 13667 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:50.970165  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:50.951232 13554 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:03:50.948009 13499 raft_consensus.cc:3055] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:50.947764 13762 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:51.013597 13499 raft_consensus.cc:740] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:51.014552 13499 consensus_queue.cc:260] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.12, Last appended by leader: 12, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:51.015817 13499 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:50.988821  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:50.989866  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:51.017568 13658 raft_consensus.cc:697] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:50.953680 13755 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:03:50.992614  8220 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:50.993867  8332 raft_consensus.cc:2410] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:03:50.999017  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:50.952992  8295 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:50.996795 13654 consensus_queue.cc:260] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:51.003268 13739 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:51.004966 13860 log.cc:927] Time spent T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:03:50.983121 13757 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:03:50.963629 13736 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:50.964313 13800 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:51.008100  9006 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:03:50.999523 13838 consensus_queue.cc:1048] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:03:50.936134 13675 consensus_queue.cc:260] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:51.012382 13672 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:51.017812  8328 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:03:50.916260 13842 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:50.986855 12973 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:51.023808  8323 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:50.991330  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:50.943388 13669 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:51.028961  8220 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:50.997485 13463 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:51.028653  9000 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:03:51.006078 13579 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:50.963167 13306 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:03:51.034437  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 81550.5us
I20260709 13:03:51.006546 13807 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:51.037528 13800 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:51.062860  8251 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:51.063395  8255 raft_consensus.cc:1275] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:03:51.064432  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:51.064553  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:51.065464  8256 raft_consensus.cc:2393] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 17: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:51.065671  8254 raft_consensus.cc:2393] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 16: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:51.039855  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:50.952986 13764 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:51.054157  8220 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:51.036123 13814 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:51.066569  8405 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:51.066525  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:51.040462 13659 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:51.071127  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:51.076357  8254 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:03:51.072177  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:51.073853  8220 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:51.078164  8251 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:03:51.073289  8399 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:51.079648 13768 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:51.080857 13364 consensus_queue.cc:1048] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:51.082273  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:51.082619  8295 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:51.083030  8257 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:03:51.084044  8295 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:51.084690  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:51.009462 13680 consensus_queue.cc:237] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:03:51.044152 13676 consensus_queue.cc:1059] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:03:51.089679  8250 raft_consensus.cc:2393] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 15: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:51.056596 13736 raft_consensus.cc:697] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 16 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:51.060599 13631 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:51.088816 13734 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:51.093820 13676 consensus_queue.cc:1243] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:03:51.076521 13817 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:51.088639 13275 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:51.055363 13658 consensus_queue.cc:237] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:51.095443 13734 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:51.092890 13631 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:51.111092 13475 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:51.111534 13774 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:51.099931 13275 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:51.121488  8295 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:51.098852 13669 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:51.126978  8295 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:03:51.132877  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 427242us
I20260709 13:03:51.101912 13846 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:51.169876  8972 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:51.122201 13631 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:51.159173 13659 raft_consensus.cc:697] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:51.159986  8254 raft_consensus.cc:1240] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:51.185272  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:51.162245  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 212807us
I20260709 13:03:51.164484  9000 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:03:51.118938 13734 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:51.188680  8222 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:51.190997  8369 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:51.158229  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:51.178872  8398 raft_consensus.cc:1275] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:03:51.176676  8220 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:51.127422 13275 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:51.185887  8249 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:51.187861  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:51.203006  8220 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:51.183033  8372 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:51.203526  8405 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:51.204355  8399 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:51.194640  8251 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:03:51.198437  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:51.205739  8330 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
I20260709 13:03:51.172926 13538 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:03:51.191987 13772 consensus_meta.cc:287] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.671s	user 0.008s	sys 0.002s
I20260709 13:03:51.211091 13846 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:51.211310 13806 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:51.212497 13755 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:51.213541  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:51.216080  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:03:51.226981  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 57202.1us
I20260709 13:03:51.273898  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:51.210618 13827 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:51.229691 13538 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:51.226774 13465 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:51.315371  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:51.314186  8400 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:51.319801  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 46019.5us
W20260709 13:03:51.316793  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 103418us
I20260709 13:03:51.321663  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:51.321811  8972 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:51.207628 13665 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.324227  8249 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:03:51.329361  8399 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:03:51.208263 13671 consensus_queue.cc:1048] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.112s
I20260709 13:03:51.272742 13672 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.343128  8972 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:51.234920 13873 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:51.232263 13846 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:03:51.349939  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 127048us
I20260709 13:03:51.355513  8369 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:51.271160 13667 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:51.324962 13813 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:51.324822 13554 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:51.325629 13817 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:51.181659 13736 consensus_queue.cc:237] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:51.326633 13549 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:51.317938 13538 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:51.283746 13306 consensus_queue.cc:1059] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.074s
I20260709 13:03:51.338729 13832 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:51.370394  8222 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:51.374424  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:51.233824 13772 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:51.383797  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 180877us
I20260709 13:03:51.358604 13778 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:51.393052  8220 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:51.368199 13842 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:51.346554 13762 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:03:51.384244 13875 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:51.387954 13866 log.cc:927] Time spent T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.270s	user 0.000s	sys 0.001s
I20260709 13:03:51.387997 13878 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:51.388834 13755 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:51.338467 13676 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:03:51.364041 13846 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:51.351275 13669 consensus_queue.cc:1059] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
W20260709 13:03:51.404513  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.395874 13873 raft_consensus.cc:697] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:03:51.407650 13723 consensus_meta.cc:287] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.718s	user 0.009s	sys 0.002s
I20260709 13:03:51.407673 13873 consensus_queue.cc:237] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:51.403482  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:03:51.336089 13665 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:51.380887 13549 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:03:51.389340 13869 log.cc:927] Time spent T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
I20260709 13:03:51.374226 13877 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:51.357849 13672 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:51.400838  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 57826.1us
I20260709 13:03:51.390321 13306 raft_consensus.cc:3055] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:51.347399 13659 consensus_queue.cc:237] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:51.424126 13895 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:51.431331 13672 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:51.431411 13549 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:51.403228 13806 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:51.437600 13892 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.439469 13772 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:51.444231  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:51.445374  8398 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:51.451561 12973 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:51.405300 13770 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:51.464732  8256 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:51.466109  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:51.457126 13579 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:51.469991 12973 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:51.472384  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:51.473304  8405 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:51.475253  8972 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:51.448568 13306 raft_consensus.cc:740] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:51.453840 13895 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:51.478032  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:51.453804 13465 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:51.481383  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:51.470301 13762 raft_consensus.cc:3055] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:51.466027 13770 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:51.486483 13895 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:51.494606 13739 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:51.495330 13873 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:51.497494 13876 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:51.491482 13806 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:51.482350 13778 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:51.479285 13891 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:51.532814 13778 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.533440 13778 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:51.535483 13778 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:51.537627 13865 raft_consensus.cc:3055] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:51.538053 13865 raft_consensus.cc:740] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:51.538887 13865 consensus_queue.cc:260] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:51.542073  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:51.544281 13549 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:51.511420 13890 consensus_queue.cc:1059] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:03:51.545688 13890 raft_consensus.cc:3055] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:51.546033 13890 raft_consensus.cc:740] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:51.546790 13890 consensus_queue.cc:260] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:51.547178 13800 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:51.548008 13890 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:51.548372  8330 raft_consensus.cc:2393] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 14: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:51.549630 13306 consensus_queue.cc:260] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:51.507802 13770 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:51.550879 13306 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:51.494150 13762 raft_consensus.cc:740] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:51.554201 13762 consensus_queue.cc:260] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:51.555545 13762 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:51.519480 13806 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:51.556339 13806 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:51.558353  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:51.469496 13892 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:51.559535  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:51.563155  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:51.564282  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:51.566694 13814 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:51.568202 13892 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:51.568742 13499 consensus_meta.cc:287] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.553s	user 0.008s	sys 0.002s
I20260709 13:03:51.569589  8332 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 10 index: 12. Preceding OpId from leader: term: 11 index: 12. (term mismatch)
I20260709 13:03:51.569909  8332 pending_rounds.cc:85] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 11
I20260709 13:03:51.570102  8332 pending_rounds.cc:107] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 10.12
I20260709 13:03:51.570307  8332 raft_consensus.cc:2889] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:51.571540 13665 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:51.490021 13465 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:51.579262 13813 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:51.579800 13813 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:51.579679 13806 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:51.581935 13806 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:51.528738 13876 raft_consensus.cc:697] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:03:51.585031 13870 log.cc:927] Time spent T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.303s	user 0.000s	sys 0.001s
I20260709 13:03:51.585371 13876 consensus_queue.cc:237] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:51.587996 13764 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:51.590356 13538 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:51.595624 13807 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:51.596181 13827 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:51.597605  8297 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:51.577242 13465 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:51.600682 13465 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.600889  8972 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:51.601173 13465 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:51.601739 13882 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.601940  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:51.602268 13882 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:51.603489 13909 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:51.604152 13909 raft_consensus.cc:697] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 14 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:51.604423 13882 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:51.602734 13862 log.cc:927] Time spent T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.768s	user 0.003s	sys 0.000s
I20260709 13:03:51.605849 13579 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:51.607456  8405 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18.
I20260709 13:03:51.609978 13675 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:51.605110 13909 consensus_queue.cc:237] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:51.614749 13734 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:51.615833  8327 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:51.618515 13846 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:51.614017 13897 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:51.619441 13631 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:51.619817 13897 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:51.620636 13846 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:51.621757  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:51.622485  8249 raft_consensus.cc:2393] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 14: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:51.622524 13897 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:51.622836  8326 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:51.603586 13465 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:51.613178 13813 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:51.627044 13813 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:51.627434  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:51.628211  8328 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:03:51.630457  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2478) took 1045 ms. Trace:
I20260709 13:03:51.631008  8325 rpcz_store.cc:276] 0709 13:03:50.584628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.584751 (+   123us) service_pool.cc:224] Handling call
0709 13:03:50.610478 (+ 25727us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:50.688819 (+ 78341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:50.688852 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:50.689291 (+   439us) log.cc:844] Serialized 26 byte log entry
0709 13:03:51.026204 (+336913us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:51.026236 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.584470 (+558234us) spinlock_profiling.cc:243] Waited 33.1 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:51.630028 (+ 45558us) spinlock_profiling.cc:243] Waited 78.1 ms on lock 0x7b54002c9a54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:51.630103 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:51.630168 (+    65us) raft_consensus.cc:1685] finished
0709 13:03:51.630188 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.630384 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":255680128,"thread_start_us":326581,"threads_started":1,"wal-append.queue_time_us":462150}
I20260709 13:03:51.637733 13734 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:51.645570  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2473) took 1058 ms. Trace:
I20260709 13:03:51.646178  8324 rpcz_store.cc:276] 0709 13:03:50.583002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.586365 (+  3363us) service_pool.cc:224] Handling call
0709 13:03:51.641138 (+1054773us) spinlock_profiling.cc:243] Waited 82.1 ms on lock 0x7b5400364a54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:51.641318 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":188761984}
I20260709 13:03:51.647782  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:51.649644  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:51.550879 13800 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:51.654733 13631 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:51.657401  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:51.659235  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:51.661865 13275 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:51.663280 13770 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:03:51.665814 13770 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:51.671708 13762 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.676412  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:51.678857 13762 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:51.681172 13762 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:51.684789  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:51.686905  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:51.694319  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:03:51.695346  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 97836.6us
I20260709 13:03:51.695431  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:51.696651 13902 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:51.697984  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:03:51.699687 13879 log.cc:927] Time spent T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.366s	user 0.001s	sys 0.000s
I20260709 13:03:51.700361  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:51.700356 13675 consensus_queue.cc:1059] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:51.701704  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:51.702688  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:51.704046  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:51.704610  8331 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:51.699550  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 99296.2us
I20260709 13:03:51.705899  8295 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:51.707051  8295 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:03:51.710345  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.711038  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:51.712014  8331 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:03:51.713321 13607 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:51.710722 13765 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:51.717656 13765 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:51.718079 13765 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:51.713090  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:51.722918  8400 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:51.723088  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:51.724373  8406 raft_consensus.cc:1240] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:51.727425  8298 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:51.712086  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:51.728694  8330 raft_consensus.cc:2410] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:03:51.735937  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.737131  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:51.738386  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2498) took 1026 ms. Trace:
I20260709 13:03:51.738795  8329 rpcz_store.cc:276] 0709 13:03:50.711703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.021221 (+309518us) service_pool.cc:224] Handling call
0709 13:03:51.022867 (+  1646us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:51.215291 (+192424us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:51.215446 (+   155us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:51.218586 (+  3140us) log.cc:844] Serialized 26 byte log entry
0709 13:03:51.357717 (+139131us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:51.357758 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.727611 (+369853us) spinlock_profiling.cc:243] Waited 63.7 ms on lock 0x7b5400058cd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:51.727682 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:51.727740 (+    58us) raft_consensus.cc:1685] finished
0709 13:03:51.727762 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.738320 (+ 10558us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":31388,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":5586,"spinlock_wait_cycles":146426240,"thread_start_us":265693,"threads_started":2,"wal-append.queue_time_us":185988}
I20260709 13:03:51.740658  8331 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:03:51.741914 13762 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:51.742360 13762 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:51.742779 13762 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:51.743651 13890 consensus_queue.cc:1059] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:51.745278 13577 consensus_queue.cc:1059] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:51.745635  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:51.746318  8249 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:51.738806 13875 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.749948 13875 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:51.752234 13654 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:51.753734  8406 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:51.754932  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:51.755764 13831 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:51.757668  8369 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:51.768411  8323 raft_consensus.cc:2368] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19.
I20260709 13:03:51.773496  8399 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:03:51.774718  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:51.593151 13538 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:51.783510  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:51.784034  8328 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:03:51.786978  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:51.789606 13739 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:51.790349 13739 raft_consensus.cc:697] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 17 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:51.790843  8249 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:03:51.665194 13275 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:51.806731  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:51.811497  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:51.794190 13306 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:03:51.819325  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:51.820497  8401 raft_consensus.cc:1275] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:03:51.820801  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:51.822803  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:51.798293 13831 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:51.833181  8330 pending_rounds.cc:85] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 6
I20260709 13:03:51.795207 13667 raft_consensus.cc:2764] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:03:51.833535  8330 pending_rounds.cc:107] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:03:51.875703  8330 raft_consensus.cc:2889] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:51.791536 13739 consensus_queue.cc:237] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:51.835062  8331 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:03:51.877547  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:51.878763  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.858397  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:51.858636  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:51.858879  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:51.859254  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:51.859539  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:51.860734  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:51.862147  8972 leader_election.cc:400] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19.
I20260709 13:03:51.863597  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:51.886498  8402 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:51.856555  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:51.887902  8403 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
W20260709 13:03:51.904541  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.804787 13658 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:03:51.958562  8402 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
W20260709 13:03:51.970412  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 94117us
I20260709 13:03:51.804991 13778 consensus_queue.cc:1059] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.140s
I20260709 13:03:51.975124  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:51.981798  8251 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:51.983951  8222 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:51.948860 13770 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.994522  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:51.999509  8327 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:03:51.946836 13774 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:51.950978 13306 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.001856  8402 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:03:52.004845  8222 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:51.804361 13889 consensus_queue.cc:1059] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.224s
I20260709 13:03:52.006572  8372 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:52.008127 13579 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:52.009761  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:51.948182 13675 consensus_queue.cc:1059] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:03:51.977458 13831 raft_consensus.cc:697] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:51.874648 13895 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:52.017625  8372 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:51.951154 13800 consensus_queue.cc:1059] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:03:52.010277 13897 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:52.017230 13817 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:52.000854 13665 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:03:52.026075 13806 raft_consensus.cc:3055] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:51.977635 13659 consensus_queue.cc:1048] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:03:52.029022 13549 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:52.010916 13306 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:52.086361  8372 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:52.005659 13770 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:52.095134  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 74330.1us
I20260709 13:03:52.110363  8256 raft_consensus.cc:1240] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:03:52.112294  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:52.118907  8222 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:52.077123  8251 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:03:52.027726 13905 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:03:52.120652  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.124040 13831 consensus_queue.cc:237] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:52.107978 13806 raft_consensus.cc:740] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:52.130643 13770 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:52.130318 13806 consensus_queue.cc:260] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:52.131168 13817 raft_consensus.cc:697] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 18 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:52.131610 13806 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:52.132102 13817 consensus_queue.cc:237] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:52.018473 13909 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 13:03:52.104032 13656 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:52.135632 13656 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:52.136363 13895 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:52.137522  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:52.137768 13656 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:52.100104 13813 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.138257  8327 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 19: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:52.138415 13813 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:52.140488 13813 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:52.141405 13865 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:52.145391 13275 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:52.147984 13275 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:52.148674  8222 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:52.149449 13868 log.cc:927] Time spent T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.867s	user 0.000s	sys 0.011s
I20260709 13:03:52.149501 13895 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:52.151249 13895 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:52.151630  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2256) took 1690 ms. Trace:
I20260709 13:03:52.152801  8222 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:52.153303 13275 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:52.153626 13892 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:52.154094 13892 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:52.154505 13892 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:52.152007  8252 rpcz_store.cc:276] 0709 13:03:50.460976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.500087 (+ 39111us) service_pool.cc:224] Handling call
0709 13:03:50.539045 (+ 38958us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:50.539557 (+   512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:50.539586 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:50.540316 (+   730us) log.cc:844] Serialized 52 byte log entry
0709 13:03:50.684151 (+143835us) spinlock_profiling.cc:243] Waited 1.53 ms on lock 0x7b1800361ea0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e0d1 00007f8828779a0f
0709 13:03:51.030762 (+346611us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:51.036625 (+  5863us) log.cc:844] Serialized 14 byte log entry
0709 13:03:51.040242 (+  3617us) log.cc:844] Serialized 14 byte log entry
0709 13:03:51.040477 (+   235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.040544 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:51.644411 (+603867us) spinlock_profiling.cc:243] Waited 4.14 ms on lock 0x7b1800361ea0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882569f2a6 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1
0709 13:03:51.645029 (+   618us) spinlock_profiling.cc:243] Waited 3.54 ms on lock 0x7b1800361ea0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:03:52.148880 (+503851us) spinlock_profiling.cc:243] Waited 32.9 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:52.151404 (+  2524us) raft_consensus.cc:1685] finished
0709 13:03:52.151427 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.151561 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5176,"prepare.queue_time_us":611338,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":61,"raft.queue_time_us":8408,"raft.run_cpu_time_us":427,"raft.run_wall_time_us":25856,"spinlock_wait_cycles":96909056,"thread_start_us":346418,"threads_started":1,"wal-append.queue_time_us":599559}
I20260709 13:03:52.162724  8222 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:52.163357 13734 raft_consensus.cc:3055] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:52.163763 13813 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:52.163786 13734 raft_consensus.cc:740] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:52.165581 13734 consensus_queue.cc:260] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:52.167130 13734 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:52.172047 13607 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:52.175640  8222 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:52.176226 13762 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:52.178043 13914 log.cc:927] Time spent T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:03:52.180476 13892 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:52.184816 13813 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:52.186627  8222 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:52.187165  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 155793us
I20260709 13:03:52.191172  8327 raft_consensus.cc:1240] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 16. Current term is 17. Ops: []
W20260709 13:03:52.193341  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 56238.9us
I20260709 13:03:52.195150  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:52.231145 13875 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:52.232967 13774 raft_consensus.cc:3055] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:52.233512 13774 raft_consensus.cc:740] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:52.233733  8252 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:03:52.234400 13774 consensus_queue.cc:260] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:52.235558 13774 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:52.235711 13306 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:52.236555 13770 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:03:52.237869  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.237790 13676 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:52.074501 13873 consensus_queue.cc:1059] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:03:52.238802 13667 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:52.238615 13676 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:52.239534 13765 consensus_meta.cc:287] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.521s	user 0.009s	sys 0.000s
I20260709 13:03:52.240020 13765 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:52.241201  8323 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:52.242774  8251 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:03:52.245561 13827 consensus_queue.cc:1059] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:52.246493 13827 raft_consensus.cc:3055] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:52.246822  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:52.238842 13910 log.cc:927] Time spent T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260709 13:03:52.247495  8251 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
W20260709 13:03:52.243732  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 49693.6us
I20260709 13:03:52.248864  8251 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:03:52.243554 13676 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:52.250068  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:52.250013  8331 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:52.258003  8252 raft_consensus.cc:2393] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 13: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:52.258257  8402 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:52.260293  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:52.260978  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:52.261761  8331 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:52.243930  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:52.264761  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:52.274564  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:52.280596  8251 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:52.289412  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:52.290059  8397 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:52.290858  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:52.291196  8403 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:52.292340  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:52.269243  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:52.293960  8252 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:03:52.271080  8369 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:52.288763  8223 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:52.297813  8295 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:52.295285  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:52.308688  8326 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:03:52.310694  8397 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:03:52.250022 13905 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:52.312361  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:52.320236  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:52.320614  8256 raft_consensus.cc:2393] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 16: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:52.321677  8252 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 16: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:52.246831 13827 raft_consensus.cc:740] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:52.245582 13770 raft_consensus.cc:697] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:52.326864  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:52.328549  8369 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:52.329049  8972 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:52.328168  8252 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:52.268563 13846 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.331090  8295 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:52.332305  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:52.333186  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:52.358405  8252 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:52.353467  9006 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:03:52.376447  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 45455.6us
I20260709 13:03:52.312117 13669 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:52.239570 13667 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:52.386531  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:52.387574  8223 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:52.389863  8252 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:03:52.408349  9000 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:03:52.407176  8295 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:52.330417 13772 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:52.408666  8397 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:52.290189 13778 consensus_queue.cc:1059] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.085s
I20260709 13:03:52.354242 13275 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:52.410185  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:52.443573  8252 raft_consensus.cc:2393] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 17: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:52.444764  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:52.445729  8252 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 13: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:52.297231 13813 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.093s
I20260709 13:03:52.376564 13846 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:52.390270 13832 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.019s	sys 0.033s
I20260709 13:03:52.374743 13890 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:52.389008 13807 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:52.382359 13672 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:52.408604 13669 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:52.368299 13656 consensus_queue.cc:1059] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.088s
W20260709 13:03:52.453136 13915 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.225s	user 0.001s	sys 0.001s
I20260709 13:03:52.447275 13607 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:52.367928 13875 consensus_queue.cc:1059] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.100s
I20260709 13:03:52.409188 13665 consensus_queue.cc:1059] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 13:03:52.487476  8404 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:03:52.507761 13825 raft_consensus.cc:3055] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:52.508282 13825 raft_consensus.cc:740] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:52.459442 13800 consensus_queue.cc:1059] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:03:52.489188 13770 consensus_queue.cc:237] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:03:52.518838 13275 raft_consensus.cc:3055] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:52.519342 13275 raft_consensus.cc:740] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:52.483243 13669 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:52.480959 13827 consensus_queue.cc:260] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:52.522188 13827 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:52.523838 13607 raft_consensus.cc:697] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:52.524994 13607 consensus_queue.cc:237] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:52.526855  8252 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:52.527817  8256 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:52.529404  8252 raft_consensus.cc:1275] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:03:52.509188 13825 consensus_queue.cc:260] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:52.530557  8252 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:52.532166  8252 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:52.544379 13891 raft_consensus.cc:2955] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Committing config change with OpId 18.11: config changed from index 6 to 11, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:52.547173 13825 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:52.548404 13762 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:52.550530 13912 log.cc:927] Time spent T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
W20260709 13:03:52.552438 13923 log.cc:927] Time spent T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:03:52.552733 13549 raft_consensus.cc:3055] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:52.553154 13549 raft_consensus.cc:740] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:03:52.554842 13917 log.cc:927] Time spent T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
W20260709 13:03:52.468698 13832 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.022s	sys 0.036s
W20260709 13:03:52.558068  8405 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.006s	sys 0.020s
W20260709 13:03:52.574656  8405 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.006s	sys 0.020s
I20260709 13:03:52.554090 13549 consensus_queue.cc:260] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:52.556053 13672 raft_consensus.cc:697] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:03:52.702132  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:52.716440  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 387925us
W20260709 13:03:52.720361  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:52.720517  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 395999us
I20260709 13:03:52.692420 13762 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:52.692070 13892 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:52.692163 13669 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:52.757910 13913 log.cc:927] Time spent T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.303s	user 0.001s	sys 0.000s
W20260709 13:03:52.769770  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.742646 13762 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:52.764250 13911 log.cc:927] Time spent T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.309s	user 0.001s	sys 0.000s
W20260709 13:03:52.767838 13918 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.300s	user 0.001s	sys 0.000s
I20260709 13:03:52.790292  8251 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15. Ops: []
W20260709 13:03:52.787506  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 400007us
I20260709 13:03:52.789041  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:03:52.803509 13921 log.cc:927] Time spent T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:03:52.803951 13607 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:52.804690 13607 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:52.805696  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2774) took 1108 ms. Trace:
I20260709 13:03:52.806874 13669 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:52.806104  8329 rpcz_store.cc:276] 0709 13:03:51.696846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.765512 (+ 68666us) service_pool.cc:224] Handling call
0709 13:03:51.765966 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:51.766299 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:51.766344 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:51.799158 (+ 32814us) log.cc:844] Serialized 26 byte log entry
0709 13:03:52.325254 (+526096us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:52.352433 (+ 27179us) log.cc:844] Serialized 14 byte log entry
0709 13:03:52.375800 (+ 23367us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.375885 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:52.805469 (+429584us) raft_consensus.cc:1685] finished
0709 13:03:52.805492 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.805637 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23059,"prepare.queue_time_us":40811,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":23395,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":31571,"thread_start_us":450337,"threads_started":1,"wal-append.queue_time_us":860331}
I20260709 13:03:52.754364 13672 consensus_queue.cc:237] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:52.807776 13607 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:52.773254 13667 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:52.818362  8372 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:52.792832  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:52.827696  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:52.839059  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:52.796968 13734 consensus_meta.cc:287] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.629s	user 0.010s	sys 0.011s
I20260709 13:03:52.842321 13669 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:52.847175  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 851) took 1064 ms. Trace:
W20260709 13:03:52.850207 13928 log.cc:927] Time spent T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:03:52.858278 13930 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:03:52.869963 13831 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:52.870042 13942 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.002s
W20260709 13:03:52.885133  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 94373.1us
W20260709 13:03:52.969901  8295 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.022s	sys 0.059s
W20260709 13:03:52.970213  8295 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.022s	sys 0.059s
W20260709 13:03:52.970736  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 563615us
I20260709 13:03:52.790028 13772 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.979645 13772 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:52.982180 13772 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:52.900039 13762 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:52.835616 13913 log.cc:927] Time spent T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:03:52.907907  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 80251.1us
W20260709 13:03:52.983513 13922 log.cc:927] Time spent T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.387s	user 0.001s	sys 0.001s
I20260709 13:03:52.984031  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:52.984649  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2501) took 1589 ms. Trace:
W20260709 13:03:52.921142  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 43692.7us
I20260709 13:03:52.904580  8257 rpcz_store.cc:276] 0709 13:03:51.782614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.782934 (+   320us) service_pool.cc:224] Handling call
0709 13:03:51.802846 (+ 19912us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:51.803237 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:51.803271 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:51.803936 (+   665us) log.cc:844] Serialized 52 byte log entry
0709 13:03:52.076451 (+272515us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:52.116941 (+ 40490us) log.cc:844] Serialized 14 byte log entry
0709 13:03:52.193618 (+ 76677us) log.cc:844] Serialized 14 byte log entry
0709 13:03:52.193789 (+   171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.193855 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:52.550168 (+356313us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b18003c3700. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:03:52.846894 (+296726us) raft_consensus.cc:1685] finished
0709 13:03:52.846925 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.847102 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":415502,"prepare.queue_time_us":38651,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":79,"raft.queue_time_us":168768,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":9529,"spinlock_wait_cycles":27341056,"thread_start_us":272081,"threads_started":1,"wal-append.queue_time_us":548579}
I20260709 13:03:52.985036  8325 rpcz_store.cc:276] 0709 13:03:51.394986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.660471 (+265485us) service_pool.cc:224] Handling call
0709 13:03:51.660878 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:51.767679 (+106801us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:51.767729 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:51.783581 (+ 15852us) log.cc:844] Serialized 26 byte log entry
0709 13:03:52.250312 (+466731us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:52.250354 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.250422 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:52.984430 (+734008us) raft_consensus.cc:1685] finished
0709 13:03:52.984453 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.984595 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":106239,"raft.queue_time_us":112022,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":11491,"thread_start_us":418087,"threads_started":1,"wal-append.queue_time_us":671354}
I20260709 13:03:52.984066  8295 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:52.888763  8223 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:52.977231  8406 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:03:52.992157 13944 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.992704 13944 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:52.994385 13774 consensus_meta.cc:287] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.758s	user 0.012s	sys 0.002s
I20260709 13:03:52.994751 13944 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:52.995015 13774 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:03:52.995538 13932 log.cc:927] Time spent T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.204s	user 0.002s	sys 0.000s
I20260709 13:03:52.823036 13306 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:52.982352 13773 raft_consensus.cc:3055] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:52.998920 13773 raft_consensus.cc:740] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:53.000037 13773 consensus_queue.cc:260] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:52.904155 13669 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:52.996632  8295 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:53.018800 13669 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:53.019948  8295 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:53.020979  8295 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:03:52.996403 13865 consensus_meta.cc:287] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.854s	user 0.006s	sys 0.005s
W20260709 13:03:52.966607 13806 consensus_meta.cc:287] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.835s	user 0.014s	sys 0.000s
I20260709 13:03:53.030470 12973 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:53.032354 12973 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:53.033072  8325 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: [18.11-18.11]
W20260709 13:03:53.030841  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 46822.7us
I20260709 13:03:53.021970 13607 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:53.041191  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:53.044016 13891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:45618 (request call id 8) took 4732 ms. Trace:
I20260709 13:03:53.044817  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:53.047679  8323 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:53.044171  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:53.051335  8326 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
W20260709 13:03:53.053911  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 165266us
I20260709 13:03:52.968928 13875 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:53.056211  8324 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:53.066162  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:53.026301 13807 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:53.031447 13667 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:53.088286  8223 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:03:53.089839  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.045639 13902 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:53.053544 13675 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.075075 13875 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.092970  8323 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:53.053915 13659 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:53.072857 13306 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:53.046058 13774 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:03:53.031200 13734 consensus_queue.cc:1059] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:03:53.113641  8250 raft_consensus.cc:1275] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:03:53.114941  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:53.109127  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:53.116241  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:53.076228 13897 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.106078  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:53.078105 13607 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:53.117619  8248 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:53.107668  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:53.110812  8252 raft_consensus.cc:1275] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:53.046329 13538 consensus_queue.cc:1059] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:03:53.121635 13734 raft_consensus.cc:3055] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:53.122267  8250 raft_consensus.cc:2393] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 11: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:53.127416  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:53.095930 13667 raft_consensus.cc:697] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:53.116412 13875 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:53.094880 13475 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:53.173177  8324 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2506) took 1710 ms (client timeout 2044 ms). Trace:
W20260709 13:03:53.173758  8324 rpcz_store.cc:269] 0709 13:03:51.462498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.648978 (+186480us) service_pool.cc:224] Handling call
0709 13:03:53.172940 (+1523962us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b540029aed4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:53.173099 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3231955840}
W20260709 13:03:53.125902 13827 consensus_meta.cc:287] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.603s	user 0.013s	sys 0.007s
I20260709 13:03:53.118330 13813 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:53.184680 13813 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.185259 13813 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:53.188941 13577 raft_consensus.cc:3055] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:53.189411 13577 raft_consensus.cc:740] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:53.192117  8401 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
W20260709 13:03:53.193542  8974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.064s	sys 0.100s
I20260709 13:03:53.097208 13817 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
W20260709 13:03:53.194525  8974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.064s	sys 0.100s
W20260709 13:03:53.195405  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.200502  8248 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:53.125638 13734 raft_consensus.cc:740] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:03:53.202771  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 368590us
I20260709 13:03:53.098575 13675 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:53.099470 13902 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:53.169137 13940 raft_consensus.cc:3055] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:53.210258 13940 raft_consensus.cc:740] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:03:53.207585  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.129750 13607 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:53.240918  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:53.129418 13897 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:53.118511 13659 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:53.245810  8404 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:03:53.169775 13865 consensus_queue.cc:1048] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:03:53.249275  8406 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:53.249977  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:53.172676 13770 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 13:03:53.254995  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:53.211313 13940 consensus_queue.cc:260] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:53.257321  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:53.259650  8401 raft_consensus.cc:1240] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:53.262320 13873 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:53.263054 13538 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:53.044808 13891 rpcz_store.cc:276] 0709 13:03:48.240782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.240979 (+   197us) service_pool.cc:224] Handling call
0709 13:03:48.488543 (+247564us) spinlock_profiling.cc:243] Waited 5.68 ms on lock 0x7b5400719ba0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f882873b520 00007f88287c4b42 00007f88287c3800 00007f88287bc9fa 00007f882d7844c9 00007f8826d00c54 00007f8826d00be2 00007f8826d00b11 00007f8826d00a8c 00007f8826cffd42
0709 13:03:48.603845 (+115302us) log.cc:844] Serialized 487 byte log entry
0709 13:03:49.416766 (+812921us) spinlock_profiling.cc:243] Waited 4.63 ms on lock 0x7b180031bfa0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:03:52.973188 (+3556422us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18664,"raft.queue_time_us":5647,"raft.run_cpu_time_us":1017,"raft.run_wall_time_us":48406,"spinlock_wait_cycles":23724032}
I20260709 13:03:53.298799  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:53.303802  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:53.304936  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:53.270779 13806 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:53.273468 13927 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:53.276533 13669 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:53.278811 13902 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:53.235111 13577 consensus_queue.cc:260] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:53.284147 13940 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:53.284675 13734 consensus_queue.cc:260] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:53.263657 13667 consensus_queue.cc:237] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:53.299788  8323 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
W20260709 13:03:53.333127  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.307221  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:53.308785  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:53.311414  8297 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:53.334061  8332 raft_consensus.cc:2410] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:03:53.313683  8329 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 16: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:53.335966  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:53.332419  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:53.336592  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:53.337074  8327 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:03:53.255615 13944 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.314864  9003 raft_consensus.cc:2393] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 16: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:53.268937  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:53.320156  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:53.325281  9002 raft_consensus.cc:3055] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:53.264957 13948 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:53.385231 13948 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:53.329077  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:53.388193  8326 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
W20260709 13:03:53.329978  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 238295us
I20260709 13:03:53.332129 13902 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:53.389911 13902 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:53.392881 13902 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:53.285651  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:53.392964  9002 raft_consensus.cc:740] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:53.394134  9005 raft_consensus.cc:2410] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:03:53.396219 13669 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:53.340427  8402 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:03:53.341674  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:53.341979  9004 raft_consensus.cc:2393] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 13: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:03:53.342463  8257 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:03:53.405932  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:53.400278  8399 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:53.299870 13538 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:53.375880  8404 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
I20260709 13:03:53.375340  8406 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:53.298089 13873 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:53.417830 13958 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:53.418313 13958 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:53.418679 13958 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:53.323140 13656 consensus_queue.cc:1059] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.086s
I20260709 13:03:53.427433 13659 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:53.428596 13675 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:53.375214 13475 consensus_queue.cc:1175] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e64761f43e4240319c41117cf886d22f" responder_term: 13 status { last_received { term: 13 index: 9 } last_committed_idx: 6 last_received_current_leader { term: 13 index: 9 } }
I20260709 13:03:53.411612 13762 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.430881 13762 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:53.431561 13897 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:53.433601 13875 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:53.436154 13875 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:53.438539 13538 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:53.439293 13925 log.cc:927] Time spent T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.643s	user 0.002s	sys 0.000s
I20260709 13:03:53.440774 13607 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:53.376029 13944 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:53.448457 13944 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:53.456961  8152 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45448 (request call id 285) took 26315 ms (client timeout 15000 ms). Trace:
I20260709 13:03:53.458655  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:53.460891  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 858) took 1133 ms. Trace:
I20260709 13:03:53.477697  8256 rpcz_store.cc:276] 0709 13:03:52.327201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.528683 (+201482us) service_pool.cc:224] Handling call
0709 13:03:53.460046 (+931363us) spinlock_profiling.cc:243] Waited 26.6 ms on lock 0x7b5800002a98. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699eee 0000000000375939 00007f882d70d1e0 00007f882d7d60c4 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:03:53.460269 (+   223us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.460497 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.460532 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:53.460584 (+    52us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:53.460625 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.460691 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.460823 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":61279232}
I20260709 13:03:53.463160  8153 catalog_manager.cc:5697] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:53.433099 13762 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:53.470415  8325 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:53.470907 13656 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.324397 13927 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:53.322547 13806 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:53.474650  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.475091  8401 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 16. Current term is 17. Ops: []
W20260709 13:03:53.525027  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 134917us
I20260709 13:03:53.410100 13902 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:53.515674  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:53.518021  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:53.531023  8252 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
W20260709 13:03:53.539779 13924 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.865s	user 0.000s	sys 0.010s
I20260709 13:03:53.513188  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:53.543326 13909 raft_consensus.cc:3055] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:53.543803 13909 raft_consensus.cc:740] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:03:53.475620  8152 rpcz_store.cc:269] 0709 13:03:27.141777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.158729 (+ 16952us) service_pool.cc:224] Handling call
0709 13:03:29.644300 (+2485571us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b5400200490. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f882569a0b2 00007f882e613a89 00007f882e5abda6 00007f882e57d088 00007f882e67b37e 00007f882cb3c2a4 00007f882cb3c232 00007f882cb3c161 00007f882cb3c0dc
0709 13:03:53.456851 (+23812551us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:47.393945 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:47.618190 (+224245us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:47.618275 (+    85us) write_op.cc:435] Acquired schema lock
0709 13:03:47.618302 (+    27us) tablet.cc:662] Decoding operations
0709 13:03:47.694442 (+ 76140us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:47.707641 (+ 13199us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:47.707669 (+    28us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:03:47.708849 (+  1180us) tablet.cc:701] Row locks acquired
0709 13:03:47.708890 (+    41us) write_op.cc:260] PREPARE: finished
0709 13:03:47.709038 (+   148us) write_op.cc:270] Start()
0709 13:03:47.709184 (+   146us) write_op.cc:276] Timestamp: P: 1783602227708987 usec, L: 0
0709 13:03:47.709215 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:03:47.729830 (+ 20615us) log.cc:844] Serialized 6659 byte log entry
0709 13:03:48.737512 (+1007682us) op_driver.cc:464] REPLICATION: finished
0709 13:03:49.145749 (+408237us) write_op.cc:301] APPLY: starting
0709 13:03:49.145855 (+   106us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:49.347468 (+201613us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:49.362373 (+ 14905us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:49.693410 (+331037us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:49.693844 (+   434us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=34,mrs_lookups=0
0709 13:03:49.693911 (+    67us) write_op.cc:312] APPLY: finished
0709 13:03:49.694417 (+   506us) log.cc:844] Serialized 156 byte log entry
0709 13:03:50.121352 (+426935us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f88256b40ac 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b
0709 13:03:50.303430 (+182078us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:50.303695 (+   265us) write_op.cc:454] Released schema lock
0709 13:03:50.303815 (+   120us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:50.304431 (+   616us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:50.304480 (+    49us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":587472,"reported_tablets":49,"spinlock_wait_cycles":32439680,"child_traces":[["op",{"apply.queue_time_us":383859,"apply.run_cpu_time_us":54817,"apply.run_wall_time_us":1158620,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75221,"num_ops":17,"prepare.queue_time_us":255598,"prepare.run_cpu_time_us":42815,"prepare.run_wall_time_us":571606,"raft.queue_time_us":221737,"raft.run_cpu_time_us":45700,"raft.run_wall_time_us":393963,"replication_time_us":993156,"spinlock_wait_cycles":32584576,"thread_start_us":1148012,"threads_started":5,"wal-append.queue_time_us":800341,"wal-append.run_cpu_time_us":31330,"wal-append.run_wall_time_us":2728937}]]}
W20260709 13:03:53.491044  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:53.547585  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.542903  8153 catalog_manager.cc:5697] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:53.551039 13734 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:53.551218 13806 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:53.413533 13813 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:53.554700  8248 raft_consensus.cc:2368] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14.
W20260709 13:03:53.555749 13919 log.cc:927] Time spent T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 1.077s	user 0.002s	sys 0.000s
I20260709 13:03:53.557952  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2784) took 1739 ms. Trace:
I20260709 13:03:53.559302 13927 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:53.558326  8328 rpcz_store.cc:276] 0709 13:03:51.819039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.819379 (+   340us) service_pool.cc:224] Handling call
0709 13:03:51.819674 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:51.820777 (+  1103us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:51.820807 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:51.821235 (+   428us) log.cc:844] Serialized 26 byte log entry
0709 13:03:52.077076 (+255841us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:52.077130 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.077222 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:53.555128 (+1477906us) spinlock_profiling.cc:243] Waited 2.62 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:53.557626 (+  2498us) raft_consensus.cc:1685] finished
0709 13:03:53.557658 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.557887 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39583,"raft.queue_time_us":762,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":9859,"spinlock_wait_cycles":6020352,"thread_start_us":199476,"threads_started":1,"wal-append.queue_time_us":570432}
I20260709 13:03:53.414907 13937 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:53.560448 13765 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:53.561666 13762 consensus_queue.cc:1059] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:53.562057 13669 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:53.562906 13961 raft_consensus.cc:3055] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:53.563311 13961 raft_consensus.cc:740] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:53.564265 13961 consensus_queue.cc:260] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:53.528043  8324 raft_consensus.cc:1240] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:53.564973 13873 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:53.566568 13846 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:53.567148 13961 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:53.567816 13931 log.cc:927] Time spent T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.795s	user 0.002s	sys 0.000s
I20260709 13:03:53.560276 13937 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:53.569159 13807 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:53.569914  8249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 852) took 1761 ms. Trace:
I20260709 13:03:53.569972 13902 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:53.570597 13773 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:53.571311  8369 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:53.572530 13902 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:53.573318 13654 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:53.520108 13656 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:53.573792 13654 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.574234 13654 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:53.575738 13656 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:53.570511  8249 rpcz_store.cc:276] 0709 13:03:51.808503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.815063 (+  6560us) service_pool.cc:224] Handling call
0709 13:03:51.815475 (+   412us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:51.817682 (+  2207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:51.817732 (+    50us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:51.820444 (+  2712us) log.cc:844] Serialized 78 byte log entry
0709 13:03:52.312609 (+492165us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:52.353975 (+ 41366us) log.cc:844] Serialized 14 byte log entry
0709 13:03:52.377645 (+ 23670us) log.cc:844] Serialized 14 byte log entry
0709 13:03:52.524144 (+146499us) spinlock_profiling.cc:243] Waited 8.68 ms on lock 0x7b18003fcf40. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:03:52.774249 (+250105us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b18003fcf40. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882564bc0a 00007f882ed8b219 00007f88256d3a40 00007f88256ce4e4 00007f88256ce455 00007f88287b6060 00007f88287c8716 00007f88287d9287 00007f88287d9208
0709 13:03:52.778698 (+  4449us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.778793 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:53.539124 (+760331us) spinlock_profiling.cc:243] Waited 3.98 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:53.569640 (+ 30516us) raft_consensus.cc:1685] finished
0709 13:03:53.569665 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.569850 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":506129,"prepare.queue_time_us":223428,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":78,"raft.queue_time_us":407577,"raft.run_cpu_time_us":457,"raft.run_wall_time_us":35286,"spinlock_wait_cycles":32136320,"thread_start_us":334479,"threads_started":1,"wal-append.queue_time_us":536529}
I20260709 13:03:53.580379  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:53.582356  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 238247us
I20260709 13:03:53.584298  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:53.586555  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:53.586555  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:53.588912  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 191209us
I20260709 13:03:53.590698  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:53.593329  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:53.595273  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:53.592401  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:53.601269  8398 raft_consensus.cc:2393] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 15: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:53.602272  9006 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:53.601907  8369 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:53.526409 13665 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:53.604019  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.605014  8402 raft_consensus.cc:2393] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 15: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:53.544727 13909 consensus_queue.cc:260] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:53.621178  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:53.502467 13676 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.097s
I20260709 13:03:53.624311  8298 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:53.608422  8153 catalog_manager.cc:5697] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 6 to 7, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 7 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:53.625571  8297 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:53.559058 13813 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:53.631568  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 49430.7us
I20260709 13:03:53.633123  8972 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:53.581811 13806 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:53.643392  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:53.644155  9005 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:03:53.603458 13902 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:53.642411 13877 consensus_queue.cc:1059] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:03:53.647167  9006 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:03:53.630970  8153 catalog_manager.cc:5697] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 4 to 7, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 7 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:53.635767  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.628748 13577 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:53.632691 13676 consensus_queue.cc:1243] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer 8c42259d7c4b463a9f485f42c320c81b log is divergent from this leader: its last log entry 15.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:03:53.652801  8297 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:53.651432  8298 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:53.628487 13875 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:53.642783 13909 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:53.633590 13956 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:03:53.628032  8222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.018s	sys 0.028s
W20260709 13:03:53.667520  8222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.018s	sys 0.029s
W20260709 13:03:53.668046  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 551184us
I20260709 13:03:53.662731  8297 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:53.650894 13807 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:53.652220 13475 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:53.680908  8402 raft_consensus.cc:1275] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:53.626896 13665 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:53.687585  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:53.664726 13755 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:53.647712 13944 consensus_queue.cc:1048] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:03:53.650758 13825 consensus_queue.cc:1059] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 13:03:53.690754  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:53.664436 13865 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:53.665678 13800 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:53.654091 13965 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:53.632098 13958 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:53.648099 13806 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:53.666945 13956 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:53.660450 13765 consensus_queue.cc:1059] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:03:53.651532 13902 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:53.673144 13875 raft_consensus.cc:697] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:03:53.859419  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 77906.6us
I20260709 13:03:53.818545  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:53.729445  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.700726 13817 consensus_queue.cc:1048] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.248s
W20260709 13:03:53.949864  8298 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.4:33825 timed out after 1.595s (SENT) [suppressed 7 similar messages]
I20260709 13:03:53.714723 13895 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.950770  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:53.950662 13895 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:53.711098 13665 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:53.711002  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:53.854825 13897 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.703593 13891 raft_consensus.cc:3055] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:53.766764  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2384) took 1477 ms. Trace:
I20260709 13:03:53.885836  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:53.947880 13902 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:53.687193 13538 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:53.952713 13891 raft_consensus.cc:740] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:53.886694  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:53.729032 13825 consensus_queue.cc:1243] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer 8c42259d7c4b463a9f485f42c320c81b log is divergent from this leader: its last log entry 15.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:03:53.884567  8255 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
W20260709 13:03:53.880877 13966 log.cc:927] Time spent T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.198s	user 0.000s	sys 0.003s
I20260709 13:03:53.958918 13895 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:53.953504  8400 rpcz_store.cc:276] 0709 13:03:52.289370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.291171 (+  1801us) service_pool.cc:224] Handling call
0709 13:03:52.293095 (+  1924us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:52.293455 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:52.293483 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:52.293909 (+   426us) log.cc:844] Serialized 26 byte log entry
0709 13:03:53.469610 (+1175701us) spinlock_profiling.cc:243] Waited 5.74 ms on lock 0x7b18001cd020. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e0d1 00007f8828779a0f
0709 13:03:53.659474 (+189864us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:53.659534 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.659636 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:53.766392 (+106756us) raft_consensus.cc:1685] finished
0709 13:03:53.766417 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.766678 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3288,"raft.queue_time_us":4441,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":4941,"spinlock_wait_cycles":13207424,"thread_start_us":189655,"threads_started":1,"wal-append.queue_time_us":1457259}
I20260709 13:03:53.886269  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:53.966645 13891 consensus_queue.cc:260] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:53.954464  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2284) took 1634 ms. Trace:
I20260709 13:03:53.969810 13891 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:53.970238  8251 rpcz_store.cc:276] 0709 13:03:52.319607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.323725 (+  4118us) service_pool.cc:224] Handling call
0709 13:03:52.324117 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.790748 (+466631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.954140 (+1163392us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b5400406854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:53.954379 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1071581056}
I20260709 13:03:53.882478  8142 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 81804f59e3124d27998ea80fc1c36663 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 9)
W20260709 13:03:53.972363  8142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 89919.9us
I20260709 13:03:53.952739 13897 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:53.977176 13897 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:53.815356 13877 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.978019 12973 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:53.978422 13877 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:53.979140 13654 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:53.979784 13948 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:53.981367 13654 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:53.984902  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:53.985476 13972 log.cc:927] Time spent T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.273s	user 0.002s	sys 0.001s
W20260709 13:03:53.986598  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.000717  8400 raft_consensus.cc:2410] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:03:54.000190  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:53.998238  8398 raft_consensus.cc:2410] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:03:53.896047  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:53.840176  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 181266us
I20260709 13:03:53.718714  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:53.885561  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:53.992763  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:54.009845  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.009974  9006 raft_consensus.cc:2410] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:03:53.956135 13806 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:53.993538  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:53.994889  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:54.012080  8974 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:54.012401  8406 raft_consensus.cc:2410] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:03:54.012966 13927 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:54.013360 13927 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.013747 13927 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:54.014109  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:54.016036 13873 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.016702 13873 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:54.019430 13873 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:54.021556  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:53.693076  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 62618.6us
I20260709 13:03:53.712605  8253 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 16.
I20260709 13:03:53.839733  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:03:53.901201 13973 log.cc:927] Time spent T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:03:54.031116  8332 raft_consensus.cc:1275] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:03:54.022904  8974 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:53.767311 13658 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.040251  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:53.987021 12973 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:54.011400 13806 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:54.024634  8405 raft_consensus.cc:2393] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 17: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:54.026134  8295 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:54.041219 12973 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:53.999516 13680 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:54.047509  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:53.944666 13813 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.027456 13877 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:54.029004  8325 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 14
W20260709 13:03:53.913159 13940 consensus_meta.cc:287] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.583s	user 0.010s	sys 0.000s
I20260709 13:03:53.952441  9003 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:53.964846 13865 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:53.854674 13475 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.945835 13956 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:54.035501  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:53.856446 13960 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:54.058512 13475 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:54.066591 13960 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:54.066810  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:54.067723  8332 raft_consensus.cc:2393] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 17: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:54.068339 13475 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:54.068642 13960 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:54.069330  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:54.071173  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:54.037662  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:54.071874  8256 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:03:54.038769  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:53.814846 13959 consensus_queue.cc:1059] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.232s
I20260709 13:03:54.039763  8257 pending_rounds.cc:85] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 13
I20260709 13:03:54.038503  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:54.006307 13902 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.071235  8404 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
I20260709 13:03:53.719641 13937 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:54.050629  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 100805us
W20260709 13:03:54.084735  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 52605.6us
I20260709 13:03:54.001287 13607 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:54.050130  8295 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:03:53.995909  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 319316us
I20260709 13:03:54.090291  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:54.091862  8249 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:54.057396  8252 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:03:54.052289 13979 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:54.048237 13654 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:54.060967  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:54.061318  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:54.100214  8405 raft_consensus.cc:2393] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 15: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:54.025179 13800 consensus_queue.cc:1059] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:03:54.050760 13538 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.102907  8220 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:54.102705 13538 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:54.104131  8220 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:54.104966 13538 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:54.066109  8331 raft_consensus.cc:2410] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:03:54.071250  8326 raft_consensus.cc:2410] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:03:54.073621  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:54.106321  8250 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
I20260709 13:03:54.106681 13840 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:54.107116 13840 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.107476 13840 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:54.107720 13772 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:54.097986 13607 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:54.105265  8220 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:53.779148 13824 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.079028  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:54.077111  8254 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
W20260709 13:03:54.085598  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 76227.9us
I20260709 13:03:54.076650 13877 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:54.058132 12973 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:54.140308  8325 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:54.054909 13865 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:54.095352  8298 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:54.102072  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 52049.1us
I20260709 13:03:54.142199  8295 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:54.074391  8257 pending_rounds.cc:107] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 15.14
I20260709 13:03:54.112475  8220 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:54.112914  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 2336) took 1012 ms. Trace:
I20260709 13:03:54.054509 13774 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:54.115231  8400 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:03:54.119577 13962 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:54.120217  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:54.122072 13672 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:54.071669 13813 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:54.087859 13902 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:54.094841 13891 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:03:54.051761 13658 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:54.124395  8222 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:54.054762 13806 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:54.125377  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:54.117653 13538 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:54.126179  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:54.195753  9005 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:54.126403 13800 consensus_queue.cc:1243] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:03:54.111147 13607 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:54.135406 13824 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:54.143266  8257 raft_consensus.cc:2889] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:54.143441  8255 rpcz_store.cc:276] 0709 13:03:53.100900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.107351 (+  6451us) service_pool.cc:224] Handling call
0709 13:03:54.112732 (+1005381us) spinlock_profiling.cc:243] Waited 749 ms on lock 0x7b5400178754. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:54.112852 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1722651904}
W20260709 13:03:54.148224  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 53001.4us
I20260709 13:03:54.149195  8327 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:54.131953 13275 consensus_queue.cc:260] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:03:54.205193  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:54.151659 13774 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.154958  8220 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:54.155417  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 98791.1us
I20260709 13:03:54.054155 13680 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:54.156332  8295 leader_election.cc:400] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14.
I20260709 13:03:54.114051 13654 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:54.217932  8295 leader_election.cc:403] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:03:54.218942  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 62833.9us
W20260709 13:03:54.155913  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 70107.3us
I20260709 13:03:54.191978  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:54.198006  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 73688.3us
I20260709 13:03:54.220362  8400 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:03:54.223861  9003 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:54.224090  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:54.193650  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:54.152064 13968 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:54.152494 13962 consensus_queue.cc:1243] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer 8c42259d7c4b463a9f485f42c320c81b log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:03:54.205928  9007 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:03:54.094012 13755 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:54.154170 13877 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:54.159052 13963 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:54.208078  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.242020 13959 raft_consensus.cc:3055] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:03:54.213271  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.242517 13959 raft_consensus.cc:740] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:54.152307 13554 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:54.229693 13800 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14.
I20260709 13:03:54.223598 13960 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:54.207288 13817 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:54.275168 13817 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:54.217360  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 62491.3us
W20260709 13:03:54.276875 13577 consensus_meta.cc:287] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.615s	user 0.011s	sys 0.000s
I20260709 13:03:54.277031  8220 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:54.278893 13909 consensus_meta.cc:287] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.613s	user 0.008s	sys 0.004s
I20260709 13:03:54.280836  8298 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:54.281833  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:54.157313  8297 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:54.244114  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:54.197163 13927 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:54.246091  8369 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:54.286865 13757 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:54.287444 13927 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:54.288215  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:54.288480 13902 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:54.288679 13962 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:54.246692  9002 consensus_queue.cc:260] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:54.289402 13669 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:54.290086  9002 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:54.290264  9006 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:03:54.291342  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:54.292084 13475 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.292737  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:54.293632 13800 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:54.294124 13800 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.294535 13800 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:54.098162 13875 consensus_queue.cc:237] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:54.297600  9000 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:03:54.259946  8325 raft_consensus.cc:1275] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:03:54.298982  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:54.300043  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:54.300397 13824 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:54.300674  8325 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
W20260709 13:03:54.301683  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 53766.9us
I20260709 13:03:54.302387  8972 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:54.303249 13577 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:54.304171 13877 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:54.305059 13877 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:54.260051  8331 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:54.207036 13770 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:54.243765  8223 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:03:54.339244  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 95576.9us
W20260709 13:03:54.339629  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 62697.9us
I20260709 13:03:54.267002 13654 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:54.268250 13813 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:54.270730 13806 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:54.271898 13958 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:54.229997 13275 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:54.390448  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:54.274726 13680 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:54.220319 13774 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:54.406088  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:54.320464 13877 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.260946  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:54.153872 13956 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:54.307824  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 150562us
I20260709 13:03:54.337901 13607 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:54.290126 13669 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:54.292688 13475 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:54.325734  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:54.420169  8404 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:03:54.420751  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:54.326957  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:54.327632  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:54.422513  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 94951.1us
I20260709 13:03:54.423033  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:54.331350  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:54.430773  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:54.432348  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:54.344020  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:54.434053  8331 raft_consensus.cc:2410] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:03:54.434856  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:54.277256 13817 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:54.391252  8327 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
W20260709 13:03:54.394960  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 64524.4us
I20260709 13:03:54.436841  8972 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:54.437987  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:54.438671 13948 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:54.439342 13948 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:54.439704 13948 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.440061 13948 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:54.440773  8404 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:03:54.407063  8252 raft_consensus.cc:2393] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 14: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:54.294610  9005 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:03:54.299001 13658 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:54.476552 13956 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:54.448457 13654 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:54.462427  8372 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:54.306067 13824 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:54.477618 13549 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:54.477847 13824 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.478312 13824 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:54.413592  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:54.414717  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:54.402405  9002 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 13:03:54.441340  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:54.443172  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:54.386636 13306 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:54.385974 13962 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.446591  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:54.447814  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:54.449313  8972 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:54.451980  8297 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:54.415638 13877 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:54.482290  8326 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:54.488206  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:54.488759  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:54.489403  8326 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:03:54.489602  8252 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 13: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:54.490329  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:54.491053  8326 raft_consensus.cc:2393] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 17: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:54.385530 13960 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:54.487576  8406 raft_consensus.cc:2368] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 16. Current term is 17.
I20260709 13:03:54.491425  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:54.491988  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 44282.8us
I20260709 13:03:54.495649  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:54.502701  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:54.502964  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:54.503742  8253 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:54.503906  8972 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:54.500663  9002 pending_rounds.cc:85] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: Aborting all ops after (but not including) 14
I20260709 13:03:54.509097  8372 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:54.385298 13757 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:03:54.508883  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:54.517510  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:54.511206  8972 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:54.414129 13607 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.518435  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:54.409358 13806 consensus_queue.cc:1048] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.099s
W20260709 13:03:54.543032  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.549932  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:54.507543 13658 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.527850  9002 pending_rounds.cc:107] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 13:03:54.550776  8327 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:54.562443  9002 raft_consensus.cc:2889] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:54.507784 13654 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:54.490831 13680 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:54.501803 13979 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:54.545405  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.508018 13956 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:54.563301  9002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 952) took 1293 ms. Trace:
I20260709 13:03:54.511624 13825 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:54.511878 13968 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:54.513947 13927 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:54.514173 13960 raft_consensus.cc:697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:54.542904 13940 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.553766 13937 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:54.583142  9002 rpcz_store.cc:276] 0709 13:03:53.270281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.300172 (+ 29891us) service_pool.cc:224] Handling call
0709 13:03:53.325129 (+ 24957us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.964524 (+639395us) spinlock_profiling.cc:243] Waited 1.19 ms on lock 0x7b4c00485bc0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8826a8aa20 00007f8826a9ba29 00007f8826a61644 00007f8826a88f2d 00007f8826a88b93 00007f88287b918b 00007f88287b9cb5 00007f88287b4f30 00007f88287b67de 00007f88287c0b49 00007f88287c1416
0709 13:03:54.246250 (+281726us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b54005cfba0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f882873c06f 00007f88287b4f60 00007f88287b67de 00007f88287c0b49 00007f88287c1416 00007f88287bdd8f 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62
0709 13:03:54.563007 (+316757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.563237 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43090176}
I20260709 13:03:54.554368 13607 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:54.550709 13577 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:54.554886 13806 consensus_queue.cc:1243] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer ff7a6fe533e94c1bae555b4cae427728 log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:03:54.577818  8974 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:03:54.636819  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 59103.4us
I20260709 13:03:54.507086 13962 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:54.561131 13846 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:03:54.586529 13944 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 13:03:54.630906 13948 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:54.589689 13902 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.616207 13937 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:54.582149 13658 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:54.653312 13979 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:54.650578 13846 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.615042 13968 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:54.632675 13940 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:54.679234 13840 consensus_meta.cc:287] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.571s	user 0.011s	sys 0.000s
I20260709 13:03:54.671186 13937 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:54.689633 13846 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:54.720790 13765 raft_consensus.cc:3055] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:54.721377 13765 raft_consensus.cc:740] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:54.692284 13979 raft_consensus.cc:697] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 17 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:54.675839 13902 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:54.728376 13983 log.cc:927] Time spent T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 13:03:54.702279 13840 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:54.690022 13960 consensus_queue.cc:237] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:54.780675 13948 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:03:54.786368 13995 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 13:03:54.791079 13989 log.cc:927] Time spent T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:03:54.796705 13962 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:54.796959 13877 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:54.801121 13877 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:54.815938 13902 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:54.818822 13959 consensus_queue.cc:260] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:54.834033 13959 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:54.781348 13979 consensus_queue.cc:237] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:54.822566 13968 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:54.824141 13940 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:54.919405  8220 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:54.928328  8251 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.004s	sys 0.012s
W20260709 13:03:54.928990  8251 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.004s	sys 0.012s
I20260709 13:03:54.932320  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:54.934104  8330 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.007s	sys 0.009s
I20260709 13:03:54.934336  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:54.934676  8330 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.007s	sys 0.009s
I20260709 13:03:54.935470  8256 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:03:54.935829  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:03:54.937883  8332 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.003s	sys 0.010s
W20260709 13:03:54.938354  8332 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.003s	sys 0.011s
I20260709 13:03:54.938609  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:54.939841  8327 raft_consensus.cc:2410] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
W20260709 13:03:54.943019  8402 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.021s	sys 0.015s
W20260709 13:03:54.943624  8402 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.021s	sys 0.015s
W20260709 13:03:54.937325  8329 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.004s	sys 0.009s
W20260709 13:03:54.946036  8329 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.005s	sys 0.009s
I20260709 13:03:54.948639  8323 raft_consensus.cc:1240] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:54.951987  8330 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
W20260709 13:03:54.928387  8248 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.826s	user 0.005s	sys 0.017s
I20260709 13:03:54.952574  8330 pending_rounds.cc:85] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 10
W20260709 13:03:54.952802  8248 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.006s	sys 0.017s
I20260709 13:03:54.953086  8330 pending_rounds.cc:107] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:03:54.954030  8330 raft_consensus.cc:2889] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:54.957242 12973 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:54.957542 13817 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.957813  9002 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:54.958366 12973 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:54.958550 13817 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:54.961297 13306 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:54.962551 13817 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:54.962412 13306 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:54.966010 13275 consensus_meta.cc:287] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.572s	user 0.007s	sys 0.002s
W20260709 13:03:54.934738  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:54.992040  8249 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.002s	sys 0.009s
W20260709 13:03:54.928406  8257 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.019s	sys 0.027s
I20260709 13:03:54.996587 12973 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:54.996816 13773 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:55.001726 13680 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.002590 13680 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:54.995107 13306 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:55.006160 13813 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:55.006304  8324 raft_consensus.cc:1275] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:03:55.006939  8324 pending_rounds.cc:85] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 8
I20260709 13:03:55.007318  8324 pending_rounds.cc:107] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:03:55.007197 13813 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:55.008347  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:55.011313  8248 pending_rounds.cc:85] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 7
I20260709 13:03:55.011674  8248 pending_rounds.cc:107] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:03:55.011973  8248 raft_consensus.cc:2889] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:55.001760  8252 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:03:55.022486 13669 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:55.023187  8257 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.020s	sys 0.027s
I20260709 13:03:54.969988 13654 raft_consensus.cc:697] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:55.027333 13654 consensus_queue.cc:237] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:55.028584 13475 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:55.029942 13939 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.007812  8324 raft_consensus.cc:2889] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:55.030979 13939 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:55.031867  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:55.035462 13773 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:55.036149 13773 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:54.992604  8249 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.002s	sys 0.009s
I20260709 13:03:55.016920 13944 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:55.064335  8250 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.009s	sys 0.005s
W20260709 13:03:55.066087  8250 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.009s	sys 0.005s
W20260709 13:03:55.067272  8254 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.006s	sys 0.009s
W20260709 13:03:55.067776  8254 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.006s	sys 0.009s
I20260709 13:03:55.031088  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:55.005373 13680 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:55.021027  8972 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:55.075523  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:55.080052  8256 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:03:55.086088  9002 raft_consensus.cc:2410] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
W20260709 13:03:55.087178  8222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.783s	user 0.000s	sys 0.025s
W20260709 13:03:55.087648  8222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.000s	sys 0.025s
W20260709 13:03:55.094341  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.096099 13937 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:55.100389  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 2341) took 1063 ms. Trace:
I20260709 13:03:55.102599 13658 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:55.101383  8251 rpcz_store.cc:276] 0709 13:03:54.037393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.037638 (+   245us) service_pool.cc:224] Handling call
0709 13:03:54.037951 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:54.051272 (+ 13321us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:54.079779 (+ 28507us) log.cc:844] Serialized 14 byte log entry
0709 13:03:54.930087 (+850308us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:54.931549 (+  1462us) log.cc:844] Serialized 26 byte log entry
0709 13:03:54.931865 (+   316us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:54.931926 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.932073 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:55.099897 (+167824us) raft_consensus.cc:1685] finished
0709 13:03:55.099938 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:55.100246 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8836,"prepare.queue_time_us":32664,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":116722,"raft.run_cpu_time_us":417,"raft.run_wall_time_us":5508,"thread_start_us":789744,"threads_started":1,"wal-append.queue_time_us":602585}
W20260709 13:03:55.104832  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 84285us
I20260709 13:03:55.110110 13948 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:03:55.113610 13875 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.005s	sys 0.019s
W20260709 13:03:55.114162 13875 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.005s	sys 0.019s
I20260709 13:03:55.114364 13948 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:55.115310 13948 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:55.117045  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:55.117993  8400 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:55.110239  8327 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: [18.11-18.11]
W20260709 13:03:55.113235 13865 consensus_meta.cc:287] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.952s	user 0.009s	sys 0.002s
W20260709 13:03:55.101841 13538 consensus_meta.cc:287] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.902s	user 0.009s	sys 0.001s
I20260709 13:03:55.126873 13680 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:55.168275  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 2937) took 1184 ms. Trace:
I20260709 13:03:55.170908  8406 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:03:55.172827  8402 rpcz_store.cc:276] 0709 13:03:53.983571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.990550 (+  6979us) service_pool.cc:224] Handling call
0709 13:03:53.990843 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:54.230588 (+239745us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:54.230630 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:54.285289 (+ 54659us) log.cc:844] Serialized 26 byte log entry
0709 13:03:54.944483 (+659194us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:54.944614 (+   131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.167822 (+223208us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b540019a5d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:55.167872 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:55.167921 (+    49us) raft_consensus.cc:1685] finished
0709 13:03:55.167932 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:55.168198 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4408,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":11387,"spinlock_wait_cycles":550445824,"thread_start_us":658378,"threads_started":1,"wal-append.queue_time_us":472607}
I20260709 13:03:55.180400 14015 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:55.181394 14015 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:55.184372 14015 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:55.185629 13948 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:55.186288 13967 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.186466  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:55.187026 13967 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:03:55.188336 13895 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:55.189124 13895 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:54.743858 13800 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:55.192685 13800 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:55.192919  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:55.193539 13877 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:55.194068  8331 raft_consensus.cc:2368] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 15. Current term is 16.
I20260709 13:03:55.194496 13877 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:55.197089  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 523596us
W20260709 13:03:55.200486 13607 consensus_meta.cc:287] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.560s	user 0.010s	sys 0.001s
I20260709 13:03:55.154594 13275 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:55.201924 13275 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:55.204463 13275 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:55.205678  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:55.214006  8253 consensus_meta.cc:287] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.588s	user 0.008s	sys 0.008s
I20260709 13:03:55.214586  8253 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:03:55.222355  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:55.229583 14007 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.248639  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2794) took 1424 ms. Trace:
I20260709 13:03:55.250597  8324 rpcz_store.cc:276] 0709 13:03:53.816283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.817096 (+   813us) service_pool.cc:224] Handling call
0709 13:03:53.859898 (+ 42802us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:55.031749 (+1171851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.232606 (+200857us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54002c9054. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:55.240596 (+  7990us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2554895744}
I20260709 13:03:55.201098 13607 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:55.156332 13865 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:55.230420 14007 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:55.253516  8325 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:55.141530 13944 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:55.257194  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:55.259516  8400 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 20.
I20260709 13:03:55.260735 13773 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:55.261650 13680 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:55.281908 13680 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:55.282894 13865 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:55.286580  8326 raft_consensus.cc:2410] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:03:55.287144 13739 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:55.287858 14007 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:55.287807 13739 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:55.288704  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 96187.2us
I20260709 13:03:55.289608 13875 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:55.290249 13875 raft_consensus.cc:697] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:55.290459 13658 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.291143 13658 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:55.291430 13875 consensus_queue.cc:237] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:55.292752  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:55.293684  9006 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:03:55.293623  8222 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:55.294734  8298 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:55.296103  8222 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:55.296388 13865 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:55.297586 12973 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:55.262229 13607 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:55.299577 13607 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:55.300138 13607 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:55.300729 13607 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:55.262079  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:55.266942 13937 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:55.301864  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:55.269906  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:55.270746 13981 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:55.271381 13997 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:55.271620  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:55.248422 13813 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:55.248413 13939 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:55.277837 13554 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.278759  8399 raft_consensus.cc:2410] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:03:55.279354 13665 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:55.279541 13773 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:55.297199 13865 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:55.269083 13895 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:55.275781  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:55.267283  8220 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:55.304718 13275 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 13:03:55.233603 13846 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:55.310160  8401 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:03:55.300817  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:55.304349 13981 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:55.291148 13944 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:55.313885 12973 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:55.331008  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:55.339869  8332 raft_consensus.cc:2393] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 15: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:55.345572  8295 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:03:55.347239  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 98391us
I20260709 13:03:55.350461 13800 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:55.351022 13800 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:55.351423 13800 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:55.356750  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:55.332548  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:55.333333  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:55.327569 13658 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:55.334512 13554 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:55.333856  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:55.362528 13939 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:55.361757  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:55.345453  8372 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:55.255869 13538 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:55.333017  8297 leader_election.cc:400] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 16. Current term is 17.
I20260709 13:03:55.366462  8297 leader_election.cc:403] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:03:55.371240  8222 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:55.368645  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:55.382740 13846 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:55.383034  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:55.389856  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:55.390779  8401 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 17: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:55.394141 13739 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:55.396768 13554 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:55.382756  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:55.364107 13939 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:55.408234  9004 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:03:55.367259 13877 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:55.409736  9009 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
W20260709 13:03:55.375805 14023 log.cc:927] Time spent T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:03:55.419654  8222 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:55.410691  8323 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
W20260709 13:03:55.415732  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 70506.1us
I20260709 13:03:55.365108 14007 consensus_queue.cc:1059] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 13:03:55.422385 13959 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 16. Current term is 17.
W20260709 13:03:55.424152 13824 consensus_meta.cc:287] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.945s	user 0.014s	sys 0.005s
I20260709 13:03:55.424602 13824 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:55.428606  8326 pending_rounds.cc:85] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 8
I20260709 13:03:55.429248  8326 pending_rounds.cc:107] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 10.9
W20260709 13:03:55.429342  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 96762us
I20260709 13:03:55.429594  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 2350) took 1088 ms. Trace:
I20260709 13:03:55.429651  8326 raft_consensus.cc:2889] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:55.324155 13997 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:55.410444 13939 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:55.430052  8248 rpcz_store.cc:276] 0709 13:03:54.341298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.010179 (+668881us) service_pool.cc:224] Handling call
0709 13:03:55.010850 (+   671us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:55.041807 (+ 30957us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:55.041838 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:55.079959 (+ 38121us) log.cc:844] Serialized 52 byte log entry
0709 13:03:55.255355 (+175396us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:55.318087 (+ 62732us) log.cc:844] Serialized 14 byte log entry
0709 13:03:55.318383 (+   296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.318481 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:55.404064 (+ 85583us) spinlock_profiling.cc:243] Waited 1.35 ms on lock 0x7b18004899a0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:03:55.429153 (+ 25089us) raft_consensus.cc:1685] finished
0709 13:03:55.429263 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:55.429417 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":126748,"prepare.queue_time_us":62760,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":50,"raft.queue_time_us":53111,"raft.run_cpu_time_us":349,"raft.run_wall_time_us":55909,"spinlock_wait_cycles":4265984,"thread_start_us":115499,"threads_started":1,"wal-append.queue_time_us":195154}
I20260709 13:03:55.443946  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 830) took 1147 ms. Trace:
I20260709 13:03:55.444520  8331 rpcz_store.cc:276] 0709 13:03:54.296077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.434627 (+138550us) service_pool.cc:224] Handling call
0709 13:03:55.443685 (+1009058us) spinlock_profiling.cc:243] Waited 758 ms on lock 0x7b540014c654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:55.443877 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1743553280}
I20260709 13:03:55.431347 13554 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:55.446049 13970 log.cc:927] Time spent T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:03:55.446985 13981 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:55.448009 13892 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:55.454047 13770 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:55.455379 13997 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:55.457453 13549 consensus_meta.cc:287] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.979s	user 0.010s	sys 0.008s
I20260709 13:03:55.458367  8405 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:03:55.458544 13939 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:55.459617 13961 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.460117  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 2351) took 1116 ms. Trace:
I20260709 13:03:55.460563  8255 rpcz_store.cc:276] 0709 13:03:54.344097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.014174 (+670077us) service_pool.cc:224] Handling call
0709 13:03:55.014746 (+   572us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:55.249993 (+235247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:55.250022 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:55.283511 (+ 33489us) log.cc:844] Serialized 26 byte log entry
0709 13:03:55.406206 (+122695us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:55.406248 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.406323 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:55.459785 (+ 53462us) raft_consensus.cc:1685] finished
0709 13:03:55.459806 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:55.459946 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":233470,"raft.queue_time_us":235349,"spinlock_wait_cycles":101478656,"thread_start_us":122227,"threads_started":1,"wal-append.queue_time_us":169930}
I20260709 13:03:55.462108 13765 consensus_queue.cc:260] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:55.462895  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:55.463661 13765 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:55.464408 13757 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:55.465026 13757 raft_consensus.cc:697] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:55.421149 13865 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:55.466053 14007 consensus_queue.cc:1243] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 18.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:03:55.468122  8372 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:55.468709 13873 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:55.469103 13846 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:55.469748  8372 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:55.469848  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:55.470302  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:55.469825 13846 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.471652 13846 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:55.473774 13944 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:55.460165 13961 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:55.474447 13944 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:55.466343 13757 consensus_queue.cc:237] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:55.477710 13961 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:55.478259 13962 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:55.478837 13962 raft_consensus.cc:697] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:55.479313 13873 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:55.480302 13824 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:55.479894 13962 consensus_queue.cc:237] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:55.483505  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:55.489005 13959 raft_consensus.cc:3055] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:55.489748  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:55.491405 13840 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:55.492158 13607 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:55.492869  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 879) took 1167 ms. Trace:
I20260709 13:03:55.477232  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:55.477794  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 210697us
I20260709 13:03:55.516245  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 882) took 1105 ms. Trace:
I20260709 13:03:55.437906 13654 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:55.535820  8405 rpcz_store.cc:276] 0709 13:03:54.325123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.326597 (+  1474us) service_pool.cc:224] Handling call
0709 13:03:55.492592 (+1165995us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54003e6f54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:55.492766 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2383134080}
I20260709 13:03:55.567837  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:55.568197  9009 raft_consensus.cc:2410] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:03:55.565805  8252 rpcz_store.cc:276] 0709 13:03:54.410843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.018607 (+607764us) service_pool.cc:224] Handling call
0709 13:03:55.019337 (+   730us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:55.171262 (+151925us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:55.171291 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:55.171974 (+   683us) log.cc:844] Serialized 52 byte log entry
0709 13:03:55.317889 (+145915us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:55.317937 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.318020 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:55.515828 (+197808us) raft_consensus.cc:1685] finished
0709 13:03:55.515863 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:55.516155 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":216064,"raft.queue_time_us":152490,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":135,"thread_start_us":145434,"threads_started":1,"wal-append.queue_time_us":268727}
W20260709 13:03:55.573747  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 122788us
W20260709 13:03:55.428647 13774 consensus_meta.cc:287] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.009s	user 0.008s	sys 0.004s
I20260709 13:03:55.575328  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:55.574934  8325 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:55.572206  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:55.575047  8222 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:55.546015 13959 raft_consensus.cc:740] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:55.580597  8330 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:55.490567 13846 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:55.584064  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:55.582062  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:55.582648  8328 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:03:55.586746 13846 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:55.581470  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:55.587731  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:55.588352  9002 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:55.588450  8399 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:03:55.569768 13873 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:55.590451  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:55.594035  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:55.595978  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:55.569733 13944 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:55.571769 13654 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.604777  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:55.612811  8331 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:03:55.607390  8972 leader_election.cc:400] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 15. Current term is 16.
I20260709 13:03:55.615056  8972 leader_election.cc:403] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:03:55.607825  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.609469  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:55.597393  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:55.608227  8323 raft_consensus.cc:2393] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 17: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:55.444018 13967 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:55.599870  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:55.627642  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:55.628312  8250 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:55.628253  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:55.629101  8249 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:55.629408  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:55.632314  8323 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:03:55.633317  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:55.630138 13554 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:55.626935  9002 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
I20260709 13:03:55.633742  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:55.640836  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:55.643168  8331 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:55.600257 13840 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:55.633991  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:55.633863  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:55.656623  8257 raft_consensus.cc:2368] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14.
I20260709 13:03:55.627651  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:55.636561  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:55.658422  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:55.658701  8398 raft_consensus.cc:2393] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 16: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:55.641811  8372 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:55.641042  8330 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
I20260709 13:03:55.661703  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:55.605917 13812 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:55.663008 13962 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:55.663456 13812 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:55.664202 13962 raft_consensus.cc:697] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:55.665647  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:55.665474 13962 consensus_queue.cc:237] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:55.666862  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:55.667484  8252 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:55.651945 13840 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:55.669512 13840 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:55.662987  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:55.672962  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:55.618093 13654 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:55.682122  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:55.680895  8402 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:55.584286 13774 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:55.655943 13944 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:03:55.690961  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:55.697774  8248 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:03:55.685978  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:55.708190  8250 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:03:55.708599  8328 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:55.700255  8220 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:55.712625  8331 raft_consensus.cc:2393] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 16: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:55.717142 13873 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:55.717595 13812 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:55.717744 13873 raft_consensus.cc:697] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:55.718508 14033 consensus_queue.cc:1048] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:03:55.718933 13873 consensus_queue.cc:237] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:55.717144  8403 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:55.679010 13658 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:03:55.722357 13840 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:55.723578  8254 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:03:55.714722 13959 consensus_queue.cc:260] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:55.725355 13840 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:55.726487 13959 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:55.733100  8403 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:03:55.717420  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:55.734583  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:55.735244  8250 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:03:55.735280  9009 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:03:55.736114  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:55.737061  8253 raft_consensus.cc:2410] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:03:55.738045  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:55.739248 13607 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:55.739599  8253 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:03:55.739189  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:55.745898  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:55.713514  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:55.741983  8369 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:55.683030 13549 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:55.737073  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:55.633374 13939 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:55.757522 13846 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:55.758172 13846 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.758617 13846 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:55.746518 13892 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:55.760953 13892 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:55.765919  8327 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15. Ops: [14.9-14.9]
W20260709 13:03:55.766049  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 65886.5us
I20260709 13:03:55.767153  8220 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:55.770488  8250 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
I20260709 13:03:55.752914  9000 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:55.778069  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:55.779372  8329 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:55.752671 13549 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.780923 13549 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:55.782150  8295 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:55.783615  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:55.771477 13554 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:55.797509 13607 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:55.798991  8372 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:55.813556  8220 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:55.817658  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:55.819518  8297 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:55.822739 13658 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:55.823321 13658 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.823827 13658 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:55.824633  8297 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:55.826644  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:55.795262 13812 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:55.828469 13812 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:55.830569 13967 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:55.831218 13967 raft_consensus.cc:697] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:55.816486  8248 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:55.832670  8297 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:55.840994  8397 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:55.844281 13800 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:55.845573 13770 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:55.846241 13770 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:55.846737  8330 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:55.848414 13770 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:55.849633 13960 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.850461 13960 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:55.831719 13812 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:55.857903  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:55.859663  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 81578us
I20260709 13:03:55.834225 13865 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:55.869935  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:55.870973  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:55.873972  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:55.871897  9000 raft_consensus.cc:1813] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Handling vote request from an unknown peer 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:55.872889  8328 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
I20260709 13:03:55.841770 13981 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:55.857393 13607 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:55.888271  9000 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 20: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:55.871141 13865 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.896595  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:55.892740 13768 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:55.909200  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:55.915915  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 114174us
I20260709 13:03:55.915072  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:55.919715  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:55.924763  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:55.905205 13865 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:55.925525  8253 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:03:55.925083  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:55.903719 13607 raft_consensus.cc:697] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 16 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:55.937664 13607 consensus_queue.cc:237] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:55.938591  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:55.939852  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:55.941612  8295 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:03:55.943104 14025 log.cc:927] Time spent T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.270s	user 0.000s	sys 0.001s
I20260709 13:03:55.945966 13680 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:55.946568 13680 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:55.947031 13680 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:55.930080  8249 raft_consensus.cc:2410] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:03:55.934520  8974 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:55.941864 14037 log.cc:927] Time spent T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:03:55.933095  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:55.954438 13774 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:55.910051 13800 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:55.965840 13960 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:55.922894 13967 consensus_queue.cc:237] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:55.972285 13680 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:55.975191  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.930514  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:55.981983  9005 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:03:55.985695  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:55.986555  8328 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 20: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:55.998242  8248 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 20.
I20260709 13:03:55.999534  8255 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:56.008396  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:56.009203 13680 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:56.009848  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:56.011520 13953 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:55.924393 14007 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.041373  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:56.042369  8328 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:56.046007  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:56.032483  8329 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:03:56.033833  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:56.047895  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:56.050025  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:56.056160 13892 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:56.058482  8326 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:56.059799  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:56.060783  8331 raft_consensus.cc:2368] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 18. Current term is 19.
I20260709 13:03:56.066093 13672 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:56.066555 13579 consensus_queue.cc:1059] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:56.066895 13672 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:56.067953 13579 consensus_queue.cc:1059] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:56.069065 13961 raft_consensus.cc:3055] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:56.069523 13961 raft_consensus.cc:740] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:56.069885 14008 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:56.070659 13961 consensus_queue.cc:260] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:56.070765 14008 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:56.072050 13937 raft_consensus.cc:3055] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:56.072536 13937 raft_consensus.cc:740] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:56.082338 13654 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:56.083033  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:56.029112 13944 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 15. Current term is 16.
I20260709 13:03:56.095887 13961 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:56.097777 13680 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.098310  8404 raft_consensus.cc:2368] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16.
I20260709 13:03:56.099045  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:56.103757  8252 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:03:56.127522  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:56.128924  8252 raft_consensus.cc:2393] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 16: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:56.034584 13939 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:56.140105  8248 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:56.087812 13549 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:56.122844 13654 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:56.177170  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:56.178241  8325 raft_consensus.cc:2393] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 15: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:56.175228  9004 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:03:56.178653  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:56.179629  8252 raft_consensus.cc:2393] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 11: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:56.179705  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:56.181838  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:56.098358 13680 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:56.184321  9002 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:03:56.191577  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:56.192381  9004 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:56.196795 13937 consensus_queue.cc:260] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:56.197520  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:56.199536  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:56.200204  8248 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:03:56.200336  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:56.201297  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:56.201995  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:56.203168  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:56.203886  8254 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:56.207022  8220 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:56.186565 13680 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:56.210299  8220 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:56.211129  8403 raft_consensus.cc:2368] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16.
I20260709 13:03:56.168478 13275 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.107678 14007 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:56.220373  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:56.220893  9005 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:56.221027  8220 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:56.168670 13672 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:56.227998  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:56.184362 13554 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:56.220700  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:56.225353  8220 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:56.193965 13962 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.235556 13962 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:56.226817 13960 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:56.226073 13275 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:56.231119 14008 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:56.243392  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:56.244264  8399 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:03:56.244671 13275 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:56.246511 13549 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:56.250954 14007 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:56.252638 13658 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:56.223871 13846 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:56.255362 13658 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:56.255950 13846 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:56.228268 13892 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:56.260445  8223 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:56.262040  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:56.217936 13768 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:56.271184  8254 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:56.263418  8297 leader_election.cc:400] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16.
I20260709 13:03:56.271136  9004 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:03:56.264580  8328 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:03:56.272356  8297 leader_election.cc:403] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 13:03:56.268242  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 337479us
I20260709 13:03:56.268564  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:56.263736  8220 leader_election.cc:400] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 18. Current term is 19.
I20260709 13:03:56.275429  8220 leader_election.cc:403] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:56.281158  8223 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:56.267004 13770 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:56.289245  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:56.294708  8325 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:03:56.311651  8297 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:56.274737 13960 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:56.309420 14007 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:56.267220 13813 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:03:56.333768 13962 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:56.283701 13840 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 18. Current term is 19.
I20260709 13:03:56.310667 13800 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.341935 13800 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:56.350728  8220 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:56.356642  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:56.307581 13812 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:56.378551  8220 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:56.381459  8222 leader_election.cc:400] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16.
I20260709 13:03:56.381999  8222 leader_election.cc:403] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:03:56.358394 13967 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:56.386181 13967 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.386586 13967 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:56.386659 14033 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:56.387250 14033 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.387566 14033 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:56.379829 13554 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:03:56.393661  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.393204 13873 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:56.407550 13892 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:56.429376  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 477119us
I20260709 13:03:56.409569 13865 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:56.438608  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 248084us
I20260709 13:03:56.450376  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:56.451390  8404 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:03:56.453455  8297 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:56.460377  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.436658 13680 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:56.436177 13800 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:56.462507 13937 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:56.461581 13962 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.490696 13774 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:56.528328 13680 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:56.509917 13875 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 13:03:56.530582 13962 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:56.532724 13962 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:56.537671 13774 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.538203 13774 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:56.500706  8147 catalog_manager.cc:5697] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 10 to 12, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 12 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:56.540805  8974 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:56.544691 14046 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:56.545249 14046 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:56.545778 14046 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:56.548544  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 473551us
I20260709 13:03:56.583986  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:03:56.584645  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 200982us
W20260709 13:03:56.590034  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 308927us
I20260709 13:03:56.588321  8372 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:56.602732  8399 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:03:56.604223  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:56.578567 13554 raft_consensus.cc:697] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:56.608616  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:56.616359  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 102502us
I20260709 13:03:56.617471  8974 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:56.601078 14007 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16.
W20260709 13:03:56.644726  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 355567us
I20260709 13:03:56.645118  8298 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:56.654985  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:56.656659  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:56.659243  8399 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:03:56.658978 14033 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:56.661543 13765 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:56.666487 13865 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:56.620291 13824 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:56.680959 13824 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:56.683537 13824 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:56.619944 13980 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:56.634150 13944 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:56.695354  8369 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:56.698845  8972 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:56.705111  8295 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:56.675405 13768 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:56.607172 14053 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 13:03:56.716512  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:56.717166  8404 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:56.666635 14054 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:56.651495 13607 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.715682  8295 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:56.685649 13865 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:56.672879 13981 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:56.718850  9006 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:56.719225 13953 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:56.719749 13806 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 13:03:56.720100  8972 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:56.709595 13765 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:56.720781 13939 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:56.708415 13980 raft_consensus.cc:697] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:56.727352 13774 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:56.676604  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:56.736929  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:56.671334 13812 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:03:56.739171  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 43915.3us
I20260709 13:03:56.742405  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:56.749133  8295 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:56.754166  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:56.756565 13981 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:56.759238 14055 log.cc:927] Time spent T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:03:56.754618  8147 catalog_manager.cc:5697] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:56.743717 14033 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:56.786275  8330 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:03:56.793910 13680 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:56.767066 13806 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:56.796713 13806 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:56.800161 13806 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:56.801787 14051 log.cc:927] Time spent T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
W20260709 13:03:56.802440 14049 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:03:56.750360 13607 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:56.743651 13800 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:56.777269 13812 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:56.778463  8369 leader_election.cc:400] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14.
I20260709 13:03:56.778980 13765 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:56.819216  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:56.820710 13875 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:03:56.789196 14070 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.104s	user 0.003s	sys 0.000s
I20260709 13:03:56.772801 13967 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:56.814254 13980 consensus_queue.cc:237] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:56.816344  9009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 971) took 1047 ms. Trace:
I20260709 13:03:56.809618 14066 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:56.841357  8328 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:56.781107 13774 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:56.827121  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:56.828128  8404 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:03:56.833876  8329 raft_consensus.cc:1240] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:56.836827  8323 raft_consensus.cc:1275] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:03:56.840934  8327 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:56.760617 13554 consensus_queue.cc:237] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:56.842069  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.821283 13875 raft_consensus.cc:697] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:56.848898  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:56.849980 13875 consensus_queue.cc:237] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:56.851795 14046 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:56.746595 14054 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:56.853077  9004 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:56.853696  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:56.860080 13892 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16.
I20260709 13:03:56.874506  9004 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:56.839002  9009 rpcz_store.cc:276] 0709 13:03:55.768485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.768647 (+   162us) service_pool.cc:224] Handling call
0709 13:03:55.791554 (+ 22907us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:55.812298 (+ 20744us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:55.812328 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:55.859424 (+ 47096us) log.cc:844] Serialized 26 byte log entry
0709 13:03:56.275085 (+415661us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:56.275141 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:56.275236 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:56.792841 (+517605us) spinlock_profiling.cc:243] Waited 4.42 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:56.795650 (+  2809us) raft_consensus.cc:1685] finished
0709 13:03:56.795678 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:56.816242 (+ 20564us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18929,"raft.queue_time_us":12135,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":3961,"spinlock_wait_cycles":10156928,"thread_start_us":359545,"threads_started":1,"wal-append.queue_time_us":768274}
I20260709 13:03:56.888532  8295 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:56.905795  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:56.906523  8403 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:56.906771  8405 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:56.890420  8298 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:56.926738  8402 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
W20260709 13:03:56.932083  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 43699.6us
I20260709 13:03:56.933040  8325 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:03:56.933034  8295 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:56.937280  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:56.937952  8327 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:56.939917  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:56.913184 14070 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.067s	user 0.002s	sys 0.000s
I20260709 13:03:56.865351 14066 raft_consensus.cc:697] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 20 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:03:56.934746 13800 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:56.942535  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:56.941164  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:56.958168  8297 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:57.018359  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 60262.2us
I20260709 13:03:56.965332 14065 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:56.965641  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:57.023451  9002 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:03:56.969969  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:56.973554  8250 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:03:56.974802  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:56.980598 14062 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.030761  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:57.031687  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 141420us
I20260709 13:03:56.988355  8331 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:56.997808 13812 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:56.999478  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:57.000033  8153 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45456 (request call id 283) took 68048 ms (client timeout 15000 ms). Trace:
I20260709 13:03:56.942296 13757 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:57.002454 13967 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:57.044509  8253 raft_consensus.cc:1275] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:03:57.002379  8327 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:03:57.003326 14054 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:57.004634 14038 log.cc:927] Time spent T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.214s	user 0.001s	sys 0.001s
I20260709 13:03:57.005694  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:57.049297  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:57.049988  8329 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:57.008471  8330 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
W20260709 13:03:57.011307  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 78350.7us
I20260709 13:03:57.051376  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:57.051950  8327 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:57.052606  8295 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:57.011552  8974 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:57.028025  8248 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:03:57.013020 13774 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:57.039642  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:57.041975  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:57.042891 14069 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:57.043718 13757 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:57.036217  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:57.063710  8249 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:56.960407 14068 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:57.031055 13865 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:57.024874 13846 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:03:57.057988  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 46549.5us
I20260709 13:03:57.081212  8974 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:57.060396  8298 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:57.060608  8295 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:57.062983  8325 raft_consensus.cc:2368] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 16. Current term is 17.
I20260709 13:03:57.084489  8327 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:57.071174  8403 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:03:57.064829 14007 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:57.075624  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:57.035624 13680 consensus_queue.cc:1059] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:03:57.086920  8220 leader_election.cc:400] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 16. Current term is 17.
I20260709 13:03:57.097925  8220 leader_election.cc:403] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:57.061900 14069 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:57.089665 13757 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:57.090044  8397 raft_consensus.cc:2393] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 16: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:03:57.092734 13953 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:57.095010 13865 raft_consensus.cc:697] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:57.095582 13846 raft_consensus.cc:697] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:03:57.036245  8153 rpcz_store.cc:269] 0709 13:02:48.951235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.972153 (+ 20918us) service_pool.cc:224] Handling call
0709 13:03:56.999759 (+68027606us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:53.879379 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:53.880052 (+   673us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:53.880091 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:03:53.880112 (+    21us) tablet.cc:662] Decoding operations
0709 13:03:53.880797 (+   685us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:53.880856 (+    59us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:53.880883 (+    27us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:03:53.881287 (+   404us) tablet.cc:701] Row locks acquired
0709 13:03:53.881309 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:03:53.881903 (+   594us) write_op.cc:270] Start()
0709 13:03:53.882009 (+   106us) write_op.cc:276] Timestamp: P: 1783602233881415 usec, L: 0
0709 13:03:53.882030 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:03:53.882698 (+   668us) log.cc:844] Serialized 1677 byte log entry
0709 13:03:54.742458 (+859760us) op_driver.cc:464] REPLICATION: finished
0709 13:03:54.870192 (+127734us) write_op.cc:301] APPLY: starting
0709 13:03:54.870304 (+   112us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:54.883042 (+ 12738us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:54.883225 (+   183us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:55.096422 (+213197us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:55.254420 (+157998us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 13:03:55.273709 (+ 19289us) write_op.cc:312] APPLY: finished
0709 13:03:55.300864 (+ 27155us) log.cc:844] Serialized 49 byte log entry
0709 13:03:55.301252 (+   388us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:55.301454 (+   202us) write_op.cc:454] Released schema lock
0709 13:03:55.301585 (+   131us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:55.302472 (+   887us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:55.302507 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1329353,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":119288,"apply.run_cpu_time_us":12260,"apply.run_wall_time_us":432098,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20855,"num_ops":4,"prepare.queue_time_us":221644,"prepare.run_cpu_time_us":18463,"prepare.run_wall_time_us":189790,"raft.queue_time_us":571301,"raft.run_cpu_time_us":12213,"raft.run_wall_time_us":178322,"replication_time_us":841982,"thread_start_us":777503,"threads_started":4,"wal-append.queue_time_us":216291,"wal-append.run_cpu_time_us":32491,"wal-append.run_wall_time_us":2198822}]]}
I20260709 13:03:57.109270 13865 consensus_queue.cc:237] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:57.111159  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:57.111877  8406 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:03:57.109472  8325 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:03:57.085006 13607 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:57.088826  8297 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:57.062588 13806 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:57.101574 14069 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:57.106621 13772 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:03:57.107249  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:57.116712  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:57.109905  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:57.148819  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:57.149677  8251 raft_consensus.cc:2393] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 14: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:57.154934  8332 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 18: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:03:57.158349  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 69596.2us
I20260709 13:03:57.159721  8397 raft_consensus.cc:2368] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 13. Current term is 14.
I20260709 13:03:57.050891 13800 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:57.161233  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:57.178484  8251 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:57.180493  8257 raft_consensus.cc:1240] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:57.183213  8248 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:57.184597  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:57.189541  8974 leader_election.cc:400] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 13. Current term is 14.
I20260709 13:03:57.102516 14046 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:57.191679  8250 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:57.189891  8974 leader_election.cc:403] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:03:57.153867 14068 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:57.197829  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:57.196489  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 135937us
I20260709 13:03:57.201575  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:57.202606  8295 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:57.202689  8257 raft_consensus.cc:2410] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:03:57.205276  8295 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:57.212764  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:57.216981  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:57.218186  8323 raft_consensus.cc:2368] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 13. Current term is 14.
I20260709 13:03:57.182873 14007 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:57.224691  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:57.191284 14061 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:57.189881 13873 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:57.152915 14053 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 13:03:57.228296  8972 leader_election.cc:400] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 13. Current term is 14.
I20260709 13:03:57.229874  8329 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:03:56.960186  8147 catalog_manager.cc:5697] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 8 to 12, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 12 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:57.159602 13658 consensus_queue.cc:1059] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:03:57.162827 13939 consensus_queue.cc:1048] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:03:57.200495 13607 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:57.236852  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 2434) took 1219 ms. Trace:
I20260709 13:03:57.244743  8402 rpcz_store.cc:276] 0709 13:03:56.017106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.039621 (+ 22515us) service_pool.cc:224] Handling call
0709 13:03:57.236691 (+1197070us) spinlock_profiling.cc:243] Waited 774 ms on lock 0x7b540019bed4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:57.236793 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1779545344}
I20260709 13:03:57.183228 14054 consensus_queue.cc:1059] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:03:57.248265  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:57.249266  8257 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:57.257028  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:57.224195 14046 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 13. Current term is 14.
I20260709 13:03:57.222208 13275 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:57.276338  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:57.276945  8398 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:57.275367  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 98522.6us
W20260709 13:03:57.275621  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 215283us
I20260709 13:03:57.279273  8298 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:57.279975  8249 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:03:57.092270 14060 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.151s
I20260709 13:03:57.284250  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:57.280862  8250 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:03:57.042665 14062 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:56.940487 14050 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.319s
I20260709 13:03:57.284937  8248 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
W20260709 13:03:57.291584  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 100727us
I20260709 13:03:57.288317  8223 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:57.073375 14065 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:57.240468 13824 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 13:03:57.310050  8251 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:03:57.304172  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:57.297212 14083 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:57.243984 13873 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:57.317150  9009 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:03:57.294297 14054 raft_consensus.cc:3055] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:57.302783  8297 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:57.295482  8369 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:57.329466  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:57.292248 13607 raft_consensus.cc:697] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:57.258670 14061 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:57.302767 14007 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:57.336642  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:57.093971 14066 consensus_queue.cc:237] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:57.345989  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 66803.7us
I20260709 13:03:57.347568  8298 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:57.351016 13607 consensus_queue.cc:237] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:57.352898 13800 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:57.333781 14054 raft_consensus.cc:740] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:03:57.354657 14077 log.cc:927] Time spent T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:03:57.356374 13873 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:57.293300 13846 consensus_queue.cc:237] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:57.300082 13275 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:57.354178 14054 consensus_queue.cc:260] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:57.304853  8147 catalog_manager.cc:5697] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 12 to 15, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 15 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:03:57.371439  8398 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:03:57.342355 13980 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:57.360654 13865 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:57.343046 13939 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:57.378427  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:57.378749  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:57.373530  8297 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:57.376694 13800 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:57.381100 13275 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:57.382952 13962 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:03:57.370769 14079 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:03:57.346666 14088 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:57.389394  8257 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:57.368357 14007 raft_consensus.cc:697] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:57.408793  8298 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:57.416815 14084 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:57.418036 14007 consensus_queue.cc:237] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:57.420490  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:57.422124  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:57.417017 14082 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:57.426456 14082 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:57.426743  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:57.395991 13865 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:57.426925 14082 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:57.402042 14054 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:57.407347 14088 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:57.408073 13980 raft_consensus.cc:697] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:03:57.459932  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 48386.9us
I20260709 13:03:57.470203  8298 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:57.435431 13865 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:57.432680 14083 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:57.467268 14079 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 13:03:57.523618  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 53480.6us
I20260709 13:03:57.525861 14061 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:57.526757 14092 log.cc:927] Time spent T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:03:57.468713 14088 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:57.480350 13800 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:57.557731  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:57.574366 14106 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:57.540714 14076 log.cc:927] Time spent T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:03:57.537482 13812 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:57.551607 13768 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.573271 13962 raft_consensus.cc:697] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:57.532891 14083 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:57.596860 14065 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:57.597360 14101 log.cc:927] Time spent T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:03:57.587036 14062 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:57.594364 13772 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 16. Current term is 17.
I20260709 13:03:57.471791  8147 catalog_manager.cc:5697] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:57.617337  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:03:57.619479 14085 log.cc:927] Time spent T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:03:57.578560 13800 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:57.649557 14089 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:57.650203 14089 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:57.652030 13944 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:57.652616 14089 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:57.652845 13944 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:57.522899 13939 raft_consensus.cc:2955] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Committing config change with OpId 18.11: config changed from index 6 to 11, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:57.654131  8147 catalog_manager.cc:5697] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:57.656477 13944 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:57.656100  8147 catalog_manager.cc:5697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 7 to 12, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 12 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:57.605120 13967 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:57.598551 13812 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:57.597229 13768 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:57.586972 13980 consensus_queue.cc:237] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:57.660171 14033 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:57.688807  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 400537us
I20260709 13:03:57.639879 14069 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:57.693437  8223 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:57.658495  8147 catalog_manager.cc:5697] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 11 to 15, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 15 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:57.703425 14072 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 13:03:57.707005 13981 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:57.706920 13768 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:57.708365  8147 catalog_manager.cc:5697] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 8 to 14, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 14 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:57.711124  8147 catalog_manager.cc:5697] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 9 to 11, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 11 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:57.713248 14083 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:57.713965 13800 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:57.714871  8326 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:57.715951 14069 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:57.675799 13967 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:57.726956  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:57.736042  8223 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:57.661263 13962 consensus_queue.cc:237] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:57.770624  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 434020us
I20260709 13:03:57.772102  8369 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:57.774727  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:57.707208 13812 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:57.768366 14072 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:57.768378 14065 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:57.816272  8972 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:57.779945 14084 raft_consensus.cc:697] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:03:57.850047  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 293371us
I20260709 13:03:57.778110 14089 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:57.810568 13680 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.859990  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:57.870803 14117 log.cc:927] Time spent T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:03:57.870944  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:57.871703 14099 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:03:57.873463 14105 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.874068 14105 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:57.876410 14105 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:57.882586 13967 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:57.894532 13680 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:57.896816 13680 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:57.898701 14094 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:57.902179  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:57.903550  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:57.908728  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:57.910678  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:57.713058  8147 catalog_manager.cc:5697] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 11 to 17, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 17 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:57.909407  8255 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:03:57.915477  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:57.920146  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:57.916383  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:57.917060  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 108253us
I20260709 13:03:57.921094  8253 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:03:57.856252 14095 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 13:03:57.930008  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:57.930606  8255 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
I20260709 13:03:57.932512  8255 raft_consensus.cc:1275] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:03:57.859808 14089 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:57.913605 13813 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:03:57.942687  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 55219.8us
W20260709 13:03:57.943692  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 78581.7us
I20260709 13:03:57.950619  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:57.952900  8254 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:03:57.953768 14122 consensus_queue.cc:1048] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:57.959919  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:57.961614  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:57.962272  9002 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:03:57.967087  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:57.916834 14084 consensus_queue.cc:237] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:58.048995  8399 raft_consensus.cc:1275] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:03:57.971904  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:57.972920  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:58.050660  8398 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:03:57.979480  8220 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:03:57.999475 14082 consensus_meta.cc:287] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.522s	user 0.009s	sys 0.000s
I20260709 13:03:58.001789  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:57.955536 14088 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:57.967696 14105 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:57.961787 13813 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:57.958942 14115 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:58.011513  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:58.061555  8222 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:57.963662 13865 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:03:58.016863 14054 consensus_meta.cc:287] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.548s	user 0.010s	sys 0.000s
I20260709 13:03:58.020112 14089 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:58.020247  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:58.031651 13774 raft_consensus.cc:3055] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:58.038355  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:58.054627  8256 raft_consensus.cc:1275] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:03:58.054430  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:58.056319  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:58.056607  8254 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:58.058606  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:58.059115  8255 raft_consensus.cc:2368] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 13. Current term is 14.
W20260709 13:03:58.059628  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 80234.7us
I20260709 13:03:58.073982  8402 raft_consensus.cc:1275] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:03:58.074381  8402 pending_rounds.cc:85] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 8
I20260709 13:03:58.074602  8402 pending_rounds.cc:107] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:03:58.074851  8402 raft_consensus.cc:2889] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:58.062745  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:58.055255  8295 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:57.916810  8147 catalog_manager.cc:5697] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 10 to 15. New cstate: current_term: 15 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:58.076184 14069 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:58.076566 14069 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:58.076946 14069 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:58.076983  8147 catalog_manager.cc:5697] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 10 to 15. New cstate: current_term: 15 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:58.078626  8147 catalog_manager.cc:5697] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 9 to 16. New cstate: current_term: 16 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:58.080374  8147 catalog_manager.cc:5697] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 7 to 17. New cstate: current_term: 17 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:58.069845  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:58.082304  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:58.083276  8147 catalog_manager.cc:5697] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:03:58.085062  8147 catalog_manager.cc:5697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:58.087025  8147 catalog_manager.cc:5697] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 11 to 16, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 16 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:58.087069 14069 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:58.088817  8147 catalog_manager.cc:5697] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 10 to 12, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 12 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:58.072888  8399 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
I20260709 13:03:58.090147 13774 raft_consensus.cc:740] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:58.090276  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:58.090862  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:58.093184 14115 raft_consensus.cc:697] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:58.095209 13813 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:58.095669 13981 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:58.154969  8328 raft_consensus.cc:2393] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 15: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:58.128592  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:58.138963  8331 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:58.140838  8398 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:03:58.155946  9000 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18.
W20260709 13:03:58.137907  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 327454us
I20260709 13:03:58.167711  8254 raft_consensus.cc:1275] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:03:58.145004  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 195609us
I20260709 13:03:58.068583 14121 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:58.123409  8325 raft_consensus.cc:2393] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 15: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:58.063434 14105 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:58.181136  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:58.182849  8254 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:03:58.150126 13812 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.065550 14082 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:58.184655  8255 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 11. Current term is 12. Ops: [10.7-11.8]
I20260709 13:03:58.199368  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:58.195323 13812 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:58.204931 14088 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:58.142024 14122 consensus_queue.cc:1048] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:03:58.205814  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:58.206419  8328 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
I20260709 13:03:58.206559 13812 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:58.208324  8327 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:03:58.209707 14115 consensus_queue.cc:237] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:58.211938 14109 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:58.212469 14109 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:58.214396 13961 raft_consensus.cc:3055] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:58.214617 14109 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:58.214835 13961 raft_consensus.cc:740] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:58.215754 13961 consensus_queue.cc:260] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:58.216398 14105 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:58.216850 13961 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:58.220816  9009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56954 (request call id 974) took 1261 ms. Trace:
I20260709 13:03:58.221302  9009 rpcz_store.cc:276] 0709 13:03:56.959861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.969658 (+  9797us) service_pool.cc:224] Handling call
0709 13:03:58.220585 (+1250927us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b54004f2ed4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:58.220721 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":672226688}
I20260709 13:03:58.187220 14121 raft_consensus.cc:697] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:58.225589  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2597) took 1117 ms. Trace:
I20260709 13:03:58.226373  8326 rpcz_store.cc:276] 0709 13:03:57.108285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.110088 (+  1803us) service_pool.cc:224] Handling call
0709 13:03:57.110373 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:57.715299 (+604926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.225181 (+509882us) spinlock_profiling.cc:243] Waited 604 ms on lock 0x7b54003ca854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:03:58.225505 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1389761408}
I20260709 13:03:58.228379 14069 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:58.230414  8147 catalog_manager.cc:5697] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 11 to 14, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 14 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:58.232571  8147 catalog_manager.cc:5697] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:03:58.234531 13865 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:58.155097 13981 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:58.236447 13981 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:58.237705 13774 consensus_queue.cc:260] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:58.238756 13774 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:58.181288 13765 consensus_queue.cc:1048] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:03:58.248322  8972 leader_election.cc:400] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 13. Current term is 14.
I20260709 13:03:58.249765  8223 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:58.253842  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:58.254555  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:03:58.310873  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 47657.1us
I20260709 13:03:58.181516 14089 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:03:58.315513  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.317819  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:58.326597  8253 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:58.328477  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:58.264994  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:58.333248  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 69625.4us
W20260709 13:03:58.335273  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 70423us
I20260709 13:03:58.336253  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:58.266433 13770 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:58.351858  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:03:58.359416  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 437756us
I20260709 13:03:58.363036  8400 raft_consensus.cc:1275] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:03:58.363286  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:58.334426 13806 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.311617 14046 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:03:58.311863 14069 consensus_queue.cc:1048] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 13:03:58.387224  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:58.329576 13813 consensus_queue.cc:1048] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:03:58.342011 13770 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:03:58.443106  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 62915us
I20260709 13:03:58.446694  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:58.448578  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:58.449880  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 70408.2us
I20260709 13:03:58.371749 13891 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:03:58.492117  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 44985.8us
I20260709 13:03:58.373610 13800 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:58.463527  8326 raft_consensus.cc:2410] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 17 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:03:58.386927 14066 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:58.225695 14121 consensus_queue.cc:237] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:58.350837 14089 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:58.518397  8297 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:58.516871  8253 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:03:58.449252 14119 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.479944 14065 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.450556 13770 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:58.542114 14142 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:58.480166 13875 consensus_queue.cc:1059] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.093s
W20260709 13:03:58.648784  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 95755.7us
I20260709 13:03:58.658051  8297 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:58.386797 13806 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:58.655498  8330 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:03:58.661988  8325 raft_consensus.cc:1240] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:03:58.648680  8298 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:03:58.663107  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 326441us
I20260709 13:03:58.663802  8298 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:58.665292  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:58.665652  8297 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:58.666288  8327 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:03:58.676169  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:58.547580 14084 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:58.664000  8972 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:58.556128 14109 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:58.646937 13765 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:58.692377  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:58.541455 13800 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:58.706104  8325 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:58.707150  8323 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:58.656847 14122 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.551175 14119 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:58.658190 14133 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:58.537547 13891 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:58.553579 14065 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:58.731618  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:58.688308 14087 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:58.688629 13680 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:58.691011 13757 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:58.682581 14088 consensus_queue.cc:1048] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:03:58.745417  8330 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:03:58.747813  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.714488 14106 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:58.520845  8147 catalog_manager.cc:5697] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 6 to 12, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 12 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:03:58.699961 13765 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:58.731568 14109 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:58.645779 14142 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:58.735810 14127 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.714726 14122 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:58.741066 14087 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:03:58.761875  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.712811 14084 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:58.747684 13757 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:58.747804 14082 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:58.750182 14106 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:03:58.756407 14138 log.cc:927] Time spent T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 13:03:58.772152  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 108432us
I20260709 13:03:58.768203 14087 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:58.790457 13806 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:58.795885 13774 consensus_meta.cc:287] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.557s	user 0.007s	sys 0.002s
I20260709 13:03:58.795951 13824 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:58.796622 13824 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:58.797005 14089 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:58.797344 14129 log.cc:927] Time spent T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.257s	user 0.000s	sys 0.001s
I20260709 13:03:58.797827  8147 catalog_manager.cc:5697] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 7 to 13, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 13 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:58.798900 13824 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:58.799533  8147 catalog_manager.cc:5697] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
W20260709 13:03:58.801909 14137 log.cc:927] Time spent T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.221s	user 0.001s	sys 0.000s
W20260709 13:03:58.803465 14131 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
W20260709 13:03:58.805042 14139 log.cc:927] Time spent T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.250s	user 0.000s	sys 0.000s
I20260709 13:03:58.774531 14144 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:58.782794 14084 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:58.766443 14127 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:58.805714 14144 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:58.775532 14066 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:03:58.806155 14144 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:58.806344 14066 consensus_queue.cc:1243] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer ff7a6fe533e94c1bae555b4cae427728 log is divergent from this leader: its last log entry 19.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:03:58.807444 14127 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:58.775254 14106 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:58.809213 14142 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:58.809988 14082 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:58.810470 14082 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:58.810772 13800 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:58.810962  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.812515 14082 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:03:58.787029 13981 consensus_meta.cc:287] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.550s	user 0.004s	sys 0.007s
I20260709 13:03:58.813778 13981 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:58.814247 14065 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:58.814780  8250 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:03:58.816502 13981 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:58.817133 13981 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:03:58.817900 13961 consensus_meta.cc:287] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.601s	user 0.010s	sys 0.001s
I20260709 13:03:58.813182 13757 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:58.817677 13981 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:58.835417  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 92788.8us
I20260709 13:03:58.868271 14106 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:58.833703 14146 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:58.893956  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:03:58.905951  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 240391us
I20260709 13:03:58.906831  8369 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:58.907716  8297 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:58.882550 14066 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:58.882161 14145 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:03:58.908397  8220 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.002s	sys 0.019s
I20260709 13:03:58.888031 14127 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:58.893218 13658 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.919687  8297 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:03:58.931442  8220 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.003s	sys 0.020s
W20260709 13:03:58.935124  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 561882us
I20260709 13:03:58.936651  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:58.914681 14109 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:58.939033  8220 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:58.944283  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:58.889721  8147 catalog_manager.cc:5697] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:58.948511  8400 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:03:58.951234  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:58.952940  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:58.953708  8253 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
I20260709 13:03:58.930584 13875 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:58.956660  8220 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:58.960419  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:58.935772 14082 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:58.970274  8220 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:58.927450 13774 consensus_queue.cc:1048] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:03:58.938385 13658 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:58.940955 14109 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.943116 14065 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:58.920305 14066 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:58.947240 14053 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:58.941455 14090 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:58.989904  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 83169.9us
I20260709 13:03:58.904126 14146 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:58.975996  8220 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:58.919342 14154 consensus_queue.cc:1059] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:03:58.976783 13768 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:58.968531 13875 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:59.004427 13875 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:58.981943 14109 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:58.955498 14088 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:58.938095 14127 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:58.977080 13840 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:59.009505 13891 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:59.011358 14127 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:58.967613 14106 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:59.012442 14065 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:59.014838 14065 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:59.016436 13840 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:59.020017 14134 log.cc:927] Time spent T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.651s	user 0.003s	sys 0.004s
I20260709 13:03:59.020866 14149 consensus_queue.cc:1059] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 13:03:58.991252 13765 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:58.997816 14110 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.022326 14110 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:59.023195 14149 raft_consensus.cc:3055] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:59.023566 14149 raft_consensus.cc:740] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:59.024435 14149 consensus_queue.cc:260] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:58.998956 13658 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:58.999564  8404 raft_consensus.cc:1275] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:03:58.923321  8401 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:46846
I20260709 13:03:59.026348 13981 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:59.009061 14088 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:59.028326  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:59.029013 14088 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:59.029695 14135 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:03:58.998414  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:59.025146 14110 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:59.026374  8401 raft_consensus.cc:606] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:58.981720 14116 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:03:59.023999  8147 catalog_manager.cc:5697] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 9 to 15. New cstate: current_term: 15 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:59.038264 13774 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.043838  9008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 1118) took 1228 ms. Trace:
I20260709 13:03:58.924078 14145 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:58.963357 14160 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:03:59.056387  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 61919.3us
I20260709 13:03:59.076154  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:59.064252  8220 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:59.020246 14066 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:59.080857  8332 raft_consensus.cc:1275] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:03:59.081506 14152 consensus_queue.cc:1048] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:59.084061  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:59.084499 13770 consensus_meta.cc:287] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.502s	user 0.010s	sys 0.000s
I20260709 13:03:59.073487 13774 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:59.088694 14156 consensus_queue.cc:1059] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:03:59.219844  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:59.220250 14156 raft_consensus.cc:3055] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:59.220798  8255 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:03:59.090586  8369 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:59.222045  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 131542us
I20260709 13:03:59.222265  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:03:59.222823  8369 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:59.223455 13770 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:59.223888  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:59.224726  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:03:58.996464 14053 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:59.094709  8147 catalog_manager.cc:5697] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 8 to 14, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 14 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:03:59.099001 14087 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:59.101229 14111 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:59.055899 14122 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:59.106592  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:59.113361 14146 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:59.067830 14116 consensus_queue.cc:1243] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:03:59.116936  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:59.120827  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:59.239964  8327 raft_consensus.cc:2393] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 15: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:59.124032 14119 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:59.125382  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:03:59.127763  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:59.129313  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:59.131832  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:59.065795  9008 rpcz_store.cc:276] 0709 13:03:57.815873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.816145 (+   272us) service_pool.cc:224] Handling call
0709 13:03:58.220749 (+404604us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:58.221274 (+   525us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:58.221316 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:58.221905 (+   589us) log.cc:844] Serialized 26 byte log entry
0709 13:03:58.367793 (+145888us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:58.386560 (+ 18767us) log.cc:844] Serialized 14 byte log entry
0709 13:03:58.653523 (+266963us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.653786 (+   263us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:58.820882 (+167096us) raft_consensus.cc:1685] finished
0709 13:03:58.820912 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.043562 (+222650us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b54004f2ed0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:03:59.043730 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2819,"prepare.queue_time_us":20138,"prepare.run_cpu_time_us":2540,"prepare.run_wall_time_us":37978,"raft.queue_time_us":329772,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":12892,"spinlock_wait_cycles":842462976,"thread_start_us":311599,"threads_started":2,"wal-append.queue_time_us":320871}
W20260709 13:03:59.136631 14140 log.cc:927] Time spent T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.581s	user 0.000s	sys 0.001s
I20260709 13:03:59.166836 14128 raft_consensus.cc:3055] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:59.198014  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:59.204018  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:59.108948 14145 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:59.220918 14156 raft_consensus.cc:740] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:59.089617 13812 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:59.268896  8220 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:59.224006  8369 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:03:59.270459  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 46515.3us
I20260709 13:03:59.271135  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:59.272375  8369 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:03:59.091444 14125 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:59.230525 13961 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:59.077521 13875 raft_consensus.cc:3055] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:59.277518 14059 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:59.234077  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.229919 13962 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:59.242340  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:59.248636  8331 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 17: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:59.249823  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:03:59.252269  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 126966us
I20260709 13:03:59.245570 14111 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:59.256145 14128 raft_consensus.cc:740] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:59.267879 14073 consensus_queue.cc:1048] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:03:59.269912 14146 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:59.318045 14156 consensus_queue.cc:260] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:59.319443 14156 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:59.307672 14122 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.320158 14122 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:59.293941 13665 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:59.322850 14111 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:59.323722  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2359) took 1062 ms. Trace:
I20260709 13:03:59.324110  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:59.325006  8329 raft_consensus.cc:2393] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:03:59.324159  8252 rpcz_store.cc:276] 0709 13:03:58.261002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.261152 (+   150us) service_pool.cc:224] Handling call
0709 13:03:58.261559 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:58.261886 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:58.261919 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:58.310986 (+ 49067us) log.cc:844] Serialized 26 byte log entry
0709 13:03:58.376083 (+ 65097us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:58.376129 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.376201 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:59.135854 (+759653us) spinlock_profiling.cc:243] Waited 66.8 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:03:59.323426 (+187572us) raft_consensus.cc:1685] finished
0709 13:03:59.323448 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.323662 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17267,"raft.queue_time_us":482,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":47466,"spinlock_wait_cycles":153596160,"thread_start_us":64718,"threads_started":1,"wal-append.queue_time_us":173729}
I20260709 13:03:59.281373 13875 raft_consensus.cc:740] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:59.273067  8256 raft_consensus.cc:2393] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 18: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:03:59.236049 14149 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:59.274595  8369 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:59.345656  8147 catalog_manager.cc:5697] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:59.276268  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:59.350225  8398 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:03:59.242908 13607 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:59.350461  8147 catalog_manager.cc:5697] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 9 to 15, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 15 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:03:59.351595 13607 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:59.352213  8147 catalog_manager.cc:5697] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 7 to 14, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 14 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:59.336231 14144 consensus_meta.cc:287] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.529s	user 0.011s	sys 0.000s
I20260709 13:03:59.353906 14144 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:59.356129 14144 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:59.354379 13607 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:59.356892  9005 raft_consensus.cc:1240] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:59.357713  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:59.287475  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:59.253886 14088 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.359544 14088 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:59.360898  8329 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 16.
I20260709 13:03:59.296772  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:59.297467  8403 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:03:59.302490 13770 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:59.267342  8257 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19. Ops: []
W20260709 13:03:59.341663 14115 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.760s	user 0.003s	sys 0.003s
I20260709 13:03:59.345450  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:59.224797  8250 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:03:59.226680  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:59.367206  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 68143us
I20260709 13:03:59.367417  8324 raft_consensus.cc:2410] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 15 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:03:59.265312  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:03:59.365103  8974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.043s	sys 0.063s
W20260709 13:03:59.368101  8974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.043s	sys 0.064s
I20260709 13:03:59.368643  8974 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:03:59.370280  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.315330 14146 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:03:59.373118  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 98628.4us
I20260709 13:03:59.373255  8295 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:03:59.373301  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:59.380040  8369 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:03:59.386184  8326 raft_consensus.cc:1275] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:03:59.388515  8325 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:59.379567  8328 raft_consensus.cc:2955] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Committing config change with OpId 18.11: config changed from index 6 to 11, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:59.391291  8295 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:03:59.376454 14115 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.004s	sys 0.003s
I20260709 13:03:59.397116  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:59.396556  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:59.399945  8331 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:03:59.401571  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:59.401814  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:59.416937  8974 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:59.378618 14146 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:59.422648  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:03:59.383906 14059 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:59.426532  8222 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:59.393117 14116 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:59.432217  8331 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:59.401288 14007 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:59.415864 13961 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:59.443791  8330 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:59.434249  8222 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:59.393728 14144 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:59.462831  8222 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:59.466565  8222 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:59.437165 13770 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:59.471959  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:59.437763 13953 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:59.443310 14007 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.479597  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:59.480010 14132 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:59.480077 14007 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:59.480696 14132 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:59.481140 14132 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:59.446861 13961 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.493285  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:59.454473 14059 raft_consensus.cc:697] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:59.495160  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:59.467669 14033 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:59.497908  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:59.416369 13875 consensus_queue.cc:260] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:59.474551 14111 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:59.469488 14116 raft_consensus.cc:697] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:03:59.512717  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:59.485085 14084 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:59.492545 14160 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:59.496270 13961 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:59.541712 14175 log.cc:927] Time spent T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:03:59.531096 14111 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:59.505558 14053 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:59.544551 14059 consensus_queue.cc:237] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:59.545732 14073 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:59.548192 14084 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:59.533699 14087 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:59.500982 14088 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:59.572000  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 126886us
I20260709 13:03:59.578711  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:59.580725  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:59.581501  8330 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:59.565603 13875 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:59.580765  8220 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:59.476786  8147 catalog_manager.cc:5697] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 11 to 14, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 14 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:03:59.567302 14111 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:59.564616 14160 raft_consensus.cc:697] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:59.595769  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:03:59.598819  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:03:59.599594  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:59.588469 14088 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.602209 14088 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:59.393972 14128 consensus_queue.cc:260] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:03:59.593290 14109 consensus_meta.cc:287] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.588s	user 0.009s	sys 0.002s
I20260709 13:03:59.593708 13768 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:59.623690  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:59.630242  8298 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:59.597360 14119 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:59.621739 14171 log.cc:927] Time spent T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:03:59.612476 14165 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:59.593725 14105 consensus_queue.cc:1059] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:03:59.645763  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:59.646315 14105 raft_consensus.cc:3055] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:59.646708 14105 raft_consensus.cc:740] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:03:59.647475 14105 consensus_queue.cc:260] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:59.629894 14116 consensus_queue.cc:237] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:59.611443 14066 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:03:59.652647 14066 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:59.599731 13812 consensus_queue.cc:1059] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:03:59.653890 13812 raft_consensus.cc:3055] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:59.654258 13812 raft_consensus.cc:740] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:03:59.619474 13658 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:03:59.655143 13812 consensus_queue.cc:260] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:59.629824  9000 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:03:59.639568  8298 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:59.588470 14073 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:59.658119 13812 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:59.596370 14072 consensus_queue.cc:1048] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:03:59.662511  8298 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:59.638826  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:59.663022 14152 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:59.658192 14105 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:59.661296 14169 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:59.675036 14169 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:59.677953 14073 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:03:59.682310  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 43555us
I20260709 13:03:59.657392 13865 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:59.628315 14109 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:59.694322 14084 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:59.693832 14160 consensus_queue.cc:237] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:59.698436  8298 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:59.663523 14128 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:59.712631  8298 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:59.705058 14132 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:59.681299 14087 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:59.657698 14119 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:59.717173  9005 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:59.718601 14145 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:03:59.719447  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:59.688186 13757 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:59.726464  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:59.704183 14120 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.728981 14109 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:59.729620  8327 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 20. Current term is 21. Ops: [18.12-20.13]
I20260709 13:03:59.730038 14127 consensus_queue.cc:1059] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:59.707828 14110 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:03:59.736871  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:59.677276 14169 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:59.739696 14111 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:59.745893 14146 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:59.749714 13680 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:59.750234  8298 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:59.751403 14007 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:03:59.673322 14033 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:59.757021  8403 raft_consensus.cc:1275] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:03:59.760353  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:03:59.769038  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.796237  8401 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:46846
I20260709 13:03:59.754662 13865 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:59.821769  8403 pending_rounds.cc:85] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 9
I20260709 13:03:59.824446  8401 raft_consensus.cc:606] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:03:59.824903  8401 raft_consensus.cc:614] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:03:59.829180  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 79020.6us
I20260709 13:03:59.830212  8298 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:03:59.794740 13757 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:03:59.832175  8298 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:03:59.761235 14145 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:59.797464 14120 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:00.084764  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:59.997972  8295 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:03:59.940902  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:59.921927  8401 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:46846
W20260709 13:04:00.091406  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 259283us
I20260709 13:04:00.091838  8401 raft_consensus.cc:606] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:00.092303  8401 raft_consensus.cc:614] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:59.872987  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2611) took 1189 ms. Trace:
I20260709 13:04:00.012322  8326 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:04:00.008215  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:59.940398  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:04:00.090674  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 92804.4us
I20260709 13:04:00.099057  8403 pending_rounds.cc:107] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:04:00.099759  8403 raft_consensus.cc:2889] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:59.857813  9002 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:59.923425  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:03:59.991855  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:00.043216 14122 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:00.135932  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:00.094442  8328 rpcz_store.cc:276] 0709 13:03:58.683294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.683431 (+   137us) service_pool.cc:224] Handling call
0709 13:03:58.684085 (+   654us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:58.757335 (+ 73250us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:58.885170 (+127835us) log.cc:844] Serialized 14 byte log entry
0709 13:03:59.335347 (+450177us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:59.379211 (+ 43864us) log.cc:844] Serialized 513 byte log entry
0709 13:03:59.379437 (+   226us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:59.452670 (+ 73233us) log.cc:844] Serialized 14 byte log entry
0709 13:03:59.465295 (+ 12625us) log.cc:844] Serialized 14 byte log entry
0709 13:03:59.577986 (+112691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.578078 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:59.872683 (+294605us) raft_consensus.cc:1685] finished
0709 13:03:59.872705 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.872907 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":145646,"prepare.queue_time_us":306005,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":83,"raft.queue_time_us":145399,"raft.run_cpu_time_us":503,"raft.run_wall_time_us":21796,"thread_start_us":552752,"threads_started":3,"wal-append.queue_time_us":483709}
I20260709 13:04:00.147934  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:00.151470  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:00.153970  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:00.154853  8256 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:00.048573 14188 log.cc:927] Time spent T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.223s	user 0.003s	sys 0.000s
I20260709 13:04:00.177124  8251 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:03:59.856587 14109 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:59.933807 14007 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:59.937534 14111 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:59.803593 14110 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:03:59.992230  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:00.018952 14105 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.160s
I20260709 13:03:59.926476 14088 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:59.856361 14087 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:00.239588  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:59.942411  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:03:59.922132 14132 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:03:59.931485 14169 raft_consensus.cc:3055] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:00.022969  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:00.181128  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:03:59.872599  8331 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:00.033691  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:00.184720  8402 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:03:59.935393 14146 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:03:59.939504 14119 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:00.124738 13757 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:00.133570 13680 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:00.189255  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:00.191521 13658 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:00.191812 14120 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:59.802524 13875 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.375s
I20260709 13:04:00.193378 14145 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:00.195945  8256 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:03:59.880844 13774 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:03:59.987473 13768 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:00.016484 14090 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.188s
I20260709 13:04:00.212257 14072 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.216797 14149 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:00.218237  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:00.180569 14144 consensus_queue.cc:1059] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:04:00.296682 14144 raft_consensus.cc:3055] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:00.297194 14144 raft_consensus.cc:740] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:00.298319 14144 consensus_queue.cc:260] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:00.299119  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:00.233923  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:00.244302  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:03:59.933329  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:00.249190  8249 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:04:00.257418  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:00.247633  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:00.238746 14152 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:00.242882 13961 consensus_meta.cc:287] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.676s	user 0.012s	sys 0.000s
W20260709 13:04:00.293762  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 194754us
I20260709 13:04:00.293888  8256 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:00.299826  8400 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:04:00.300314  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:00.306325  8325 raft_consensus.cc:1275] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:04:00.322034  8325 pending_rounds.cc:85] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 7
I20260709 13:04:00.322363  8325 pending_rounds.cc:107] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:04:00.322700  8325 raft_consensus.cc:2889] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:00.310410  8327 raft_consensus.cc:1240] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:00.314854 14152 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:00.328642 14152 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:00.315121 13961 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:00.332003 13961 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:00.299543 14144 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:00.334136 13961 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:00.341060 14145 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:00.341765 14145 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:00.342468  8369 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:00.344161 14145 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:00.344300 14132 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:00.349646 14132 consensus_queue.cc:1048] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:00.315568  8324 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 17: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:00.352128 14132 consensus_queue.cc:1059] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:00.316169  8295 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:00.355705  8400 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:46846
I20260709 13:04:00.356464  8400 raft_consensus.cc:606] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:00.356997  8400 raft_consensus.cc:614] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:00.360647  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:00.361389  8327 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:00.368585  8327 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:04:00.369897  8400 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:46846
I20260709 13:04:00.377591  8257 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:04:00.248539 14169 raft_consensus.cc:740] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:00.378942  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 864) took 1036 ms. Trace:
I20260709 13:04:00.425920  8400 raft_consensus.cc:606] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:00.259325 13680 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:00.388446  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 72392.2us
I20260709 13:04:00.426901  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:00.274420 14189 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:00.289608 14146 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:00.394562  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:00.291035 14133 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:00.401831  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:00.292605 14119 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:00.376812 14116 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:00.409072  8223 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:00.411283  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:00.413278  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 191651us
I20260709 13:04:00.349676  8405 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:04:00.275274 13658 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:00.377465 13768 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:00.382445 14149 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:00.417591  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:00.392776 14151 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:00.392354 13961 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.429280  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:00.426319  8329 rpcz_store.cc:276] 0709 13:03:59.342183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.342365 (+   182us) service_pool.cc:224] Handling call
0709 13:04:00.378777 (+1036412us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b540014cb54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:00.378889 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5843328}
I20260709 13:04:00.431526  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:00.546684  8257 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:00.437002  8295 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:00.490502  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:00.494062  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:00.544667  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 135657us
I20260709 13:04:00.548840  8223 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:00.549959  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 215235us
I20260709 13:04:00.551632  8252 raft_consensus.cc:2410] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:04:00.382615 14072 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:00.556311  8369 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:00.556622 13875 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:04:00.552241  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 115346us
I20260709 13:04:00.558652  8295 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:00.429913 13680 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:00.555074  8256 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:04:00.560770  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:00.555496 14087 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:00.561899  8369 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:00.575464  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:00.603690  8253 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:04:00.599202  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:00.607419  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:00.609457  8249 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:00.610915  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:00.424132 14127 consensus_queue.cc:1059] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.136s
I20260709 13:04:00.608587  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:00.619175  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:00.619520  8405 pending_rounds.cc:85] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 8
I20260709 13:04:00.619777  8295 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:00.620538  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:00.621047  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:00.621714  8397 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:00.621794  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:00.622695  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:00.622992  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:04:00.624543  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 62804.7us
I20260709 13:04:00.625631 14196 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:00.627213  8369 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:00.628609  8369 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:00.629480 14132 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:00.630272 14132 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:00.631851 14196 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:00.633761  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:00.633163  8369 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:00.635164  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:00.639019  8398 raft_consensus.cc:3055] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:00.639575  8398 raft_consensus.cc:740] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:00.619824  8405 pending_rounds.cc:107] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:04:00.640420  8405 raft_consensus.cc:2889] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:00.645139  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:00.645414  8257 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:04:00.574759 14165 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:00.573261 14087 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:04:00.656226  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 254771us
I20260709 13:04:00.574594 14156 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:00.489568 14199 consensus_queue.cc:1059] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.149s
I20260709 13:04:00.580807 13770 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:00.579679 14152 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:00.654868  8295 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:00.640375  8398 consensus_queue.cc:260] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:00.490907 14146 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:00.665519  8398 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:00.667276  8295 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:00.543977 14119 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:00.693773  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:00.642194 14196 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:00.699900  8248 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:04:00.647142 13774 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.492643 14133 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:00.643118 14203 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:00.642657 13865 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:00.489833 14189 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:00.668440 14111 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:00.663113 14087 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:00.719591  8974 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:00.712033 14128 consensus_meta.cc:287] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.954s	user 0.012s	sys 0.000s
I20260709 13:04:00.715937  8329 raft_consensus.cc:1813] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Handling vote request from an unknown peer 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:00.725485  8329 raft_consensus.cc:2410] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:04:00.704066 14110 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:00.574424 13961 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:00.658648 14122 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:00.703631 13770 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.701865 14072 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:00.728220  9004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 1173) took 1011 ms. Trace:
I20260709 13:04:00.698791 14007 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:00.734503  9002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 982) took 1290 ms. Trace:
I20260709 13:04:00.756101  8398 raft_consensus.cc:1275] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:04:00.694769 14156 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:00.715183 14152 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:04:00.815372  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 148150us
I20260709 13:04:00.816488  9004 rpcz_store.cc:276] 0709 13:03:59.716691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.718049 (+  1358us) service_pool.cc:224] Handling call
0709 13:03:59.718536 (+   487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:00.176483 (+457947us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:00.176527 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:00.176590 (+    63us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:00.176636 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.727962 (+551326us) spinlock_profiling.cc:243] Waited 458 ms on lock 0x7b54004ec854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:00.728015 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:00.728157 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1052611328}
I20260709 13:04:00.822233  8398 pending_rounds.cc:85] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 9
I20260709 13:04:00.838238  8398 pending_rounds.cc:107] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:04:00.838546  8398 raft_consensus.cc:2889] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:00.821870  9002 rpcz_store.cc:276] 0709 13:03:59.444478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.444590 (+   112us) service_pool.cc:224] Handling call
0709 13:03:59.445336 (+   746us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:00.101120 (+655784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.734092 (+632972us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b54005cf8d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:00.734412 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":947950976}
I20260709 13:04:00.841312 14215 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:00.843360 14215 consensus_queue.cc:1059] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:00.844175 14215 consensus_queue.cc:1243] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer ff7a6fe533e94c1bae555b4cae427728 log is divergent from this leader: its last log entry 19.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:04:00.845594 14007 raft_consensus.cc:697] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:00.846717 14007 consensus_queue.cc:237] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:00.822036 14128 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:00.850244 14128 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
W20260709 13:04:00.851920 14204 log.cc:927] Time spent T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 13:04:00.852396 14128 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:00.837492 14152 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:00.853713 14152 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:00.821816 13770 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:00.853742 14195 log.cc:927] Time spent T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.275s	user 0.001s	sys 0.000s
I20260709 13:04:00.858218 14149 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:00.858229  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2875) took 1274 ms. Trace:
I20260709 13:04:00.860195 14119 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:00.859133  8330 rpcz_store.cc:276] 0709 13:03:59.583421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.987344 (+403923us) service_pool.cc:224] Handling call
0709 13:03:59.987745 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:00.004143 (+ 16398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:00.004177 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:00.019867 (+ 15690us) log.cc:844] Serialized 26 byte log entry
0709 13:04:00.333119 (+313252us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:00.333182 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.333453 (+   271us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:00.857962 (+524509us) raft_consensus.cc:1685] finished
0709 13:04:00.857992 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:00.858161 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10111,"raft.queue_time_us":14147,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":15619,"thread_start_us":285041,"threads_started":1,"wal-append.queue_time_us":395087}
I20260709 13:04:00.863006  8404 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:00.712186 13774 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:00.787325 14203 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:00.718542 13865 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:00.873620  8252 raft_consensus.cc:1275] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:04:00.724123 14111 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:00.890156  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:00.863436 14194 log.cc:927] Time spent T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:04:00.815429 14110 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:00.838893 14122 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:00.902209  8369 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:04:00.897024  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 69804.7us
I20260709 13:04:00.890239 14206 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:04:00.921154  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 184707us
I20260709 13:04:00.922057  8297 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:00.922349  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 202840us
I20260709 13:04:00.923480  8974 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:00.920224 14206 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:00.813580 13757 consensus_meta.cc:287] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.523s	user 0.011s	sys 0.000s
I20260709 13:04:00.917092  8972 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:00.886448 14145 raft_consensus.cc:3055] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:00.930488 14122 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:00.959533 14210 log.cc:927] Time spent T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:04:00.960187 13774 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:00.961790 14127 raft_consensus.cc:993] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f: : Instructing follower 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:04:00.962663 14133 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:00.965358 14189 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:00.967621 14207 log.cc:927] Time spent T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:04:00.969113 14144 consensus_meta.cc:287] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.635s	user 0.008s	sys 0.000s
I20260709 13:04:00.970798  8404 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 21.
I20260709 13:04:00.975761  8223 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:00.899780 14128 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:00.965097 14148 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:00.906898 14119 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:00.931059 14081 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:04:00.992278  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 70298.2us
I20260709 13:04:00.912799 14110 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:01.014371  9009 raft_consensus.cc:1240] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:01.001724  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:00.981873 14145 raft_consensus.cc:740] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:00.930109 14181 consensus_queue.cc:1048] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:04:00.986382 14156 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:01.000293  8297 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:01.015035  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:00.970438 14199 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:04:00.991226 14127 raft_consensus.cc:1081] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Signalling peer 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:04:01.021943  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 2876) took 1340 ms. Trace:
I20260709 13:04:00.889918 14203 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.028940  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:00.981029 13757 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.034093  8297 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:00.996577 13774 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:00.999599 14202 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:01.037956  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:01.030321  8323 rpcz_store.cc:276] 0709 13:03:59.681769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.001051 (+319282us) service_pool.cc:224] Handling call
0709 13:04:00.008681 (+  7630us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:00.009036 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:00.009065 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:00.009529 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 13:04:00.339151 (+329622us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:00.339223 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.339408 (+   185us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:01.021542 (+682134us) raft_consensus.cc:1685] finished
0709 13:04:01.021576 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:01.021859 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8362,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":3896,"thread_start_us":293655,"threads_started":1,"wal-append.queue_time_us":550635}
I20260709 13:04:01.038931 14007 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:01.044042 14007 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.044445 14007 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:01.048195 14209 log.cc:927] Time spent T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
I20260709 13:04:01.049024 14220 consensus_queue.cc:1048] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:04:01.049973 14205 log.cc:927] Time spent T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:04:01.051997 13961 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:01.054013 14110 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:01.039642  8222 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:01.040995  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.057984 13774 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:01.058885  8222 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:01.059906 14212 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:01.018785 14081 raft_consensus.cc:697] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:01.060400 14212 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.061022 14212 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:01.061703 14110 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:01.062374 14110 raft_consensus.cc:697] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:01.063548 14110 consensus_queue.cc:237] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.015336 14218 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:04:01.067492  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:01.068637  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:01.068820 13865 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:01.071208 14122 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:01.072012 14168 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.072535 14168 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.074586 14168 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:01.074645 14200 consensus_queue.cc:1048] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:01.060504 13774 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:01.078962  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:01.083417  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:01.084378  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 167354us
I20260709 13:04:01.086354  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:01.089594  8332 raft_consensus.cc:2393] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 18: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:01.023061 14133 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:01.028299 14144 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:01.018476 14189 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:01.090565  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:01.094111  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.094197  8332 raft_consensus.cc:2393] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 12: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:01.087489  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:01.092581  8399 raft_consensus.cc:1240] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:01.096817  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:01.032634 14146 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:01.031625 14119 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.096607 14133 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:01.101647 14222 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:01.093289 14144 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.102650 14144 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:01.102888 14133 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:01.103787  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 61908us
I20260709 13:04:01.105937  8332 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:01.109617  8332 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 17. Current term is 18. Ops: [15.11-17.12]
I20260709 13:04:01.090131 14218 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:01.103649 14189 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:01.117308  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 192073us
I20260709 13:04:01.117750 14218 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:01.118191  8974 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:01.114946  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 923) took 1124 ms. Trace:
I20260709 13:04:01.119163  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2372) took 1111 ms. Trace:
I20260709 13:04:01.119046  8255 rpcz_store.cc:276] 0709 13:03:59.990591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.991937 (+  1346us) service_pool.cc:224] Handling call
0709 13:04:00.016367 (+ 24430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:00.276472 (+260105us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:00.276521 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:00.276591 (+    70us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:00.276650 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.114706 (+838056us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b540009ecd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:01.114760 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:01.114888 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":560024448}
I20260709 13:04:01.120039 14218 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:01.121066  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:01.121528 13680 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:01.040019 14224 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:01.121961  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:01.125568  8252 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:01.126820  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:01.127578  8252 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:04:01.124784  8326 raft_consensus.cc:1240] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:04:01.124305  8369 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:01.132573  8974 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:01.119771  8253 rpcz_store.cc:276] 0709 13:04:00.008042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.008179 (+   137us) service_pool.cc:224] Handling call
0709 13:04:01.118978 (+1110799us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b54003f16d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:01.119084 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":815775232}
I20260709 13:04:01.134370  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:01.134963  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:01.122066  8255 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:01.134975  8252 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
I20260709 13:04:01.102679 14119 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:01.154966 14145 consensus_queue.cc:260] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:01.158524 14065 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.136986 14066 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:01.159248 14065 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:01.123625 14152 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.161523  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2632) took 1104 ms. Trace:
I20260709 13:04:01.161931  8328 rpcz_store.cc:276] 0709 13:04:00.056949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.141008 (+ 84059us) service_pool.cc:224] Handling call
0709 13:04:00.320665 (+179657us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:00.325660 (+  4995us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:00.325694 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:00.326642 (+   948us) log.cc:844] Serialized 26 byte log entry
0709 13:04:00.326969 (+   327us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:00.327039 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.327180 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:01.161299 (+834119us) raft_consensus.cc:1685] finished
0709 13:04:01.161322 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:01.161459 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:01.162993 14152 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:01.125149 13680 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:01.123275 13757 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:01.163391 14153 raft_consensus.cc:3055] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:01.164063 14153 raft_consensus.cc:740] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:01.165796 14053 raft_consensus.cc:3055] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:01.166302 14053 raft_consensus.cc:740] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:01.166419 14145 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:01.167286 14053 consensus_queue.cc:260] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.168711 14053 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:01.165189 14153 consensus_queue.cc:260] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:01.139439  8974 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:01.121891 14081 consensus_queue.cc:237] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:01.145881  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 928) took 1143 ms. Trace:
I20260709 13:04:01.170621 14153 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:01.170871  8250 rpcz_store.cc:276] 0709 13:04:00.002952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.003082 (+   130us) service_pool.cc:224] Handling call
0709 13:04:00.023033 (+ 19951us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:00.602906 (+579873us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:00.602951 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:00.603012 (+    61us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:00.603059 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.145642 (+542583us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b54001306d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:01.145697 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:01.145828 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1332961152}
I20260709 13:04:01.174077  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:01.174940 14215 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:01.177358  8298 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:01.178190 13680 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:01.178669  8298 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:01.178915  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2630) took 1490 ms. Trace:
I20260709 13:04:01.179358  8326 rpcz_store.cc:276] 0709 13:03:59.688178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.135868 (+447690us) service_pool.cc:224] Handling call
0709 13:04:00.378960 (+243092us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:01.131271 (+752311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.177979 (+ 46708us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b540014cb54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:01.178697 (+   718us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b540014cb50. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:01.178858 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2272566528}
I20260709 13:04:01.138810 14229 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:01.180083  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:01.180550  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:01.181059 14088 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:01.181600 14152 consensus_queue.cc:1059] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:01.182722 14152 raft_consensus.cc:3055] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:01.183102 14152 raft_consensus.cc:740] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:01.183990 14152 consensus_queue.cc:260] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:01.184899  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2625) took 1743 ms. Trace:
I20260709 13:04:01.099211 14087 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.185293  8331 rpcz_store.cc:276] 0709 13:03:59.442185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.444497 (+  2312us) service_pool.cc:224] Handling call
0709 13:03:59.444835 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:00.247637 (+802802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.184667 (+937030us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b540037cb54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:01.184843 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":983259008}
I20260709 13:04:01.186542 14065 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:01.168953 14144 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:01.187171  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:01.191202 14230 log.cc:927] Time spent T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:04:01.149106  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:01.185194 14152 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:01.191993 13770 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:01.143038 14146 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:01.198872 14144 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:01.200500  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:01.201952  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:01.202124  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:01.203697  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:01.209510  8397 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 20. Current term is 21. Ops: [18.12-20.13]
I20260709 13:04:01.210645  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:01.185629  8223 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:01.212050  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:01.212363  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:01.212781  8398 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:04:01.212033 14144 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:01.214037 14065 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:01.214315 14144 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:01.216668 14144 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:01.216969  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:01.187882 14087 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:01.217624 14154 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:01.217774  9004 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:04:01.218329 14154 raft_consensus.cc:697] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:01.219354 14154 consensus_queue.cc:237] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.130807 14202 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.222841 14202 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:01.226454  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 93232.8us
I20260709 13:04:01.150403 13658 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:01.231570  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:01.226445 14206 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:01.158437  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:01.236879  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:01.238099  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:01.238618  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:01.239957  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:01.240597  8403 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 10.
I20260709 13:04:01.191263  9005 raft_consensus.cc:1240] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:04:01.137122 14187 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:01.238730  8397 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:04:01.244231 14187 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.246996 14187 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:01.247730 14007 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.250056 14007 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:01.232744 14203 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:01.149868 14200 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:01.252048  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:01.252631  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3036) took 1565 ms. Trace:
I20260709 13:04:01.252556  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:01.254139  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:01.200805 14182 log.cc:927] Time spent T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 1.484s	user 0.000s	sys 0.037s
I20260709 13:04:01.256129  8330 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:04:01.153170  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:01.253352  8399 rpcz_store.cc:276] 0709 13:03:59.687445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.697359 (+  9914us) service_pool.cc:224] Handling call
0709 13:03:59.697769 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.098484 (+1400715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.252403 (+153919us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54003a1954. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:01.252575 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3204993408}
I20260709 13:04:01.258590  8329 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:01.153791 13665 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.269085  8974 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:01.275382  8223 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:01.275532  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:01.264032  8330 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 20. Current term is 21. Ops: [18.12-20.13]
I20260709 13:04:01.264647  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:01.265364  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:01.155983  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:01.266872  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:01.278861  8222 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:01.281061  8974 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:01.281551  8331 raft_consensus.cc:2393] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 17: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:01.281273  8326 raft_consensus.cc:2393] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 15: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:01.281906  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:01.264184  8295 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:01.256871  8972 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:01.282644  8330 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:04:01.282897  8323 raft_consensus.cc:2393] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:01.286306  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:01.285977  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:01.285989  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:01.287220  8295 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:01.292292  8295 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:01.293205  8326 raft_consensus.cc:1240] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 15. Current term is 16. Ops: []
W20260709 13:04:01.294130  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.297585  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:01.232808 14212 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.298192  8972 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:01.284189 13665 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.309228  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:01.310570  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:01.301049  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:01.301620  8974 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:01.311488  8329 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 21: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:01.312482  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:01.158429 14151 consensus_meta.cc:287] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.608s	user 0.008s	sys 0.001s
I20260709 13:04:01.313511 14151 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:01.313844 14218 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:01.314332 14218 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:01.314846 14218 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:01.316540 14221 consensus_queue.cc:1059] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:01.317869 14206 raft_consensus.cc:3055] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:01.318181 14229 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:01.318421 14206 raft_consensus.cc:740] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:01.318770 14229 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:01.319445 14229 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:01.319787 14206 consensus_queue.cc:260] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.321165 14206 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:01.323318 14215 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:01.328890 13768 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:01.330174  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:01.330899  9005 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:04:01.334734  8223 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:01.336912  8295 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:01.276449 14202 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:01.334141  8974 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:01.344970  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:01.346674  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:01.285081 13812 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:01.370225  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:01.371409  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:01.372568  8295 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:01.375751  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:01.311453 14187 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:01.311894 14153 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:01.289786 14099 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:01.275635 14200 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.387207  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:01.389920  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:01.353318 14116 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.398590  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:01.402350  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:01.349162 14228 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:01.400712  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:04:01.402870  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 96320.6us
I20260709 13:04:01.406322  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:01.347759 14007 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:04:01.403870  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 69801.1us
I20260709 13:04:01.376679 13812 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.349951 13768 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:01.384891 14153 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.386292 14152 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:01.407439  8974 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:01.345752 14212 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:01.410967  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:01.437774  8255 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:04:01.401500 13665 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:01.413769  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:01.438611  8403 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18.
I20260709 13:04:01.407038  8256 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:01.380530 14151 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:01.380584 14189 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:01.395045 14213 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:01.448416  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:01.332521 13846 consensus_queue.cc:1059] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 13:04:01.412575 14168 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:01.402221 14088 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:01.418883 14169 consensus_queue.cc:260] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:01.439512  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 74278.8us
I20260709 13:04:01.439929  8222 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:01.435859 14153 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:01.406984 14099 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:01.444517  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:01.424137 13812 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:01.408145 14228 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:01.408591 14007 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:01.434187 13768 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:01.436017 13757 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.462853  8972 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:01.464416  8256 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:04:01.466100  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:01.476145  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:01.477103  8255 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:04:01.440353 14152 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:01.409722 14081 consensus_queue.cc:1059] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:04:01.467211 13665 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:01.469815 14007 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:01.446141 14215 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:01.451439 14168 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.483935  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:01.488118  8327 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18.
I20260709 13:04:01.488168  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:01.409601 14116 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:01.493952  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:01.496484  8222 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:01.500830  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:01.471807 14228 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:01.477632 13768 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:01.504943  8401 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:01.489089 13665 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.506206 13665 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:01.510923 14116 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:01.511336 14189 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:01.513852 14053 raft_consensus.cc:3055] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:01.514390  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:01.495714 14168 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:01.516520  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:01.463698 14156 consensus_queue.cc:1059] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:04:01.520116  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:01.480370 14152 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:01.495059 14206 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.525040 14174 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:01.525416  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2642) took 1106 ms. Trace:
I20260709 13:04:01.526726  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:01.545621  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:01.528043 14234 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:04:01.529031 14200 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:01.529542 14229 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:01.531035  8252 raft_consensus.cc:1275] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:01.554093  8252 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:01.533123  9005 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
W20260709 13:04:01.548779  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 108502us
I20260709 13:04:01.486166 13757 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.563391 13757 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:01.565508  8256 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:01.544548  8324 rpcz_store.cc:276] 0709 13:04:00.418721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.492922 (+ 74201us) service_pool.cc:224] Handling call
0709 13:04:00.573567 (+ 80645us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:00.579232 (+  5665us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:00.651067 (+ 71835us) log.cc:844] Serialized 14 byte log entry
0709 13:04:01.221410 (+570343us) spinlock_profiling.cc:243] Waited 6.21 ms on lock 0x7b1800326740. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e23a 00007f88287c8a2a
0709 13:04:01.376081 (+154671us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:01.383899 (+  7818us) log.cc:844] Serialized 26 byte log entry
0709 13:04:01.384417 (+   518us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:01.388136 (+  3719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.388247 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:01.525134 (+136887us) raft_consensus.cc:1685] finished
0709 13:04:01.525222 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:01.525359 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5894,"prepare.queue_time_us":71832,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":30804,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":49533,"spinlock_wait_cycles":14289664,"thread_start_us":154319,"threads_started":1,"wal-append.queue_time_us":750602}
I20260709 13:04:01.554930  8369 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:01.521963 14053 raft_consensus.cc:740] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:01.529057 13665 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.555361  8298 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:01.543301 14187 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:01.577890 13665 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:01.551203 14206 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:01.608871 14206 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:01.609580 14116 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:01.610083 14116 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.610478 14116 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:01.608800 14218 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:01.618006 14241 log.cc:927] Time spent T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:04:01.620163 14218 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:01.620851  8257 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:55264
I20260709 13:04:01.621585  8257 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:01.621971  8257 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:01.623854  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:01.624650  8397 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:01.580783 14088 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:01.625624 14088 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:01.609629  8255 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
W20260709 13:04:01.582499  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:01.630522  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.636195 14187 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:01.639200 14187 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:01.583199  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:01.605495  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:01.700575  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 117442us
I20260709 13:04:01.554639 14221 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:01.554450 14073 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:01.622105 14053 consensus_queue.cc:260] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.639808  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:01.640162 14238 log.cc:927] Time spent T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 13:04:01.639101  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:01.659999  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:01.582670  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:01.606698 13962 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:01.726564  9000 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:04:01.701581  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:01.704661  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 118931us
I20260709 13:04:01.728178  8223 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:01.707500  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:01.708194  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:01.729202  8369 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:01.709605 13812 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:01.712930 14200 consensus_queue.cc:1059] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:01.736351  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:01.714022  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:01.714995  8397 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:04:01.716666 14152 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.705662 14218 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:01.757825 14218 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:01.759683 13768 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:01.764055 14152 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:01.766418 14200 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:01.767066 14200 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.767508 14200 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:01.769687 14202 raft_consensus.cc:3055] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:01.770290 14202 raft_consensus.cc:740] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:01.771252 14202 consensus_queue.cc:260] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.772765 14202 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:01.773973 14246 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:01.724532  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:01.781502 14154 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:01.782024 14154 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:01.702747  8257 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.784189  8257 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:01.784633 14229 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:01.788640  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:01.754428 14053 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:01.796522  8254 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:04:01.788595  8295 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:01.731061 14073 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:01.797477 14122 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:01.798035 14122 raft_consensus.cc:697] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:01.798154 14152 consensus_queue.cc:1048] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:01.799183 14073 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:01.727093 14007 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.798980 14122 consensus_queue.cc:237] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.723423 13846 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:01.801211 14007 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:01.726152 14221 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.782428 14154 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:01.803552 14221 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:01.733515  8220 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:01.808265  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 74889.3us
I20260709 13:04:01.809635 13846 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:01.810375  8220 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:01.811242 14073 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:01.697775 14066 consensus_queue.cc:1059] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.116s
I20260709 13:04:01.814595 14066 raft_consensus.cc:3055] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:01.815148 14066 raft_consensus.cc:740] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:01.789491  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:01.785338  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:01.820760  8326 raft_consensus.cc:2393] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 12: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:01.810417 13846 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:01.827838 13812 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:01.837671 14154 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.840070 14154 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:01.841126 14181 raft_consensus.cc:3055] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:01.841447 13768 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:01.847349  8257 raft_consensus.cc:2368] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13.
I20260709 13:04:01.847138  8220 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:01.850253  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:01.850963  9000 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:01.853726  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:01.816480 14066 consensus_queue.cc:260] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:01.861159  8220 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:01.842191 14181 raft_consensus.cc:740] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:01.869490  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:01.872345  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 291017us
I20260709 13:04:01.746273  8369 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:01.877374  8220 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:01.877640  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 131496us
I20260709 13:04:01.878724 14196 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:01.879235 14181 consensus_queue.cc:260] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:01.880962 14181 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:01.883810  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:01.846601 14218 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:01.862987 14221 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:01.735085 14168 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.869119 13768 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:01.896261  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:01.892966  8326 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:01.911451  8297 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:01.892962  8220 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:01.869554 14073 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.917627  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:01.922806  8220 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:01.919040  9005 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:04:01.923671  8222 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:01.926213 14149 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:01.923921 14064 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:01.927459 14213 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:01.833209 14228 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:01.932129  8297 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:01.913897 14206 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:01.868103 14153 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:01.940421 13768 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:01.876034 13846 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:01.955263  8297 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:01.890550 14120 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:01.893721 14065 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:01.886660 14088 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:01.961930  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:01.978629  9003 raft_consensus.cc:2410] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:04:01.974694  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:01.938436 14218 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:01.953882 14084 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:01.979473  8330 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:04:01.974781  8223 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:01.960090 14007 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:01.989061  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:01.988494  8329 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:01.985039  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:01.998207  8324 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:01.986286 14218 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:02.002630 14218 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:02.003734  9000 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:04:01.973672 14196 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:01.994045 14087 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:02.005733 14196 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:02.006145 14087 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:02.008003 14196 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:01.993180  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:02.009294 14168 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:02.009388  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:02.010880  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:01.972904 14122 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:02.015605 14248 log.cc:927] Time spent T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:04:01.934914 14073 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:02.018121 14073 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:02.012123  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:01.983989 14084 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:02.021857 14084 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:02.022806  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:01.995136  8405 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:04:02.025242  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:02.026531  8324 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:02.035427  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:01.962874 14206 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:02.039650  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:02.040951 14206 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:02.044256  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:01.992640 14187 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:01.956658 14213 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:02.055331  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:02.091102  8406 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:02.073669  8972 leader_election.cc:400] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13.
I20260709 13:04:02.093307  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:02.093343  8972 leader_election.cc:403] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 13:04:02.076172  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 153436us
I20260709 13:04:02.081655  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:02.045408  8257 raft_consensus.cc:2393] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 19: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:04:02.083627  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.084563  8222 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:02.095741 14198 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.096210 14065 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:02.096413 14198 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:02.098251  8220 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:02.098062  8222 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:02.106998 14198 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:02.109666 14065 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:02.111455  8220 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:02.113727 14203 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:02.114586  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:02.115468  9002 raft_consensus.cc:2393] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 20: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
W20260709 13:04:02.131673  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:02.131996  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 58547.1us
I20260709 13:04:02.139248  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:02.139984  8256 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:04:02.117476 14111 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:02.127593 14033 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:02.129140 13680 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.124183 14198 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:02.126649 14187 raft_consensus.cc:697] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:02.141976 14099 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13.
I20260709 13:04:02.136543 14059 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:02.124725 14116 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:02.157763  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2496) took 1440 ms. Trace:
I20260709 13:04:02.161046  8400 rpcz_store.cc:276] 0709 13:04:00.717555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.720884 (+  3329us) service_pool.cc:224] Handling call
0709 13:04:00.721208 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:00.862097 (+140889us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:01.015739 (+153642us) log.cc:844] Serialized 14 byte log entry
0709 13:04:01.088320 (+ 72581us) log.cc:844] Serialized 14 byte log entry
0709 13:04:01.202886 (+114566us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b18003b46a0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882564bc0a 00007f882ed8b219 00007f88256d3a40 00007f88256ce4e4 00007f88256ce455 00007f88287b6060 00007f88287c8716 00007f88287d9287 00007f88287d9208
0709 13:04:01.203047 (+   161us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:01.203103 (+    56us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:01.203168 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.157471 (+954303us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54000fdcd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:02.157521 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:02.157679 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":102340,"prepare.queue_time_us":278630,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":118352,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":7507,"spinlock_wait_cycles":327574656,"thread_start_us":225237,"threads_started":2,"wal-append.queue_time_us":91216}
I20260709 13:04:02.165611  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:02.166442  8400 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:04:02.171253  8222 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:02.179517  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:02.186676 14228 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:02.187404 14256 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:02.187897 14256 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.188268 14256 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:02.194391  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:02.194744  8974 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:02.195060  8330 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
I20260709 13:04:02.196028  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:02.196911  8297 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:02.197075  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:02.198400  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:02.199196  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:02.201287  8330 raft_consensus.cc:2368] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 14. Current term is 15.
I20260709 13:04:02.203504  8974 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:02.204108  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:02.204914  8328 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:02.197904  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:02.197736  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:02.208526  8326 raft_consensus.cc:2393] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 15: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:02.199553  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:02.168712 14145 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.213227  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:02.154359 13680 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:02.218403  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:02.219934  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:02.224161  8325 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:02.216979  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:02.165197 14059 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:02.230438 14059 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:02.230571 14152 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:02.231189 14152 raft_consensus.cc:697] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:02.231469  8398 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 13:04:02.231851  8398 pending_rounds.cc:85] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 10
I20260709 13:04:02.232138 14152 consensus_queue.cc:237] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:02.232991  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 2404) took 1324 ms. Trace:
I20260709 13:04:02.233389  8254 rpcz_store.cc:276] 0709 13:04:00.908115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.244880 (+336765us) service_pool.cc:224] Handling call
0709 13:04:01.699425 (+454545us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b5800002a98. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699e9c 00007f8825699a5b 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:04:01.699704 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.796947 (+ 97243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.232737 (+435790us) spinlock_profiling.cc:243] Waited 88.3 ms on lock 0x7b5400076cd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:02.232928 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":231783168}
I20260709 13:04:02.234007 13680 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:02.234874  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:02.223244 14215 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:02.235491 14215 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:02.235900 14215 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:02.236172  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:02.232151  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:02.238232 14213 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:02.214154 14090 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:02.239991 13962 consensus_queue.cc:1059] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:02.240091 14090 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:02.240898 13962 consensus_queue.cc:1243] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [LEADER]: Peer ff7a6fe533e94c1bae555b4cae427728 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:04:02.216274  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:02.241410 13962 raft_consensus.cc:3055] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:02.248345 13962 raft_consensus.cc:740] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:02.242210  8324 raft_consensus.cc:2393] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 18: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:02.250342  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2381) took 1355 ms. Trace:
I20260709 13:04:02.250124 13962 consensus_queue.cc:260] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:02.250699  8248 rpcz_store.cc:276] 0709 13:04:00.894515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.189183 (+294668us) service_pool.cc:224] Handling call
0709 13:04:01.257217 (+ 68034us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.438361 (+181144us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:01.438394 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:01.438441 (+    47us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:01.438475 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.250067 (+811592us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54002ee354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:02.250113 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:02.250237 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":416099712}
I20260709 13:04:02.251312 13962 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:02.252027  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:02.253986  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3052) took 1022 ms. Trace:
I20260709 13:04:02.253599 14187 consensus_queue.cc:237] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:02.254355  8401 rpcz_store.cc:276] 0709 13:04:01.231105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.258885 (+ 27780us) service_pool.cc:224] Handling call
0709 13:04:01.297075 (+ 38190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.505973 (+208898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.253750 (+747777us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b5400319f54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:02.253927 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":477434624}
I20260709 13:04:02.255195  8223 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:02.256155  8223 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:02.257403 14088 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:02.232101  8398 pending_rounds.cc:107] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:04:02.258152  8398 raft_consensus.cc:2889] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:02.248620 14200 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:02.259514 14153 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:02.260005 14120 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:02.260643 14200 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:02.223073 14242 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:02.261075  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:02.261467  8252 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:02.262247  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:02.263228  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:02.264117  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:02.264585  8257 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:02.267079  8256 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: [10.7-12.8]
I20260709 13:04:02.242501 14090 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:02.246655  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:02.260807 14212 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:02.269047  9000 raft_consensus.cc:2410] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:04:02.247691  8323 raft_consensus.cc:2393] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 15: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:02.260638 14120 raft_consensus.cc:697] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:02.271538 14120 consensus_queue.cc:237] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:02.272516  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:02.159389 14198 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:02.246390  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:02.253110  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:02.214921  8331 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17. Ops: []
W20260709 13:04:02.278023  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.279042  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:02.279603  9005 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:04:02.280248  8325 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:04:02.280884  8252 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:04:02.283639  8398 pending_rounds.cc:107] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 13:04:02.284224  8398 raft_consensus.cc:2889] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:02.284475  8404 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:04:02.286825  8295 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:02.253044 14221 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:02.237334  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:02.295334  8397 raft_consensus.cc:2410] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:04:02.283604  8256 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:04:02.298091  8254 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:04:02.219383 14218 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:02.283396  8406 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:02.293098  9005 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:04:02.299474  8401 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:02.288874  8974 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:02.303205  8256 raft_consensus.cc:1275] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:04:02.303637  8256 pending_rounds.cc:85] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 9
I20260709 13:04:02.303902  8256 pending_rounds.cc:107] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:04:02.304172  8256 raft_consensus.cc:2889] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:02.301575  8223 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:02.316215 14160 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:02.316798 14160 raft_consensus.cc:697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:02.317765 14160 consensus_queue.cc:237] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:02.325753 14146 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:02.326455 14146 raft_consensus.cc:697] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:02.224553 14145 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:02.327463 14146 consensus_queue.cc:237] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:02.328344 14198 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:02.331050 14145 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:02.305651 14065 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.334226 14065 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:02.336692 14065 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:02.344246  8257 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:04:02.347193  8295 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:02.342248  8222 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:02.356845  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2388) took 1400 ms. Trace:
I20260709 13:04:02.358059  8253 rpcz_store.cc:276] 0709 13:04:00.956560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.248011 (+291451us) service_pool.cc:224] Handling call
0709 13:04:02.355938 (+1107927us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b5800002a98. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699e9c 00007f8825699a5b 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:04:02.356183 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:02.356410 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:02.356447 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:02.356503 (+    56us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:02.356546 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.356618 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:02.356759 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24692096}
I20260709 13:04:02.299441 14088 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:02.358736 14111 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:02.355067 13680 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:04:02.396795  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 99153.1us
W20260709 13:04:02.394660  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 105852us
I20260709 13:04:02.399259  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:02.399997  8401 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 21.
I20260709 13:04:02.355165 14153 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:04:02.389554 14073 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.405756  8249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2387) took 1509 ms. Trace:
W20260709 13:04:02.406740  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 167284us
I20260709 13:04:02.409440  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:02.411170  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 214330us
I20260709 13:04:02.412914  8297 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:02.406502  8249 rpcz_store.cc:276] 0709 13:04:00.907379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.227897 (+320518us) service_pool.cc:224] Handling call
0709 13:04:02.227193 (+999296us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b5800002a98. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699e9c 00007f8825699a5b 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:04:02.227383 (+   190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:02.227608 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:02.227898 (+   290us) log.cc:844] Serialized 14 byte log entry
0709 13:04:02.399772 (+171874us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:02.399828 (+    56us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:02.399894 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.399983 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:02.405695 (+  5712us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":236,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":5038,"spinlock_wait_cycles":24817920,"thread_start_us":171136,"threads_started":1}
I20260709 13:04:02.417141  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:02.418411  8297 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:02.419893  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:02.354504 14215 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:02.424950  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:02.427454  8328 raft_consensus.cc:1275] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:04:02.428681  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:02.398977 14111 raft_consensus.cc:697] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 19 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:02.429415  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:02.433619  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:02.439706  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:02.439989  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:02.440776  9005 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:02.430032 14007 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:02.404095 13680 raft_consensus.cc:697] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:02.446663 14168 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:02.447165 14168 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.447535 14168 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:02.422770 14213 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:02.461126  9005 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:04:02.464078  8369 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:02.494037  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.414342 14073 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:02.496042 13962 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:02.439577 14116 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:02.515738  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:02.543678  8402 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 21: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:02.544250  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 3051) took 1313 ms. Trace:
I20260709 13:04:02.544628  8402 rpcz_store.cc:276] 0709 13:04:01.230780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.249208 (+ 18428us) service_pool.cc:224] Handling call
0709 13:04:02.515538 (+1266330us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b5800170798. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:04:02.544177 (+ 28639us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40949632}
I20260709 13:04:02.546453  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:02.549880 14262 log.cc:927] Time spent T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:04:02.533316 14256 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:02.525746 14198 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:02.529657 14111 consensus_queue.cc:237] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:02.455504 14059 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 13:04:02.530560  8369 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:02.531607 14127 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:02.533038 14168 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:02.480216 14268 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 13:04:02.446408 13680 consensus_queue.cc:237] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:02.537736 14073 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:02.554040 14084 consensus_meta.cc:287] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.532s	user 0.011s	sys 0.000s
I20260709 13:04:02.567233 14084 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:02.555446 14215 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:02.562515  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:02.570622  8253 raft_consensus.cc:2393] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 19: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:02.523798  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 937) took 1435 ms. Trace:
I20260709 13:04:02.565300  8369 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:02.572109  8251 rpcz_store.cc:276] 0709 13:04:01.088873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.256130 (+167257us) service_pool.cc:224] Handling call
0709 13:04:02.523162 (+1267032us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b5800002a98. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699f8c 00007f8825699ef6 0000000000375939 00007f882d70d1e0 00007f882d7d60c4 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:04:02.523287 (+   125us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:02.523480 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:02.523508 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:02.523547 (+    39us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:02.523577 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.523629 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:02.523741 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25002112}
I20260709 13:04:02.509087 14053 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:02.560622 14059 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:02.575241  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 144743us
I20260709 13:04:02.578467  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:02.579635  8400 raft_consensus.cc:2410] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:04:02.580186  8248 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:04:02.581665  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:02.582511  8399 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:04:02.584295  8400 raft_consensus.cc:1240] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:02.588682  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:02.590230  8401 raft_consensus.cc:1275] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:02.592075 14084 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:02.583319 13961 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:02.595366  8398 raft_consensus.cc:2393] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 15: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:02.598969  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:02.575940 14278 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:02.601312  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:02.597395 14081 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:02.602293 14059 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:04:02.602761  8297 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:02.615783  8297 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:02.618141  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:02.619055  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:02.619690  8323 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 21.
I20260709 13:04:02.620721  8249 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:04:02.621300  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:02.622753  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:02.625957  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:02.630522  8328 raft_consensus.cc:1240] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:02.633189  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:02.629913  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:02.637456  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:02.627068 14084 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:02.639494 14084 raft_consensus.cc:697] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:02.640534 14084 consensus_queue.cc:237] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:02.607089 14256 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:02.597091 14250 consensus_queue.cc:1059] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:04:02.644415 14250 raft_consensus.cc:3055] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:02.644825 14250 raft_consensus.cc:740] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:02.645699 14250 consensus_queue.cc:260] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:02.605013 14168 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:02.653690  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:02.631968 14059 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:02.655079  9000 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:02.655237  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2499) took 1417 ms. Trace:
I20260709 13:04:02.631059 14053 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:02.654405  9002 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:02.655715  8399 rpcz_store.cc:276] 0709 13:04:01.237343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.261845 (+ 24502us) service_pool.cc:224] Handling call
0709 13:04:02.524322 (+1262477us) spinlock_profiling.cc:243] Waited 5.04 ms on lock 0x7b5800170798. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:04:02.524510 (+   188us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:02.582879 (+ 58369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.654994 (+ 72115us) spinlock_profiling.cc:243] Waited 57.8 ms on lock 0x7b54000221d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:02.655182 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":144427648}
I20260709 13:04:02.655114  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:02.669973  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:02.685154  8399 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:02.685274  8331 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
W20260709 13:04:02.691653  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 64458.6us
I20260709 13:04:02.681905  8223 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:02.693950  8223 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:02.632462 14276 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:02.699988  8253 raft_consensus.cc:1275] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:02.702023  8972 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:02.706192  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:02.666720 14145 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:02.692337 14246 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:04:02.772675  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.624791 14278 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:02.701669 14200 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:04:02.818020  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.714640 14169 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:02.686743 13846 consensus_queue.cc:1059] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:04:02.692634 14268 consensus_queue.cc:1059] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:04:02.717523 14215 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:02.722502 14256 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:02.732582 14218 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:02.716264 14109 consensus_queue.cc:1048] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.137s
I20260709 13:04:02.950686  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3055) took 1708 ms. Trace:
I20260709 13:04:02.841776 14200 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.951187  8403 rpcz_store.cc:276] 0709 13:04:01.242175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.501289 (+259114us) service_pool.cc:224] Handling call
0709 13:04:01.501614 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:01.581826 (+ 80212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:01.581857 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:01.582268 (+   411us) log.cc:844] Serialized 26 byte log entry
0709 13:04:01.957719 (+375451us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:01.957788 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.950241 (+992453us) spinlock_profiling.cc:243] Waited 62.2 ms on lock 0x7b5400277c54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:02.950295 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:02.950343 (+    48us) raft_consensus.cc:1685] finished
0709 13:04:02.950359 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:02.950617 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28279,"raft.queue_time_us":18426,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":94923,"spinlock_wait_cycles":143048576,"thread_start_us":315804,"threads_started":1,"wal-append.queue_time_us":531771}
I20260709 13:04:02.979875  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 2421) took 1005 ms. Trace:
I20260709 13:04:02.980454  8252 rpcz_store.cc:276] 0709 13:04:01.974271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.041192 (+ 66921us) service_pool.cc:224] Handling call
0709 13:04:02.979610 (+938418us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b5400309654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:02.979796 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":383348096}
I20260709 13:04:02.982069  8250 raft_consensus.cc:1240] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:02.703569 14220 consensus_queue.cc:1059] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.240s
I20260709 13:04:02.745042 14145 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:02.858215 14270 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.859839 14215 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.979799 14200 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:02.977146 14199 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:02.913482 14213 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:02.944248 14268 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:02.852283 14272 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:02.905208 14072 consensus_queue.cc:1059] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.106s
I20260709 13:04:02.982075 14218 raft_consensus.cc:697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:02.977697 13680 consensus_queue.cc:1048] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 13:04:03.033938 14215 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:03.022498  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 108648us
I20260709 13:04:03.132808  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:03.031832 14270 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:03.159446 14282 log.cc:927] Time spent T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
W20260709 13:04:03.161232 14284 log.cc:927] Time spent T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 13:04:03.162886 14246 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.165143 14293 consensus_queue.cc:1059] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.011s
I20260709 13:04:03.070693 14213 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:03.170204 14278 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:03.170744 14213 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:03.171602 14281 log.cc:927] Time spent T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.458s	user 0.000s	sys 0.001s
W20260709 13:04:03.173134 14283 log.cc:927] Time spent T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:04:03.163507 14246 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:03.081895 14268 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:03.175298 14246 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:03.173348 14287 raft_consensus.cc:3055] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:03.176278 14287 raft_consensus.cc:740] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:03.177511 14145 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:03.177201 14287 consensus_queue.cc:260] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:03.178604 14287 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:03.175840 14268 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:03.069010 14199 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:03.194836  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:03.229113  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:03.229861  8328 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 20.
I20260709 13:04:03.088248 14272 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:03.263411  8256 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.003s	sys 0.008s
W20260709 13:04:03.263847  8256 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.003s	sys 0.008s
W20260709 13:04:03.265782  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 253129us
I20260709 13:04:03.192667 14297 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:03.195127 14152 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:03.276155  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:03.282186 14196 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:03.287581  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49220 (request call id 943) took 2009 ms. Trace:
I20260709 13:04:03.288086  8255 rpcz_store.cc:276] 0709 13:04:01.277683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.486574 (+208891us) service_pool.cc:224] Handling call
0709 13:04:03.287416 (+1800842us) spinlock_profiling.cc:243] Waited 87.5 ms on lock 0x7b54003f1bd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:03.287524 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":201140992}
I20260709 13:04:03.284778 14196 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:03.302378 14196 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:03.302546  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2385) took 2406 ms. Trace:
I20260709 13:04:03.303059  8250 rpcz_store.cc:276] 0709 13:04:00.895711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.191642 (+295931us) service_pool.cc:224] Handling call
0709 13:04:02.247525 (+1055883us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b5800002a98. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699e9c 00007f8825699a5b 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d797fc0 00007f882d782929 00007f882d7823d2 00007f8826cf9bd4
0709 13:04:02.981799 (+734274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:02.982810 (+  1011us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.302268 (+319458us) spinlock_profiling.cc:243] Waited 734 ms on lock 0x7b5400309650. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:03.302443 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1712644096}
W20260709 13:04:03.315953  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 168138us
I20260709 13:04:03.260905 14218 consensus_queue.cc:237] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:03.310166 14242 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:03.284185 14152 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:03.335775  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 52780.6us
I20260709 13:04:03.338526  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:03.341982  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 884) took 1555 ms. Trace:
I20260709 13:04:03.342412  8329 rpcz_store.cc:276] 0709 13:04:01.787031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.787159 (+   128us) service_pool.cc:224] Handling call
0709 13:04:01.796718 (+  9559us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.997759 (+201041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.341724 (+1343965us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54000906d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:03.341921 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":440601344}
I20260709 13:04:03.346792  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 2503) took 1760 ms. Trace:
I20260709 13:04:03.347287  8397 rpcz_store.cc:276] 0709 13:04:01.586473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.883496 (+297023us) service_pool.cc:224] Handling call
0709 13:04:03.346596 (+1463100us) spinlock_profiling.cc:243] Waited 410 ms on lock 0x7b5400277254. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:03.346732 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":944017664}
I20260709 13:04:03.350662  8298 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:03.336117 14242 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:03.353626 14305 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:03.359366  8399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:46846
I20260709 13:04:03.360041  8399 raft_consensus.cc:606] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:03.354260 14305 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:03.366628 14305 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:03.360811 14213 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.367975 14213 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:03.370229 14213 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:03.370947 14122 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:03.371500 14122 raft_consensus.cc:697] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:03.372555  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2403) took 1516 ms. Trace:
I20260709 13:04:03.362210 14300 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:03.373126  8248 rpcz_store.cc:276] 0709 13:04:01.856202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.286319 (+430117us) service_pool.cc:224] Handling call
0709 13:04:02.286608 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:02.580605 (+293997us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.372277 (+791672us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b540034c6d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:03.372494 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":666262912}
I20260709 13:04:03.354800 14242 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:03.271517 14297 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:03.374009 14300 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:03.384917  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:03.386109  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.389905  8323 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:03.390300  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:03.391188  8329 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:03.393453  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:03.394429  8326 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:03.396458 14202 raft_consensus.cc:3055] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:03.396956 14202 raft_consensus.cc:740] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:03.398722  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 48158.9us
I20260709 13:04:03.398157 14202 consensus_queue.cc:260] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:03.400010  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:03.399976 14202 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:03.372373 14122 consensus_queue.cc:237] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:03.378232 14169 consensus_meta.cc:287] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.524s	user 0.007s	sys 0.004s
I20260709 13:04:03.403822  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:03.406370  9005 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:03.404812  8220 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:03.410158 13757 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.416625 13757 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:03.421355 14300 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:03.421417  8397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:46846
I20260709 13:04:03.422389  8397 raft_consensus.cc:606] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:03.422693  8397 raft_consensus.cc:614] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:04:03.424299 14280 log.cc:927] Time spent T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.572s	user 0.010s	sys 0.010s
I20260709 13:04:03.381421 14152 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:03.407368 13812 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.434206 13812 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:03.435003  8324 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 888) took 1243 ms (client timeout 1522 ms). Trace:
W20260709 13:04:03.435416  8324 rpcz_store.cc:269] 0709 13:04:02.191889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.192024 (+   135us) service_pool.cc:224] Handling call
0709 13:04:03.434653 (+1242629us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b540029a9d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:03.434913 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41982208}
I20260709 13:04:03.436419 14297 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:03.437655  8331 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:04:03.436872  8324 raft_consensus.cc:1240] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:03.436535 13812 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:03.405965 13770 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:03.448369  8329 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:04:03.450582  8324 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:04:03.406487 14199 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:03.452284  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 47580.5us
I20260709 13:04:03.456274  8399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "766ac43473084aa6a9fdce530520947c"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "ff7a6fe533e94c1bae555b4cae427728"
 from {username='slave'} at 127.0.0.1:46846
I20260709 13:04:03.456920  8399 raft_consensus.cc:606] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Received request to transfer leadership to TS ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:03.457288  8399 raft_consensus.cc:614] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:03.459694  8220 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:03.460436  8329 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:04:03.461681  8220 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:03.462635  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:03.469075  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2510) took 1493 ms. Trace:
I20260709 13:04:03.455297 14072 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:03.469368 14270 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:03.454586 13770 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:03.463152  8403 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:04:03.481873  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:03.483632  8220 leader_election.cc:400] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 14. Current term is 15.
I20260709 13:04:03.478333  8972 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:03.479058  8404 rpcz_store.cc:276] 0709 13:04:01.975542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.289214 (+313672us) service_pool.cc:224] Handling call
0709 13:04:02.289491 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:02.311484 (+ 21993us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:02.311519 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:02.311566 (+    47us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:02.311599 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.468835 (+1157236us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b54000c8854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:03.468883 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:03.469016 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50117504}
I20260709 13:04:03.487186  8326 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 16.
I20260709 13:04:03.488682  8220 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:03.519785  8972 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:03.488019  8298 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:03.500707  8404 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:46846
I20260709 13:04:03.525619 14099 consensus_queue.cc:1059] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:04:03.463443 14111 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:03.486382 14256 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:03.482383 14072 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:04:03.534821  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.485387 14272 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:03.484390 14087 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:03.534786  8222 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:03.512399 14270 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:03.485734 13770 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:03.526678 14276 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:03.539172 14311 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:03.540205 14304 consensus_queue.cc:1059] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:03.541121 14215 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:03.534459 14145 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:03.544866 14304 raft_consensus.cc:3055] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:03.545338 14304 raft_consensus.cc:740] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:03.527027 14299 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:03.546402 14292 raft_consensus.cc:3055] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:03.546713 14299 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:03.546829 14215 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:03.547000 14292 raft_consensus.cc:740] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:03.536497 14072 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:03.548352 14292 consensus_queue.cc:260] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:03.552726 14285 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.519s	user 0.002s	sys 0.000s
I20260709 13:04:03.554006  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:03.557348 14295 raft_consensus.cc:3055] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:03.557863 14295 raft_consensus.cc:740] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:03.547348 14299 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:03.473984 14200 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:03.544878 14145 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.564867  8372 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:03.527145 14316 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:03.569633  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:03.570366  8326 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:04:03.572702  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:03.543236 14276 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.573640  8326 raft_consensus.cc:2393] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 17: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:03.560890 14311 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.577952 14111 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
I20260709 13:04:03.578212 13962 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:03.569309 14145 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:03.591663 14200 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:03.590075 14292 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:03.593644  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:03.595513  8328 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:03.598392 14199 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.599151 14311 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:03.599049 14199 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:03.602373  8328 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 22.
I20260709 13:04:03.542997 14270 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:03.612859  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:03.615334  8326 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:03.612671 14276 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:03.558967 14295 consensus_queue.cc:260] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:03.573487 14189 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:04:03.631166 14291 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.627722 14309 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.636420 14270 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:03.624040 14199 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:03.643090 14319 log.cc:927] Time spent T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:04:03.644235 13757 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:03.647333  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:03.649066  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:03.650435  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:03.608628 14311 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:03.546237 14304 consensus_queue.cc:260] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:03.661456 14295 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:03.648392 14291 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:03.648370 14218 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 13:04:03.680210  8331 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:03.654235 14309 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:03.693842 14116 raft_consensus.cc:3055] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:03.694382 14116 raft_consensus.cc:740] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:03.695194 14116 consensus_queue.cc:260] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:03.696198 14116 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:03.697947 14199 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:03.698539 14199 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:03.699184 14072 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:03.701419 14072 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:03.705961  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3086) took 1144 ms. Trace:
I20260709 13:04:03.706207  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:03.706416  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:03.706833  9001 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:04:03.707193  8331 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:04:03.702489 14218 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.701697 14199 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:03.708343 14218 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:03.704109  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:03.710340  8325 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:04:03.710985  8372 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:03.706375  8402 rpcz_store.cc:276] 0709 13:04:02.561622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.561877 (+   255us) service_pool.cc:224] Handling call
0709 13:04:02.573934 (+ 12057us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:02.597934 (+ 24000us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:02.597967 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:02.598013 (+    46us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:02.598047 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.705735 (+1107688us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b540019b9d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:03.705783 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:03.705911 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54443392}
W20260709 13:04:03.717229  8369 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.815s (SENT)
I20260709 13:04:03.724814 14291 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:03.704542 14311 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:03.734604 14066 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:03.703014 14309 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:03.735064  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:03.735841  8332 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:04:03.705453 13891 consensus_queue.cc:1059] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 13:04:03.730144  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:03.743201  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 2430) took 1113 ms. Trace:
I20260709 13:04:03.757319  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:03.766688  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:03.770646  9001 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:03.746923  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:03.715467 14081 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:04:03.785135  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 67921us
I20260709 13:04:03.787820  9002 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:04:03.734500 14145 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:03.703714 14299 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:03.796774  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.779450 14081 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.789742 14199 consensus_queue.cc:1059] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.013s
W20260709 13:04:03.801563  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.802381 14152 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:03.802961 14152 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:03.804762 14315 log.cc:927] Time spent T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.240s	user 0.001s	sys 0.000s
I20260709 13:04:03.817617  8249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2414) took 1196 ms. Trace:
I20260709 13:04:03.818154  8249 rpcz_store.cc:276] 0709 13:04:02.620980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.621851 (+   871us) service_pool.cc:224] Handling call
0709 13:04:02.622194 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:03.434164 (+811970us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:03.434205 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:03.434255 (+    50us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:03.434292 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.817347 (+383055us) spinlock_profiling.cc:243] Waited 812 ms on lock 0x7b54002ee354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:03.817400 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:03.817553 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1866772224}
I20260709 13:04:03.819063 14299 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:03.790503 14291 raft_consensus.cc:2764] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:04:03.824851 14145 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:03.826148  9004 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:03.828078  9009 raft_consensus.cc:2393] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 17: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:03.840766  8297 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:03.842717  8329 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:04:03.856998 14299 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.800283 14081 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:03.857761 14299 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:03.858572 14109 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:03.859004 14145 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.886843 14073 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:03.887828  8326 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:03.889554 14250 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:03.890872  8220 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:03.859688 14246 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:03.902160  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:03.902268 14246 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:03.893570  9002 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:04:03.904881 14246 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:03.898165 14218 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:03.899359 14299 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:03.901454 14322 log.cc:927] Time spent T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 13:04:03.859508 14145 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:03.912667  8297 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:03.934089  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:03.911703 14317 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.313s	user 0.001s	sys 0.002s
I20260709 13:04:03.937106 14270 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:03.928223 14295 consensus_queue.cc:1059] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:04:03.946956 14276 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:03.947152 14270 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:03.948611  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:03.949326 14276 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:03.911998 14073 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:03.943404  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:03.960983  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:03.930080  8972 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:03.945376  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:03.961335  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:03.963142  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:03.975984  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:03.963510  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:03.976891  8398 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:03.963753  9004 raft_consensus.cc:2393] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 20: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:03.968540  9006 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:04:03.961874  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:03.980022  8400 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 16.
I20260709 13:04:03.981680  8323 raft_consensus.cc:2393] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 17: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:03.983708  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:03.977447  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:03.985097  8251 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 20.
I20260709 13:04:03.959236  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:03.987746  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:03.988421  8325 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:03.919404 14168 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.991509  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:03.992358  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:03.992784  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:03.992712  8331 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:04:03.997030  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:03.929878 14246 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:03.936386 14088 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:04.011602  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:04.012140  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:04.012498  8255 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:04.014112  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:04.000631  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:03.990096  8220 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:04.004235  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.016960  8402 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:04.005174  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:03.999244  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:04.017814  8220 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:04.023593  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:04.024240  8398 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 22.
I20260709 13:04:04.026787  8251 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:04:03.956739 14294 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:03.949396 14270 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:03.959483 14169 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:03.961398 13846 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:04.039197  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.040726  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:04.042721  8401 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:04.026680  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:03.959374 14154 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:03.963037 14073 raft_consensus.cc:697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:03.981012 14218 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:04.046193  9002 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:04:04.001484 14276 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:04:04.021817 14325 log.cc:927] Time spent T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:04:03.955516 14301 consensus_queue.cc:1059] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 13:04:04.011372 13770 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.082773  9003 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:04:04.027091 14311 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:04.027323 14256 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:04.097409 14059 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:04.074887  8402 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:04:04.130466  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:04.014034 14087 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 13:04:04.011590 14168 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:04.024060 14246 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:04.165102  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.051453 14218 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:04.051533 13875 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:04.108037  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.040555 14294 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:04.163069 14281 log.cc:927] Time spent T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:04:04.090919 14309 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.096941 14297 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.183771  8398 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:04:04.129521 14202 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:04.056137 14169 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:04.097558 13757 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:04.187879  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 79485.4us
I20260709 13:04:04.160557 14292 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:04.201747 14292 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:04.202229 14169 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:04.204644 14292 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:04.169795 14218 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:04.206140  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:04.195794 14202 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:04.210945 14202 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:04.212100 13757 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:04.214674 13757 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:04.216539 14218 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:04.217852  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:04.219270  9003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 1195) took 2240 ms. Trace:
I20260709 13:04:04.219847  9003 rpcz_store.cc:276] 0709 13:04:01.980386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.980490 (+   104us) service_pool.cc:224] Handling call
0709 13:04:01.980802 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:04.095732 (+2114930us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.218902 (+123170us) spinlock_profiling.cc:243] Waited 2.1 s on lock 0x7b5400533254. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:04.219141 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4833170816}
I20260709 13:04:04.223037  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:03.743640  8256 rpcz_store.cc:276] 0709 13:04:02.629542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.634912 (+  5370us) service_pool.cc:224] Handling call
0709 13:04:02.635195 (+   283us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:02.635503 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:02.635529 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:02.663943 (+ 28414us) log.cc:844] Serialized 52 byte log entry
0709 13:04:03.264222 (+600279us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:03.741986 (+477764us) spinlock_profiling.cc:243] Waited 975 us on lock 0x7b180038af40. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88283664bc 00007f88256b448b 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256ce455 00007f882c1b63c7 00007f88287c89a7 00007f88287d80bf
0709 13:04:03.742161 (+   175us) log.cc:844] Serialized 14 byte log entry
0709 13:04:03.742596 (+   435us) log.cc:844] Serialized 14 byte log entry
0709 13:04:03.742783 (+   187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.742864 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:03.742987 (+   123us) raft_consensus.cc:1685] finished
0709 13:04:03.743004 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:03.743143 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":119887,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":21584,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":4311,"spinlock_wait_cycles":50473600,"thread_start_us":588035,"threads_started":2,"wal-append.queue_time_us":619074}
W20260709 13:04:04.226230  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.231510  8406 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:04:04.172165 14189 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:04:04.232190  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 190919us
I20260709 13:04:04.232215  8372 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:04.090272 14276 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:04.235224  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:04.162429 14059 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:04.115533 13770 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:04.060210 14154 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:04.248085  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.188161 14203 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.133750 14152 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:04.174291 13875 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:04.295884  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:04.307590  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.087669 13846 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:04.231082 14246 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:04.320812  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:04.286165 14303 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:04.326601  8250 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:04:04.327059  8250 pending_rounds.cc:85] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 9
I20260709 13:04:04.315866  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:04.327280  8250 pending_rounds.cc:107] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:04:04.327555  8250 raft_consensus.cc:2889] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:04.307709 14156 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.192330 14309 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:04.287501 14189 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:04.311208 14120 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:04.196254 14297 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:04.322995 14200 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:04.317853 14111 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:04.289862 13757 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:04:04.328697 14202 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:04.355253  8255 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 22.
I20260709 13:04:04.192998 14073 consensus_queue.cc:237] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:04.329500 14303 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:04:04.362804  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.344164 14306 raft_consensus.cc:3055] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:04.364384  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.388944  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:04.304441 14203 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:04.393211  8253 raft_consensus.cc:3055] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:04.393633  8253 raft_consensus.cc:740] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:04.396196  8257 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:04.361830 14256 raft_consensus.cc:697] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:04.359539 14335 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:04.385550  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:04.350818 14156 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:04.386561  8330 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 14. Current term is 15. Ops: []
W20260709 13:04:04.397758  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.408983  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:04.397927  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:04.398463  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:04.365455  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.394412  8253 consensus_queue.cc:260] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:04.411736  8327 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 20.
I20260709 13:04:04.412114  8253 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:04.386001  8255 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:04:04.404857  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.417052  8255 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:04:04.387781 14303 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:04.387202 14198 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.422042  8324 raft_consensus.cc:2393] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 17: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:04.390914 14306 raft_consensus.cc:740] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:04.405472 14199 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.434267  8323 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:04.400358 14152 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.458429 14152 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:04.460831 14152 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:04.431957  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:04.470183 14331 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:04:04.471537 14329 log.cc:927] Time spent T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:04:04.472190  8974 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:04.473742 14154 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:04.474395 14270 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:04.474628 14154 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:04.357163 14120 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:04.366056 14200 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:04.443913  8401 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:04:04.445178  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:04.448678  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:04.376492 14111 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:04.480322  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 48513.9us
W20260709 13:04:04.529291  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 49916.8us
I20260709 13:04:04.483183  8297 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:04.532727  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 49656.1us
W20260709 13:04:04.487789 14081 consensus_meta.cc:287] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.630s	user 0.006s	sys 0.007s
W20260709 13:04:04.488504 14327 log.cc:927] Time spent T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc: Append to log took a long time: real 0.297s	user 0.001s	sys 0.000s
I20260709 13:04:04.488960 14059 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:04.489717 14228 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.491189 14309 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:04.491617 13875 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:04.501955 14276 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:04.447321 14198 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:04.478196 14168 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:04.502414 13846 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:04.503572 14306 consensus_queue.cc:260] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:04.503572 14294 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:04.577400  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:04.578156 14306 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:04.418089 14335 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:04.484638  8974 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:04.583833  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 99260.4us
I20260709 13:04:04.384533 14202 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:04.586300 14270 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:04.587870  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:04.588622 14202 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:04.590138 14335 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:04.527734  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:04.592012 14304 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:04.592621  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:04.594213  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:04.595697  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:04.596606  8401 raft_consensus.cc:2368] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16.
I20260709 13:04:04.534536 13962 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:04.548966  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.597764 13962 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:04.598445  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.599555  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.599931 13962 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:04.600528  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.601655  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.549872  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:04.602480  8251 raft_consensus.cc:2376] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13. Re-sending same reply.
I20260709 13:04:04.602737  8974 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:04.603401  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.569880  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.604357  8257 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:04:04.604555  8974 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:04.605206  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.606087  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.606712  8251 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:04:04.607483  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.573971  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:04.608937  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 75111.1us
I20260709 13:04:04.610739  8257 raft_consensus.cc:2393] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 18: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:04.612159  8297 leader_election.cc:400] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16.
I20260709 13:04:04.612623  8297 leader_election.cc:403] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:04.575073 14198 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:04.537680 14338 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:04:04.577293  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:04.504769 14256 consensus_queue.cc:237] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:04.623648  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.522586  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:04.537945 14152 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:04.538527 14199 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:04.638854  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:04.631704  8974 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:04.615331  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:04.618017  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 159613us
I20260709 13:04:04.641573  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:04.643378  8297 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:04.644673 14276 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:04.623111 14122 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16.
I20260709 13:04:04.646245  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:04.649597  8401 raft_consensus.cc:2393] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 17: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:04.610626 14278 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:04.621949  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:04.632689  8369 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:04.637043  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:04.619244  8253 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:04:04.653379  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:04.656884  8253 pending_rounds.cc:85] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 10
I20260709 13:04:04.657217  8253 pending_rounds.cc:107] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:04:04.657570  8253 raft_consensus.cc:2889] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:04.658386  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.558220 14033 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.087s
I20260709 13:04:04.659138  8255 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:04:04.659253  8369 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:04.660756  8297 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:04.662030  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:04.548754 14081 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:04.667738  8401 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:04.625478 13770 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:04.678354  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.619822 14294 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:04.561911 14228 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:04.684633  8369 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:04.692312  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:04.637740 14342 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:04.693974  8369 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:04.689152  8401 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:04.680799  8297 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:04.695477 14122 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:04.709702  8369 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:04.669633 14301 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:04.712842  8400 raft_consensus.cc:2393] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 18: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:04.678920 14152 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:04:04.699342 14250 consensus_meta.cc:287] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.775s	user 0.011s	sys 0.003s
W20260709 13:04:04.725071  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.744310 14218 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:04.695905 14122 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:04.746199 14122 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:04.750828 14181 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:04.751664 14181 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:04.752455 14199 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:04.753765 14156 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:04.754487 13961 raft_consensus.cc:3055] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:04.708877 14153 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:04:04.759393 14111 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:04.762030 14145 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:04.775359 14228 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:04.775906 14156 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:04:04.778826 14340 log.cc:927] Time spent T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
I20260709 13:04:04.720373 14246 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:04.717799 14332 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:04.781564 14332 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:04.782066 14332 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:04.785919  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.787969  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.689934 14270 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:04.682919 14059 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:04.687156 14169 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:04.818414  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.793947  8404 raft_consensus.cc:606] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:04.796603  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:04.797940  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:04.776597 14156 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:04.801333  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:04.802556 14181 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:04.803233  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:04.804081  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:04.805495 14341 log.cc:927] Time spent T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:04:04.702205 14154 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:04:04.718735  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 78570.2us
W20260709 13:04:04.785996 14333 log.cc:927] Time spent T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260709 13:04:04.852109 14154 raft_consensus.cc:697] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:04.854382  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.702813 13757 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:04.755100 13961 raft_consensus.cc:740] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:04.876420  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:04.852542  8404 raft_consensus.cc:614] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:04.824198 14059 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:04.877770  8403 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:04:04.854589  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:04.854108 14154 consensus_queue.cc:237] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:04.830689  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.905967  8330 raft_consensus.cc:2393] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 16: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:04.847533  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:04.853278  8397 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:04:04.814254  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.860050  8329 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:04:04.860992  8326 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:04:04.865164  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:04.910370  9009 raft_consensus.cc:1813] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Handling vote request from an unknown peer 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:04.910895  9009 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 21: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:04.815613  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:04.811805  8295 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:04.867156  8972 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:04.792289 14189 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:04.880554  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:04.918661  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:04.696131 14294 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:04.884624 14296 raft_consensus.cc:3055] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:04.885672  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:04.850412 14203 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:04.886845  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:04.935009  8332 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:04.935477  9005 raft_consensus.cc:2393] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 18: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:04.902519 13875 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.902273 14270 raft_consensus.cc:697] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 17 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:04.941293  8220 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:04.900560  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:04.906374 14228 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
W20260709 13:04:04.912276  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 100573us
W20260709 13:04:04.913021  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 97636.4us
I20260709 13:04:04.931501 14296 raft_consensus.cc:740] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:04.830744 13665 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:04.923216  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 56151.8us
I20260709 13:04:04.886080 14156 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:04.867180 13757 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:04.868263 14168 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.917759 13865 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:04.861794 14303 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:04.939168 13875 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:04.945139 14294 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:04.932658 14301 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:04.889410  8297 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:04:04.850966 14116 consensus_meta.cc:287] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.154s	user 0.021s	sys 0.007s
I20260709 13:04:04.932211 14145 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:04.952272  8369 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:04.850076 14297 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:04.953462  8295 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:04.956720  8972 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:04.958909 14311 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.769s	user 0.004s	sys 0.004s
I20260709 13:04:04.943240 14278 raft_consensus.cc:697] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 22 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:04.987761  8295 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:04.946796 13961 consensus_queue.cc:260] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:04.990890 14116 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:04.986238 13865 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:04.964855 14156 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.960805 13757 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:04.955574 14316 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:04.972204  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 82933.2us
I20260709 13:04:04.976552  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:46846 (request call id 10) took 1469 ms. Trace:
I20260709 13:04:04.973855 14301 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:04.959578 13665 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:04.989979  8369 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:04.986654 14311 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.005s	sys 0.004s
I20260709 13:04:04.992815  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:05.006467  8972 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:04.963505 14168 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:04.996507  8295 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:04.998202  8404 rpcz_store.cc:276] 0709 13:04:03.500224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.500501 (+   277us) service_pool.cc:224] Handling call
0709 13:04:04.969692 (+1469191us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54000febd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287b8853 00007f882d78660a 00007f8826d0c7d4 00007f8826d0c762 00007f8826d0c691 00007f8826d0c60c 00007f8826d0b8c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:04.969864 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2911063808}
I20260709 13:04:04.977991  8323 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:04:05.004232  8249 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:05.000926  8220 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:05.008836  8220 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:04.994993 14181 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:05.010339  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.995625 14145 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:05.014508  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:05.017972  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:05.032975  8255 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
I20260709 13:04:05.020236  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.999946 14116 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:05.025969  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:05.030018  8254 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:04:05.013254  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:04.991516 13865 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:05.001914 14301 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:04.997746 14156 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:05.039091  8972 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:05.050144  8406 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:04:05.050540  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:05.015057 14198 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:05.028812 14059 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:05.047143  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:05.005081 14311 raft_consensus.cc:697] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:05.052122  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:05.046976 13665 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:05.049646 13865 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:05.029412 14073 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:05.033995 14291 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:05.060139 14156 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:05.052809 14353 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:05.094437 14353 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:05.094885 14353 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:05.066684  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:05.062070 14257 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:05.097775  8327 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:04:05.097886 14257 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:05.064754 13757 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:05.069854 14303 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:05.098395 14257 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:05.056713 14152 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 13:04:04.996836 14270 consensus_queue.cc:237] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:05.033293 14342 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:05.079484 14278 consensus_queue.cc:237] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:05.047238 14332 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:05.100978  8405 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:46846
I20260709 13:04:05.007467 14296 consensus_queue.cc:260] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:05.103447 14296 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:05.104710  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:05.058336 14218 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:05.105393  8252 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:04:05.074951  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:05.105813 14311 consensus_queue.cc:237] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:05.080456  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:05.086450  8972 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:05.069974 14347 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:04:05.079133 14073 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.108357 14073 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:05.092391 14291 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.032008 14276 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:05.109617  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:05.087155 14349 consensus_queue.cc:1059] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:05.110059 14291 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:05.110805 14349 raft_consensus.cc:3055] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:05.111246 14349 raft_consensus.cc:740] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:05.112461 14349 consensus_queue.cc:260] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:05.114229 14349 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:05.114987  8398 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:04:05.102730 14332 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:05.116529  8398 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:04:05.125860  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:05.126546  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:05.128680  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:05.129051  9009 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:04:05.129503  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:04:05.130024  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 43682.1us
I20260709 13:04:05.130229  8256 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:05.132282  8402 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:05.133685  8332 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:04:05.134820  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:05.136735  8399 raft_consensus.cc:1240] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:05.137691  8405 raft_consensus.cc:606] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:05.132385  8372 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:05.141804  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:05.142758  8250 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:05.124539 14199 consensus_queue.cc:1059] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 13:04:05.123697 14168 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:05.136632  8369 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:05.170928  8974 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:05.171227  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:05.168602  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:04:05.172410  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 161018us
I20260709 13:04:05.159734 14246 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:05.176218 14246 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.176635 14246 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:05.175609  8972 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:05.167246 14306 consensus_meta.cc:287] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.585s	user 0.012s	sys 0.000s
I20260709 13:04:05.167528 14154 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:05.185040  8256 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:04:05.169374 13962 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.192107  8329 raft_consensus.cc:2410] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:04:05.166613 13757 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:05.191022 14145 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:05.187867  8250 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:04:05.189248  8295 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:05.198855 13962 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:05.199838 14218 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:05.183413 14250 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:05.176229 14297 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:05.197451 14276 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:04:05.208607  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:05.209909  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:05.186810 14351 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:05.230753  8295 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:05.192585 14154 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.209615 14306 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:05.230123 14116 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:05.234467 14297 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:05.280248  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 90303.4us
I20260709 13:04:05.242691 14153 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:05.237061 14351 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.243343 14154 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:05.234573 14218 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:05.292325 14297 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:05.303202 13962 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:05.305636 14200 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:05.307827 14073 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:05.310107 14073 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:05.311038 14088 raft_consensus.cc:3055] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:05.311548 14088 raft_consensus.cc:740] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:05.312368 14088 consensus_queue.cc:260] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:05.240207 14122 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:05.299342 14351 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:05.291016 14306 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:05.318969 14154 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:05.320835 14306 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:05.315366 14088 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:05.278609 14359 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:04:05.318655 14122 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:05.283425 14301 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:05.304991 14257 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:05.309096 14353 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:05.326659 14218 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:05.329614 14301 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:05.330856 14291 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:05.338886 14353 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:05.370455  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.375397  8324 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:05.392424  8400 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:05.399149  8249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 972) took 1423 ms. Trace:
I20260709 13:04:05.399762  8249 rpcz_store.cc:276] 0709 13:04:03.975509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.976420 (+   911us) service_pool.cc:224] Handling call
0709 13:04:03.976706 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.008138 (+1031432us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.398237 (+390099us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400177d54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:05.398730 (+   493us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2362668416}
W20260709 13:04:05.404673  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.410283  9009 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:05.411476  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2533) took 1061 ms. Trace:
I20260709 13:04:05.411931  8399 rpcz_store.cc:276] 0709 13:04:04.350199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.732823 (+382624us) service_pool.cc:224] Handling call
0709 13:04:04.733083 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.137350 (+404267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.410744 (+273394us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b54003a1bd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:05.411178 (+   434us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":927701760}
I20260709 13:04:05.244297 13865 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:05.418535  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:05.419818  8403 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 3101) took 1481 ms (client timeout 1842 ms). Trace:
W20260709 13:04:05.420373  8403 rpcz_store.cc:269] 0709 13:04:03.938541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.938735 (+   194us) service_pool.cc:224] Handling call
0709 13:04:05.418980 (+1480245us) spinlock_profiling.cc:243] Waited 928 ms on lock 0x7b54000febd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:05.419533 (+   553us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2135273728}
I20260709 13:04:05.424289 14111 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:05.425755 14111 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:05.427491 14120 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:05.430688  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:05.435271 14111 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:05.436043 14120 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.330257 14169 raft_consensus.cc:993] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: : Instructing follower 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:04:05.436956 14120 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:05.438300 14169 raft_consensus.cc:1081] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Signalling peer 8c42259d7c4b463a9f485f42c320c81b to start an election
I20260709 13:04:05.438490 14154 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:05.325086 14246 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:05.444500 14291 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:05.428915  8369 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:05.464224  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:05.421131 13865 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:05.446321  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:05.447099 14120 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:05.445416 14246 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:05.447705 14257 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:05.464339 14081 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:05.484011 14066 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:05.485328 14066 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:05.487540  8298 connection.cc:256] RPC call timeout handler was delayed by -1.0222276720000991s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:04:05.475895 14309 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:05.501613 13961 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:05.496404  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 68103us
W20260709 13:04:05.511415  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.510797  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:05.511468 14309 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:05.514853 14309 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:05.520103  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:05.521238  9002 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
I20260709 13:04:05.522058 14309 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:05.525514  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:05.525883 14203 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:05.526793 14203 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:05.530428  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:05.532536  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:05.533751  8327 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 11.
I20260709 13:04:05.505141 13875 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:05.542655 14203 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:05.545388 14122 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.546350  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:05.547365  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 308273us
I20260709 13:04:05.552680  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 951) took 1037 ms. Trace:
I20260709 13:04:05.553367  8404 rpcz_store.cc:276] 0709 13:04:04.514926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.007591 (+492665us) service_pool.cc:224] Handling call
0709 13:04:05.007885 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.057557 (+ 49672us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:05.057607 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:05.057670 (+    63us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:05.057717 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.551719 (+494002us) spinlock_profiling.cc:243] Waited 49.4 ms on lock 0x7b54003a20d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:05.551781 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.552315 (+   534us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113676032}
I20260709 13:04:05.512635  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2430) took 1080 ms. Trace:
I20260709 13:04:05.558228  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:05.562054 14359 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.562318  8325 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:05.557228  8248 rpcz_store.cc:276] 0709 13:04:04.385687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.385826 (+   139us) service_pool.cc:224] Handling call
0709 13:04:04.386126 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:04.558764 (+172638us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:04.558797 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:04.558848 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:04.558881 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.396985 (+838104us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b540026c854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:05.397082 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.466291 (+ 69209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":396534656}
I20260709 13:04:05.552372 14122 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:05.577812  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.580188  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:05.579049  8323 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:04:05.563566 14359 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:05.577284  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:05.584714  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:05.587437  8328 raft_consensus.cc:2410] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:04:05.599248 14088 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.575027 14120 consensus_queue.cc:1059] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:04:05.612520  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:05.623081  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:05.608628 14122 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:05.632362  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2541) took 1165 ms. Trace:
I20260709 13:04:05.638098  8400 rpcz_store.cc:276] 0709 13:04:04.467436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.949930 (+482494us) service_pool.cc:224] Handling call
0709 13:04:04.950233 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.393146 (+442913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.631966 (+238820us) spinlock_profiling.cc:243] Waited 442 ms on lock 0x7b54003e7954. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:05.632295 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1015600000}
I20260709 13:04:05.639299 14088 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:05.624980  8369 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:05.643362 14111 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:05.644353 14298 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:05.644218 14111 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:05.654454 14066 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:05.657838  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:05.665949 14111 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:05.668439 14088 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:05.645157 14298 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:05.671772 14298 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:05.674681 14372 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 13:04:05.677137  8298 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:05.678292  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 68146.6us
I20260709 13:04:05.681725 14088 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:05.682238 14088 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.682658 14088 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:05.677189  8295 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:05.688038 14122 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:05.694269  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 100876us
I20260709 13:04:05.694504 13961 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:05.695747  8369 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:05.697253 13961 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:05.702392 14090 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:05.703616 14090 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:05.705361 14359 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:05.707089  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:05.707724 14090 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:05.707952  8403 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:04:05.710574  8403 raft_consensus.cc:1240] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:05.711386 14359 consensus_queue.cc:1059] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:05.712733  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:05.712917 14369 raft_consensus.cc:3055] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:05.713696 14369 raft_consensus.cc:740] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:05.714219  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:05.714632  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:05.715471  8406 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:04:05.714993 14369 consensus_queue.cc:260] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:05.716787  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:05.718513  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:05.718132  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:05.719653  8397 raft_consensus.cc:1813] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Handling vote request from an unknown peer 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:05.722787  8397 raft_consensus.cc:2410] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:04:05.719709  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:05.725222  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:05.716862 14369 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:05.697584  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:05.739284  8325 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:04:05.752723  9006 raft_consensus.cc:2410] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:04:05.755335  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:05.756246  8403 raft_consensus.cc:1275] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:04:05.749024  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:05.759806  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:05.761139 14088 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:05.762734  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:05.764104  8402 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:04:05.764216  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:05.764643  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:05.765362  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:05.765842  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:05.766140  8406 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:05.785284  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:05.791209  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:05.791886  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:05.786612  8405 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:05.777882 14203 raft_consensus.cc:3055] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:04:05.802695  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.809137  8402 raft_consensus.cc:1240] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:05.814014 14342 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:05.815218 13757 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:05.817655 14088 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:05.818535 14297 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:05.821877  9005 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
W20260709 13:04:05.830521  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 133101us
I20260709 13:04:05.832659  8222 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:05.807535  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:05.838222  8401 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:04:05.833222  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:05.846801  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:05.847563  9002 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:05.821250 14297 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:05.802263 14203 raft_consensus.cc:740] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:05.858757  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 181734us
I20260709 13:04:05.857776  8220 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:05.866338 14065 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:05.884191  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 82057us
I20260709 13:04:05.861297 14088 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:04:05.895948  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 280802us
I20260709 13:04:05.899186  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:05.903373 14065 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:05.883754 14372 consensus_queue.cc:1059] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:04:05.913534 14059 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:05.915158  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:05.919423  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 86387us
W20260709 13:04:05.920207  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 87956.3us
I20260709 13:04:05.919683  9003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 1171) took 1055 ms. Trace:
I20260709 13:04:05.921336  8220 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:05.924760  8404 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 22.
W20260709 13:04:05.961525  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.963089 14353 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:04:05.968796  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 90587.1us
I20260709 13:04:05.963614 14353 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.971282 14353 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:05.977820  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:05.974279 14145 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.106s
I20260709 13:04:05.963599 14122 raft_consensus.cc:3055] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:05.994292  8249 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:04:05.925848 14088 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:05.960695 14372 consensus_queue.cc:1059] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:04:06.004446  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:06.010926  8248 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:04:05.988130 14122 raft_consensus.cc:740] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:06.013387  9002 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 21.
I20260709 13:04:06.013913  8297 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:05.997867  8295 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:06.015651  8406 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
W20260709 13:04:06.022549  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.019980 14122 consensus_queue.cc:260] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:05.922443  9003 rpcz_store.cc:276] 0709 13:04:04.863725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.863850 (+   125us) service_pool.cc:224] Handling call
0709 13:04:04.864180 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:04.877206 (+ 13026us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:04.877239 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:05.551034 (+673795us) spinlock_profiling.cc:243] Waited 8.44 ms on lock 0x7b1800470920. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882569f2a6 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a
0709 13:04:05.629956 (+ 78922us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b2c00047a38. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:04:05.630746 (+   790us) log.cc:844] Serialized 26 byte log entry
0709 13:04:05.768773 (+138027us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:05.768813 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.773788 (+  4975us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:05.916801 (+143013us) raft_consensus.cc:1685] finished
0709 13:04:05.916838 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.919423 (+  2585us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12410,"raft.queue_time_us":17503,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":9935,"spinlock_wait_cycles":58470528,"thread_start_us":111039,"threads_started":1,"wal-append.queue_time_us":171188}
I20260709 13:04:05.989099 14203 consensus_queue.cc:260] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:06.035295 14372 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:06.036583  9005 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 13:04:06.037031  9005 pending_rounds.cc:85] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: Aborting all ops after (but not including) 14
I20260709 13:04:06.035266 14116 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:06.037643  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 58371.7us
I20260709 13:04:06.037305  9005 pending_rounds.cc:107] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:04:06.039491  9005 raft_consensus.cc:2889] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:06.043354  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:06.044222  8256 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:06.044466 14088 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:06.046733  8220 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:06.051151 14298 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:06.055508 14203 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:06.056375  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:06.057207 13865 raft_consensus.cc:3055] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:06.057829 13865 raft_consensus.cc:740] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:06.058871 13865 consensus_queue.cc:260] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:06.060540 13865 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:06.052316 14351 consensus_meta.cc:287] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.657s	user 0.018s	sys 0.000s
I20260709 13:04:06.062795 14351 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:06.065816  9009 raft_consensus.cc:2393] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 19: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:06.074227  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:06.077932  8369 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:06.073266  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 59453.9us
I20260709 13:04:06.080147 14059 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:06.081893  8297 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:06.065332 14351 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:06.086894 14116 consensus_queue.cc:1059] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:06.087505 14088 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:06.089828 14372 raft_consensus.cc:3055] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:06.089975 14088 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.090309 14372 raft_consensus.cc:740] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:06.090451 14088 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:06.080562 14122 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:06.091377 14372 consensus_queue.cc:260] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:06.092927 14372 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:06.098826 14065 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:06.098384 14353 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:06.075156 14376 log.cc:927] Time spent T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.144s	user 0.006s	sys 0.000s
W20260709 13:04:06.111153  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 127074us
I20260709 13:04:06.088167 14120 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:06.117130 14065 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:06.117789 14065 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:06.121542  8248 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:04:06.127182  8297 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:06.129914 14065 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:06.125877  8298 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:06.139701 14359 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.140291 14359 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:06.142417 14359 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:06.143054 14359 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:06.143555 14359 raft_consensus.cc:697] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 21 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:06.144575 14359 consensus_queue.cc:237] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:06.150203  8406 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:04:06.117058 14120 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:06.154559  8256 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:04:06.162289  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:06.175541 14120 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:06.178314  8222 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:06.180784 14111 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:06.181373 14111 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:06.181814 14111 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:06.186932  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 75952.6us
I20260709 13:04:06.206657 14257 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:06.205910 14116 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:06.220865  8402 raft_consensus.cc:2393] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 19: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:06.213204 14298 raft_consensus.cc:697] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:06.234071  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 119970us
I20260709 13:04:06.226277 14120 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:06.241671  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:06.245337  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:06.228456 14116 raft_consensus.cc:697] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:06.231482 14122 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:06.222683 14088 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:06.250345  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 69503.2us
I20260709 13:04:06.252966  8248 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:06.252523  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 129830us
I20260709 13:04:06.233778 14351 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 13:04:06.277545 14111 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:06.261896 14122 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:06.263197 14116 consensus_queue.cc:237] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:06.284408  8256 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:04:06.260733 14298 consensus_queue.cc:237] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:06.285682  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 81256us
I20260709 13:04:06.265498 14353 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:06.284475 14122 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:06.228206 14257 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:06.294759 14257 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:06.269753  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:06.295686  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:06.296449  9005 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:04:06.297930  9001 raft_consensus.cc:2393] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 21: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:06.298702 14111 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:06.272996 14088 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:06.244259 14065 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 13:04:06.279610  8369 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:06.281767 14120 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:06.301569 14369 consensus_meta.cc:287] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.553s	user 0.006s	sys 0.006s
I20260709 13:04:06.309558  8298 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:06.312295 14246 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:06.328673 14246 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:06.330116  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:06.276708  8248 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 16.
I20260709 13:04:06.326828  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:04:06.328207  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.333863  8252 raft_consensus.cc:2393] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 19: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:06.331264 14246 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:06.337610  8222 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:06.338888  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.342929  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:06.343634  8252 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:04:06.348380  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:06.350425 14381 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:06.350977 14381 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:06.351372 14381 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:06.354171 14059 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:06.354341 14257 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:06.354885 14257 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:06.354833 14059 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:06.355585 14257 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:06.341877 14309 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:06.358089 14309 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:06.360338 14309 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:06.361990  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 106451us
I20260709 13:04:06.362891  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:06.363257  8253 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:04:06.364382  8369 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:06.365307 14246 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:06.366074 14246 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:06.375422 14120 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:06.375893  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 119457us
I20260709 13:04:06.376201 14120 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:06.376685 14120 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:06.377128 14120 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:06.383811  8222 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:06.377775 14378 consensus_queue.cc:1059] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:06.385344  8250 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:04:06.385865  8249 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:04:06.390249  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:06.390823  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:06.385114  8155 catalog_manager.cc:5697] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 4 to 6, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 6 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:06.391611  8254 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:04:06.391495  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:06.313724 14359 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:04:06.394573  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.402208 14378 raft_consensus.cc:3055] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:06.402875 14378 raft_consensus.cc:740] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:06.399396  8257 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
I20260709 13:04:06.357306 14059 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:06.418569  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:06.394798 14369 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:06.423619  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:06.424557  8369 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:06.429464  8252 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:06.406216 14359 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:06.432355  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 146424us
W20260709 13:04:06.432717  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.434270 14359 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:06.435278 14384 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:06.436216 14359 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:06.436152 14384 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:06.438339  8329 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
W20260709 13:04:06.446772  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.444166  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:06.447896  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:06.448450  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:06.444242 13865 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.450114  8328 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:06.450012 13865 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:06.452095  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.452212 13865 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:06.452884 14122 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:04:06.453958  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 57125.3us
I20260709 13:04:06.453734 14122 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:06.455971 14122 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:06.458731  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:06.448477  8406 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:06.461707 14059 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:04:06.462224  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 84794.9us
I20260709 13:04:06.444593  8325 raft_consensus.cc:1275] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:04:06.430495 14298 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:06.476253  8328 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 21 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 19. Current term is 21. Ops: []
W20260709 13:04:06.481652  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 48017.5us
I20260709 13:04:06.486100  8323 raft_consensus.cc:1240] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:06.486122  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:06.437209 14359 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:06.494530  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:06.495612  8327 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:04:06.501574  8222 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:06.503149  8222 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:06.505626 14379 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:06.506417 14379 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:06.506745 14379 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:06.507074 14379 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:06.510643  9002 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:06.512665  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:06.515304 14257 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:06.483246 14059 raft_consensus.cc:697] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 18 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:06.517100 14059 consensus_queue.cc:237] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:06.482719 13865 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:06.520893 13865 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:06.523545 13865 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:06.423763 14378 consensus_queue.cc:260] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:06.526579 14385 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:06.527244 14378 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:06.430745 14369 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:06.529791  8972 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:06.530663 14351 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:06.531198 14369 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:06.533771 14309 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:06.534269 14309 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:04:06.536113  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.523792  8248 raft_consensus.cc:2410] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 23 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:04:06.531874  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:06.532611  8155 catalog_manager.cc:5697] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 2 to 5, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 5 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:06.548704  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:06.550428  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:06.551338  8325 raft_consensus.cc:2393] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 18: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:06.550585  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:06.553083  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:06.554813  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:06.553710  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:06.559835  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:06.534723 14309 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:06.561148  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:06.532615 14257 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:06.565302  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 60119.3us
I20260709 13:04:06.569690  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:06.570034  8220 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:06.570304  8331 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:06.571231  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:06.572060  8325 raft_consensus.cc:2410] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:04:06.572715 14353 consensus_queue.cc:1059] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:06.573890 14353 raft_consensus.cc:3055] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:06.574244 14353 raft_consensus.cc:740] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:06.559921 14298 consensus_queue.cc:1048] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:04:06.575122 14353 consensus_queue.cc:260] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:06.575800 14381 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:06.576532 14353 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:06.576718  8401 raft_consensus.cc:2393] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 17: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:06.569276  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:06.565939 14297 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:06.587505  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:06.585101 14065 consensus_queue.cc:1059] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 13:04:06.591766 14065 raft_consensus.cc:3055] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:06.592161 14065 raft_consensus.cc:740] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:06.597787 14297 raft_consensus.cc:697] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:06.599980 14297 consensus_queue.cc:237] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:06.601619  8331 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:04:06.604909 14392 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:06.593982 14298 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:06.605682 14298 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:06.606118 14298 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:06.609160  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:06.593793 14384 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:06.608287 14392 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:06.611819  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:06.612850  8329 raft_consensus.cc:2393] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 17: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:06.615087  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:06.634382  8406 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:06.642163 14120 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:06.648056 14379 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:06.652416  8255 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:06.652262 14381 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:06.666628  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:06.667850  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:06.668437  9001 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:06.593037 14065 consensus_queue.cc:260] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:06.683492  9001 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:04:06.685621  8328 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 13:04:06.686715  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 228048us
I20260709 13:04:06.673223 14120 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:06.690881 14088 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:06.698087 14088 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:06.700781 14270 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.701540 14270 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:06.697139 14111 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.705555 14111 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:06.708395 14111 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:06.692097  8298 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:06.700987 14088 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:06.727098  8255 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:06.731611 14065 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:06.744318  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:06.745247  8331 raft_consensus.cc:2410] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:04:06.745544  8325 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:04:06.746083  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:06.746412  8323 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:04:06.748111  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:06.748677  8253 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:06.751519  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:06.753254  8253 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:06.755406  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:06.756911  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 187953us
I20260709 13:04:06.758826  8222 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:06.759831 14111 consensus_queue.cc:1048] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:04:06.761708  8147 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45448 (request call id 286) took 15770 ms (client timeout 15000 ms). Trace:
I20260709 13:04:06.764567 14111 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:06.765225  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:06.760358  8222 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:06.778440 14359 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:06.778098 14111 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:06.779663  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 89905.8us
W20260709 13:04:06.785529  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.786150  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:06.782634  8222 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:06.786936 14359 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:06.788769 14359 raft_consensus.cc:697] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:06.775734 14401 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.728615 14270 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:06.808983 14401 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:06.810438  8402 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 21.
I20260709 13:04:06.812183 14401 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:06.798825 14384 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:06.800120  9005 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:04:06.805440  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 77286.5us
I20260709 13:04:06.813215 14384 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:06.813719  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:06.814373  9003 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
I20260709 13:04:06.814558  8369 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:06.815696 14384 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:06.816263 14384 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:04:06.816694  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:06.816883  8254 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:55264
I20260709 13:04:06.817258  8329 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:06.817656  8254 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:06.818135  8254 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:06.817584  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:06.808209 14379 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:06.818856  8253 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:06.819666  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:06.820262  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:06.822659  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:06.825413  8295 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:06.827179  9005 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 21: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:06.829460  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:06.829859  8397 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:04:06.827323  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:06.831153  8974 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:06.833724  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:06.824985  8372 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:06.829452  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:06.834905 14120 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:06.835311  8248 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:04:06.836097  8331 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:04:06.836688 14398 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:06.837144 14398 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.837535 14398 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:06.837709  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:06.839433  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:06.841562  8220 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:06.842444 14120 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:06.843076 14120 raft_consensus.cc:697] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:06.842849  8972 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:06.848129 14392 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:06.848186 14270 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:06.859627  8253 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:04:06.859694  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:06.851254  8329 raft_consensus.cc:2410] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:04:06.860736  8323 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:04:06.853343  8254 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:06.854149  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 71587.7us
I20260709 13:04:06.863438 14381 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:06.864027  8248 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:06.863746  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:06.864590  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:06.870219  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:06.818364 14111 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:06.877616  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:06.826201 14401 raft_consensus.cc:2764] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:04:06.886163  8404 raft_consensus.cc:2368] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 18. Current term is 19.
I20260709 13:04:06.852491 14120 consensus_queue.cc:237] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:06.890586  8297 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:06.918962  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:06.919133  8297 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:06.919610  8398 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:04:06.923476  8254 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:06.762322  8147 rpcz_store.cc:269] 0709 13:03:50.991549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.991826 (+   277us) service_pool.cc:224] Handling call
0709 13:04:06.761378 (+15769552us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:00.491945 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:00.622002 (+130057us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:00.622060 (+    58us) write_op.cc:435] Acquired schema lock
0709 13:04:00.622079 (+    19us) tablet.cc:662] Decoding operations
0709 13:04:00.624465 (+  2386us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:00.624585 (+   120us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:00.624617 (+    32us) tablet.cc:685] Acquiring locks for 31 operations
0709 13:04:00.969095 (+344478us) tablet.cc:701] Row locks acquired
0709 13:04:00.969155 (+    60us) write_op.cc:260] PREPARE: finished
0709 13:04:00.969362 (+   207us) write_op.cc:270] Start()
0709 13:04:00.969556 (+   194us) write_op.cc:276] Timestamp: P: 1783602240969276 usec, L: 0
0709 13:04:00.969584 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:04:01.065893 (+ 96309us) log.cc:844] Serialized 11972 byte log entry
0709 13:04:01.547754 (+481861us) op_driver.cc:464] REPLICATION: finished
0709 13:04:01.962400 (+414646us) write_op.cc:301] APPLY: starting
0709 13:04:01.990439 (+ 28039us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:03.167791 (+1177352us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:03.167910 (+   119us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:03.740982 (+573072us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:03.741333 (+   351us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=62,key_file_lookups=62,delta_file_lookups=62,mrs_lookups=0
0709 13:04:03.741377 (+    44us) write_op.cc:312] APPLY: finished
0709 13:04:03.812029 (+ 70652us) log.cc:844] Serialized 268 byte log entry
0709 13:04:04.024090 (+212061us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:04.087207 (+ 63117us) write_op.cc:454] Released schema lock
0709 13:04:04.099146 (+ 11939us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:04.191357 (+ 92211us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:04.193694 (+  2337us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":520011,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":359876,"apply.run_cpu_time_us":89771,"apply.run_wall_time_us":2236655,"cfile_cache_hit":126,"cfile_cache_hit_bytes":133769,"num_ops":31,"prepare.queue_time_us":374449,"prepare.run_cpu_time_us":39992,"prepare.run_wall_time_us":886022,"raft.queue_time_us":86552,"raft.run_cpu_time_us":21419,"raft.run_wall_time_us":159146,"replication_time_us":577962,"spinlock_wait_cycles":29952,"thread_start_us":827086,"threads_started":5,"wal-append.queue_time_us":658102,"wal-append.run_cpu_time_us":14204,"wal-append.run_wall_time_us":2564724}]]}
I20260709 13:04:06.891731  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:06.891816 14116 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.927920  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:06.928212 14116 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:06.929557  8406 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:06.930015  8255 raft_consensus.cc:1240] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:06.930676  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:06.933588  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:06.892256  8295 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:06.891242 14369 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:06.937882  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:06.937953 14369 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:06.900708 14392 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:06.901444 14359 consensus_queue.cc:237] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:06.857616 14309 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:06.916096  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:06.916951  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:06.917151  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:06.864627  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:06.916951  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:06.921056  8297 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:06.881464 14298 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:06.926822 14088 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:06.927582 14393 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:06.927467  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:06.942456  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 50292.1us
I20260709 13:04:06.978121  8295 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:06.946909  8222 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:06.980203  8222 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:06.957259 14384 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:06.930867 14116 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:06.989802  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:06.991285 13865 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:06.991742 13865 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:06.992093 13865 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:06.975775  8332 raft_consensus.cc:2368] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 11. Current term is 12.
I20260709 13:04:06.993743  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:06.994556  8369 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:06.995893  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:06.997241 14257 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:06.997804 14059 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:06.998473 14059 raft_consensus.cc:697] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:06.999558 14059 consensus_queue.cc:237] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:06.955948  8253 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:07.003244  8295 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:07.004190  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 161442us
I20260709 13:04:06.984807  8329 raft_consensus.cc:1240] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:06.987186  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:06.987694 14111 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:06.989979 14309 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:06.997817 14257 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:06.981913  8297 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:07.017844  8256 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:06.989212  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:06.961954  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:07.014458  8295 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:07.020121  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:07.020162  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:07.020780  8399 raft_consensus.cc:3055] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:06.966329 14381 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:07.021296  8399 raft_consensus.cc:740] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:07.001706 14246 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:06.976673 14369 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:07.004523 14353 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:07.060461  8222 leader_election.cc:400] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 18. Current term is 19.
I20260709 13:04:07.026885  8397 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:04:06.975683 14406 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:07.071563 14406 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:07.073848 14406 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:07.074150  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:07.060904  8222 leader_election.cc:403] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:07.074853  8405 raft_consensus.cc:2368] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 16. Current term is 17.
I20260709 13:04:07.077620 14359 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.067382 13865 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:07.034528 14380 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:07.079617  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:07.080358 13865 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:07.082710 14257 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:07.083798 14413 log.cc:927] Time spent T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:04:07.085525  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:07.086361  8406 raft_consensus.cc:2393] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 17: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:07.088076  8402 raft_consensus.cc:3055] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:07.088460  8402 raft_consensus.cc:740] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:07.089282  8402 consensus_queue.cc:260] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:07.090298  8402 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:07.030670 14384 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:07.036190  8295 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:07.069841  8399 consensus_queue.cc:260] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:07.070283 14270 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:07.061403  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 58505us
I20260709 13:04:07.078166 14359 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:07.101096  8399 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:07.100579  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 64515.2us
I20260709 13:04:07.096414  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:07.103626  8295 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:07.104035  8329 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
W20260709 13:04:07.107175  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 45104.3us
I20260709 13:04:07.107955  8295 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:07.109326  8398 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:04:07.114509  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:07.095360 14384 raft_consensus.cc:697] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:07.118036  9001 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14. Ops: []
W20260709 13:04:07.118384  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 115515us
I20260709 13:04:07.117751 13865 raft_consensus.cc:2764] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:04:07.123660 14116 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:07.124511 14298 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:07.124681 14359 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:07.119102 14246 raft_consensus.cc:697] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 19 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:07.107225 14270 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:07.117640 14381 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:07.097958 14088 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:07.130663  8972 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:07.135514  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:07.140663  8256 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:04:07.141283  8253 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:04:07.140704  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:07.136521  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.143687  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:07.144362  8256 raft_consensus.cc:2393] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 17: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:07.145133  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:07.146281  8222 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:07.145850  8252 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:04:07.148448  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:07.149791  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:07.152104  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:07.152626  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:07.152877  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 171059us
I20260709 13:04:07.152828  8257 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:07.153707  8297 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:07.153869  8222 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:07.155455  8222 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:07.157003  8222 leader_election.cc:400] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 16. Current term is 17.
I20260709 13:04:07.138448 14378 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.157353  8222 leader_election.cc:403] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:04:07.167106 14351 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:07.170238  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:07.171167  8972 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:07.173955  8331 raft_consensus.cc:1275] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:04:07.189306  8253 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:04:07.189455  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:07.145157 14390 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 18. Current term is 19.
I20260709 13:04:07.188545 14066 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:04:07.194268  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:07.246304  8330 raft_consensus.cc:2410] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:04:07.194941  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:04:07.195349  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.158005 14298 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:07.199815  8253 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:04:07.225931  8402 raft_consensus.cc:1275] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:04:07.250020  8402 pending_rounds.cc:85] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 7
I20260709 13:04:07.250267  8402 pending_rounds.cc:107] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:04:07.250504  8402 raft_consensus.cc:2889] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:07.161010 13865 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:07.251854  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:07.252737  8323 raft_consensus.cc:2393] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 17: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:07.162889 14406 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:07.165650 14379 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:04:07.224859  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 68034.7us
I20260709 13:04:07.159622 14297 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:07.145223 14369 consensus_queue.cc:1059] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:04:07.164714 14392 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:07.176402 14353 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 16. Current term is 17.
I20260709 13:04:07.200944 14145 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:07.162555 14065 consensus_queue.cc:1059] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.091s
I20260709 13:04:07.168387 14378 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:07.301569  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:07.200511 14381 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:04:07.254043 14309 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:07.303875  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:07.250797 14066 consensus_queue.cc:1243] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer ff7a6fe533e94c1bae555b4cae427728 log is divergent from this leader: its last log entry 19.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:04:07.310226  8399 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
W20260709 13:04:07.314898  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 166495us
I20260709 13:04:07.317224  8369 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:07.319562  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:07.320240  8398 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:04:07.325882  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:07.326581  8331 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 20.
I20260709 13:04:07.348268  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:07.351460  8295 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:07.353320  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:07.354194  8325 raft_consensus.cc:2410] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:04:07.263965 14384 consensus_queue.cc:237] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:07.308351 14120 consensus_queue.cc:1048] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:04:07.363018  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:07.364902  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 47797.9us
I20260709 13:04:07.366185  8369 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:07.367178  8295 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:07.370978 14423 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:07.369556  8295 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:07.371095 14420 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:07.372140 14369 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:07.372587 14369 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.372946 14369 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:07.373138 14419 consensus_queue.cc:1059] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 13:04:07.375294 14417 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:04:07.375576 14411 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 13:04:07.376266 13865 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:07.376704 13865 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.377061 13865 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:07.378157 14414 log.cc:927] Time spent T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:04:07.383867  8295 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:07.382800 14369 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:07.385941 14418 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:07.386754 14420 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:04:07.354516 14309 raft_consensus.cc:697] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:07.321974 14246 consensus_queue.cc:237] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:07.421358 14386 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:04:07.421454 14066 consensus_queue.cc:1059] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 13:04:07.426587 14066 raft_consensus.cc:3055] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:07.426990 14066 raft_consensus.cc:740] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:07.427932 14066 consensus_queue.cc:260] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:07.355093 14359 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:04:07.430480  8331 pending_rounds.cc:85] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 11
W20260709 13:04:07.429738  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 162642us
I20260709 13:04:07.431478  8331 pending_rounds.cc:107] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 13:04:07.431885  8331 raft_consensus.cc:2889] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:07.349836  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 189659us
I20260709 13:04:07.413416 14369 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:07.438249  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:07.439090  8257 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:07.419023 14393 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.440598  8222 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:07.429409 14066 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:07.449204  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 83095.5us
I20260709 13:04:07.435055 14088 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.455134  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:07.437103 14423 raft_consensus.cc:697] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 19 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:04:07.450701  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.457741  8972 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:07.445307 14379 raft_consensus.cc:697] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:07.457859 14398 consensus_meta.cc:287] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.620s	user 0.012s	sys 0.000s
I20260709 13:04:07.472545  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:07.462677 14309 consensus_queue.cc:237] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:07.475075 14379 consensus_queue.cc:237] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:07.458468 14393 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:07.465634 14145 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:07.482666  8257 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
I20260709 13:04:07.456171 14088 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:07.467398 14378 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:07.485287  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:07.472150 14203 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:07.507333 14257 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:07.509253 14378 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:07.511552 14423 consensus_queue.cc:237] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:07.512594  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 49170.1us
I20260709 13:04:07.470060 14398 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:07.535009  8369 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:07.521034 14257 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.538838  8329 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 13:04:07.539209  8329 pending_rounds.cc:85] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 10
I20260709 13:04:07.541626  8329 pending_rounds.cc:107] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:04:07.541988  8329 raft_consensus.cc:2889] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:07.546743 14429 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:07.547389 14429 raft_consensus.cc:697] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:07.548835 14088 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:07.529891 14203 raft_consensus.cc:697] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:07.578992  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:07.577198  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:07.581199  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:07.582260  8400 raft_consensus.cc:2368] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 21. Current term is 22.
I20260709 13:04:07.582151  8249 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:07.583042  8327 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:07.536298 14257 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:07.587126  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:07.588361  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:07.591269 14122 raft_consensus.cc:3055] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:07.591760 14122 raft_consensus.cc:740] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:07.592620 14122 consensus_queue.cc:260] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:07.594460 14417 raft_consensus.cc:3055] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:07.594883 14417 raft_consensus.cc:740] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:04:07.595096 14380 consensus_meta.cc:287] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.516s	user 0.004s	sys 0.005s
I20260709 13:04:07.595608 14380 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:07.599617  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:07.600335  8398 raft_consensus.cc:2393] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 17: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:07.601166  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:07.601784 14246 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:07.602341 14246 raft_consensus.cc:697] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 18 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:07.602212 13865 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:07.603955  8329 raft_consensus.cc:2368] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 18. Current term is 19.
I20260709 13:04:07.605201  8249 raft_consensus.cc:2393] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 18: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:07.540535 14309 consensus_queue.cc:1059] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:04:07.618867  8369 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:07.621217  8332 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:04:07.539254 14378 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:07.622246  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:07.637571 14428 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:07.603372 14246 consensus_queue.cc:237] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:07.640050 14122 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:07.655488 14203 consensus_queue.cc:237] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:07.661623 14425 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.662238 14425 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:07.595819 14417 consensus_queue.cc:260] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:07.664798 14417 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:07.669879 13865 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:07.670621 14257 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:07.673063 14435 log.cc:927] Time spent T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
W20260709 13:04:07.677237 14426 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:04:07.548314 14429 consensus_queue.cc:237] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:07.686697 14401 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:07.703086 14436 log.cc:927] Time spent T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.159s	user 0.003s	sys 0.000s
I20260709 13:04:07.727365  8253 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:07.728938  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:07.729975  8252 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:07.732609  8295 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:07.731406 14393 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:07.745134 14393 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:07.745671 14393 raft_consensus.cc:697] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:07.746577 14393 consensus_queue.cc:237] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:07.725036 14385 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:07.733605 14419 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:07.766692  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 111324us
I20260709 13:04:07.767740  8369 leader_election.cc:400] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 11. Current term is 12.
I20260709 13:04:07.768260  8369 leader_election.cc:403] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:07.777475 14378 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:07.779135 14270 consensus_meta.cc:287] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.632s	user 0.010s	sys 0.002s
I20260709 13:04:07.765758 14385 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:07.741678 14380 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:07.808549  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.810746  8369 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:07.821952  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:07.830560  8297 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:07.842818  8220 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:07.839671 14451 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:07.811574 14059 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 11. Current term is 12.
I20260709 13:04:07.824420 14379 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.815807 14270 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:07.858326  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:07.858619 14445 consensus_queue.cc:1048] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:04:07.860870 14385 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:07.874280  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 43826.9us
I20260709 13:04:07.879998  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:07.881203  8252 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
W20260709 13:04:07.844647  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 112120us
I20260709 13:04:07.903597  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:07.857589 13865 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:07.899616 14425 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:07.911816  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:07.901088 14257 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:07.916616  8326 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:04:07.966323  8326 pending_rounds.cc:85] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 8
I20260709 13:04:07.966626  8326 pending_rounds.cc:107] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:04:07.966851  8326 raft_consensus.cc:2889] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:07.968004  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:07.908905 14059 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:07.911811  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:07.982666  8325 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:07.896145 14449 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:07.824343 14380 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:07.843547 14447 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:07.908857 14451 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:07.908267  8369 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:07.940920  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:07.913610  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:04:07.945190  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 219667us
I20260709 13:04:07.880100 14379 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:07.943730  8295 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:07.989188  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 81000us
I20260709 13:04:07.989861  8332 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 22
W20260709 13:04:07.991539  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 47933.6us
I20260709 13:04:07.994623  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:07.995396  8400 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:07.997855  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:08.003923  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:07.990466 14059 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:08.021243 14059 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:08.009473 14447 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:08.021839 14455 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:04:08.018160  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 146334us
I20260709 13:04:08.018330  8295 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:08.037359  8332 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 22.
I20260709 13:04:08.040081  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:08.032137  8220 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:08.045344  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.043251  8250 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:04:08.007853 13865 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:08.067881 14460 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.068512 14460 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:08.044718 14088 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:08.070065 14088 raft_consensus.cc:697] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:08.070807 14460 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:08.071213 14088 consensus_queue.cc:237] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:08.073626 14459 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.074285 14459 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:08.076773 14459 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:08.079246 14145 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:08.079874 14270 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:08.079795 14145 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:08.054817 13865 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:08.080865 13865 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:08.082518  8297 leader_election.cc:400] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 21. Current term is 22.
I20260709 13:04:08.082862  8297 leader_election.cc:403] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:08.092231  8325 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:04:08.104794  8399 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:04:08.117928  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:08.118970  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 86924.4us
I20260709 13:04:08.091907 14459 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 21. Current term is 22.
I20260709 13:04:08.120302  8220 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:08.129308  8220 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:08.124107 14390 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.133289 14464 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.133244 14390 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:08.134160 14464 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:08.137600 14445 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:08.138921  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:08.140813  9003 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:04:08.142164 14145 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:08.143244 14406 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:08.181798  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 55820.4us
I20260709 13:04:08.112808 14417 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:08.181025  8298 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:08.221086  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 89239.1us
W20260709 13:04:08.233944  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 104710us
I20260709 13:04:08.270952  8405 raft_consensus.cc:1275] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:04:08.280692  8406 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16. Ops: []
W20260709 13:04:08.221709 14437 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.057s	sys 0.062s
I20260709 13:04:08.277252 14417 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:08.298614  8257 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:04:08.228382 14298 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:08.300499  8254 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:04:08.302196  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 121903us
I20260709 13:04:08.302460  8400 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:08.302700  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:08.222213 14460 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:04:08.294747 14437 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.057s	sys 0.063s
I20260709 13:04:08.313948  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:08.317803  9001 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:04:08.319638  8220 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:08.304461 14463 log.cc:927] Time spent T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:04:08.327153  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:08.325809  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:08.332639  8325 raft_consensus.cc:1275] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:04:08.337337  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:08.337996  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:08.339120  8325 raft_consensus.cc:2410] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:04:08.305608 14459 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:08.345558  9003 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:04:08.351598  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:08.356076  8330 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:08.352306  8331 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:04:08.358410  8372 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:08.311767 14393 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:04:08.326092 14460 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:08.436864  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 131417us
I20260709 13:04:08.360086 14298 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:08.406111 13757 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:08.446414 13757 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:08.441510 14425 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 13:04:08.450551 14270 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.456272 14145 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.459703 14122 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:08.461083 13757 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:08.489820  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 48438.1us
I20260709 13:04:08.488551 14145 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:08.503072 14145 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:08.540952  8220 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:08.493063 14456 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:08.544160 14470 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:08.545074 14120 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:08.544741 14470 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:08.485755 14270 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:08.493384 14434 consensus_queue.cc:1059] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:04:08.547885 14120 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:08.548484 14466 log.cc:927] Time spent T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 13:04:08.548287  8220 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:08.551445 14474 log.cc:927] Time spent T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
W20260709 13:04:08.541693 14477 log.cc:927] Time spent T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:04:08.505378 14122 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:08.567756 14478 log.cc:927] Time spent T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:04:08.123634  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:08.558889 14145 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:08.578886 14145 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:08.582737 14120 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:08.584400 14145 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:08.612684 14465 consensus_queue.cc:1048] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:04:08.614706 14270 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:08.623270  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 75040.9us
I20260709 13:04:08.614554 14487 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:08.606902 14460 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:08.611156 14486 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:08.657368 14486 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:08.657981 14486 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:08.660722 14122 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:08.637403 14470 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:08.667076 13865 consensus_meta.cc:287] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.586s	user 0.003s	sys 0.008s
I20260709 13:04:08.667608 13865 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:08.746968 14120 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.810874  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:08.846446  8325 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 22: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:08.746204 14423 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:08.801918 14492 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:08.899353  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:08.915985  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:08.917258  8328 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 20.
I20260709 13:04:08.803144 13757 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:08.865020 13865 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:08.906937 14465 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.906332 14392 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:08.914301 14066 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:08.950920 14066 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:08.955201  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:08.956007  9003 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:04:08.955838 14392 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:08.959867 14392 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:08.937723 14451 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:08.970971 14451 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:08.944679 14465 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:08.978440 14465 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:08.853791 14120 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:08.969844  9003 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:04:08.866071 14492 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:08.855736 14423 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:08.992213  8372 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:08.953230 13757 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:08.991036 14487 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:08.999363 14487 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:09.001252 14492 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:08.995867 14420 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.006232 14420 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:09.006908 14120 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:09.008549 14420 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:08.992686 14353 raft_consensus.cc:3055] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:09.012059 14066 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:09.013885 13757 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:04:09.035876  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 43751.9us
I20260709 13:04:09.018303 14353 raft_consensus.cc:740] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:09.039608 14487 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:09.038689 14451 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:09.114280 13757 raft_consensus.cc:697] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:04:09.123560 14497 log.cc:927] Time spent T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:04:09.123658 13757 consensus_queue.cc:237] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:09.108976 14465 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:09.023546 14486 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:09.148871 14486 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:09.144704 14465 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:09.143661 14353 consensus_queue.cc:260] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:09.160666  8331 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:09.168018 14398 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:09.169795  8325 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 13:04:09.170219  8325 pending_rounds.cc:85] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 10
I20260709 13:04:09.170437  8325 pending_rounds.cc:107] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:04:09.170374 14464 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:09.170845  8325 raft_consensus.cc:2889] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:09.171777  8329 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2750) took 1594 ms (client timeout 1541 ms). Trace:
W20260709 13:04:09.172314  8329 rpcz_store.cc:269] 0709 13:04:07.577786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.578755 (+   969us) service_pool.cc:224] Handling call
0709 13:04:09.171479 (+1592724us) spinlock_profiling.cc:243] Waited 24.2 ms on lock 0x7b540008aa54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:09.171642 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55556352}
I20260709 13:04:09.173872 14465 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:09.164525  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:09.182307  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:09.182837 14353 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:09.152722 14492 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.186259 14492 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:09.188465 14492 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:09.150362 14420 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.193838  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:09.194499  9009 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:09.200837  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:09.201042  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:09.204452  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:09.205114  9009 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 22: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
W20260709 13:04:09.198904  8369 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.541s (SENT) [suppressed 1 similar messages]
I20260709 13:04:09.213065 14449 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:09.213744 14449 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:09.215899 14449 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:09.195381 14406 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.219365  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:09.221849  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:09.222739  8253 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:09.223380  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:09.226230  8252 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:09.224903  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:09.232399 14449 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.233000 14449 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:09.234869 14417 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:09.238950  8253 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:09.221949 14066 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.240866 14066 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:09.241335  8256 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:04:09.241371 14487 consensus_queue.cc:1059] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:09.242905  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:09.243472 14066 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:09.243600  8252 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:09.243472 14487 raft_consensus.cc:3055] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:09.244771 14492 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:09.245597 14492 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:09.248219 14492 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:09.222330 14406 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:09.249461 14432 raft_consensus.cc:3055] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:09.249938 14432 raft_consensus.cc:740] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:09.250795 14406 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:09.251449 14381 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:09.250883 14432 consensus_queue.cc:260] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:09.252017 14381 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:09.256799 14449 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:09.254729 14381 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:09.259182  8328 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:04:09.264544  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:09.196000 14420 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:09.244952 14487 raft_consensus.cc:740] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:09.280769  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:09.275202  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 52341.7us
I20260709 13:04:09.282953  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:09.293253  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:09.294902  8332 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:09.270032 14492 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:09.299115  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:09.306464  8324 raft_consensus.cc:1275] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:04:09.306949  8324 pending_rounds.cc:85] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 9
I20260709 13:04:09.307192  8324 pending_rounds.cc:107] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:04:09.307440  8324 raft_consensus.cc:2889] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:09.309219 14380 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:09.312934  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.313882  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:09.314625  8256 raft_consensus.cc:2393] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 17: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:09.315529 14420 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:09.273910 14465 consensus_queue.cc:1048] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:04:09.313589 14492 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:09.332980  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:09.336530  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:09.337491  8325 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:09.343704  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:09.347168  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:09.341841 14429 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:09.341010 14492 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:09.360913  8223 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:09.365542  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:09.346745 14088 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.376408  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:09.376984  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:09.385293  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.378068  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.378420  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:09.388047  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.389139  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.388985 14429 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:09.389591  8220 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:09.372043 14398 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.391077 14398 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:09.391499 14429 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:09.360376 14401 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:09.377660 14449 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:09.392555 14449 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:09.392445 14401 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:09.393049 14449 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:09.378114 14417 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.394531 14401 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:09.394627  9003 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:04:09.394812 14417 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:09.395875 14437 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:09.368438 13757 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:09.397903 13757 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:09.399899 13757 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:09.376652  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:09.298830 14487 consensus_queue.cc:260] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:09.393193 14398 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:09.400978  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:09.401913 14487 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:09.402392  8251 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:09.403330  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:09.380837  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:09.389324 14465 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:04:09.406167  9009 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:09.361673 14419 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:09.406796  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.380048  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:09.407119 14419 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:09.381801  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.407984  8252 raft_consensus.cc:2368] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17.
I20260709 13:04:09.408245  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 931) took 1979 ms. Trace:
I20260709 13:04:09.379464  8255 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:04:09.402398 14088 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:09.407773  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2596) took 1382 ms. Trace:
I20260709 13:04:09.385452  8256 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:04:09.402128  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:09.377306  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:09.388060  8405 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 22: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:09.412436 14423 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:09.410761  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:09.413460  8326 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:04:09.413897  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:09.414233 14390 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:09.414574  9009 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:04:09.408677  8323 rpcz_store.cc:276] 0709 13:04:07.428757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.431524 (+  2767us) service_pool.cc:224] Handling call
0709 13:04:07.438861 (+  7337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:07.483690 (+ 44829us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:07.483721 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:07.516302 (+ 32581us) log.cc:844] Serialized 26 byte log entry
0709 13:04:07.616654 (+100352us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:07.624984 (+  8330us) log.cc:844] Serialized 14 byte log entry
0709 13:04:07.638913 (+ 13929us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.699463 (+ 60550us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b18001371a0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:04:07.780766 (+ 81303us) spinlock_profiling.cc:243] Waited 7.7 ms on lock 0x7b48001701d8. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882d6e7aa1 00007f882d6deb37 00007f882d6de9ca 00007f882d7d626f 00007f882d7d5f37 00007f882d7ea55c 00007f882d7ea508 00007f882d7ea488 00007f882d7ea443 00007f882d7e96ec 00007f882c713a96
0709 13:04:09.407944 (+1627178us) spinlock_profiling.cc:243] Waited 39.4 ms on lock 0x7b540014ffd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:09.407992 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:09.408032 (+    40us) raft_consensus.cc:1685] finished
0709 13:04:09.408043 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.408170 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7944,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":1644,"raft.queue_time_us":4536,"raft.run_cpu_time_us":3193,"raft.run_wall_time_us":278508,"spinlock_wait_cycles":111648640,"thread_start_us":99952,"threads_started":1,"wal-append.queue_time_us":127397,"wal-append.run_cpu_time_us":1718,"wal-append.run_wall_time_us":1042710}
I20260709 13:04:09.411363  8399 rpcz_store.cc:276] 0709 13:04:08.025696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.027745 (+  2049us) service_pool.cc:224] Handling call
0709 13:04:08.028097 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:08.105365 (+ 77268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.407538 (+1302173us) spinlock_profiling.cc:243] Waited 63.7 ms on lock 0x7b5400276854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:09.407691 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":146501376}
I20260709 13:04:09.419433  8325 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:04:09.368480 14468 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:09.450943 14468 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:09.451803 14460 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:09.452374 14460 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:09.408381  8401 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:04:09.453572 14457 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:09.454167 14457 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:09.453205 14468 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:09.456589 14457 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:09.457404  8402 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:04:09.458317  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.459770  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.460386  8401 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:09.461248  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.461570  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:09.462432  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.462991  8401 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:04:09.464017  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:09.441196  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 2490) took 1530 ms. Trace:
I20260709 13:04:09.465584  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:09.426326  8403 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:09.466940  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.449191 14449 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:09.427122  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.469101  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:09.469379 14449 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:09.472282  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:09.428625  8398 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:04:09.474439  8398 raft_consensus.cc:3055] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:09.474787  8398 raft_consensus.cc:740] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:09.475169  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:09.475890  9002 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:04:09.429881 14491 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:09.464865  8250 rpcz_store.cc:276] 0709 13:04:07.910392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.911460 (+  1068us) service_pool.cc:224] Handling call
0709 13:04:09.440983 (+1529523us) spinlock_profiling.cc:243] Waited 80.3 ms on lock 0x7b540033ed54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:09.441102 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":184760192}
I20260709 13:04:09.461669  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:09.438526 14379 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:09.397446 14417 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:09.441462  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 53993.8us
I20260709 13:04:09.412107  8253 raft_consensus.cc:2393] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 18: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:09.479893  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:09.398104  8220 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:09.480731  9003 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
W20260709 13:04:09.481177  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 83140.2us
I20260709 13:04:09.411818 14088 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:09.481844  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.482609  8220 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:09.483234  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.483791 14390 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:09.484652 14390 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:09.409372 14419 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:09.485152 14390 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:09.485702 14390 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:09.426326  8400 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 16.
I20260709 13:04:09.488787 14088 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:04:09.443074  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 55349.8us
I20260709 13:04:09.490046  8295 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:09.442209  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:09.444519 14423 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:09.381179  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:09.491438  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 49313.1us
I20260709 13:04:09.491863 14423 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:09.492478  8372 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:09.493093 14386 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:09.490888 14384 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:09.493773 14419 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:09.493892 14384 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:09.494226 14419 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:09.494375 14384 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:09.493765 14386 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:09.495469 14386 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:09.495635 14455 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:09.496277 14455 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:09.457261 14457 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:09.489176  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:09.497889 14457 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:09.498479  9006 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:04:09.499500 14455 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:09.500202 14457 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:09.494423 14423 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:09.491452  8295 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:09.501906  8295 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:09.502749 14088 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:09.503069 13865 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:09.503181  8295 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:09.503427 14088 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:09.503623 13865 raft_consensus.cc:697] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:09.504709 13865 consensus_queue.cc:237] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:09.506906  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:09.507627  8323 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:09.509827 14492 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:09.510344 14492 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:09.510789 14492 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:09.519403  8220 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:09.520598 14445 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:09.521188 14445 raft_consensus.cc:697] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:09.522037 14445 consensus_queue.cc:237] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:09.523608 14419 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:09.493175  8404 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:04:09.530463 14430 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:09.530853  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:09.531258 14430 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:09.425177 14378 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:09.533188  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:09.533066 14378 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:09.534965  8298 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:09.535254 14378 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:09.536190 14088 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:09.579357 14088 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:09.579872 14088 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:09.580411  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:09.581144  9005 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:09.538748  8372 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:09.499841  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.542646  8295 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:09.544159  8222 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:09.544279 14432 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:09.544999  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.546937 14297 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:09.547008  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.548269 14430 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:09.548178  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:09.550455  8297 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:09.551415  8974 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:09.552268 14427 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:09.552794  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.530543  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:09.558744 14390 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:09.545158  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.475479  8398 consensus_queue.cc:260] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:09.564865  8323 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
I20260709 13:04:09.465193  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.566639 14386 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:09.497910  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.571262 14492 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:09.491233 14460 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:09.589344  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 390424us
W20260709 13:04:09.591806  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 49246.2us
W20260709 13:04:09.595441  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 44179.2us
W20260709 13:04:09.590796  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 52134.1us
I20260709 13:04:09.633931  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:09.595436  8400 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:09.798730  8974 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:09.592485  8402 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:04:09.589994  8399 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
W20260709 13:04:09.590691  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 46624.8us
I20260709 13:04:09.620529  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:09.766083  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.830004  8250 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:04:09.620965  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:09.599347  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.591387 13865 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:09.597483  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.594437  9003 raft_consensus.cc:2393] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 15: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:09.593412  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.832679  8397 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
W20260709 13:04:09.748708  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.596378  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.603556  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.838629  8404 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:04:09.593708  8297 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:09.598023  8398 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:09.844064  8369 leader_election.cc:400] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17.
I20260709 13:04:09.602216  8405 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:09.844995  8369 leader_election.cc:403] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:09.618184  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:09.618649  8223 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:09.807943  8295 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:09.846798  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 228214us
I20260709 13:04:09.605651 14386 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:09.594594 14457 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:09.851293  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 52643.1us
I20260709 13:04:09.590838 14449 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:09.852324  8974 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:09.852411 14378 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:09.853328 14386 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:09.854202 14386 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:09.854673 14386 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:09.855206 14501 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:09.855405 14386 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:09.855832  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.856107 14501 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:09.857504 14475 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:09.858115 14475 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:09.858848 14491 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:09.859740 14491 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:09.860625 14475 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:09.844475 14066 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:09.862011 14501 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:09.862942  8372 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:09.646626 14384 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:09.865222 14423 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:09.865638 14423 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:09.866014 14423 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:09.869493 14417 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:09.870992 14491 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:09.873558  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:09.733560 14447 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:09.874468  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:09.874850 14447 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:09.877122 14447 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:09.869395  8398 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:04:09.878625  8406 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:04:09.879604 14384 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:09.879968 14369 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:09.880501 14386 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:09.883611  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:09.911666  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 95563us
I20260709 13:04:09.590708 14489 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:09.631765 14492 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:09.893867  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 46374.7us
I20260709 13:04:09.638509 14451 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.931788  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.933862  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:09.910277  9005 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
I20260709 13:04:09.597102 14390 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:09.893138  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 299493us
I20260709 13:04:09.939745  8297 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:09.941012  8297 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:09.641608 14432 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:09.912969  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:09.946413  8399 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:09.591971 14455 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:09.892781 14487 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:09.941481  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:09.941954  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:09.958595  8297 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:09.957496  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:09.959163  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 128587us
I20260709 13:04:09.957224 13865 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:09.594960 14427 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:09.589470 14088 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:09.592379 14297 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:09.965996  8222 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:09.938040 14525 log.cc:927] Time spent T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.191s	user 0.002s	sys 0.009s
W20260709 13:04:09.958801  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 45919.8us
W20260709 13:04:09.915692 14526 log.cc:927] Time spent T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.161s	user 0.006s	sys 0.000s
W20260709 13:04:09.913832 14520 log.cc:927] Time spent T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.160s	user 0.006s	sys 0.000s
W20260709 13:04:09.950315  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.977962  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:09.966571  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:09.937249 14489 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:09.968379  8223 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:09.627125 14419 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:09.911478 14460 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:09.956890 14384 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:09.930692 14450 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:04:09.993697 14384 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:09.994251 14246 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:09.994819 14246 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:09.996248 14384 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:09.997085 14246 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:09.975142  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:09.975845  8222 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:09.909615 14369 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:09.812692 14430 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:09.883570 14498 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.087s
I20260709 13:04:09.970288 14432 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:10.005463  9007 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:04:10.006431  8325 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:04:10.007232 14460 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:10.007506 14088 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:10.007694 14450 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17.
I20260709 13:04:10.044303 14460 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:10.041252 14369 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:10.013239 14492 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:09.973122  8369 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:10.013734  8155 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45434 (request call id 282) took 80264 ms (client timeout 15000 ms). Trace:
W20260709 13:04:10.049593  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 76566.7us
I20260709 13:04:09.930003 14398 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 13:04:09.965111 14487 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:10.014345  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 46104.4us
I20260709 13:04:10.051997  8223 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:10.014829 14406 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:09.963339 14420 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:09.966900 14500 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:09.951800 14451 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:10.024405  8329 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:04:09.982496 14457 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:10.027067  8401 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
W20260709 13:04:10.028806  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 176538us
I20260709 13:04:10.067281  8974 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:09.985875 14449 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:09.985698 14489 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:09.970598 14455 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:09.990566  8402 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:04:10.043802  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 68050.9us
I20260709 13:04:10.073216  8222 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:10.074128  9005 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:10.075466  8222 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:10.011986  8297 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:10.078140  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 66226.3us
I20260709 13:04:10.041059 14501 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.042346 14430 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:10.045285 14390 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:10.055095  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:10.057713  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:10.053519 14379 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:10.051252 14450 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:10.070915  8974 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:10.053905 14487 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:10.079084  8297 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:10.101992  8297 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:04:10.106135  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 54628.3us
I20260709 13:04:10.108139  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:10.063091 14500 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:10.101815  8974 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:10.055953 14420 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:10.123601  8369 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:10.124125  9001 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:04:10.125963  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:10.049213  8155 rpcz_store.cc:269] 0709 13:02:49.749782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.753209 (+  3427us) service_pool.cc:224] Handling call
0709 13:02:50.556208 (+802999us) spinlock_profiling.cc:243] Waited 39.9 ms on lock 0x7b54003bf454. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828728090 00007f88287c6f3a 00007f882e57d1cc 00007f882e67b37e 00007f882cb3c2a4 00007f882cb3c232 00007f882cb3c161 00007f882cb3c0dc 00007f882cb3b392 00007f8826af948c 00007f8826af88c6
0709 13:04:10.013633 (+79457425us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:06.649517 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:06.650195 (+   678us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:06.650246 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:04:06.650284 (+    38us) tablet.cc:662] Decoding operations
0709 13:04:06.650795 (+   511us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:06.650889 (+    94us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:06.650916 (+    27us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:04:06.651392 (+   476us) tablet.cc:701] Row locks acquired
0709 13:04:06.651440 (+    48us) write_op.cc:260] PREPARE: finished
0709 13:04:06.651610 (+   170us) write_op.cc:270] Start()
0709 13:04:06.651755 (+   145us) write_op.cc:276] Timestamp: P: 1783602246651549 usec, L: 0
0709 13:04:06.651793 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 13:04:06.767742 (+115949us) log.cc:844] Serialized 908 byte log entry
0709 13:04:06.973826 (+206084us) op_driver.cc:464] REPLICATION: finished
0709 13:04:07.124367 (+150541us) write_op.cc:301] APPLY: starting
0709 13:04:07.124485 (+   118us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:07.127037 (+  2552us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:07.127201 (+   164us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:07.128681 (+  1480us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:07.129010 (+   329us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:04:07.129100 (+    90us) write_op.cc:312] APPLY: finished
0709 13:04:07.130056 (+   956us) log.cc:844] Serialized 33 byte log entry
0709 13:04:07.151593 (+ 21537us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:07.158804 (+  7211us) write_op.cc:454] Released schema lock
0709 13:04:07.168482 (+  9678us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:07.307843 (+139361us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:07.311128 (+  3285us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":189747,"reported_tablets":50,"spinlock_wait_cycles":91668224,"child_traces":[["op",{"apply.queue_time_us":150127,"apply.run_cpu_time_us":6261,"apply.run_wall_time_us":135030,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12491,"num_ops":2,"prepare.queue_time_us":44413,"prepare.run_cpu_time_us":36875,"prepare.run_wall_time_us":318590,"raft.queue_time_us":1197,"raft.run_cpu_time_us":2246,"raft.run_wall_time_us":9759,"replication_time_us":321765,"thread_start_us":175292,"threads_started":4,"wal-append.queue_time_us":183021,"wal-append.run_cpu_time_us":4559,"wal-append.run_wall_time_us":1271861}]]}
I20260709 13:04:10.056267 14498 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:04:10.136823 14423 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:10.137745  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:10.135025  8372 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:10.081255 14486 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:10.098608 14378 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:10.098603 14475 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:10.098711 14088 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:10.122884 14457 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:10.143894  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:10.142405  8974 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:10.118403 14459 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:10.121198 14270 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:10.124155 14417 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:10.127218 14447 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:10.129204 14500 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:10.135773 14420 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:10.121948 14390 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:10.120893 14418 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.028s
W20260709 13:04:10.149576 14518 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:04:10.147483 14378 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.234494 14378 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:10.147881 14450 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:10.060536  8148 catalog_manager.cc:5697] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:10.122822 14501 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:10.149839 14475 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:10.246137 14475 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:10.247048  8406 raft_consensus.cc:2393] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 16: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:10.151749 14088 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.134272 14491 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:04:10.149168 14528 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:10.152199 14459 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:10.153261 14457 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:10.165901 14418 consensus_queue.cc:1243] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 19.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:04:10.185949 14419 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:10.186748 14390 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:10.185004 14270 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:10.221552 14386 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:10.223696  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:10.225029 14432 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:10.227401  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:10.230079 14455 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:10.144861 14498 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:10.135552 14380 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:04:10.231901 14451 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:10.144363 14384 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:04:10.237689  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 148406us
I20260709 13:04:10.266781 14379 raft_consensus.cc:697] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 22 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:10.245399 14501 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:10.279522 14379 consensus_queue.cc:237] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:10.247728  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 278721us
I20260709 13:04:10.282253 14384 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:10.252957 14088 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:10.259274  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 108411us
I20260709 13:04:10.290232 14406 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:10.285272 14384 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:10.294018 14430 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:10.242717 14450 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:10.242377 14417 raft_consensus.cc:697] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 22 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:10.247290 14447 raft_consensus.cc:697] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 18 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:10.260804 14459 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:10.255367 14528 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:10.264381 14457 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:10.269663 14270 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:10.301734  8298 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:10.316475  8298 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:10.292889 14460 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:10.308037  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:10.307623  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:10.310248  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:10.311138  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:10.324880  8253 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
W20260709 13:04:10.313108  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 166379us
I20260709 13:04:10.328488 13865 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:10.329098 13865 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:10.303026 14450 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:10.324918 14380 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:10.367463 14541 consensus_queue.cc:1048] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:10.371196  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:10.374267  8249 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:04:10.373828 14417 consensus_queue.cc:237] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:10.374327  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:10.375428  8257 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:04:10.380611  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:10.314318 14528 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:10.322356 14270 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:10.384080  8257 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:10.385177  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:10.385885  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:10.328111 14418 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:10.406968  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:10.411288 13865 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:10.422500  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:10.407639  8249 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:10.423327  8329 raft_consensus.cc:2368] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17.
I20260709 13:04:10.425930  8257 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:04:10.426038  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:10.407018 14380 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.428933 14380 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:10.430052  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:10.431159 14380 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:10.431625  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:10.437656  8148 catalog_manager.cc:5697] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:10.438997 14088 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:10.441629 14447 consensus_queue.cc:237] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:10.428931  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:04:10.445854  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 147544us
I20260709 13:04:10.446144  8250 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:10.447268  8332 raft_consensus.cc:2368] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15.
I20260709 13:04:10.447495  8249 raft_consensus.cc:2393] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 18: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:10.441398  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:10.445263  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:10.388744 14427 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:10.454905  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:10.474516  9004 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:04:10.466066  8249 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:04:10.407400 14423 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:10.441396 14088 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:10.491729  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:10.407601 14369 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:10.498423  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.440431  8148 catalog_manager.cc:5697] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 4 to 15, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 15 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:10.503212  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:10.503935  8148 catalog_manager.cc:5697] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 5 to 11, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 11 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:10.506084  8148 catalog_manager.cc:5697] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:10.423936 14390 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:10.515045  8369 leader_election.cc:400] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17.
I20260709 13:04:10.520970  8250 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:10.515564  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:10.522156  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:10.527184  8295 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:10.508646  8148 catalog_manager.cc:5697] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 9 to 13, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 13 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:10.545680 14475 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:10.536193  8295 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:10.547739 14492 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:10.547029  8148 catalog_manager.cc:5697] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 9 to 11, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 11 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:10.548179 14492 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:10.548264 14475 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:10.548671 14492 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:10.537201 13865 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:10.540980  8253 raft_consensus.cc:1275] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:04:10.542197  8254 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:10.550299 14460 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:10.558287  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:10.553882  8148 catalog_manager.cc:5697] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:10.559249  8254 raft_consensus.cc:2393] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 14: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:10.559082 14459 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:10.560984  8325 raft_consensus.cc:2393] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 19: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:10.560099  8148 catalog_manager.cc:5697] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 9 to 12, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 12 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:10.556221  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:10.556797  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:10.561880 14500 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:10.562103 14378 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:10.562623  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:10.589990  8325 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 22.
I20260709 13:04:10.555126  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:10.582360  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3180) took 1052 ms. Trace:
I20260709 13:04:10.560307  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:10.519393 14088 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:10.587086  8249 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 18. Ops: []
I20260709 13:04:10.595829  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:10.596422  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:10.596850  8250 raft_consensus.cc:1240] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:10.598812  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:10.561276 14459 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:10.602948  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:10.604001  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:10.607956  8369 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:10.609773  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:10.612641  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:10.563712 14447 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:10.591174  8405 rpcz_store.cc:276] 0709 13:04:09.581739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.945247 (+363508us) service_pool.cc:224] Handling call
0709 13:04:09.970518 (+ 25271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.970756 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:09.997418 (+ 26662us) log.cc:844] Serialized 14 byte log entry
0709 13:04:10.581927 (+584509us) spinlock_profiling.cc:243] Waited 4.17 ms on lock 0x7b1800329800. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88283664bc 00007f88256b448b 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256ce455 00007f88287b6060 00007f88287c8716 00007f88287d9287
0709 13:04:10.582026 (+    99us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:10.582077 (+    51us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:10.582123 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.582190 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.582303 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26301,"prepare.queue_time_us":27601,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":81857,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":8403,"spinlock_wait_cycles":9596160,"thread_start_us":243946,"threads_started":2,"wal-append.queue_time_us":227292}
I20260709 13:04:10.618618  8249 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:04:10.618541  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:10.538344 14380 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:04:10.620389  8399 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:10.618860  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:10.624035  8248 raft_consensus.cc:1240] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:10.626956  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:10.630695  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:10.632421  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:10.582916 14460 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:04:10.645910 13865 raft_consensus.cc:697] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:10.646023  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:10.619360 14537 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:10.589885 14487 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:10.605778 14088 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:10.615799 14420 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:10.659389  8255 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:04:10.665129  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:10.712661  8256 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:04:10.715022 14088 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:10.671453 14549 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.715305 14420 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:10.647666  9005 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15. Ops: []
W20260709 13:04:10.648458  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 201427us
I20260709 13:04:10.761438  8972 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:10.762522  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:10.642293 14297 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:10.672875  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:10.717571 14459 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.709723  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:10.767362  8253 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:04:10.700338  8324 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:04:10.692540  9006 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:04:10.771224  8298 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:04:10.755884  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 190262us
I20260709 13:04:10.623773 14423 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:10.756942  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:10.753801  8223 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:10.760682  9009 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:10.762961  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:10.789381  8324 raft_consensus.cc:1275] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:10.789719  8295 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:10.757541 13865 consensus_queue.cc:237] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:10.775105  8250 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:04:10.775591  8326 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:10.791937  8398 raft_consensus.cc:1275] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:10.792554  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:10.799049  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:10.799979  8401 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:10.795259  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:10.801307  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:10.806506  8328 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:04:10.770396 14369 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:10.818233  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:10.819831  8253 raft_consensus.cc:2410] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:04:10.761930 14486 consensus_queue.cc:1059] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 13:04:10.792419 14529 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:10.821005  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:10.798195 14297 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:10.797531 14420 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:10.791587 14088 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:10.837505  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:04:10.842120  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 68484us
I20260709 13:04:10.815583 14547 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:10.867314  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:10.867412  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:10.817116 14552 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:10.784528 14459 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:10.821151 14559 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:10.899255  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:10.902969  8256 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:10.897796  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:10.923820  8252 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:10.902149  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:10.944252  8295 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:10.945659  8249 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:10.854674 14297 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.863551 14500 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:10.825691 14529 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:10.862998 14390 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:10.918200 14420 raft_consensus.cc:697] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:04:10.958297 14489 consensus_meta.cc:287] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.859s	user 0.013s	sys 0.000s
I20260709 13:04:10.829982 14491 consensus_queue.cc:1059] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.134s
I20260709 13:04:10.942828 14486 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:10.982183  8252 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:04:10.829773 14550 consensus_queue.cc:1059] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.131s
I20260709 13:04:10.954605 14460 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:10.959601 14297 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:10.866524 14088 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:11.013833  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:11.013651  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:11.014647  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:10.957868 14447 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:10.910811 14270 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:10.901372 14502 consensus_queue.cc:1059] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 13:04:11.020889  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:10.976692 14529 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:11.034654  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 179155us
W20260709 13:04:11.019879 14450 consensus_meta.cc:287] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.641s	user 0.010s	sys 0.000s
I20260709 13:04:11.000154 14560 raft_consensus.cc:3055] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:11.032967 14550 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:11.073662 14088 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:11.040249 14570 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:11.038007 14460 raft_consensus.cc:3055] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:11.078728  8249 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
W20260709 13:04:11.064816 14555 log.cc:927] Time spent T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.327s	user 0.000s	sys 0.001s
I20260709 13:04:11.033381 14539 consensus_queue.cc:1048] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:04:11.061298 14560 raft_consensus.cc:740] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:10.764358 14549 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:11.101701 14420 consensus_queue.cc:237] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.036741 14447 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:10.762338 14537 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.117631  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2616) took 1534 ms. Trace:
I20260709 13:04:10.958811 14559 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.118111  8399 rpcz_store.cc:276] 0709 13:04:09.582834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.970513 (+387679us) service_pool.cc:224] Handling call
0709 13:04:10.015392 (+ 44879us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.624691 (+609299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.116910 (+492219us) spinlock_profiling.cc:243] Waited 605 ms on lock 0x7b54003160d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:11.117558 (+   648us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1390919424}
I20260709 13:04:11.120301 14537 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:11.120576 14559 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:11.121047  8257 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 22.
I20260709 13:04:11.122272 14459 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:11.122576 14447 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:11.124912 14549 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:11.125602  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.127583  8974 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:11.085836 14550 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.088359 14570 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.065084 14450 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:11.111788 14457 consensus_meta.cc:287] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.777s	user 0.010s	sys 0.000s
I20260709 13:04:11.086491 14460 raft_consensus.cc:740] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:11.008169 14489 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.116037 14560 consensus_queue.cc:260] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.131772  8974 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:11.167461  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:11.173297  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:11.179633  8404 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 15
W20260709 13:04:11.180668  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 115916us
I20260709 13:04:11.160226 14502 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.196424  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:11.200822  8401 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18.
I20260709 13:04:11.169549 14427 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:11.200798  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:11.202426  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:11.202886  9006 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 22.
I20260709 13:04:11.170275 14552 raft_consensus.cc:697] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:11.210021  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:11.211331  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:11.212108 14549 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:11.211941 14502 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:11.212610  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:11.213124  8399 pending_rounds.cc:85] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 9
I20260709 13:04:11.216357  8399 pending_rounds.cc:107] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 13:04:11.216687  8399 raft_consensus.cc:2889] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:11.178261 14547 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:11.214179  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 293902us
I20260709 13:04:11.220160  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:11.220804  9005 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:11.221256  8974 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:11.222576  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:11.222970  8398 raft_consensus.cc:2393] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 15: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:11.183241 14569 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:11.227475  8369 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:11.228307  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:11.184067 14270 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:11.194654 14489 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:11.184669 14570 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:11.170166 14457 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:11.271728  8974 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:11.227999 14427 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:11.277477  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:11.229246 14501 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:11.212750 14549 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.287994  8323 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:11.269884 14554 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:11.209136 14552 consensus_queue.cc:237] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:11.282032 14066 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:11.404457 14066 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.407150 14066 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:11.285223 14425 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:11.413919 14578 consensus_queue.cc:1059] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:11.415119  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:11.416378  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:11.416466 14578 raft_consensus.cc:3055] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:11.418151 14487 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:11.419764  8298 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:11.420766  8298 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:11.421190  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:11.421753 14576 consensus_queue.cc:1059] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:11.422937  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:11.295037  8404 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:04:11.423748 14574 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:11.424245 14574 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:11.424583  8404 raft_consensus.cc:3055] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:11.425453 14559 raft_consensus.cc:3055] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:11.425916 14559 raft_consensus.cc:740] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:11.426832 14559 consensus_queue.cc:260] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.428164 14559 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:11.430809  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2771) took 1050 ms. Trace:
W20260709 13:04:11.294473 14564 log.cc:927] Time spent T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:04:11.286533 14386 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:11.297850  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:11.297814 14550 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:11.316586  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:11.339025 14378 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:11.345643  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:11.480268  9005 raft_consensus.cc:2393] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 21: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:11.349944 14568 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:11.364001 13865 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.366323  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:11.367230  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:11.369351  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:11.369946  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:11.371605  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:11.195052 14460 consensus_queue.cc:260] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.495991 14460 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:11.497541 14459 consensus_queue.cc:1059] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:11.497939  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:11.478937 14425 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:11.498970 14425 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:11.499541  8401 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:11.489251 13865 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:11.501642 14558 consensus_queue.cc:1175] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e64761f43e4240319c41117cf886d22f" responder_term: 15 status { last_received { term: 13 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15. Ops: [14.11-14.11]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:04:11.503741 13865 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:11.507184 14459 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:11.508279 14423 raft_consensus.cc:3055] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:04:11.508580 14492 consensus_meta.cc:287] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.959s	user 0.010s	sys 0.002s
I20260709 13:04:11.508787 14423 raft_consensus.cc:740] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:11.274842 14537 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:11.402900 14572 raft_consensus.cc:3055] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:11.423152 14576 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:11.424664 14574 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:11.296476 14549 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:11.431654  8974 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:11.525784  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:04:11.526435  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 94873.1us
I20260709 13:04:11.297798 14457 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:11.425351  8404 raft_consensus.cc:740] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:11.529515 14537 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.530820  8404 consensus_queue.cc:260] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:11.532269 14537 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:11.532413  8404 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:11.433449  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:11.408931  8297 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:11.435915  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:11.479643 14563 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.541131 14563 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:11.543218 14564 log.cc:927] Time spent T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:04:11.477840  8223 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:04:11.548722  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 71003us
W20260709 13:04:11.550972  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 142151us
I20260709 13:04:11.287390 14297 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.272616 14406 consensus_queue.cc:1048] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:04:11.431257  8332 rpcz_store.cc:276] 0709 13:04:10.379990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.380356 (+   366us) service_pool.cc:224] Handling call
0709 13:04:11.375993 (+995637us) spinlock_profiling.cc:243] Waited 65.8 ms on lock 0x7b540011e9d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:11.430743 (+ 54750us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":151234304}
I20260709 13:04:11.468387  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:11.346527  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.478808  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 44098.6us
I20260709 13:04:11.477317  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:11.417201 14578 raft_consensus.cc:740] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:11.492007  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:11.492197  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:11.492846  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:11.493182 14557 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:11.519433 14568 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.581220 14557 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.583930  8372 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:11.494976  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:11.507872 14459 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.586941 14459 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:11.511610  8401 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:11.521873 14572 raft_consensus.cc:740] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:11.535730 14066 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:11.508809 14550 raft_consensus.cc:3055] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:11.614327 14550 raft_consensus.cc:740] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:11.615301 14550 consensus_queue.cc:260] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:11.509053 14492 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:11.616359 14297 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:11.617524  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:11.617856 14492 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:11.470140 14502 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:11.517629  8222 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:11.523937 14576 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:11.619143  8329 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:04:11.619369 14576 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:11.559863 14537 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:11.623198 14537 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:11.623651 14537 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:11.619191  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 101635us
I20260709 13:04:11.571342 14563 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:11.627491 14578 consensus_queue.cc:260] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:11.588732 14450 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:11.551854  8297 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:11.630879  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 79116.1us
I20260709 13:04:11.631731  8297 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:11.632562 14492 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:11.552572 13865 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:11.451238  8323 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18.
I20260709 13:04:11.268723 14569 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.576982  8404 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:04:11.634199  8404 pending_rounds.cc:85] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 8
I20260709 13:04:11.634476  8404 pending_rounds.cc:107] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:04:11.634748  8404 raft_consensus.cc:2889] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:11.634598 14572 consensus_queue.cc:260] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:11.573747  8295 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:11.578740  8252 raft_consensus.cc:2368] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18.
I20260709 13:04:11.636317 14572 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:11.588567  8223 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:11.636482  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 62837.6us
I20260709 13:04:11.636893 14563 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.520028 14475 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:11.633028  8297 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:11.637329  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 56707.8us
I20260709 13:04:11.638785  8223 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:11.640241  8223 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:11.592382  8401 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:11.472573  9001 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:04:11.641539  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:11.598032  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:11.598124 14539 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:11.603713 14568 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:11.604698 14584 log.cc:927] Time spent T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.091s	user 0.003s	sys 0.000s
I20260709 13:04:11.577275 14498 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:11.545528 14425 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:11.487761 14529 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:11.611356  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.665729  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:11.666251  9008 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:04:11.383646  8325 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:04:11.432462 14554 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.633471 14492 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:11.588347 14423 consensus_queue.cc:260] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:11.635782 14569 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:11.644903  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:11.637311  8295 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:11.647825 13865 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:04:11.653314  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 425907us
I20260709 13:04:11.679462 13865 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:11.613756 14557 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:11.704217 14582 log.cc:927] Time spent T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc: Append to log took a long time: real 0.406s	user 0.001s	sys 0.002s
I20260709 13:04:11.655345 14378 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:11.611761 14491 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:11.629328 14578 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:11.669618  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:11.706189 14378 raft_consensus.cc:697] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 18 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:11.672648  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:11.667470  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:11.679968 13865 raft_consensus.cc:697] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:11.661878 14475 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.709478  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:11.710330 13865 consensus_queue.cc:237] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:11.711203  8972 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:11.679744  8369 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:11.711719 14475 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:11.711925  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2787) took 1208 ms. Trace:
I20260709 13:04:11.712566 14569 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:11.712795  8369 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:11.647240 14549 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:11.660668  8297 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:11.653983 14420 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:04:11.715027  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 54496.2us
I20260709 13:04:11.715154 14420 raft_consensus.cc:697] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:11.715865  8369 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:11.716172 14420 consensus_queue.cc:237] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:11.717041 14459 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:11.681943  8295 leader_election.cc:400] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18.
I20260709 13:04:11.720713  8295 leader_election.cc:403] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:04:11.722995 14560 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:11.723891  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.669504 14498 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.682866 14457 consensus_queue.cc:1059] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:11.725625 14459 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18.
I20260709 13:04:11.725919 14457 raft_consensus.cc:3055] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:11.726416 14457 raft_consensus.cc:740] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:11.726562 14550 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:11.727185 14498 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:11.727818  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 3039) took 1023 ms. Trace:
I20260709 13:04:11.712342  8329 rpcz_store.cc:276] 0709 13:04:10.503436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.630388 (+126952us) service_pool.cc:224] Handling call
0709 13:04:11.711752 (+1081364us) spinlock_profiling.cc:243] Waited 987 ms on lock 0x7b54002c9054. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:11.711865 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2271142144}
I20260709 13:04:11.728250  8323 rpcz_store.cc:276] 0709 13:04:10.704368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.707147 (+  2779us) service_pool.cc:224] Handling call
0709 13:04:11.727636 (+1020489us) spinlock_profiling.cc:243] Waited 506 ms on lock 0x7b5400365954. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:11.727753 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1163196160}
I20260709 13:04:11.684283 14529 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:11.738870 14450 consensus_queue.cc:1059] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:11.685571  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:11.687166  9009 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:11.739620 14574 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.740275 14450 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:11.742697  9002 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:11.663776 14406 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:11.702566 14554 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:11.704118 14528 consensus_meta.cc:287] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 1.254s	user 0.013s	sys 0.000s
I20260709 13:04:11.747328 14460 consensus_queue.cc:1059] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:11.747665 14528 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.748296 14460 raft_consensus.cc:3055] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:11.748682 14460 raft_consensus.cc:740] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:11.749307  9004 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:11.750061 14528 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:11.749902 14460 consensus_queue.cc:260] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.750488  9004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 1062) took 1155 ms. Trace:
I20260709 13:04:11.750865  9004 rpcz_store.cc:276] 0709 13:04:10.594784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.596841 (+  2057us) service_pool.cc:224] Handling call
0709 13:04:10.622957 (+ 26116us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:11.749725 (+1126768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.750271 (+   546us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400642b54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:11.750432 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2589923200}
I20260709 13:04:11.707372 14378 consensus_queue.cc:237] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:11.647025 14581 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:11.718750 14585 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.189s	user 0.000s	sys 0.002s
I20260709 13:04:11.662846 14487 consensus_queue.cc:1059] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:04:11.729905  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:11.732046  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:11.735008  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:11.754403  9006 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:11.738387 14425 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:11.738072 14457 consensus_queue.cc:260] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:11.770823  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:11.771997  8329 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:04:11.739982  8972 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:11.688042 14576 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.637496 14563 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.788441  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:11.743510  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:11.743791  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:11.667202  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2784) took 1178 ms. Trace:
I20260709 13:04:11.751264 14460 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:11.753759  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:11.737938  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:11.717732 14418 consensus_queue.cc:1059] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:04:11.796833  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 67000.8us
I20260709 13:04:11.797797  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:11.759326  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:11.745250 14539 raft_consensus.cc:3055] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:11.798815 14539 raft_consensus.cc:740] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:11.799919 14539 consensus_queue.cc:260] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:11.779057  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:11.779356  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:11.802732 14574 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:11.807757 14576 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:11.810015 14563 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:11.810226 14457 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:11.810294  8972 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:11.811234  8253 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:11.813597  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.813989 14501 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:11.817853  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3190) took 1049 ms. Trace:
I20260709 13:04:11.818477  8400 rpcz_store.cc:276] 0709 13:04:10.768412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.768679 (+   267us) service_pool.cc:224] Handling call
0709 13:04:10.769008 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:10.798091 (+ 29083us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:10.798120 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:11.327585 (+529465us) spinlock_profiling.cc:243] Waited 2.04 ms on lock 0x7b18003297a0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882569f2a6 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a
0709 13:04:11.328659 (+  1074us) spinlock_profiling.cc:243] Waited 6.02 ms on lock 0x7b2c000e1c48. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:04:11.388325 (+ 59666us) log.cc:844] Serialized 26 byte log entry
0709 13:04:11.654248 (+265923us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:11.654281 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.654343 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:11.817538 (+163195us) raft_consensus.cc:1685] finished
0709 13:04:11.817571 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:11.817780 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61552,"raft.queue_time_us":32338,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":10939,"spinlock_wait_cycles":18543360,"thread_start_us":141458,"threads_started":1,"wal-append.queue_time_us":326468}
I20260709 13:04:11.820647 14418 raft_consensus.cc:3055] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:11.712342 14475 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:11.821285 14418 raft_consensus.cc:740] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:11.822717 14418 consensus_queue.cc:260] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:11.823563  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:11.824224  9008 raft_consensus.cc:1240] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:11.775997  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:11.790524  8325 rpcz_store.cc:276] 0709 13:04:10.489326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.641498 (+152172us) service_pool.cc:224] Handling call
0709 13:04:10.641909 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:11.325394 (+683485us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b5400266c20. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f8828735dc0 00007f88287c01c0 00007f88287c1231 00007f88287bdd8f 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c
0709 13:04:11.666873 (+341479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.667124 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":264566144}
W20260709 13:04:11.823983 14459 consensus_queue.cc:1175] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f865c4d761f5446eb12b3a029234772e" responder_term: 17 status { last_received { term: 16 index: 12 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 16. Current term is 17. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:04:11.778311 14528 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:11.830500  9006 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:04:11.831565  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:11.832415 14498 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:11.832990  8251 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:11.833169 14425 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:11.833403  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:11.834398  8369 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:11.835281 14587 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:11.835645 14587 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:11.835965 14587 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:11.837965 14425 consensus_queue.cc:1059] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:11.839130 14246 raft_consensus.cc:3055] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:04:11.839347  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 45444.2us
I20260709 13:04:11.839560  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:11.840628  9005 raft_consensus.cc:2393] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 19: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:11.841813  8369 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:11.839548 14246 raft_consensus.cc:740] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:11.794873  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:11.848296  8324 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:11.848707  8972 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:11.850756  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:11.851027  8252 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:11.851727  8331 raft_consensus.cc:2393] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 17: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:11.857062  8327 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:04:11.859288  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:11.864898  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:11.867177  8327 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:04:11.871410  8972 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:11.845803  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:11.872351  8332 raft_consensus.cc:1240] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:04:11.872429  8295 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:11.818555 14540 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:11.788187 14554 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:11.887579  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:11.874169 14475 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:11.895164  8253 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
I20260709 13:04:11.897938  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:11.898597  9003 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:11.878262 14539 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:11.885776 14489 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:11.915540  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:04:11.916822  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.833016 14498 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.072s
I20260709 13:04:11.884541 14559 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:11.888547 14538 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:11.888258 14557 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:11.890300 14475 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:11.877918 14450 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:04:11.935209  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:11.931533  8295 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:11.937968  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:11.934203  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:11.941291  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 59140.9us
W20260709 13:04:11.941475  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 58168.7us
I20260709 13:04:11.942478  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:11.943056  8254 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 13:04:11.943466  8254 pending_rounds.cc:85] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 11
I20260709 13:04:11.943675  8254 pending_rounds.cc:107] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 13:04:11.943845  8369 leader_election.cc:400] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15.
I20260709 13:04:11.943974  8254 raft_consensus.cc:2889] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:11.944154  8369 leader_election.cc:403] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:11.944414 13865 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:11.945173  8248 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:04:11.945745 13865 raft_consensus.cc:697] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 16 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:11.917475 14489 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:11.899019 14554 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:11.982066  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.017673  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:12.017992  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:11.935030 14475 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:12.058485  9003 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 23.
I20260709 13:04:12.062322 13865 consensus_queue.cc:237] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:11.980057 14489 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:12.022702 14398 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15.
I20260709 13:04:11.950933 14570 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.113s
I20260709 13:04:11.933538 14538 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:12.085954  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 369323us
I20260709 13:04:11.980581 14417 consensus_queue.cc:1059] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.130s
I20260709 13:04:11.951344 14491 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:12.093564  8297 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:11.941535 14246 consensus_queue.cc:260] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:12.110642 14578 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:12.119169 14246 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:12.144629  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 142006us
I20260709 13:04:12.155345  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:12.157616  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:12.158859  8332 raft_consensus.cc:2410] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:04:12.161978  8329 raft_consensus.cc:1240] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:12.163851  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:12.165287  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:12.166590  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:12.167703  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:12.168524  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:12.171924  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.176131  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:12.177448  8328 raft_consensus.cc:1240] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:12.179046  8369 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:12.179152  8331 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18.
I20260709 13:04:12.108857 14537 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:12.184432  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:12.183388  8332 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:04:12.145371 14550 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:04:12.263479  8256 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:12.149261 14570 consensus_queue.cc:1059] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.112s
I20260709 13:04:12.266589 14588 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:12.188992 14498 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:12.285082 14600 log.cc:927] Time spent T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
W20260709 13:04:12.286862 14597 log.cc:927] Time spent T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:04:12.180604 14541 consensus_queue.cc:1059] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.094s
W20260709 13:04:12.280846  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 92815.2us
I20260709 13:04:12.291837  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:12.187366 14569 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.093s
I20260709 13:04:12.296440  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:12.297253 14568 consensus_queue.cc:1048] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:04:12.297375 14501 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:12.306278 14603 log.cc:927] Time spent T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:04:12.306975 14611 raft_consensus.cc:3055] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:12.278219 14369 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:04:12.309630 14460 consensus_meta.cc:287] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.513s	user 0.010s	sys 0.000s
I20260709 13:04:12.316226 14578 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:12.293757 14588 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:12.319895 14576 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:12.353070 14576 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:12.332100  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 141496us
I20260709 13:04:12.359372  8220 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:12.360595 14379 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:12.362437 14498 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:12.363021  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:12.364639 14541 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:12.364830 14588 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:12.367111  8323 raft_consensus.cc:2410] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:04:12.366827 14578 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:12.336149 14611 raft_consensus.cc:740] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:12.370496 14611 consensus_queue.cc:260] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:12.371906 14611 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:12.361236  8220 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:12.332309 14475 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:12.340847 14608 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:12.344797 14560 consensus_meta.cc:287] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.621s	user 0.011s	sys 0.001s
W20260709 13:04:12.317232 14594 log.cc:927] Time spent T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.279s	user 0.000s	sys 0.002s
I20260709 13:04:12.299762 14491 consensus_queue.cc:1059] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:04:12.337180 14554 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:12.327191 14417 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.074s
I20260709 13:04:12.374455 14369 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:12.380422  8330 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
W20260709 13:04:12.346432 14587 consensus_meta.cc:287] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.510s	user 0.009s	sys 0.005s
I20260709 13:04:12.387128  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2629) took 1225 ms. Trace:
I20260709 13:04:12.401381  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:12.401719  8252 raft_consensus.cc:1275] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:04:12.386255 14486 raft_consensus.cc:3055] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:12.395681  8249 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 21.
I20260709 13:04:12.399650  8220 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:12.391036 13865 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:12.350744 14538 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:12.403574  8397 rpcz_store.cc:276] 0709 13:04:11.162156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.167010 (+  4854us) service_pool.cc:224] Handling call
0709 13:04:11.182283 (+ 15273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:11.198578 (+ 16295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:11.198624 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:11.428083 (+229459us) spinlock_profiling.cc:243] Waited 4.78 ms on lock 0x7b18004274e0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882569f2a6 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a
0709 13:04:11.429278 (+  1195us) spinlock_profiling.cc:243] Waited 4.47 ms on lock 0x7b2c000e1c48. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:04:11.429842 (+   564us) log.cc:844] Serialized 26 byte log entry
0709 13:04:11.766981 (+337139us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:11.767042 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.767152 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:12.386891 (+619739us) raft_consensus.cc:1685] finished
0709 13:04:12.386920 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.387065 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15843,"raft.queue_time_us":20280,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":1940,"spinlock_wait_cycles":21279872,"thread_start_us":336714,"threads_started":1,"wal-append.queue_time_us":490770}
I20260709 13:04:12.409667  8220 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:12.407797 13865 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:12.369330 14578 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:12.465204  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 55619.7us
W20260709 13:04:12.467267  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 104304us
W20260709 13:04:12.488008  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.384800 14617 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:12.490067  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:12.491214  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:12.492162  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:12.493831  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:12.499800  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:12.395591 14369 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:12.363030 14498 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:12.407315 14486 raft_consensus.cc:740] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:12.464046 14560 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:12.552857  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:12.544972  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:12.488623 14457 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.489094 14576 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:12.559988 14621 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:12.560647 14621 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:12.563084 14621 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:12.565830 14537 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:12.566257 14541 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:12.567655 14457 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:12.569895 14398 raft_consensus.cc:3055] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:12.570267 14457 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:12.570343 14398 raft_consensus.cc:740] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:12.486572 14588 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.070s
I20260709 13:04:12.549605  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:12.551133 13865 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:12.389726 14563 consensus_queue.cc:1059] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.128s
I20260709 13:04:12.545260  8400 raft_consensus.cc:2368] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18.
I20260709 13:04:12.575285 14563 raft_consensus.cc:3055] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:12.575783 14563 raft_consensus.cc:740] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:12.556797  8397 raft_consensus.cc:2393] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 17: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:12.472998 14615 consensus_queue.cc:1059] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.150s
I20260709 13:04:12.517800 14608 consensus_queue.cc:1059] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:04:12.665153 14608 raft_consensus.cc:3055] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:12.666318 14608 raft_consensus.cc:740] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:12.666921 14560 raft_consensus.cc:3055] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:12.667351 14560 raft_consensus.cc:740] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:12.667706  8297 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:12.668160 14560 consensus_queue.cc:260] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:12.668967 14608 consensus_queue.cc:260] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:12.670832 14618 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:12.671936 14560 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:12.672391  8297 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:12.673137 14618 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:12.674373  8297 leader_election.cc:400] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18.
I20260709 13:04:12.558429 14486 consensus_queue.cc:260] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:12.676829 14486 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:12.678126 14608 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:12.559577 14539 consensus_meta.cc:287] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.642s	user 0.011s	sys 0.000s
I20260709 13:04:12.409147 14587 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:12.483323 14423 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:12.698107  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:12.476256 14549 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:12.541450 14459 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:12.547343  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:12.613461 14418 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:12.547802 14460 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:12.612885 14378 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.617969  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:12.626894 14270 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.625090 14475 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:12.628648  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:12.629005  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:12.616287 14498 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:12.576771 14563 consensus_queue.cc:260] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:12.636812  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:12.637607  8974 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:12.618088  8298 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:12.635653  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:12.396965 14489 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:12.653014  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.660727  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:12.740108  8256 raft_consensus.cc:2393] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 20: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:12.662034  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:12.703502  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:12.714138  8402 raft_consensus.cc:2368] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 16. Current term is 17.
W20260709 13:04:12.743728  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 125735us
I20260709 13:04:12.744666 14539 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.745239 14539 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:12.746483  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 108971us
I20260709 13:04:12.743752  8405 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 20.
I20260709 13:04:12.750563  8298 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:12.717717 14270 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:12.753796  8406 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:12.757893  9007 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 22.
I20260709 13:04:12.762458  8403 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:12.742975 14417 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:12.766542  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:12.768375  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 965) took 1025 ms. Trace:
I20260709 13:04:12.769032 14559 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:12.718186 14460 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:12.715193 14459 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:12.781569  8298 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:12.791091  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:12.779704 14270 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:12.780927  8974 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:12.747433 14492 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:12.783670  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:12.786013  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:12.801043  8405 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 13:04:12.801324  8405 pending_rounds.cc:85] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 10
I20260709 13:04:12.786463  8323 rpcz_store.cc:276] 0709 13:04:11.743056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.743202 (+   146us) service_pool.cc:224] Handling call
0709 13:04:12.768188 (+1024986us) spinlock_profiling.cc:243] Waited 576 ms on lock 0x7b54003c8f54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:12.768321 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1325891968}
I20260709 13:04:12.793955  8372 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:12.755445 14475 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:12.792769  8223 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:12.797981  8406 raft_consensus.cc:1275] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:12.717515 14378 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:12.801530  8405 pending_rounds.cc:107] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 13:04:12.809255  8405 raft_consensus.cc:2889] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:12.797189  8974 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:12.813320  8406 raft_consensus.cc:1275] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 13:04:12.816323  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:12.818173  8223 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:12.802505 14559 raft_consensus.cc:697] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 23 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:12.802208 14546 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:12.813024 14502 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:12.816313 14270 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:12.816565 14568 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:12.797567 14417 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:12.855450 14417 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:12.829813  8406 pending_rounds.cc:85] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 10
I20260709 13:04:12.833015 14560 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:12.834976  8404 raft_consensus.cc:2410] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:04:12.831418 14270 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:12.846818 14459 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:12.867340  8223 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:12.848755 14587 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:12.853132 14539 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:12.884518 14460 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:12.889657  9007 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:12.890580  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:12.892344  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.892481  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:12.823766 14618 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:12.857962  8406 pending_rounds.cc:107] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:04:12.901124  8406 raft_consensus.cc:2889] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:12.862329  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:12.864329  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:12.866464  8297 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:12.857348 14270 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:12.863518  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:12.945276  8256 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:12.945634  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:12.947274  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 46426.1us
I20260709 13:04:12.870625  8372 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:04:12.897316  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 239021us
I20260709 13:04:12.899899  8405 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:12.942167  9000 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 13. Current term is 14. Ops: []
W20260709 13:04:12.949306  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 78893.1us
I20260709 13:04:12.950531  8372 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:12.953482  8248 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:12.953559  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:12.955346  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:12.955855  9000 raft_consensus.cc:2410] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:04:12.956429  8330 raft_consensus.cc:2410] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:04:12.957720  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:12.961344  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:12.962173  8323 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:04:12.975082  8256 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 23.
I20260709 13:04:12.979544  8248 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:04:12.963196  8406 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:04:12.980758  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:12.984247 14535 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:12.991426 14066 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:12.992276 14066 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:12.995286 14066 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:12.999293  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 132910us
I20260709 13:04:12.999503 14460 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:13.000155  8297 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:13.000325 14559 consensus_queue.cc:237] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:13.001299  8297 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:13.004632  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 56733.8us
I20260709 13:04:13.005383 14621 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:13.005859 14576 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:13.006166 14621 consensus_queue.cc:1048] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:13.007036 14576 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:13.009133 14626 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.009641 14626 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:13.010460 14460 consensus_queue.cc:1048] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:13.011150 14378 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:13.011819  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:13.011824  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:12.939013 14560 consensus_queue.cc:1059] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.084s
I20260709 13:04:13.023676 14560 raft_consensus.cc:3055] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:13.024098 14560 raft_consensus.cc:740] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:12.965665 14539 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:13.027740 14614 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:13.028493 14614 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:13.029150 14450 raft_consensus.cc:3055] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:13.029618 14450 raft_consensus.cc:740] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:13.030452 14450 consensus_queue.cc:260] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:13.031402 14614 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:13.031422 14450 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:13.042057 14489 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:12.893712 14581 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:13.044814  9004 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:12.984828 14535 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:13.047554  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:13.048229  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 69455.5us
I20260709 13:04:13.051028  8330 raft_consensus.cc:2393] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 17: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:13.052582  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:13.054718  8250 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:13.058382  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:04:13.060456  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 77720.5us
I20260709 13:04:13.057013 13865 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.052278  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:13.060184  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:13.067632  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:13.065169 13865 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:13.067173  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:13.073516  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:13.075889  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:13.068234  8329 raft_consensus.cc:2376] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21. Re-sending same reply.
I20260709 13:04:12.979029 14502 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:13.081107  8330 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:04:13.024866 14560 consensus_queue.cc:260] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:13.082250 13865 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:13.091631  8256 raft_consensus.cc:1275] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:13.092198  8253 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 16. Current term is 17. Ops: []
W20260709 13:04:13.087080  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 106390us
I20260709 13:04:13.048956 14492 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.108819 14492 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:13.112005 14492 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:13.112164  9002 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 13:04:13.049029 14550 consensus_queue.cc:1048] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:04:13.113538  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:13.114125  8326 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:04:13.114517  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:13.115156  8404 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
W20260709 13:04:13.115419  8326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 2798) took 1270 ms (client timeout 1507 ms). Trace:
I20260709 13:04:13.116123  8401 raft_consensus.cc:1275] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:04:13.115617 14270 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:13.116520  8401 pending_rounds.cc:85] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 9
I20260709 13:04:13.116833  8401 pending_rounds.cc:107] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:04:13.117013  9009 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:13.117072  8401 raft_consensus.cc:2889] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:13.096158  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.131340  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:13.132339  9002 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:04:13.106783 14618 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:13.141479 14618 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:13.096374  8369 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:13.097036  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:13.145782 14502 raft_consensus.cc:697] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:13.146019  8253 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
W20260709 13:04:13.115809  8326 rpcz_store.cc:269] 0709 13:04:11.845039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.845564 (+   525us) service_pool.cc:224] Handling call
0709 13:04:13.115273 (+1269709us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b540014c654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:13.115368 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2833204480}
I20260709 13:04:13.146888 14502 consensus_queue.cc:237] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:13.138157 14423 consensus_queue.cc:1059] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:13.079083 14608 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:13.159893 14570 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:13.160640 14570 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:13.161688 14568 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:04:13.164163  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 66493.1us
I20260709 13:04:13.164383 14570 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:13.165371 14570 consensus_queue.cc:1059] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:13.166994 14430 raft_consensus.cc:3055] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:13.167459 14430 raft_consensus.cc:740] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:13.168464 14430 consensus_queue.cc:260] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:13.170266 14430 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:13.172123  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:13.172626  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:13.173214  9007 raft_consensus.cc:1813] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Handling vote request from an unknown peer 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:13.173717  9007 raft_consensus.cc:2368] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 22. Current term is 23.
I20260709 13:04:13.174868  8298 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:13.181720 14492 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:13.182376 14492 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.183009 14492 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:13.189345  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:13.191506 14297 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:13.192284 14297 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:13.194777 14297 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:13.195356 14297 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:13.196596 14297 raft_consensus.cc:697] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:13.197597 14297 consensus_queue.cc:237] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:13.208860  8257 raft_consensus.cc:1275] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:04:13.213419  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:13.235152  8257 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:13.244850  8326 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:13.248309  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:13.249229  8325 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:13.251948  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:13.258263  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 161960us
I20260709 13:04:13.259344  8369 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:13.261264  8369 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:04:13.266927  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.217067 14560 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:13.269500  8222 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:13.268292 14491 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:13.272424 14491 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:13.272888 14491 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:13.285796  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:13.286468  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:13.291217  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 3059) took 1126 ms. Trace:
I20260709 13:04:13.297279  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:13.299160  8331 rpcz_store.cc:276] 0709 13:04:12.165197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.167508 (+  2311us) service_pool.cc:224] Handling call
0709 13:04:13.291046 (+1123538us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b540037c654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:13.291163 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24834816}
I20260709 13:04:13.218518 14535 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:13.264781 14460 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.290921 14501 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.295158  8972 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:13.269191 14587 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:13.302299  8404 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:04:13.332388  8222 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:13.333581 14535 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:13.334093 14535 raft_consensus.cc:697] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:13.335022 14535 consensus_queue.cc:237] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:13.328624  8223 leader_election.cc:400] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 22. Current term is 23.
I20260709 13:04:13.340449  8223 leader_election.cc:403] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:13.280597 14486 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:04:13.346524  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 51471.7us
I20260709 13:04:13.299187 14572 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:13.348245  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:13.356339  8404 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 19: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:04:13.348310  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 137854us
I20260709 13:04:13.352342  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:13.423823  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 71672.1us
I20260709 13:04:13.424672  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:13.436910  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 78999.7us
I20260709 13:04:13.437891  8295 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:13.439750  8295 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:13.440689 13865 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:13.329823 14540 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:13.458343 13865 consensus_queue.cc:1048] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:13.462909 14623 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:13.348436 14570 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:04:13.464217  8295 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:13.348646 14475 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:04:13.515714  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.516359  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:13.519438  8398 raft_consensus.cc:1813] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Handling vote request from an unknown peer 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:13.520103  8398 raft_consensus.cc:2368] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 22. Current term is 23.
I20260709 13:04:13.350184 14486 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:13.306250 14501 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:13.436601 14427 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:13.521095  8369 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:13.465027 14418 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:13.516417  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 52319.7us
I20260709 13:04:13.525262 14528 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:13.339535 14460 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:13.526782  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:13.451967 14572 raft_consensus.cc:697] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:13.545236  8295 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:13.308100 14398 consensus_queue.cc:260] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:13.550666 13865 consensus_queue.cc:1059] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:13.528008 14486 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:13.529078 14369 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:13.546749  8369 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:13.556977  8249 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2525) took 2080 ms (client timeout 1851 ms). Trace:
I20260709 13:04:13.537580 14608 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:13.557232  8295 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:13.564029 13865 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:13.564427 13865 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.564739 13865 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:13.569895 14572 consensus_queue.cc:237] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:13.573216  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:13.573947  8251 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 21.
I20260709 13:04:13.574692 14623 consensus_queue.cc:1048] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:13.576792 14623 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:13.557464  8249 rpcz_store.cc:269] 0709 13:04:11.476778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.476960 (+   182us) service_pool.cc:224] Handling call
0709 13:04:13.556806 (+2079846us) spinlock_profiling.cc:243] Waited 821 ms on lock 0x7b5400177d54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:13.556913 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1887790336}
I20260709 13:04:13.508967 14540 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:13.582100  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 275679us
I20260709 13:04:13.526572 14297 consensus_queue.cc:1059] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 13:04:13.552223 14418 consensus_queue.cc:1048] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:04:13.595734 14492 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:13.595773 13865 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:13.559141 14420 raft_consensus.cc:3055] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:04:13.601151  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 143524us
I20260709 13:04:13.579882 14552 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:13.603070 14420 raft_consensus.cc:740] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:13.599740  8405 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "ff7a6fe533e94c1bae555b4cae427728"
 from {username='slave'} at 127.0.0.1:46846
I20260709 13:04:13.608939 14420 consensus_queue.cc:260] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:13.609918  8405 raft_consensus.cc:606] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Received request to transfer leadership to TS ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:13.610397  8405 raft_consensus.cc:614] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:13.615206 13865 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:13.577248 14623 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:13.610352 14420 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:13.597414 14608 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:04:13.620414  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.577307 14618 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:13.626348 14491 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:13.674468 13865 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.687244 13865 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:13.688221  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 262421us
I20260709 13:04:13.709841  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:13.710798  8256 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 21.
W20260709 13:04:13.710707  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 98206.8us
I20260709 13:04:13.717447 13865 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:13.717583  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:13.718941  8248 raft_consensus.cc:2393] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 20: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:04:13.718742  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 78775.3us
I20260709 13:04:13.692482 14501 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:13.721164 14487 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:13.799016 14540 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:13.801247 14644 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:04:13.805692 14654 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.806340 14654 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:13.807013 14570 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 22. Current term is 23.
W20260709 13:04:13.797508  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 180761us
I20260709 13:04:13.808688 14654 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:13.868957  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:13.740584 14647 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.773358 14501 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:04:13.911674  8402 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 2648) took 1417 ms (client timeout 1850 ms). Trace:
I20260709 13:04:13.922799 14501 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:13.910998  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 1029) took 1248 ms. Trace:
I20260709 13:04:13.926249  8400 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 12. Current term is 13. Ops: []
W20260709 13:04:13.874971 14638 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.292s	user 0.000s	sys 0.001s
I20260709 13:04:13.927702 14647 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:13.868180 14487 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:13.929581 14618 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:13.925132 14501 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:13.923000  8402 rpcz_store.cc:269] 0709 13:04:12.494672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.499440 (+  4768us) service_pool.cc:224] Handling call
0709 13:04:13.911399 (+1411959us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54002be154. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:13.911596 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":383945984}
I20260709 13:04:13.923856  8403 rpcz_store.cc:276] 0709 13:04:12.662744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.664043 (+  1299us) service_pool.cc:224] Handling call
0709 13:04:12.664313 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:12.766303 (+101990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.910621 (+1144318us) spinlock_profiling.cc:243] Waited 97.9 ms on lock 0x7b54001cc854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:13.910898 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":225158528}
W20260709 13:04:13.882437 14627 log.cc:927] Time spent T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:04:13.942325  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:13.896132 14528 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:13.947440  8295 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:13.883337 14623 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:13.957406 14487 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:13.960505  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:13.961064  9006 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:13.963964  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:13.884194 14492 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:13.978524  8325 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:04:13.981705 14487 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:13.984033 14487 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:13.981995 14581 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.987082 14581 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:13.989359  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:13.990083  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:13.990839  8327 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:13.990689 14486 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:13.990124  8402 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:04:13.992424  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:13.958698 14550 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:13.996335  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:13.997121  8402 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
W20260709 13:04:14.001905  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.005089  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:13.989326 14581 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:13.994205 14535 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.994815 14487 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:13.996285 14550 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:04:14.019505  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 295583us
I20260709 13:04:14.020354  8369 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:13.999735 14608 raft_consensus.cc:697] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:14.026332 14486 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:14.027837  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:14.028826  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:14.031121 14460 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:14.013155 14655 consensus_queue.cc:1059] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.025s
I20260709 13:04:14.034554  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:14.034798  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:14.139398 14270 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:14.168952  9000 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:14.035575  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:14.043828 14663 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:14.044204  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:14.046799 14666 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:14.054220  9004 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:14.054812 14537 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:14.054912  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:14.056694  9003 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:04:14.057492 14670 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:14.015324 14535 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:14.066869 14665 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.021510 14550 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:14.010895 14491 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:14.069196 14450 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:14.073701 14657 log.cc:927] Time spent T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:04:14.067864  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.076751  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:14.186249  8248 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:14.078413  8401 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:04:14.082741 14651 log.cc:927] Time spent T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260709 13:04:14.096763  8397 raft_consensus.cc:1240] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:14.105861 14626 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:14.118147  8222 leader_election.cc:400] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 22. Current term is 23.
W20260709 13:04:14.195065  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 77123.1us
I20260709 13:04:14.124929  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:14.130224  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:14.196854  8402 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:14.197971  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:04:14.131332 14631 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 1.247s	user 0.009s	sys 0.016s
I20260709 13:04:14.138652  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:14.200398  8251 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:14.139469  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:14.201749  8256 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:14.139964  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:14.203372  8249 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:04:14.140228  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:14.141855  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:14.172668  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:14.176240  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 132147us
W20260709 13:04:14.176494  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.207537  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:14.167766 14406 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:14.225493 14066 raft_consensus.cc:3055] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:14.226171 14066 raft_consensus.cc:740] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:14.192252 13865 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.205354  8400 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:04:14.227097 13865 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:14.227893  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:14.228437  8400 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:14.170982 14666 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:14.184669 14450 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:14.231515 14666 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:14.194927 14665 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:14.209019  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:14.227128 14066 consensus_queue.cc:260] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:14.225836 14406 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:14.236716  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:14.237340  9002 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:04:14.237818 14406 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:14.183382 14664 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:14.238534 14664 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:14.240599 14664 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:14.214576  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:04:14.211454  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.189936 14563 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:14.188300 14670 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:14.243620 14670 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:14.176007 14537 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:14.188026 14663 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 13:04:14.182157 14668 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:14.247377 14537 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:14.248114 14668 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:14.248888 14420 consensus_meta.cc:287] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.622s	user 0.011s	sys 0.000s
I20260709 13:04:14.248709 14668 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:14.250871 14668 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:14.229568 13865 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:14.254114 14647 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:14.255298  8372 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:14.198520 14608 consensus_queue.cc:237] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:14.256356 14489 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:14.256808 14489 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.257146 14489 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:14.234946  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:14.259667  8401 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
I20260709 13:04:14.260644  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:14.211467  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:14.198590  8402 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:04:14.263530  8326 raft_consensus.cc:2393] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 13: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:14.263805  9004 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:04:14.234473 14665 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:14.267143  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 3066) took 1584 ms. Trace:
I20260709 13:04:14.267607  8328 rpcz_store.cc:276] 0709 13:04:12.682334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.682456 (+   122us) service_pool.cc:224] Handling call
0709 13:04:12.712271 (+ 29815us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:12.712742 (+   471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:12.712782 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:12.744580 (+ 31798us) log.cc:844] Serialized 52 byte log entry
0709 13:04:12.822229 (+ 77649us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:12.822270 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.822343 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:14.130543 (+1308200us) spinlock_profiling.cc:243] Waited 6.01 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:04:14.266807 (+136264us) raft_consensus.cc:1685] finished
0709 13:04:14.266841 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:14.267071 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1099,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":21195,"spinlock_wait_cycles":13814144,"thread_start_us":77261,"threads_started":1,"wal-append.queue_time_us":139270}
I20260709 13:04:14.272538  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2809) took 1471 ms. Trace:
W20260709 13:04:14.278402  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 144946us
I20260709 13:04:14.278606 14568 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:14.279317  8220 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:14.279183 14568 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:14.233297 14450 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:14.281534  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 108150us
I20260709 13:04:14.283383  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:14.284034  8397 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:14.285797 14617 consensus_queue.cc:1059] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:14.287761 14549 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:14.288311 14549 raft_consensus.cc:697] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:14.289407  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 93353.9us
I20260709 13:04:14.289623 14560 raft_consensus.cc:3055] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:14.290004 14560 raft_consensus.cc:740] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:14.291999  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.273017  8332 rpcz_store.cc:276] 0709 13:04:12.801320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.809933 (+  8613us) service_pool.cc:224] Handling call
0709 13:04:13.291760 (+481827us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:13.303256 (+ 11496us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:13.598867 (+295611us) log.cc:844] Serialized 14 byte log entry
0709 13:04:13.937986 (+339119us) log.cc:844] Serialized 14 byte log entry
0709 13:04:13.946156 (+  8170us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:13.970085 (+ 23929us) log.cc:844] Serialized 26 byte log entry
0709 13:04:13.978550 (+  8465us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:13.978603 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.978674 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:14.251341 (+272667us) raft_consensus.cc:1685] finished
0709 13:04:14.251364 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:14.272345 (+ 20981us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b540037c650. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:14.272471 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9319,"prepare.queue_time_us":365056,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":2984,"raft.queue_time_us":117984,"raft.run_cpu_time_us":379,"raft.run_wall_time_us":48676,"spinlock_wait_cycles":979472128,"thread_start_us":550174,"threads_started":3,"wal-append.queue_time_us":157595}
I20260709 13:04:14.303972  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:14.306962  8295 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:14.310971  8295 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:14.311659  9000 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:14.311959 13865 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:14.312500 13865 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.312909 13865 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:14.323127  8403 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 20.
I20260709 13:04:14.339716  8332 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:04:14.340382  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:14.328467  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:14.341925  8329 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:04:14.331095  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:14.331455  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:14.332319  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:14.345867  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:14.332891  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:14.346427  8298 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:14.347281  8256 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:04:14.333199  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:14.349398  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:14.350287  9000 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:04:14.338732  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:14.359472  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:14.360666  8329 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:04:14.319857 14568 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:14.374372  8298 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:14.376125  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:14.376993  8398 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
I20260709 13:04:14.378791  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:14.379652  8398 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:14.380637  8398 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "ff7a6fe533e94c1bae555b4cae427728"
 from {username='slave'} at 127.0.0.1:46846
I20260709 13:04:14.330627  8251 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:14.352335  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:14.354574  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:14.355206  8328 raft_consensus.cc:2393] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 21: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:14.318601 14625 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.312201  8295 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:14.389986  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 77895.9us
I20260709 13:04:14.390865  8295 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:14.367904  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:14.370815  8400 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:14.333144 14460 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:14.289211 14549 consensus_queue.cc:237] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:14.386238  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:14.388199  8256 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:04:14.389115  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:14.392616  8398 raft_consensus.cc:606] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Received request to transfer leadership to TS ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:14.392289 13865 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:14.394588  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:14.397120 14625 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:14.402087  8295 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:14.360136 14664 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:04:14.419456  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.408932  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:14.407838  8398 raft_consensus.cc:614] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:14.427418 13865 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:14.370635  8402 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:14.436323  8369 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:14.340176 14647 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:04:14.440819  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:14.410741  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:14.443584  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:14.446223  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:14.454312  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 43691.4us
W20260709 13:04:14.456010  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 136539us
I20260709 13:04:14.456251  8399 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:14.457509  8220 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:14.394306 14369 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:14.359102 14486 consensus_queue.cc:1059] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 13:04:14.460712  8220 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:14.434973 14559 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.432s	user 0.007s	sys 0.026s
I20260709 13:04:14.404860 14420 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:14.425552 14492 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:14.494179  8248 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:14.406883 14460 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.408805 14487 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:14.396957 14568 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:04:14.454341 14668 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:04:14.490479 14559 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.487s	user 0.007s	sys 0.027s
I20260709 13:04:14.462260 14270 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:14.462448 14369 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.426434 14491 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.079s
I20260709 13:04:14.458375 14406 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:14.454499 14663 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:14.547068 13865 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:04:14.546970  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 144969us
W20260709 13:04:14.541299  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:14.556304  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 115593us
I20260709 13:04:14.491802 14420 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.395164 14560 consensus_queue.cc:260] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:14.533116 14460 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:14.545140 14378 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:14.599171 14599 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:14.593001 14664 raft_consensus.cc:697] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 16 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:14.598686 14613 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:14.603093  8324 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 23 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 13:04:14.540287 14668 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.556959 14369 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:14.556712 14270 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:14.650637 14420 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:14.596508 14637 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:04:14.537623 14487 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:14.560624 14406 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:04:14.594267  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.560341 14663 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:14.626046 14297 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.599654 14643 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:04:14.667066  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 347485us
I20260709 13:04:14.692427  8222 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:14.693357  8222 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:14.694334  8222 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:14.663246  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:14.673485 13865 raft_consensus.cc:697] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:14.557775 14572 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:14.661051 14646 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.668959 14686 raft_consensus.cc:3055] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:14.678880 14668 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:14.655419 14581 consensus_queue.cc:1059] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:04:14.704013  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:14.698261 13865 consensus_queue.cc:237] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:14.681319 14270 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:14.740661  8369 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:14.688181 14406 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:14.686457 14535 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:14.692173 14663 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:14.744807  8369 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:14.691864 14574 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:14.683053 14599 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:14.755506  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 263703us
I20260709 13:04:14.742038 14643 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:14.754768 14691 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:14.749296 14646 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:14.745908 14572 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:14.710160 14686 raft_consensus.cc:740] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:14.737736 14378 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:14.772123 14691 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:14.753163 14539 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:04:14.755192 14674 log.cc:927] Time spent T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 13:04:14.786283  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:14.790952 14489 consensus_meta.cc:287] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.000s
W20260709 13:04:14.793396  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.706220 14297 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:14.692369 14613 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:14.787245 14550 consensus_meta.cc:287] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.604s	user 0.001s	sys 0.010s
I20260709 13:04:14.806872  8328 raft_consensus.cc:1275] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
W20260709 13:04:14.791356 14563 consensus_meta.cc:287] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.550s	user 0.009s	sys 0.001s
I20260709 13:04:14.756551 14535 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:04:14.803205  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 108006us
W20260709 13:04:14.834071 14684 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 13:04:14.835049 14297 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:14.835146 14671 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:14.802752 14599 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:14.828986  8328 pending_rounds.cc:85] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 9
I20260709 13:04:14.836997  8328 pending_rounds.cc:107] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:04:14.837286  8328 raft_consensus.cc:2889] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:14.837673 14686 consensus_queue.cc:260] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:14.839103 14686 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:14.839684  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.785665 14625 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:14.795047 14691 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:14.837195 14675 log.cc:927] Time spent T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 13:04:14.806675  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.833271  8250 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:14.710464 14425 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 13:04:14.821168 14489 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:14.781922 14574 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:14.926469  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 169505us
W20260709 13:04:14.930814  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 54142us
I20260709 13:04:14.929128  8220 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:14.932969  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 436216us
I20260709 13:04:14.935400  8972 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:14.843371 14535 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.944145  8404 raft_consensus.cc:2393] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 16: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:14.872444 14570 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.770031 14664 consensus_queue.cc:237] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:15.020182  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 83245.9us
W20260709 13:04:15.022995  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 84413.4us
W20260709 13:04:15.020069  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 49798.6us
I20260709 13:04:14.831094 14550 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:15.032483  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:14.933602 14678 log.cc:927] Time spent T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.333s	user 0.001s	sys 0.000s
I20260709 13:04:15.026197 14535 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:14.944145 14670 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.039927  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:15.018879 14297 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:15.040769  8328 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:04:15.050051  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:15.033058 14679 log.cc:927] Time spent T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.379s	user 0.000s	sys 0.002s
I20260709 13:04:15.059121 14698 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:15.059870 14682 log.cc:927] Time spent T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 13:04:15.059705 14698 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:15.062054 14489 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:15.062222  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2558) took 1008 ms. Trace:
I20260709 13:04:15.063118 14700 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:15.063441 14693 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
W20260709 13:04:15.065564 14689 log.cc:927] Time spent T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.319s	user 0.001s	sys 0.000s
I20260709 13:04:15.065877 14672 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.062747  8257 rpcz_store.cc:276] 0709 13:04:14.053394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.139578 (+ 86184us) service_pool.cc:224] Handling call
0709 13:04:14.139891 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:14.140234 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:14.140266 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:14.140740 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 13:04:14.342563 (+201823us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:14.342623 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.342700 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:15.062024 (+719324us) raft_consensus.cc:1685] finished
0709 13:04:15.062044 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:15.062165 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63329,"raft.queue_time_us":21299,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":6698,"thread_start_us":201432,"threads_started":1,"wal-append.queue_time_us":351620}
I20260709 13:04:15.073944 14487 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:15.060343 14574 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:15.086735 14599 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.087284 14599 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:15.088891  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 732534us
I20260709 13:04:15.089357 14599 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:15.105959 14698 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:15.101519 14672 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:15.107802  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:04:15.084961  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.109037  8401 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:15.114024  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:15.108026 14672 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:15.104038 14643 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:15.120434 14691 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:15.121182  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:15.123745  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:15.125002 14420 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:15.126706 14599 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:15.127414 14420 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:15.128046 14420 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.128582 14420 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:15.130046 14671 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:15.131273 14676 log.cc:927] Time spent T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.339s	user 0.000s	sys 0.000s
I20260709 13:04:15.127321 14599 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:15.132735  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3231) took 1864 ms. Trace:
I20260709 13:04:15.133205  8406 rpcz_store.cc:276] 0709 13:04:13.268087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.579132 (+311045us) service_pool.cc:224] Handling call
0709 13:04:13.579410 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:14.011135 (+431725us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:14.011166 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:14.011650 (+   484us) log.cc:844] Serialized 26 byte log entry
0709 13:04:14.462093 (+450443us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:14.462153 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.462584 (+   431us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:15.132468 (+669884us) raft_consensus.cc:1685] finished
0709 13:04:15.132497 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:15.132651 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5083,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":5280,"thread_start_us":723764,"threads_started":2,"wal-append.queue_time_us":677253}
I20260709 13:04:15.134346 14599 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:15.136024 14692 log.cc:927] Time spent T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.208s	user 0.000s	sys 0.000s
W20260709 13:04:15.138116 14688 log.cc:927] Time spent T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.265s	user 0.000s	sys 0.000s
I20260709 13:04:15.139173  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:15.141376  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:15.142272 14699 log.cc:927] Time spent T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:04:15.144325 14459 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:15.144865 14459 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:15.145298 14691 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:15.145920  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2661) took 2274 ms. Trace:
I20260709 13:04:15.146311  8399 rpcz_store.cc:276] 0709 13:04:12.873048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.873418 (+   370us) service_pool.cc:224] Handling call
0709 13:04:13.912767 (+1039349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:14.456936 (+544169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:15.144419 (+687483us) spinlock_profiling.cc:243] Waited 543 ms on lock 0x7b54002be154. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:15.145715 (+  1296us) spinlock_profiling.cc:243] Waited 967 ms on lock 0x7b54002be150. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:15.145838 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3473984512}
I20260709 13:04:15.146974 14459 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:15.148090 14550 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:15.123471 14663 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:15.153717  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:15.159667  8401 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
W20260709 13:04:15.168403  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 90337.8us
I20260709 13:04:15.063222 14662 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.033717 14570 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:15.101553 14066 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:15.050523 14670 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:15.178059  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 2667) took 1054 ms. Trace:
I20260709 13:04:15.178630  8404 rpcz_store.cc:276] 0709 13:04:14.123749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.124630 (+   881us) service_pool.cc:224] Handling call
0709 13:04:15.177838 (+1053208us) spinlock_profiling.cc:243] Waited 748 ms on lock 0x7b5400267ad4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:15.177992 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1719273344}
I20260709 13:04:15.179641 14670 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:15.182379  8974 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:04:15.182956 14460 consensus_meta.cc:287] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.514s	user 0.008s	sys 0.000s
I20260709 13:04:15.182758 14662 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:15.183375 14460 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:15.184435  8328 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:15.186625 14460 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:15.062054 14613 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:15.173460  8405 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
W20260709 13:04:15.173734 14690 log.cc:927] Time spent T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.444s	user 0.000s	sys 0.001s
I20260709 13:04:15.101536 14702 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:15.192028  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:15.185165 14662 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:15.189181  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.190673  8297 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:15.194641  9007 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:15.197484  8972 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:15.189456 14692 log.cc:927] Time spent T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:04:15.198686  8297 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:15.202129  8297 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:15.185266 14646 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:15.210527  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:15.188547  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 169648us
I20260709 13:04:15.190156 14655 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:15.192281 14626 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:15.223804  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:15.195514 14297 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:04:15.228246  8400 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.016s	sys 0.016s
W20260709 13:04:15.228945  8400 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.016s	sys 0.016s
I20260709 13:04:15.228933  8222 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:15.230043  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:15.233793  9003 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:04:15.212265 14662 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:15.231396  9009 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:04:15.212038 14460 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:15.217063 14528 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:15.230607 14623 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:15.249380  8222 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:15.225492  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 195755us
I20260709 13:04:15.243681 14460 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:15.243343 14662 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:15.247748 14703 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:15.251444 14697 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:15.266536  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:15.217787 14406 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:15.264353  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 50097.9us
I20260709 13:04:15.268791  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:15.260193 14369 consensus_meta.cc:287] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.570s	user 0.009s	sys 0.002s
I20260709 13:04:15.199793 14702 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:15.271684  8220 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:15.267083  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:15.291520  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:15.249655 14528 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:15.284788  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:15.229599 14655 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:15.231362 14626 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:15.306188  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:15.269881 14706 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:15.313078  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:15.314967  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:15.315217  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:15.316172  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:15.316648  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:15.317086  8331 raft_consensus.cc:2393] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:04:15.318033  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.292166  8327 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:04:15.291213  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:04:15.285563  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.297608  8250 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:04:15.297709  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:15.269750 14711 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:15.299387  8254 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:15.303009  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:15.303718  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:15.289244  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:15.326565  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:15.328174  8250 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:15.277743 14460 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:15.275493 14662 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:15.322955  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:15.288201 14549 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.326400 13865 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.290030  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:15.338415  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:15.287027 14559 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:15.334336  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:15.345861  8249 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:15.260308 14623 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:15.335294 13865 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:15.342911  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:15.339170  8330 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
W20260709 13:04:15.310264 14270 consensus_meta.cc:287] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.556s	user 0.010s	sys 0.000s
I20260709 13:04:15.299345 14528 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:15.331388  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:15.294338 14665 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:15.356248  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2564) took 1216 ms. Trace:
I20260709 13:04:15.305871 14537 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:15.396880  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 168593us
I20260709 13:04:15.291641 14710 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:15.495704  8255 rpcz_store.cc:276] 0709 13:04:14.139384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.166640 (+ 27256us) service_pool.cc:224] Handling call
0709 13:04:14.167002 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:14.171454 (+  4452us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:14.357174 (+185720us) log.cc:844] Serialized 14 byte log entry
0709 13:04:14.877261 (+520087us) log.cc:844] Serialized 14 byte log entry
0709 13:04:15.132033 (+254772us) spinlock_profiling.cc:243] Waited 5.72 ms on lock 0x7b18002c9560. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882564bc0a 00007f882ed8b219 00007f88256d3a40 00007f88256ce4e4 00007f88256ce455 00007f882c1b63c7 00007f88287c89a7 00007f88287d80bf 00007f88287d8038
0709 13:04:15.132239 (+   206us) log.cc:844] Serialized 14 byte log entry
0709 13:04:15.211512 (+ 79273us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:15.211973 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 13:04:15.212097 (+   124us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:15.212139 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:15.217741 (+  5602us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:15.355964 (+138223us) raft_consensus.cc:1685] finished
0709 13:04:15.355996 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:15.356175 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":336960,"prepare.queue_time_us":319633,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":117,"raft.queue_time_us":331467,"raft.run_cpu_time_us":537,"raft.run_wall_time_us":50317,"spinlock_wait_cycles":20649600,"thread_start_us":623291,"threads_started":3,"wal-append.queue_time_us":417212}
I20260709 13:04:15.504292  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:15.306759 14297 raft_consensus.cc:697] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 20 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:15.327303 14697 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:15.290258 14572 consensus_queue.cc:1048] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:04:15.292834 14369 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:15.416718 13865 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:15.418476  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:15.345539 14487 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.424206  8220 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:15.407490 14683 log.cc:927] Time spent T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:04:15.552992 14487 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:15.555361 14487 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:15.555856 14670 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:15.556386 14670 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:15.557801 14406 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:15.559485 14297 consensus_queue.cc:237] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:15.356690 14378 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.298830 14552 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:15.441952  8249 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
W20260709 13:04:15.575306  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 151197us
I20260709 13:04:15.408051 14663 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:15.462911  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2677) took 1272 ms. Trace:
I20260709 13:04:15.580302  8400 rpcz_store.cc:276] 0709 13:04:14.190807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.448398 (+257591us) service_pool.cc:224] Handling call
0709 13:04:14.450671 (+  2273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:14.450912 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:15.231801 (+780889us) log.cc:844] Serialized 14 byte log entry
0709 13:04:15.462424 (+230623us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:15.462604 (+   180us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:15.462654 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:15.462740 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:15.462854 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8848,"prepare.queue_time_us":764223,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":7461,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":9957,"thread_start_us":910101,"threads_started":2,"wal-append.queue_time_us":239171}
I20260709 13:04:15.356843 14625 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.091s
I20260709 13:04:15.506311 14665 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:15.347959 14549 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:15.584578 14665 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:15.585757 14713 log.cc:927] Time spent T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:04:15.586740 14378 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:15.516801 14537 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:15.588876 14378 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:15.590842 14535 consensus_meta.cc:287] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.553s	user 0.008s	sys 0.001s
I20260709 13:04:15.591308 14535 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:15.358695 14559 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:15.605674 14623 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:15.358029 14711 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:15.465224  9007 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:04:15.485930  8250 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
I20260709 13:04:15.355191 14706 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:15.616981  8372 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:15.618470 14498 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:15.619175 14706 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:15.619148 14498 raft_consensus.cc:697] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:15.489353 14671 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:15.459059 14420 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:15.474581 14703 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:15.620167 14671 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:15.620322 14703 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:15.620502 14498 consensus_queue.cc:237] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:15.622694 14703 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:15.500694 14655 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:15.501565 14702 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:15.520474  8255 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:04:15.523761  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:15.528275  8332 raft_consensus.cc:2393] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 16: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:15.528733 14369 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:15.493857 14270 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:15.532074  8248 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:15.538750  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:15.628789  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:15.546305  8251 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 21.
W20260709 13:04:15.552580 14668 consensus_meta.cc:287] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.806s	user 0.009s	sys 0.002s
I20260709 13:04:15.570029  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:15.571841  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:15.630463  8298 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:15.573468  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:15.325395 14666 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.086s
I20260709 13:04:15.578244 13865 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:15.636544 13865 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:15.596227  8253 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:04:15.599912 14720 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:15.600984  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:15.593393 14535 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:15.603221  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:15.612314  8220 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:15.586779 14549 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:15.650429  9005 raft_consensus.cc:2393] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 16: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:04:15.612421  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 256013us
I20260709 13:04:15.615111  8252 raft_consensus.cc:2393] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 20: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:15.616818  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:15.595981 14710 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:15.652863  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.653187  8220 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:15.653651 13865 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:15.654187  8220 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:15.607131 14559 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:15.629079  9009 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
W20260709 13:04:15.550221 14714 log.cc:927] Time spent T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:04:15.659749  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:15.632128  8398 raft_consensus.cc:2410] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:04:15.635251  8406 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 19: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:15.660880  9005 raft_consensus.cc:2393] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 14: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:15.638938  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:15.664187  9001 raft_consensus.cc:2393] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 21: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:15.640897  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:15.641750  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:15.644066 14270 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:15.646536  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:15.601400 14670 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:15.646979  9006 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:15.666307  9003 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:15.661721  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:15.645912 14702 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:15.668720  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:15.593710 14552 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:15.673681  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:15.679680  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:15.663437 14693 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:15.659825 14647 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:15.656828 14700 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:15.680603  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:15.680944  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:15.675434  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:15.682930  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:15.646083 14668 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:15.684860  8406 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:04:15.686169  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:15.687382  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:15.690970  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:15.692281  8295 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:15.701416  8400 raft_consensus.cc:2393] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 20: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:15.691663  8327 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:04:15.703327  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:15.704032  8330 raft_consensus.cc:2393] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 20: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:15.704759  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:15.686825 14537 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:15.706869  8323 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:15.711521 14552 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:15.684180 14693 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:15.713717 14693 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:15.713948  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:15.714625  9001 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 23.
I20260709 13:04:15.717711 14668 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:15.663196 14720 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:15.720626 14720 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:15.734756  8220 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:15.678805 14528 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:15.744328  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 1066) took 1301 ms. Trace:
I20260709 13:04:15.744895 14528 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:15.746877  9004 raft_consensus.cc:2368] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19.
I20260709 13:04:15.746949 14552 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:15.750276  8220 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:15.752816 14549 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:15.755802 14549 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:15.756206 14549 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:15.761664 14537 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:15.762262 14537 raft_consensus.cc:697] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 19 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:15.762817  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:15.751546 14671 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:15.763597  8406 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 23: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:15.763095 14537 consensus_queue.cc:237] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:15.764470  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:15.744774  8256 rpcz_store.cc:276] 0709 13:04:14.443029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.445418 (+  2389us) service_pool.cc:224] Handling call
0709 13:04:14.445787 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:14.707498 (+261711us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:14.707534 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:14.707599 (+    65us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:14.707631 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:15.744047 (+1036416us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b54001306d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:15.744099 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:15.744249 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":601395968}
I20260709 13:04:15.712924 14460 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:15.746534 14711 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:15.767558 14687 log.cc:927] Time spent T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.981s	user 0.003s	sys 0.000s
I20260709 13:04:15.771772 14549 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:15.775362  8405 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:04:15.777017  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:15.777568  8327 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:15.780900  8405 pending_rounds.cc:85] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 12
I20260709 13:04:15.781219  8405 pending_rounds.cc:107] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 13:04:15.781436  8405 raft_consensus.cc:2889] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:15.785686  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:15.786610  8406 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:04:15.725909 14662 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:15.809969 14705 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:15.810668 14705 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:15.814618 14705 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:15.816453  9003 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:04:15.816695  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2815) took 2897 ms. Trace:
I20260709 13:04:15.817034  8324 rpcz_store.cc:276] 0709 13:04:12.919598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.942933 (+ 23335us) service_pool.cc:224] Handling call
0709 13:04:12.943295 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:14.670178 (+1726883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:15.815954 (+1145776us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b540033b420. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f8828735dc0 00007f88287c1a02 00007f88287bddd6 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:04:15.816439 (+   485us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b54003c9e54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:15.816616 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3795956736}
I20260709 13:04:15.817679  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:15.821857 14460 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:15.822872  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:15.824115  8406 raft_consensus.cc:2393] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 18: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:04:15.832139  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.833971 14704 log.cc:927] Time spent T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.763s	user 0.002s	sys 0.000s
I20260709 13:04:15.835549  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:15.843544  8330 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:15.845156  8327 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18.
I20260709 13:04:15.794935 14549 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:15.851019  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:15.851907  8324 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 19
W20260709 13:04:15.863348 14731 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:04:15.864617 14662 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:15.866928 14693 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:15.870220 14695 log.cc:927] Time spent T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.808s	user 0.000s	sys 0.002s
I20260709 13:04:15.874897 14693 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:15.884490 14398 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:15.888943 14626 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:15.889283 14560 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:15.896718 14489 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.897420 14489 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:15.903810 14489 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:15.943027  8220 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:15.941727  8147 catalog_manager.cc:5697] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:15.946372  8257 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:15.949542  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:15.950812  8323 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:04:15.952554  8326 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:15.954308  8397 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:04:15.955394  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:15.961165 14693 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:15.974493 14709 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:15.975762 14709 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:15.978469  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:15.979861  8399 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:15.984592  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:15.985235  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:15.986301  8405 raft_consensus.cc:2393] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 13: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:15.989710 14693 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:15.971526 14698 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:15.996414 14698 raft_consensus.cc:697] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 18 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:15.996516 14693 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:15.998337 14698 consensus_queue.cc:237] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:16.002599  8223 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:16.004278  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:16.009650  8222 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:16.014030 14486 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:16.015491 14486 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:16.016172 14486 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.016426 14709 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:16.016811 14486 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:16.020650  8297 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:16.025748 14664 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:16.026701 14664 raft_consensus.cc:697] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:16.033174  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:16.038527  8222 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:16.039718 14693 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:04:16.040995  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.042462  8222 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:16.053057  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:16.054252  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 1047) took 1269 ms. Trace:
I20260709 13:04:16.055308 14574 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.056483  8297 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:16.058104 14491 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:16.064855 14617 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:16.071692  9006 raft_consensus.cc:2393] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 19: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:16.071309 14691 consensus_queue.cc:1059] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.002s
I20260709 13:04:16.066340  8397 rpcz_store.cc:276] 0709 13:04:14.785400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.786040 (+   640us) service_pool.cc:224] Handling call
0709 13:04:16.053991 (+1267951us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54000fd7d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:16.054158 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2652915968}
I20260709 13:04:15.979894 14560 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:16.089886 14560 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:16.091616  8399 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 22.
I20260709 13:04:16.083115  8974 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:16.092036 14574 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:16.094007  8974 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:16.095546 14574 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:16.096488 14066 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:16.084810 14617 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:16.099786 14617 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:16.086704  8223 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:16.100953 14066 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:16.088979  8324 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
I20260709 13:04:16.105892  8974 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:16.107993 14587 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:16.111930 14560 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:16.112074  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2844) took 1326 ms. Trace:
I20260709 13:04:16.109026 14587 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:16.028816 14664 consensus_queue.cc:237] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:16.112690  8329 rpcz_store.cc:276] 0709 13:04:14.785861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.785996 (+   135us) service_pool.cc:224] Handling call
0709 13:04:14.786311 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:15.184221 (+397910us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:15.184251 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:15.184696 (+   445us) log.cc:844] Serialized 26 byte log entry
0709 13:04:15.251363 (+ 66667us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:15.253004 (+  1641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.111389 (+858385us) spinlock_profiling.cc:243] Waited 398 ms on lock 0x7b540014c654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:16.111476 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:16.111569 (+    93us) raft_consensus.cc:1685] finished
0709 13:04:16.111587 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:16.111872 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1552,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"spinlock_wait_cycles":914329472,"thread_start_us":66291,"threads_started":1,"wal-append.queue_time_us":111130}
I20260709 13:04:16.116165 14587 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:16.118297 14691 raft_consensus.cc:3055] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:04:16.118517  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 76374.1us
I20260709 13:04:16.118889 14691 raft_consensus.cc:740] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:16.119586 14563 raft_consensus.cc:2764] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:04:16.120293 14691 consensus_queue.cc:260] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:16.122695 14691 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:16.125182  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 57354.6us
I20260709 13:04:16.126498 14570 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:16.127370 14570 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:16.128746  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:16.129261  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:16.130276 14406 consensus_queue.cc:1059] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:16.131906 14406 raft_consensus.cc:3055] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:16.132526 14406 raft_consensus.cc:740] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:16.132637  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:16.133016  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:16.133526  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:16.134331  8400 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18.
I20260709 13:04:16.135126  8252 raft_consensus.cc:2368] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15.
I20260709 13:04:16.136468 14406 consensus_queue.cc:260] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:16.140098  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:16.140097 14406 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:16.141860  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:16.142594  8972 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:16.143056  8324 raft_consensus.cc:2368] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14.
I20260709 13:04:16.145052 14626 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:16.149468  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:16.149039  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:16.151400  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:16.154136  8399 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:16.162218  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2674) took 1984 ms. Trace:
I20260709 13:04:16.151504  8254 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:04:16.163033  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:16.162292  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:16.162740  8403 rpcz_store.cc:276] 0709 13:04:14.177227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.395476 (+218249us) service_pool.cc:224] Handling call
0709 13:04:14.395977 (+   501us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:14.398961 (+  2984us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:15.772062 (+1373101us) spinlock_profiling.cc:243] Waited 4.81 ms on lock 0x7b18004253e0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256ce455 00007f882c1b63c7 00007f88287c89a7 00007f88287d80bf 00007f88287d8038
0709 13:04:15.847305 (+ 75243us) log.cc:844] Serialized 14 byte log entry
0709 13:04:16.054952 (+207647us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:16.055815 (+   863us) log.cc:844] Serialized 26 byte log entry
0709 13:04:16.056178 (+   363us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:16.056228 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.056409 (+   181us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:16.061974 (+  5565us) raft_consensus.cc:1685] finished
0709 13:04:16.068649 (+  6675us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:16.162021 (+ 93372us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9777,"prepare.queue_time_us":789251,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":15037,"raft.run_cpu_time_us":526,"raft.run_wall_time_us":15226,"spinlock_wait_cycles":11066240,"thread_start_us":193765,"threads_started":2,"wal-append.queue_time_us":58755}
I20260709 13:04:16.168504  8974 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:16.169183  8324 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 17. Current term is 18. Ops: [17.10-17.10]
W20260709 13:04:16.172271  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 62002.2us
I20260709 13:04:16.174070  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:16.179119  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:16.184521 14427 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:16.185201 14427 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.185649 14427 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:16.185799 14701 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:16.186640 14701 raft_consensus.cc:697] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:16.187975 14701 consensus_queue.cc:237] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:16.188983  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 132711us
I20260709 13:04:16.177840  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:16.190176  8401 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:04:16.190801  8298 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:16.191869  8297 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:16.193008 14297 consensus_queue.cc:1059] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:16.193974 14297 consensus_queue.cc:1243] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer ff7a6fe533e94c1bae555b4cae427728 log is divergent from this leader: its last log entry 18.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:04:16.194796 14554 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:16.200206  8323 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:04:16.202008 14369 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:16.208377 14574 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:16.208918 14574 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.209297 14574 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:16.214567  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:16.215500  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:16.226027  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:16.224328  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:16.237181  8399 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:04:16.241701 14475 raft_consensus.cc:3055] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:16.243846  8256 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 23: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:16.244827  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:04:16.247846  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 56564.8us
I20260709 13:04:16.248834  8402 raft_consensus.cc:2393] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 18: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:16.249850  8222 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:16.251082 14406 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:16.251664 14406 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:16.253496 14406 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:16.255838 14369 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:16.265105  8254 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:16.270095 14486 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:16.272763 14486 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:16.243845 14475 raft_consensus.cc:740] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:16.299878 14475 consensus_queue.cc:260] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:16.273859  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:16.303323 14450 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:16.303587  8402 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "ff7a6fe533e94c1bae555b4cae427728"
 from {username='slave'} at 127.0.0.1:46846
I20260709 13:04:16.304026 14450 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:16.274294  8147 catalog_manager.cc:5697] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 15 to 17, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 17 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:16.306401  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.278031 14671 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:16.281980  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:16.308050 14671 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:16.306946  8147 catalog_manager.cc:5697] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 10 to 13, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 13 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:16.309633 14491 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:16.310786 14491 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:16.309990  8147 catalog_manager.cc:5697] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 7 to 17, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 17 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:16.304579  8402 raft_consensus.cc:606] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Received request to transfer leadership to TS ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:16.281980  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:16.339444  8402 raft_consensus.cc:614] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:16.340070  8256 raft_consensus.cc:2410] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 22 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 10.
I20260709 13:04:16.313897 14491 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:16.314106  9008 raft_consensus.cc:1275] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:04:16.315896  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:16.342826  8329 raft_consensus.cc:2393] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 19: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:16.301421 14475 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:16.318924  8324 raft_consensus.cc:1275] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:04:16.320127  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:16.298233  8323 raft_consensus.cc:2393] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 18: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:16.305171  8254 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:04:16.326088 14427 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:16.351804 14406 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:16.333571 14450 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:16.344010  8147 catalog_manager.cc:5697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:16.359351  8147 catalog_manager.cc:5697] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 14 to 18, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 18 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:16.361166  8147 catalog_manager.cc:5697] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 15 to 17, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 17 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:16.354368  8222 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:16.353485 14427 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:16.331162 14671 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:16.346573  8148 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55754 (request call id 201) took 46746 ms (client timeout 15000 ms). Trace:
I20260709 13:04:16.366888  8222 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:16.372361  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:16.373250 14626 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:16.374432 14626 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:16.376847 14626 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:16.376993  8295 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:16.372424  8298 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:16.372748  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:16.393304  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:16.379233 14486 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:16.429837  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:04:16.431154  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 58882.7us
I20260709 13:04:16.431324 14549 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:16.431838 14549 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:16.432250 14549 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:16.433375  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 56667.4us
I20260709 13:04:16.434283  8295 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:16.415934 14475 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.436367 14475 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:16.436414 14730 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:16.438155 14730 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:16.438678 14730 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:16.419720 14486 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:16.393988 14450 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:04:16.447002  8253 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 23.
I20260709 13:04:16.435218  8147 catalog_manager.cc:5697] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 12 to 18, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 18 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:16.391880 14626 consensus_queue.cc:1059] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 13:04:16.462100 14559 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:16.461297  8147 catalog_manager.cc:5697] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 8 to 13, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 13 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:04:16.458614 14664 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:04:16.465081  8147 catalog_manager.cc:5697] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 17 to 18, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 18 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:04:16.468596  8402 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:04:16.451582 14486 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:16.486975  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:16.512452  8399 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:16.518306  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.494764  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:16.490785 14662 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:16.492820 14554 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.485699 14406 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:16.504081 14559 raft_consensus.cc:697] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:16.533185  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:16.538213  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:16.538938  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:16.528162  8147 catalog_manager.cc:5697] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:04:16.539521  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:16.539611  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:16.540475  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:16.519793 14626 raft_consensus.cc:3055] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:16.481295 14608 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.547410  9009 raft_consensus.cc:2393] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 19: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
W20260709 13:04:16.545678  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 90013.5us
I20260709 13:04:16.550997 14491 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:16.527256 14369 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:04:16.551752 14554 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:16.548063 14626 raft_consensus.cc:740] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:16.549082 14662 raft_consensus.cc:697] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:16.553817  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:16.553960 14626 consensus_queue.cc:260] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:16.554497  8398 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:16.554540 14662 consensus_queue.cc:237] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:16.555732 14626 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:16.555954 14475 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:16.551409 14491 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.561266  8402 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:16.570883  9004 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:16.567049  8222 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:04:16.368335  8148 rpcz_store.cc:269] 0709 13:03:29.599452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.599673 (+   221us) service_pool.cc:224] Handling call
0709 13:04:16.346183 (+46746510us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:11.082874 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:11.094525 (+ 11651us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:11.094583 (+    58us) write_op.cc:435] Acquired schema lock
0709 13:04:11.094606 (+    23us) tablet.cc:662] Decoding operations
0709 13:04:11.096081 (+  1475us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:11.096323 (+   242us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:11.096376 (+    53us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:04:11.099335 (+  2959us) tablet.cc:701] Row locks acquired
0709 13:04:11.099355 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:04:11.099501 (+   146us) write_op.cc:270] Start()
0709 13:04:11.099817 (+   316us) write_op.cc:276] Timestamp: P: 1783602251099450 usec, L: 0
0709 13:04:11.099849 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 13:04:11.100691 (+   842us) log.cc:844] Serialized 3604 byte log entry
0709 13:04:11.702731 (+602040us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:04:12.288165 (+585434us) op_driver.cc:464] REPLICATION: finished
0709 13:04:12.557936 (+269771us) write_op.cc:301] APPLY: starting
0709 13:04:12.558032 (+    96us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:12.652859 (+ 94827us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:12.652888 (+    29us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:12.775802 (+122914us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:12.776039 (+   237us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=18,mrs_lookups=0
0709 13:04:12.776083 (+    44us) write_op.cc:312] APPLY: finished
0709 13:04:12.776356 (+   273us) log.cc:844] Serialized 89 byte log entry
0709 13:04:12.776519 (+   163us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:12.776646 (+   127us) write_op.cc:454] Released schema lock
0709 13:04:12.776691 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:12.777108 (+   417us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:12.777134 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":341539,"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":269473,"apply.run_cpu_time_us":14434,"apply.run_wall_time_us":219167,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41765,"num_ops":9,"prepare.queue_time_us":333999,"prepare.run_cpu_time_us":20905,"prepare.run_wall_time_us":154643,"raft.queue_time_us":554733,"raft.run_cpu_time_us":44593,"raft.run_wall_time_us":142203,"replication_time_us":1188137,"spinlock_wait_cycles":304536832,"thread_start_us":856459,"threads_started":4,"wal-append.queue_time_us":167251,"wal-append.run_cpu_time_us":42387,"wal-append.run_wall_time_us":2521882}]]}
I20260709 13:04:16.584677  8147 catalog_manager.cc:5697] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 8 to 17, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 17 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:16.587884  8147 catalog_manager.cc:5697] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 9 to 16. New cstate: current_term: 16 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:04:16.563046 14491 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:16.591817  8403 raft_consensus.cc:3055] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:16.607329  8403 raft_consensus.cc:740] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:16.599079 14608 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:16.602388 14554 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:16.614209  8399 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:16.615927  8222 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:16.605468 14406 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:16.624715  8147 catalog_manager.cc:5697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 18 to 20, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 20 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:16.621219  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:16.610148  8403 consensus_queue.cc:260] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:16.568854 14570 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:16.572927 14574 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:16.638003  8401 raft_consensus.cc:2393] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 19: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:16.634967 14608 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:16.640703  8400 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:04:16.627624 14698 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:16.658892  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:16.655638  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:16.663350  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 103722us
I20260709 13:04:16.689877  9007 raft_consensus.cc:2393] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 19: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:16.703090  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:16.704659  8403 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:16.706431 14570 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:16.711488 14427 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:16.712330 14427 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:16.639976 14406 raft_consensus.cc:697] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 19 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:16.720650  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 57556.1us
I20260709 13:04:16.704263 14698 raft_consensus.cc:697] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 19 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:16.766064 14698 consensus_queue.cc:237] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:16.749660 14730 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:16.752027 14570 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:16.757413  8254 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:04:16.774324  8369 leader_election.cc:400] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14.
I20260709 13:04:16.774740  8369 leader_election.cc:403] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:04:16.757917 14427 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:16.760377  8402 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 16.
I20260709 13:04:16.761988  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:16.776175  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:16.762777  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:16.723408 14559 consensus_queue.cc:237] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:16.779502  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 346835us
W20260709 13:04:16.777447  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 315962us
W20260709 13:04:16.780556  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 319817us
I20260709 13:04:16.743994 14563 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.784368  8974 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:16.779153  8147 catalog_manager.cc:5697] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 16 to 21. New cstate: current_term: 21 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:16.788460  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 66824.9us
I20260709 13:04:16.791468 14701 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:16.794011 14701 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.794384 14701 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:16.796557 14563 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:16.798203  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:16.798849  8255 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 23.
I20260709 13:04:16.799188 14486 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:16.799947  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:16.800506  8253 raft_consensus.cc:1240] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:16.800716  8249 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 21.
I20260709 13:04:16.803404  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:16.805902  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:16.807109  8254 raft_consensus.cc:2368] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14.
I20260709 13:04:16.808430 14574 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:16.809134 14730 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:16.809532  8256 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:04:16.810263  8297 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:16.818187 14406 consensus_queue.cc:237] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:16.820236 14730 consensus_queue.cc:1059] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:04:16.824990  8398 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
I20260709 13:04:16.828011  8401 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:04:16.839282  8253 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 18 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:16.830335 14671 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:16.847762 14486 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:16.853530 14708 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:16.831177 14730 raft_consensus.cc:3055] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:16.827919 14475 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.030s
I20260709 13:04:16.836621 14549 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:16.858372 14621 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:16.844534  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:16.867733  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:16.863595  9004 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:04:16.867424 14730 raft_consensus.cc:740] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:16.870944 14730 consensus_queue.cc:260] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:16.872030 14730 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:16.873003  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:16.876653  8403 raft_consensus.cc:1275] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:04:16.877106  8403 pending_rounds.cc:85] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 9
I20260709 13:04:16.866967  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:16.864574  8147 catalog_manager.cc:5697] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 9 to 20. New cstate: current_term: 20 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:16.887537  8257 raft_consensus.cc:1240] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:16.881871 14427 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.003s
I20260709 13:04:16.890215  8329 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 21.
I20260709 13:04:16.886986 14621 raft_consensus.cc:697] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 22 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:16.888459  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:16.903076  8253 raft_consensus.cc:2393] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 18: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:16.905241  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:16.880582 14708 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:16.877328  8403 pending_rounds.cc:107] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:04:16.914543  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:16.922322  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:16.922767  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:16.923679  8972 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:16.925191  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:16.927057 14427 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:16.927848 14427 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:16.930465  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:16.906651  8147 catalog_manager.cc:5697] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 12 to 14, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 14 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:04:16.913313  8403 raft_consensus.cc:2889] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:16.931461  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:16.933800 14297 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:16.937114  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:16.934242 14297 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:16.942011  8329 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:04:16.943361  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:16.943787 14475 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:16.942175 14549 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:16.944439 14297 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:16.951123  8147 catalog_manager.cc:5697] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 5 to 22, leader changed from <none> to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 22 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:16.958027  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:16.961177  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:16.962232  8329 raft_consensus.cc:2393] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 23: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:16.962323 14708 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:16.964515 14701 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:16.958765  8400 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:16.911844 14491 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:16.900218 14621 consensus_queue.cc:237] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:16.944308 14475 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:16.974092  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.982970  8147 catalog_manager.cc:5697] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 7 to 17, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 17 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:04:16.968163 14563 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:17.058766 14486 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:17.059608 14486 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:16.969969 14691 consensus_queue.cc:1048] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.093s
I20260709 13:04:17.076385 14701 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:16.982998 14475 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:17.083597  8295 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:17.094177  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:17.094892  8399 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:17.090046  8147 catalog_manager.cc:5697] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: config changed from index 6 to 11, term changed from 15 to 22, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to <none>, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New cstate: current_term: 22 committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:17.072405 14708 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.065135 14549 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.114005  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:17.114770  8252 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:17.120508 14549 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:17.122855  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 65321.9us
I20260709 13:04:17.115561  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:17.112003 14730 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:04:17.134004  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 287723us
I20260709 13:04:17.146891  8256 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 19 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19. Ops: []
W20260709 13:04:17.135746 14492 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.022s	sys 0.048s
I20260709 13:04:17.133224 14486 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:17.145674  8147 catalog_manager.cc:5697] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:04:17.127868 14708 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:17.158958  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 335588us
I20260709 13:04:17.160121  8298 leader_election.cc:400] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19.
I20260709 13:04:17.160503  8298 leader_election.cc:403] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:04:17.165707  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 372572us
I20260709 13:04:17.138793  8369 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:17.153066  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:17.155974 14492 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.023s	sys 0.048s
I20260709 13:04:17.146585 14730 raft_consensus.cc:697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 21 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:17.209317 14730 consensus_queue.cc:237] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:17.210992 14297 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:17.168814 14369 consensus_queue.cc:1059] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:04:17.189975 14626 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19.
I20260709 13:04:17.195745 14708 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:17.197175  8295 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:17.168179 14491 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:17.204244  8252 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:04:17.205212  9004 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:04:17.178328 14486 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:17.229921  9007 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 21. Current term is 22. Ops: []
W20260709 13:04:17.206442 14757 log.cc:927] Time spent T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:04:17.215108  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:04:17.226562  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 46501us
I20260709 13:04:17.232685  8399 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:04:17.216004  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:17.229391 14486 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:17.206897  8147 catalog_manager.cc:5697] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:17.234232  8974 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:04:17.231648  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 300229us
I20260709 13:04:17.243872  9006 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:17.238019 14427 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:17.249007  8399 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:17.243002  8972 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:17.245513 14753 raft_consensus.cc:3055] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:17.291848  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:17.253618  8147 catalog_manager.cc:5697] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 9 to 15, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 15 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } }
I20260709 13:04:17.238236 14664 consensus_queue.cc:1059] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:04:17.250460  8220 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:17.253145 14297 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:17.297652 14708 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:04:17.293188  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 50327.9us
I20260709 13:04:17.303016  8147 catalog_manager.cc:5697] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 7 to 16. New cstate: current_term: 16 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
W20260709 13:04:17.313514  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 116442us
W20260709 13:04:17.310494  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 62903.5us
I20260709 13:04:17.320323 14398 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:17.324499 14754 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:17.324611 14708 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:17.327527 14708 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:17.328877  8295 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:17.330013 14664 raft_consensus.cc:3055] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:17.330612 14664 raft_consensus.cc:740] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:17.331496 14664 consensus_queue.cc:260] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:17.333461  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:17.330098 14762 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:17.291391 14427 raft_consensus.cc:697] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 17 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:17.295634 14554 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:17.295291 14753 raft_consensus.cc:740] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:17.344273  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:17.344913 14755 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:17.346568  8323 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:04:17.347326  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:17.349547  8326 raft_consensus.cc:1275] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:04:17.348174  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:17.323266 14563 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:17.354180  8328 raft_consensus.cc:2393] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 18: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:17.332751 14664 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:17.315400 14574 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:17.356104  8147 catalog_manager.cc:5697] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:17.362746  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:17.375648  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 256038us
I20260709 13:04:17.392963  8402 raft_consensus.cc:1275] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:04:17.376634  8223 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:17.349516 14560 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.399374  8147 catalog_manager.cc:5697] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 15 to 18, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 18 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:17.405552  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 267293us
I20260709 13:04:17.357201 14754 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:17.410565  8327 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
I20260709 13:04:17.412212  8147 catalog_manager.cc:5697] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 15 to 20. New cstate: current_term: 20 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:17.425108  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 48587.9us
W20260709 13:04:17.428100  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 66161.9us
W20260709 13:04:17.432521  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 103802us
I20260709 13:04:17.436297  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:17.356904 14297 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:17.442184  8223 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:17.458261  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 63783.1us
W20260709 13:04:17.463896  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 70998.3us
I20260709 13:04:17.354216 14755 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:17.473193  8147 catalog_manager.cc:5697] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:17.443753 14574 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:17.498555 14297 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:17.521341  8295 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:17.401886 14554 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.091s
I20260709 13:04:17.531771  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:17.522522  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 232256us
I20260709 13:04:17.522773  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:17.522516  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:17.535857  8329 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:04:17.521529  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:17.522943  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:17.538655  8329 raft_consensus.cc:1240] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:17.527503  8332 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:04:17.521540  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:17.543588  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:17.547917  9006 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:04:17.525570  8400 consensus_queue.cc:237] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:17.522315 14759 log.cc:927] Time spent T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:04:17.538723  8147 catalog_manager.cc:5697] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:17.544288  9001 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 23.
I20260709 13:04:17.392089 14762 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:17.559494  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.561785  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:17.566172  8298 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:17.563323  8372 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:04:17.574198  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 138715us
I20260709 13:04:17.526290 14297 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:17.524067 14574 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:17.610666  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 44573.6us
I20260709 13:04:17.580420  8147 catalog_manager.cc:5697] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 8 to 19. New cstate: current_term: 19 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:17.541980 14617 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:17.624629  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 162940us
I20260709 13:04:17.539090 14701 consensus_queue.cc:1059] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.073s
W20260709 13:04:17.628465  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 107273us
I20260709 13:04:17.580264 14626 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:17.615792 14671 consensus_meta.cc:287] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.750s	user 0.012s	sys 0.000s
I20260709 13:04:17.632825  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:17.633781  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:17.642589  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 67557.7us
I20260709 13:04:17.523749 14758 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.186s
I20260709 13:04:17.498993 14427 consensus_queue.cc:237] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:17.445593 14560 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:17.660387  8297 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:17.585356 14486 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:17.659642  8147 catalog_manager.cc:5697] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 7 to 13, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 13 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:17.673177  8325 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:17.670320  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 203875us
W20260709 13:04:17.677064  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.635483 14563 raft_consensus.cc:697] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:17.637857 14698 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:17.680773  8223 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:17.684895  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:17.659663 14671 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14.
I20260709 13:04:17.679082  8147 catalog_manager.cc:5697] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:17.692675  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:17.675129 14691 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:17.693341  8399 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:04:17.695116 14779 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:17.695575 14717 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:17.695951 14764 log.cc:927] Time spent T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
W20260709 13:04:17.696496 14763 log.cc:927] Time spent T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
W20260709 13:04:17.696797  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.636133 14617 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:17.698212  8404 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:17.557982 14753 consensus_queue.cc:260] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:17.701207 14762 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:17.690213  8331 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:04:17.691267  8220 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:17.729857 14560 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:17.731642  8295 leader_election.cc:400] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15.
I20260709 13:04:17.750586  8295 leader_election.cc:403] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:17.725123 14574 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:17.732510  8147 catalog_manager.cc:5697] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:04:17.733556 14760 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:04:17.734179  8399 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:17.736008 14786 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:17.736725 14788 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:17.737690 14784 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:17.743392 14783 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:17.728111  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:17.753876  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 67589.1us
W20260709 13:04:17.761544  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 71955.5us
W20260709 13:04:17.756083 14777 log.cc:927] Time spent T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 13:04:17.750154 14787 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:17.755576 14560 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:17.766646  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:17.748924 14779 raft_consensus.cc:697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:04:17.774011  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 211243us
I20260709 13:04:17.764319  8223 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:17.774042  8220 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:17.746963 14698 consensus_queue.cc:1048] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:04:17.777443 14761 consensus_queue.cc:1059] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.013s
W20260709 13:04:17.778245 14475 consensus_meta.cc:287] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.682s	user 0.011s	sys 0.000s
I20260709 13:04:17.775640  8147 catalog_manager.cc:5697] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 13 to 14, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 14 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:17.820094  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 55890.4us
I20260709 13:04:17.820974  8223 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:17.762773 14492 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15.
I20260709 13:04:17.774658 14398 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:17.825666  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 98027.2us
W20260709 13:04:17.850549  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 119068us
I20260709 13:04:17.823729  8147 catalog_manager.cc:5697] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:04:17.757614 14574 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:17.861805  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 87853.7us
I20260709 13:04:17.757850 14717 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:17.911675  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 90780.8us
W20260709 13:04:17.917232  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 43589us
I20260709 13:04:17.918680  8220 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:17.924882  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:17.925657  9006 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:17.816154 14787 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:17.908540 14786 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:17.821692 14475 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:17.918499 14701 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.940174 14617 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:17.922092 14486 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:17.951687  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 43602.5us
I20260709 13:04:17.940891 14717 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:17.946098  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:17.914495 14730 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:17.948787  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:17.917564 14755 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:17.952608 14701 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:17.957276 14772 consensus_queue.cc:1048] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.177s
I20260709 13:04:17.958214 14574 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:17.958810 14701 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:17.959350  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:17.960000  8326 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18.
I20260709 13:04:17.960819 14768 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:04:17.961623 14782 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:17.962218 14782 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.962730 14782 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:17.962779 14797 consensus_queue.cc:1059] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:17.964162 14797 raft_consensus.cc:3055] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:17.964534 14797 raft_consensus.cc:740] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:17.824429 14563 consensus_queue.cc:237] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:17.821849 14783 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:17.819059 14788 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:17.945878  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:17.953603  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:17.919970 14560 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.996744 14560 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:17.966018  8295 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:18.000185  8295 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:17.932647 14785 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:18.001940 14785 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:18.003638 14806 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:18.004419 14785 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:18.004640 14778 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.270s	user 0.003s	sys 0.000s
I20260709 13:04:18.004352 14806 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:18.005285 14369 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:17.985397  9006 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
W20260709 13:04:17.987715  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 419799us
I20260709 13:04:17.989121  9001 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:04:17.991462 14475 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:17.993901  8330 raft_consensus.cc:3055] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:04:17.967732 14789 log.cc:927] Time spent T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:04:18.029984 14549 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:18.030611 14770 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.031167 14770 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:17.983896 14730 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:18.044243  8222 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:18.044677  8295 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:17.955547 14779 consensus_queue.cc:237] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:18.056792  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 2567) took 1148 ms. Trace:
I20260709 13:04:18.057193  8253 rpcz_store.cc:276] 0709 13:04:16.908670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.908806 (+   136us) service_pool.cc:224] Handling call
0709 13:04:16.910699 (+  1893us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:16.911175 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:16.911216 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:16.911720 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 13:04:17.128625 (+216905us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:17.357328 (+228703us) log.cc:844] Serialized 14 byte log entry
0709 13:04:17.357567 (+   239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:17.357645 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:18.053970 (+696325us) spinlock_profiling.cc:243] Waited 6.97 ms on lock 0x7b2400168908. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255579c4 00007f8825555acb 00007f8825710dc9 00007f8825710a55 00007f882570ffb0 00007f8828386488 00007f8828386be8 00007f882835e398 00007f882835976a 00007f8828358bb6
0709 13:04:18.056480 (+  2510us) raft_consensus.cc:1685] finished
0709 13:04:18.056584 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:18.056712 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20894,"prepare.queue_time_us":459006,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":170976,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":21889,"spinlock_wait_cycles":16023424,"thread_start_us":416456,"threads_started":2,"wal-append.queue_time_us":388290}
I20260709 13:04:18.062116 14783 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:18.010701  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:18.035640 14730 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:18.015856  9008 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:04:18.015966 14806 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:18.066581 14297 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:18.073446 14297 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:18.012533 14369 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:18.075899 14297 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:17.965584 14771 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:18.023561 14787 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:17.966406 14765 raft_consensus.cc:3055] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:18.078123 14765 raft_consensus.cc:740] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:04:17.966344 14793 log.cc:927] Time spent T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:04:18.078503 14795 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:18.026631  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:17.966651 14769 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:18.006304 14786 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:18.081893 14786 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:18.084306 14786 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:17.965410 14797 consensus_queue.cc:260] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:18.041164 14788 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:18.013120  8330 raft_consensus.cc:740] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:04:18.007335 14664 consensus_meta.cc:287] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.645s	user 0.010s	sys 0.002s
I20260709 13:04:18.116520  8325 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
I20260709 13:04:18.011577  8249 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 23.
W20260709 13:04:18.128754  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 84591us
I20260709 13:04:18.128923 14783 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:18.129645 14783 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:18.130105 14783 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:18.129998 14786 consensus_queue.cc:1059] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:18.132294 14786 raft_consensus.cc:3055] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:18.132728 14786 raft_consensus.cc:740] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:18.133684 14786 consensus_queue.cc:260] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:18.135268 14786 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:18.144081 14406 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:18.144599 14781 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:18.145396 14781 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:18.145146 14406 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:18.148250 14781 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:18.150154 14406 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:18.054811 14782 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:18.152812  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 172976us
I20260709 13:04:18.176936  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:18.177846  8397 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:04:18.187501  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:18.086890 14797 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:18.193253  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 148681us
I20260709 13:04:18.033392 14770 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:18.151966  8330 consensus_queue.cc:260] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:18.202596  8330 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:18.197754  8222 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:18.121688 14662 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:18.208192  8369 leader_election.cc:400] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 13 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 13. Current term is 14.
I20260709 13:04:18.188443  8405 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
I20260709 13:04:18.213312 14756 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.214123 14756 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:18.215485 14775 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:18.216254 14775 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:18.218902 14766 log.cc:927] Time spent T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.595s	user 0.003s	sys 0.003s
I20260709 13:04:18.220690  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 1067) took 1321 ms. Trace:
I20260709 13:04:18.221067  8398 rpcz_store.cc:276] 0709 13:04:16.899446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.904029 (+  4583us) service_pool.cc:224] Handling call
0709 13:04:16.930097 (+ 26068us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:16.936955 (+  6858us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:16.936990 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:16.943430 (+  6440us) log.cc:844] Serialized 26 byte log entry
0709 13:04:17.239815 (+296385us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:17.239866 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:17.239943 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:18.081281 (+841338us) spinlock_profiling.cc:243] Waited 7.14 ms on lock 0x7b240020c2e8. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255579c4 00007f8825555acb 00007f8825710dc9 00007f8825710a55 00007f882571009d 00007f882834210e 00007f882835988c 00007f8828358bb6 00007f88283587d8 00007f8828362221
0709 13:04:18.220329 (+139048us) raft_consensus.cc:1685] finished
0709 13:04:18.220355 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:18.220620 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3004,"raft.queue_time_us":4552,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"spinlock_wait_cycles":16428416,"thread_start_us":292470,"threads_started":1,"wal-append.queue_time_us":450121}
I20260709 13:04:18.238687  8295 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:18.211591 14662 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:04:18.239862  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 45537.8us
I20260709 13:04:18.266017  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:18.272306  8400 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
W20260709 13:04:18.281628  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 83963.5us
W20260709 13:04:18.285730  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.307116 14807 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:18.310401 14807 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.310833 14807 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:18.237491 14570 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:18.280652 14662 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:18.319672  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 90410us
I20260709 13:04:18.233054 14765 consensus_queue.cc:260] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:18.454002 14775 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:18.458895  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.475483 14787 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:18.283645 14369 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:18.476476 14486 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:18.479542 14664 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:18.480094 14570 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:18.484751 14782 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:18.313505 14771 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.476630  8369 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:18.533403  8295 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:18.552981  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:18.553496  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:18.554033  8252 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:04:18.554281  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:18.553006  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:18.554039  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:18.553169  8323 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:04:18.559809  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:18.560011  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:18.561028  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:18.561220 14486 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:18.561817  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:18.562423  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:18.562659  8250 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:04:18.563889 14486 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:18.564811  8972 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.611s (SENT) [suppressed 4 similar messages]
I20260709 13:04:18.552963 14765 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:18.570186  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:18.571923  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:18.561048  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:18.586891  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.548804 14804 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.555147 14806 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.549103 14406 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:18.577937 14369 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:18.607844  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:18.619777  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:18.621850  8330 raft_consensus.cc:1275] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:04:18.622529  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 3125) took 1273 ms. Trace:
I20260709 13:04:18.573376 14787 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:18.625586  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 149081us
I20260709 13:04:18.626770  9001 raft_consensus.cc:1275] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:04:18.626889  8369 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:18.625705  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:18.570537 14664 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:18.622838  8330 rpcz_store.cc:276] 0709 13:04:17.348605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.348705 (+   100us) service_pool.cc:224] Handling call
0709 13:04:17.349103 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:17.993670 (+644567us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b54000af8a0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f8828735dc0 00007f88287c01c0 00007f88287c1231 00007f88287bdd8f 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c
0709 13:04:18.622281 (+628611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:18.622473 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28024192}
I20260709 13:04:18.594990 14297 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:18.603281 14475 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:18.602303 14805 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:18.577677 14771 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:18.646507  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:18.630595 14813 log.cc:927] Time spent T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.480s	user 0.002s	sys 0.000s
I20260709 13:04:18.628221 14406 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:18.652834  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:18.653304  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:18.632117 14784 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 13:04:18.654837  8253 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:04:18.650053  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:18.607196 14806 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:18.609637 14804 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:18.662953  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:18.643599 14774 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:18.670881  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:18.671584  9001 raft_consensus.cc:2393] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 22: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:18.672425  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:18.672819  8330 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
I20260709 13:04:18.656755 14369 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:18.674698  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 109924us
I20260709 13:04:18.640669 14297 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:18.676011  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:18.679144  8323 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 12.
I20260709 13:04:18.643749 14475 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:18.688932 14771 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:18.690232 14808 log.cc:927] Time spent T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 13:04:18.692520  9004 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:04:18.704900 14753 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:18.681674 14369 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:18.707432 14369 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:18.709368 14297 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:18.702033 14783 consensus_meta.cc:287] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.571s	user 0.012s	sys 0.000s
W20260709 13:04:18.719561  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 51746.1us
I20260709 13:04:18.720374  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:18.720463  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:18.721015  8330 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:04:18.721868  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:18.722400  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 189470us
I20260709 13:04:18.723542  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:18.714881 14774 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:04:18.723734 14814 log.cc:927] Time spent T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
W20260709 13:04:18.727824 14817 log.cc:927] Time spent T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:04:18.731680 14475 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:18.740283 14797 consensus_meta.cc:287] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.537s	user 0.007s	sys 0.003s
I20260709 13:04:18.745723 14806 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:18.748842 14786 consensus_meta.cc:287] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.613s	user 0.009s	sys 0.006s
I20260709 13:04:18.749752  8257 raft_consensus.cc:2410] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:04:18.750270 14769 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:18.750722 14769 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:18.753747 14570 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:18.758170  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:18.758808  9004 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:18.753150 14756 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:18.761835 14756 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.761979  8298 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:18.762439 14756 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:18.763836  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:18.764305 14804 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:18.765328 14756 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:18.765661 14762 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:18.765976 14756 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:18.766753 14756 consensus_queue.cc:1243] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:04:18.770269  9006 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:04:18.770423  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:18.760929  9007 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:04:18.702260 14807 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:18.775121  8295 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:18.779104  8248 raft_consensus.cc:2368] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 18. Current term is 19.
I20260709 13:04:18.776126 14758 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:18.774206 14819 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:18.783227 14819 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:18.725925 14783 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:18.781901  8295 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:04:18.784309  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:18.787173  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 66869.1us
I20260709 13:04:18.791553  8369 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:18.793682  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:18.794772  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:18.794914  8295 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:04:18.792150  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 113115us
I20260709 13:04:18.796083  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:18.792187 14758 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:18.797693  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:18.797891  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:18.789151 14774 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.799795 14807 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:18.799439 14774 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:18.800002  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:18.800457 14797 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.800644  8252 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 20.
I20260709 13:04:18.800971 14797 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:18.801406  8223 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:18.801769  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:18.802042  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:18.818470  8255 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 24: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:18.818704  8254 raft_consensus.cc:2393] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 16: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:18.819136  8223 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:18.791918 14775 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:04:18.820279  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 57375us
I20260709 13:04:18.821353 14570 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:18.791143 14762 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:18.825309  8398 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:04:18.804567 14621 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:18.828331  8252 raft_consensus.cc:1240] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:18.803161 14797 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:18.835774  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:18.836400 14662 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:18.823865 14450 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:18.834311 14774 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:18.834702 14787 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:18.834249 14826 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:18.835654  9000 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:18.822990 14804 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:18.848495 14804 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:18.848920 14804 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:18.849088 14807 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.849828  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:18.849778 14807 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:18.851933 14783 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:18.852121 14807 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:18.838414  8295 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:18.839864  8326 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:04:18.854542  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:18.841349  9004 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 24.
I20260709 13:04:18.842038  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:18.856428  8369 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:18.834129 14762 raft_consensus.cc:697] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:18.830780 14621 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.844117  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:18.847255  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:18.859153  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:18.860653  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:18.806226 14758 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:18.858553  8406 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:04:18.862283  8330 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 24.
I20260709 13:04:18.862007  8369 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:18.827412 14570 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:04:18.866509  8400 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 23.
I20260709 13:04:18.870573  8369 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:18.843473  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:18.825640 14775 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:18.869797 14820 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:18.879643 14708 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:18.880250 14708 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:18.882668 14708 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:18.882730 14560 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:18.883522 14708 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:18.858273 14787 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:18.861830 14774 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:18.886176 14774 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:18.886577 14774 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:18.887111  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:18.831199 14786 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:04:18.876999 14805 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:18.889192 14805 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.866329 14662 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:18.889626 14805 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:18.891783  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:18.892688  9004 raft_consensus.cc:2410] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:04:18.894833 14758 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:18.849169 14819 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:18.890450 14804 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:18.866081 14621 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:18.902910 14804 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:18.904378 14769 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:18.905514 14754 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.906055 14754 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:18.906685 14406 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:18.910369 14762 consensus_queue.cc:237] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:18.911172 14769 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:18.915360 14774 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:18.917670  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:18.869997 14825 consensus_queue.cc:1059] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 13:04:18.916934  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:18.923306  8249 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:18.875478  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:18.878506  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:04:18.924178  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 45762us
W20260709 13:04:18.921046  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 81969.3us
I20260709 13:04:18.929242  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:18.865828 14450 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:18.944785  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:18.935235 14758 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:18.950244 14758 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:18.950640 14758 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:18.952057  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:18.953712  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:18.953809  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:18.955703  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:18.956310  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:18.957829 14754 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:18.958432  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.880801 14775 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:18.953320  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:18.959843  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:18.960402  8255 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:18.962587  8329 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 24: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:18.962616  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:18.967823  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:18.969841  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:18.971154  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:18.971851  8256 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:04:18.973726  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:18.929354 14797 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:18.974361  8323 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:04:18.974587  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:18.976330  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:18.977085  8250 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:04:18.982862  8399 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:18.983194  8252 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:18.976328  8328 raft_consensus.cc:2410] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:04:18.989856  8250 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:04:18.988480  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:18.992792  8249 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 22.
I20260709 13:04:18.993753  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:18.994249  8399 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:04:18.997488  8255 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:18.997876  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:18.999197  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:19.001528  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:18.998692  8328 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 22.
I20260709 13:04:19.002964  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:19.002013  8399 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:18.977809 14784 raft_consensus.cc:3055] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:19.008519  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:19.009089  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:18.981727 14754 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:19.013851  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:18.986498 14775 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:19.010069  8255 raft_consensus.cc:2393] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 17: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:19.010543  8250 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:19.011794  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:19.012763  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:19.017347  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:19.017522  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:19.018448  8399 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:19.019529  8295 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:18.995820 14450 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:18.997527 14797 raft_consensus.cc:697] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 23 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:19.021950  8256 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 24: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:19.022184  8328 raft_consensus.cc:2393] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 18: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:19.044024  8399 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 24.
I20260709 13:04:19.009859 14784 raft_consensus.cc:740] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:19.046833 14784 consensus_queue.cc:260] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:19.046100  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:18.998426 14491 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:19.049757 14491 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:19.050168  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:19.051296  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:19.051831  8323 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
I20260709 13:04:19.021790  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:19.052699  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:19.053028 14622 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:19.020933 14779 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:19.053548 14622 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:19.053988 14779 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:19.054407 14779 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:19.008618 14769 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.026213  8295 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:19.027019 14754 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:19.055294 14769 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:19.021230  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:19.056885 14754 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:19.027102  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:19.013013 14796 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:19.057749  8325 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 24: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:19.057963  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:19.057760 14796 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:19.058450  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:19.058872  8325 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:04:19.059262  8325 pending_rounds.cc:85] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 11
I20260709 13:04:19.031879  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:19.035346  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:19.060011  8398 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18.
I20260709 13:04:19.034024  9007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56940 (request call id 1262) took 1110 ms. Trace:
I20260709 13:04:19.039561  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:19.054296  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:19.035346  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:19.002592 14780 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:04:19.048785 14450 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:19.051832 14491 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:19.054229 14622 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:19.059542  8325 pending_rounds.cc:107] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:04:19.078931  8325 raft_consensus.cc:2889] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:19.033514  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:19.060865  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:19.058830  9008 raft_consensus.cc:2393] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 15: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:19.048203 14784 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:19.061811  8402 raft_consensus.cc:1813] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Handling vote request from an unknown peer 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:19.040009 14664 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:19.023386  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:19.053200  8323 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:19.095577  8249 raft_consensus.cc:2393] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 15: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:19.080195  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:19.079587  8251 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:04:19.086663  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:19.074497  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:19.060565  9007 rpcz_store.cc:276] 0709 13:04:17.924002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.924204 (+   202us) service_pool.cc:224] Handling call
0709 13:04:19.033810 (+1109606us) spinlock_profiling.cc:243] Waited 756 ms on lock 0x7b54006e23d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:19.033940 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1738207488}
I20260709 13:04:19.096855  8397 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:04:19.102160  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:19.103472  8298 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:19.103183  8402 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 24: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:19.106353  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:19.108007  8328 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:04:19.102890  8256 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:04:19.110132  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:19.110776  9003 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:04:19.110930  8399 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:04:19.114306  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:19.114882  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.116863  9009 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 24: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:19.078691 14823 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:19.121340  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:19.132081  8223 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:19.078469 14754 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:19.068735 14570 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:19.119912  8402 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:46846
I20260709 13:04:19.145375  8402 raft_consensus.cc:606] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:19.145771  8402 raft_consensus.cc:614] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:19.120487  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:19.122602  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:19.123198  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:19.124925 14827 log.cc:927] Time spent T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:04:19.155565 14406 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:19.156172 14406 raft_consensus.cc:697] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:19.155917 14398 raft_consensus.cc:2955] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Committing config change with OpId 14.12: config changed from index 5 to 12, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:19.158396  8249 raft_consensus.cc:1240] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:19.158612 14754 raft_consensus.cc:697] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:19.159509 14754 consensus_queue.cc:237] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:19.159953  8249 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:04:19.125975  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:19.126883 14796 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:19.079325 14560 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.118439 14664 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:19.127457 14297 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:19.166218 14560 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:19.166899 14297 raft_consensus.cc:697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 22 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:19.167755 14797 consensus_queue.cc:237] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:19.168377 14560 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:19.167982 14297 consensus_queue.cc:237] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:19.127948 14769 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:19.129070 14450 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:19.133795  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:19.134459 14774 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:19.157089 14406 consensus_queue.cc:237] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:19.087455 14756 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 13:04:19.042765 14775 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:19.173895  8254 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:04:19.174350 14775 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:19.135445  8256 raft_consensus.cc:1275] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:19.073771 14795 consensus_queue.cc:1059] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 13:04:19.175550 14795 raft_consensus.cc:3055] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:19.175880 14795 raft_consensus.cc:740] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:19.176663 14795 consensus_queue.cc:260] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:19.122627  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:19.178617 14795 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:19.137419  8297 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:19.181806 14758 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:19.182108  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 44783.2us
I20260709 13:04:19.183050  8297 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:19.183948 14758 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:19.184374  8297 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:19.138087  8406 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:04:19.139628  8253 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 11. (term mismatch)
I20260709 13:04:19.185869  8253 pending_rounds.cc:85] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 10
I20260709 13:04:19.186159  8253 pending_rounds.cc:107] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 21.11
I20260709 13:04:19.186431  8253 raft_consensus.cc:2889] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:19.189615  8323 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:19.189899 14664 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:19.190688 14664 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:19.191390 14664 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:19.146193 14823 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.194517 14823 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:19.194307 14622 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:19.195470  8398 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:19.198413  8326 raft_consensus.cc:2410] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:04:19.147862  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:19.199836  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.151384  8295 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:19.145211  8222 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:19.165558 14570 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:19.200847  8297 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:19.202986  8398 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:19.212389  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:19.213464 14756 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:04:19.213903  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 62656.5us
I20260709 13:04:19.214843 14756 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:19.215308 14756 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.215670 14756 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:19.216010 14560 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:04:19.216173  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 71055.1us
I20260709 13:04:19.216451 14560 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.216843 14560 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:19.216902  8222 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:19.217303 14369 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:19.218891 14369 consensus_queue.cc:1048] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:19.223981 14786 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:19.224428 14662 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:19.227469 14824 consensus_queue.cc:1059] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:04:19.228901  8222 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:19.236725  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:19.237193  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:19.239060  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:19.239233  8222 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:19.239663  8404 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:04:19.241106  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:19.242575  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:19.243432  8398 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:19.261803  8222 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:19.214795 14622 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:19.261385  8220 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:19.218918 14570 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:19.332803  8972 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:19.333832  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 236596us
I20260709 13:04:19.333931  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:19.224472 14786 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:19.348515  8974 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:19.348469  8402 raft_consensus.cc:1275] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:19.348469  8403 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:04:19.348963  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:19.338001 14826 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.339592  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:19.340161  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 284863us
I20260709 13:04:19.339586  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:19.339715  8297 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:19.348322  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 86642.8us
I20260709 13:04:19.350484  8397 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
I20260709 13:04:19.359905  8222 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:19.232116 14662 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:19.211627 14621 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:19.367488  8222 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:19.445151 14762 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:19.439734  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 100102us
I20260709 13:04:19.370141 14840 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:19.363649 14807 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:19.333712 14804 consensus_queue.cc:1048] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.145s
I20260709 13:04:19.356287 14786 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:19.369462  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:19.334784 14563 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:19.439208 14809 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:19.368248 14826 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:19.370182 14822 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:19.547230 14822 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:19.547857 14822 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:19.548096 14797 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:19.548646 14797 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.549043 14797 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:19.365422 14783 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:19.549837 14783 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:19.550359 14783 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:19.371608 14824 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.553819 14824 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:19.556694 14824 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:19.432631 14805 consensus_meta.cc:287] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.542s	user 0.009s	sys 0.001s
W20260709 13:04:19.555150  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 185689us
I20260709 13:04:19.557809 14805 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:19.560729 14805 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:19.561873  8400 raft_consensus.cc:2393] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:19.563140  8972 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:19.444845  8324 raft_consensus.cc:2368] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 23. Current term is 24.
W20260709 13:04:19.431864  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 65956.2us
W20260709 13:04:19.532713  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 165311us
I20260709 13:04:19.572293  8222 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:19.572832 14806 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:19.572288 14475 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:19.534962  8297 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:19.578996  8298 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:19.581486 14826 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:19.511423 14753 raft_consensus.cc:3055] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:04:19.587868  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 52977.8us
W20260709 13:04:19.537849 14832 log.cc:927] Time spent T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:04:19.508786 14779 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:19.538527 14807 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:19.577860  8972 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:19.610510  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 349196us
I20260709 13:04:19.611339  8220 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:19.611802  8972 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:19.620949  8972 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:19.626037  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:19.577915 14809 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:19.594023 14406 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:19.599502 14753 raft_consensus.cc:740] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:19.642277 14570 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:19.615342 14807 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:19.609215 14840 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:19.626271 14475 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:04:19.764608  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 192356us
I20260709 13:04:19.752207  8249 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:04:19.739727 14563 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:19.633145 14806 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:19.633056 14841 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:19.633476 14809 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:19.761518  8972 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:19.782210  8222 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:04:19.765115 14823 consensus_meta.cc:287] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.567s	user 0.011s	sys 0.001s
I20260709 13:04:19.821467  8222 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:19.764942 14804 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:19.767381 14475 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:04:19.860862  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 99468.1us
I20260709 13:04:19.875736  8222 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:19.825012 14861 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:19.923444 14863 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:04:19.924005 14850 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
W20260709 13:04:19.934181 14860 log.cc:927] Time spent T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
I20260709 13:04:19.771636 14840 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:19.773361 14406 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:19.938654 14797 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:20.011670 14797 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:20.013208 14847 log.cc:927] Time spent T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:04:19.889005 14762 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:19.889495 14475 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:19.894569 14621 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:20.009455 14866 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:20.021680  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 593425us
I20260709 13:04:20.023119  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3320) took 1091 ms. Trace:
I20260709 13:04:20.034214  8399 rpcz_store.cc:276] 0709 13:04:18.932179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.143723 (+211544us) service_pool.cc:224] Handling call
0709 13:04:19.143954 (+   231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:19.144130 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:19.228077 (+ 83947us) log.cc:844] Serialized 14 byte log entry
0709 13:04:20.022732 (+794655us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:20.022793 (+    61us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:20.022839 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.022920 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.023064 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":511093,"prepare.queue_time_us":77047,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":503263,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"thread_start_us":287136,"threads_started":1,"wal-append.queue_time_us":351869}
I20260709 13:04:19.835108 14823 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:20.026950  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:19.948235 14861 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:19.960626 14863 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:19.898893 14841 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.137s
I20260709 13:04:19.888039 14563 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:20.085387  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 58554.8us
I20260709 13:04:20.026239 14297 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:20.013387 14797 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:20.015704 14406 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:19.937034 14822 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:20.017880 14762 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.887015 14753 consensus_queue.cc:260] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:20.074435  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:20.088280  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 225719us
W20260709 13:04:19.852703  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 273707us
W20260709 13:04:20.055171 14560 consensus_meta.cc:287] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.834s	user 0.012s	sys 0.000s
I20260709 13:04:20.022953 14398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:32970 (request call id 0) took 2885 ms. Trace:
I20260709 13:04:20.022006 14877 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:20.016731 14840 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:20.030819 14756 consensus_meta.cc:287] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.815s	user 0.013s	sys 0.005s
I20260709 13:04:20.023411 14874 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:20.098083  9007 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 20. Current term is 21. Ops: [18.14-20.15]
W20260709 13:04:20.102524  8250 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.002s	sys 0.020s
W20260709 13:04:20.111502  8250 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.003s	sys 0.020s
I20260709 13:04:20.106302  8298 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:19.830150 14783 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:20.123185  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:20.123915  8399 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
W20260709 13:04:20.097553  8295 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.015s	sys 0.006s
I20260709 13:04:20.081841 14861 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:20.015149 14867 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.086s
I20260709 13:04:20.143044 14867 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:20.128558  8399 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 23. Current term is 24. Ops: []
I20260709 13:04:20.105665 14762 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:20.159912  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:20.120164 14874 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.109313 14779 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:20.096268 14769 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:20.081686 14863 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:20.133219  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 1013) took 1153 ms. Trace:
I20260709 13:04:20.134480 14877 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:20.135931 14859 log.cc:927] Time spent T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:04:20.171837  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 1101) took 1066 ms. Trace:
W20260709 13:04:20.174610 14845 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.237s	user 0.003s	sys 0.016s
I20260709 13:04:20.174834 14823 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:20.176779 14563 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:20.177007 14570 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:20.189093  8401 raft_consensus.cc:1275] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:04:20.101418 14297 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:20.201270  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 1089) took 1333 ms. Trace:
I20260709 13:04:20.201781 14874 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:20.233199  8398 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:20.231062  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2631) took 1008 ms. Trace:
I20260709 13:04:20.110788 14560 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:20.128485 14756 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:20.213130  8324 rpcz_store.cc:276] 0709 13:04:18.967756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.982806 (+ 15050us) service_pool.cc:224] Handling call
0709 13:04:20.111230 (+1128424us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b540008ff54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:20.120925 (+  9695us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1022904064}
I20260709 13:04:20.215916  8254 rpcz_store.cc:276] 0709 13:04:19.105552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.175349 (+ 69797us) service_pool.cc:224] Handling call
0709 13:04:19.175608 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:19.175790 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:04:19.274243 (+ 98453us) log.cc:844] Serialized 14 byte log entry
0709 13:04:20.171211 (+896968us) spinlock_profiling.cc:243] Waited 1.98 ms on lock 0x7b18003dd020. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88283664bc 00007f88256b448b 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256ce455 00007f88287b6060 00007f88287c8716 00007f88287d9287
0709 13:04:20.171353 (+   142us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:20.171420 (+    67us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:04:20.171483 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.171581 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.171755 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87658,"prepare.queue_time_us":567651,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":91558,"raft.run_cpu_time_us":9145,"raft.run_wall_time_us":69706,"spinlock_wait_cycles":4545408,"thread_start_us":176360,"threads_started":2}
I20260709 13:04:20.236418  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 3316) took 1349 ms. Trace:
W20260709 13:04:20.241972  8220 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.016s	sys 0.041s
W20260709 13:04:20.249024  8220 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.016s	sys 0.041s
W20260709 13:04:20.249410  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 604644us
I20260709 13:04:20.233728  8248 rpcz_store.cc:276] 0709 13:04:19.222266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.222447 (+   181us) service_pool.cc:224] Handling call
0709 13:04:19.224828 (+  2381us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:19.225178 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:19.225209 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:19.225817 (+   608us) log.cc:844] Serialized 52 byte log entry
0709 13:04:19.270726 (+ 44909us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:19.270793 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.270899 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:20.230851 (+959952us) raft_consensus.cc:1685] finished
0709 13:04:20.230876 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.231006 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1121,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":3113,"thread_start_us":44426,"threads_started":1,"wal-append.queue_time_us":351873}
W20260709 13:04:20.237506 14754 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.761s	user 0.010s	sys 0.033s
W20260709 13:04:20.126276  8295 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.015s	sys 0.006s
I20260709 13:04:20.244287  8400 rpcz_store.cc:276] 0709 13:04:18.886802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.937280 (+ 50478us) service_pool.cc:224] Handling call
0709 13:04:20.236165 (+1298885us) spinlock_profiling.cc:243] Waited 591 ms on lock 0x7b54002ef9d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:20.236342 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1359449472}
I20260709 13:04:20.250238  8220 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:20.270447  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:20.234464 14570 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:20.279925  8251 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 22. Current term is 23. Ops: [22.11-22.11]
I20260709 13:04:20.232343 14769 raft_consensus.cc:697] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:20.233409  8405 rpcz_store.cc:276] 0709 13:04:18.868110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.136786 (+268676us) service_pool.cc:224] Handling call
0709 13:04:19.137078 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:19.198971 (+ 61893us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:04:19.232724 (+ 33753us) log.cc:844] Serialized 14 byte log entry
0709 13:04:19.350432 (+117708us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:19.350484 (+    52us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:04:19.350533 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.011671 (+661138us) spinlock_profiling.cc:243] Waited 7.73 ms on lock 0x7b1800195640. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882564bc0a 00007f882ed8b219 00007f882c69258f 00007f88256d1571 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd
0709 13:04:20.199731 (+188060us) spinlock_profiling.cc:243] Waited 3.39 ms on lock 0x7b1800195640. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882569f2a6 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419
0709 13:04:20.200944 (+  1213us) spinlock_profiling.cc:243] Waited 61.7 ms on lock 0x7b54001e7654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:20.201002 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.201196 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30772,"prepare.queue_time_us":62933,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":7173,"raft.queue_time_us":3759,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":11363,"spinlock_wait_cycles":167451904,"thread_start_us":113536,"threads_started":1,"wal-append.queue_time_us":208264}
I20260709 13:04:20.279031  8220 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:20.277037  8972 leader_election.cc:400] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 23. Current term is 24.
W20260709 13:04:20.283557 14838 log.cc:927] Time spent T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:04:20.284334 14876 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:20.285122 14297 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:20.284869 14876 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:20.286193 14769 consensus_queue.cc:237] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:20.287127 14876 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:20.287632 14876 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:20.288060 14876 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:20.241542 14858 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:20.304200  8296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.008s	sys 0.021s
W20260709 13:04:20.274174 14754 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.797s	user 0.010s	sys 0.034s
I20260709 13:04:20.243553 14870 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:20.288767 14475 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:20.309741  8220 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:20.303164 14859 log.cc:927] Time spent T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:04:20.245990 14804 raft_consensus.cc:697] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 24 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:20.309861  8296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.008s	sys 0.021s
I20260709 13:04:20.322124  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:04:20.319900  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 445110us
I20260709 13:04:20.301201 14826 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:04:20.318280  8975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.017s	sys 0.021s
I20260709 13:04:20.298043 14730 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.325531  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:20.325527  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:20.332754  8325 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:04:20.327932  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:20.335400  8327 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:20.305382 14297 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:04:20.322253 14691 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.889s	user 0.010s	sys 0.022s
I20260709 13:04:20.314025 14563 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:04:20.340811  8975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.017s	sys 0.022s
W20260709 13:04:20.346096  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 237907us
I20260709 13:04:20.340659  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:20.347442  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:20.348091  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:20.360102  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:20.360767  8330 raft_consensus.cc:2368] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15.
I20260709 13:04:20.363732  9007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 1121) took 1267 ms. Trace:
I20260709 13:04:20.364296  9007 rpcz_store.cc:276] 0709 13:04:19.096479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.123632 (+ 27153us) service_pool.cc:224] Handling call
0709 13:04:19.123935 (+   303us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:20.110130 (+986195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.363393 (+253263us) spinlock_profiling.cc:243] Waited 974 ms on lock 0x7b54004132d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:20.363643 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2239429376}
W20260709 13:04:20.353710  8369 connection.cc:256] RPC call timeout handler was delayed by -1.0145684640000354s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18280000000000002s for a response
W20260709 13:04:20.366997  8372 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.288s	user 0.024s	sys 0.027s
W20260709 13:04:20.367349  8372 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.298s	user 0.024s	sys 0.027s
I20260709 13:04:20.367494  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:04:20.367591  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 1.30649e+06us
I20260709 13:04:20.330988 14754 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:04:20.349355 14691 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.010s	sys 0.022s
I20260709 13:04:20.337114 14875 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:20.376658  8400 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:20.377712  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:20.382076  8372 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:20.388600  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:20.392163  8328 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:04:20.394213  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:20.394791  8369 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:20.310566 14858 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:20.400900  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:20.399784  8329 raft_consensus.cc:2368] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 18. Current term is 19.
I20260709 13:04:20.339814 14730 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:20.407334  8369 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:20.401741  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 92090.1us
I20260709 13:04:20.345204 14826 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:20.108835 14398 rpcz_store.cc:276] 0709 13:04:17.137082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.137295 (+   213us) service_pool.cc:224] Handling call
0709 13:04:17.524939 (+387644us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b5800896cd0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f882871a0aa 00007f882879c447 00007f88287b970a 00007f88287ba76a 00007f88287b991b 00007f88287c503c 00007f88287c46b3 00007f88287bc9fa 00007f882d7844c9 00007f8826d00c54
0709 13:04:17.677184 (+152245us) log.cc:844] Serialized 487 byte log entry
0709 13:04:20.022825 (+2345641us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23985,"raft.run_cpu_time_us":1702,"raft.run_wall_time_us":46522,"spinlock_wait_cycles":381342976,"thread_start_us":470192,"threads_started":2,"wal-append.queue_time_us":264133,"wal-append.run_cpu_time_us":2541,"wal-append.run_wall_time_us":1068261}
I20260709 13:04:20.349241 14297 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:20.406999  8972 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:20.411749  8372 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:20.388871 14783 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:20.389325 14876 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:20.387979 14878 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:20.419584 14855 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:20.408933 14450 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:20.419864 14798 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:04:20.420192 14864 log.cc:927] Time spent T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 13:04:20.420477  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:20.420753 14868 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:20.320722 14870 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:20.403692 14708 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:20.384040 14754 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:20.415201 14807 consensus_meta.cc:287] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.596s	user 0.012s	sys 0.000s
I20260709 13:04:20.412434 14795 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:20.428709 14878 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:20.451965 14839 log.cc:927] Time spent T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 1.120s	user 0.015s	sys 0.021s
I20260709 13:04:20.451807 14560 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:20.454031 14843 log.cc:927] Time spent T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
W20260709 13:04:20.455837 14879 log.cc:927] Time spent T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:04:20.408520  8220 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:20.459106 14877 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:20.460538 14858 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:20.461311 14475 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:20.561554  8141 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 430fa172910841a49c0c80f2cbf76ae5 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:04:20.463798 14809 consensus_meta.cc:287] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.629s	user 0.012s	sys 0.000s
W20260709 13:04:20.561260 14787 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.222s	user 0.015s	sys 0.022s
I20260709 13:04:20.536615 14708 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:04:20.562574 14787 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.224s	user 0.015s	sys 0.022s
I20260709 13:04:20.562340 14809 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:20.562839 14708 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:20.563093 14787 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.563771 14787 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:20.564443 14809 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:20.464991  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:20.565151  8328 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:20.466189 14889 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:20.468427 14730 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:20.429137 14769 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:04:20.426090 14805 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:20.470005  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:20.566931 14787 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:20.569823 14889 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:20.572142 14889 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:20.475279 14798 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:20.476711 14783 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:20.575929 14783 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.576349 14783 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:20.577266 14798 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:20.533008 14874 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:20.579960 14874 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:20.580277 14754 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:20.424695 14869 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.521983 14863 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:20.423867 14786 consensus_meta.cc:287] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.812s	user 0.010s	sys 0.004s
I20260709 13:04:20.481850  8326 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 23. Current term is 24. Ops: []
I20260709 13:04:20.583566  8328 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 16.
I20260709 13:04:20.485780  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2748) took 1214 ms. Trace:
I20260709 13:04:20.486790 14297 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:20.488276 14826 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:20.494117 14756 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:20.421195 14427 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:20.547643 14795 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:04:20.495545 14886 log.cc:927] Time spent T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.071s	user 0.001s	sys 0.001s
I20260709 13:04:20.563167 14878 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:20.498418  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:20.597067  8400 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:04:20.502043 14822 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:20.508178  8297 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:20.421268 14804 consensus_queue.cc:237] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:20.598925  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 90821us
I20260709 13:04:20.547236 14807 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:20.525105  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 161271us
I20260709 13:04:20.617988 14807 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:20.405910 14896 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:20.532541  8327 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 23.
I20260709 13:04:20.546984  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:20.534746 14779 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:20.462116 14806 consensus_meta.cc:287] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.636s	user 0.009s	sys 0.002s
I20260709 13:04:20.625689 14806 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:20.578249  8403 raft_consensus.cc:2410] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:04:20.584810  8972 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:20.586087  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:20.626912  8328 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
I20260709 13:04:20.627766  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:20.628051 14570 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:20.427690  9007 raft_consensus.cc:2955] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Committing config change with OpId 14.12: config changed from index 5 to 12, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:20.628844 14779 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:20.631186 14779 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:20.590370 14450 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:20.430444 14855 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:20.633392 14851 log.cc:927] Time spent T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 1.193s	user 0.004s	sys 0.008s
W20260709 13:04:20.592437  8223 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.020s	sys 0.016s
W20260709 13:04:20.645052  8223 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.020s	sys 0.016s
W20260709 13:04:20.592733  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 184288us
I20260709 13:04:20.584580  8404 rpcz_store.cc:276] 0709 13:04:19.271600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.355894 (+ 84294us) service_pool.cc:224] Handling call
0709 13:04:19.356346 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:19.356798 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:19.356849 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:20.167989 (+811140us) spinlock_profiling.cc:243] Waited 2.13 ms on lock 0x7b2c000e1c48. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:04:20.168556 (+   567us) log.cc:844] Serialized 26 byte log entry
0709 13:04:20.322064 (+153508us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:20.322101 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.322169 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:20.485483 (+163314us) raft_consensus.cc:1685] finished
0709 13:04:20.485503 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.485718 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":591866,"raft.queue_time_us":6115,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":62410,"spinlock_wait_cycles":4905728,"thread_start_us":153113,"threads_started":1,"wal-append.queue_time_us":314912}
I20260709 13:04:20.641052  8326 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 13:04:20.646045  8326 pending_rounds.cc:85] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 15
I20260709 13:04:20.646245  8326 pending_rounds.cc:107] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 13:04:20.646464  8326 raft_consensus.cc:2889] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:20.498323 14834 log.cc:927] Time spent T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.459s	user 0.001s	sys 0.000s
I20260709 13:04:20.590943 14427 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:20.599740  8297 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:04:20.524163  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 116916us
I20260709 13:04:20.605919 14756 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:20.669800 14758 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:20.670398 14758 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:20.672842 14758 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:20.672999 14866 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:20.674201  9002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 1119) took 1818 ms. Trace:
I20260709 13:04:20.674677  9002 rpcz_store.cc:276] 0709 13:04:18.855891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.861206 (+  5315us) service_pool.cc:224] Handling call
0709 13:04:18.878288 (+ 17082us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:18.930085 (+ 51797us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:18.930118 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:18.975272 (+ 45154us) log.cc:844] Serialized 26 byte log entry
0709 13:04:19.233908 (+258636us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:19.233949 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.234020 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:20.497557 (+1263537us) spinlock_profiling.cc:243] Waited 6.33 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:04:20.673822 (+176265us) raft_consensus.cc:1685] finished
0709 13:04:20.673844 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.674138 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42425,"raft.queue_time_us":56878,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":14941,"spinlock_wait_cycles":14568192,"thread_start_us":225960,"threads_started":1,"wal-append.queue_time_us":836088}
I20260709 13:04:20.676600  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:20.677855 14427 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:20.641373 14806 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:20.682075 14806 raft_consensus.cc:697] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:20.682726 14875 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:20.683233 14875 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:20.683109 14806 consensus_queue.cc:237] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:20.686061  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 86422.4us
I20260709 13:04:20.686367 14854 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:20.687424  8297 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:20.606139 14708 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:20.606796 14861 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:20.607749  9006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 1276) took 1603 ms. Trace:
I20260709 13:04:20.608443 14870 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:20.590958 14762 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:20.587544 14786 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:20.592028 14823 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:20.581732 14869 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:20.686762 14854 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.615859 14863 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:20.593904 14877 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:20.618847  8222 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:20.619776  8329 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:04:20.645112  8972 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:20.652778  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2633) took 1425 ms. Trace:
I20260709 13:04:20.659050  8220 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:20.635692 14855 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:20.727576  8325 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:04:20.725148  8251 rpcz_store.cc:276] 0709 13:04:19.228048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.228179 (+   131us) service_pool.cc:224] Handling call
0709 13:04:19.261324 (+ 33145us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:20.280369 (+1019045us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.652527 (+372158us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400309b54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:20.652717 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2334359808}
I20260709 13:04:20.663332  8369 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:20.657826  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:20.725132  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 80107.5us
I20260709 13:04:20.657243  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2632) took 1434 ms. Trace:
I20260709 13:04:20.666508  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:20.667241  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:20.667563 14809 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:20.667963  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:20.668666  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:20.668452 14783 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:20.689082  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:20.738788  9008 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:04:20.693608  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:04:20.643142 14865 log.cc:927] Time spent T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.619s	user 0.002s	sys 0.000s
W20260709 13:04:20.745688  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 126938us
I20260709 13:04:20.695099  9006 rpcz_store.cc:276] 0709 13:04:19.003973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.004159 (+   186us) service_pool.cc:224] Handling call
0709 13:04:19.017713 (+ 13554us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:19.018751 (+  1038us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:19.018796 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:19.024431 (+  5635us) log.cc:844] Serialized 26 byte log entry
0709 13:04:19.263447 (+239016us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:19.263511 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.263615 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:20.450992 (+1187377us) spinlock_profiling.cc:243] Waited 4.08 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:04:20.607440 (+156448us) raft_consensus.cc:1685] finished
0709 13:04:20.607462 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.607689 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":561,"raft.queue_time_us":1346,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":11153,"spinlock_wait_cycles":9381248,"thread_start_us":160728,"threads_started":1,"wal-append.queue_time_us":255971}
I20260709 13:04:20.712924  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2628) took 1707 ms. Trace:
I20260709 13:04:20.714704  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:20.718194  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:20.593484 14795 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:20.687600 14822 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:20.723104 14854 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:20.759187  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:20.760066  8251 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:20.720714  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:20.661662  8400 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:04:20.692400 14691 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:20.643852 14878 consensus_queue.cc:1059] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.076s
I20260709 13:04:20.762195  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:20.764328  8254 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 20. Current term is 21. Ops: []
W20260709 13:04:20.737552  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 74283.5us
I20260709 13:04:20.690140  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:20.741813  8398 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:20.742048  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:20.712841 14622 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:20.718417 14820 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:20.783607  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:20.741255 14758 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:20.745342  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 86362.7us
I20260709 13:04:20.789215  8220 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:20.717974 14779 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:20.749670  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:20.695297 14369 consensus_queue.cc:1048] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:04:20.796149  8298 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:20.747148  8250 rpcz_store.cc:276] 0709 13:04:19.005428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.170654 (+165226us) service_pool.cc:224] Handling call
0709 13:04:19.171007 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:19.219257 (+ 48250us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:19.515452 (+296195us) log.cc:844] Serialized 14 byte log entry
0709 13:04:20.112391 (+596939us) log.cc:844] Serialized 14 byte log entry
0709 13:04:20.296439 (+184048us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:20.296517 (+    78us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:20.296585 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.642411 (+345826us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:04:20.697823 (+ 55412us) spinlock_profiling.cc:243] Waited 48 ms on lock 0x7b54003f16d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:20.712686 (+ 14863us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.712855 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":544625,"prepare.run_cpu_time_us":116,"prepare.run_wall_time_us":16373,"raft.queue_time_us":355411,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":20151,"spinlock_wait_cycles":126172672,"thread_start_us":1075691,"threads_started":3,"wal-append.queue_time_us":508094}
I20260709 13:04:20.753923  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:20.754443  8248 raft_consensus.cc:1275] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:04:20.722363 14826 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:20.733655  8252 rpcz_store.cc:276] 0709 13:04:19.222740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.222904 (+   164us) service_pool.cc:224] Handling call
0709 13:04:19.223237 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:19.227678 (+  4441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:19.227710 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:19.237834 (+ 10124us) log.cc:844] Serialized 26 byte log entry
0709 13:04:19.627563 (+389729us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:19.627605 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.627673 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:20.657000 (+1029327us) raft_consensus.cc:1685] finished
0709 13:04:20.657029 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.657183 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4074,"raft.queue_time_us":5067,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":7336,"thread_start_us":195115,"threads_started":1,"wal-append.queue_time_us":781259}
I20260709 13:04:20.765481  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:20.785583  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:20.745289  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:20.801465  8257 raft_consensus.cc:2393] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 15: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:20.787369  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:20.797231  8256 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:20.800644  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:20.803205  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:20.722086 14823 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:20.755661 14877 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:20.811759  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:20.807193  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:20.813593  8254 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:20.783722 14820 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.812745  8401 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:20.819468 14820 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:20.819919 14887 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:04:20.820513 14842 log.cc:927] Time spent T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc: Append to log took a long time: real 0.692s	user 0.000s	sys 0.005s
I20260709 13:04:20.822346  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:20.736683 14819 consensus_queue.cc:1059] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 13:04:20.827836  8252 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: []
W20260709 13:04:20.805544  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 73943.6us
I20260709 13:04:20.826591  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:20.830750  8369 leader_election.cc:400] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 18. Current term is 19.
I20260709 13:04:20.787801 14622 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:20.833336  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:20.837679  8252 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 22.
I20260709 13:04:20.810492 14774 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:20.839496 14774 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:20.839697 14904 consensus_queue.cc:1059] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.078s
I20260709 13:04:20.840839 14883 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:20.841298 14883 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:20.841696 14883 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:20.841886  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:20.842852 14622 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:20.788079 14730 consensus_queue.cc:1048] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:04:20.843266  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:20.844493  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:20.844614 14730 raft_consensus.cc:3055] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:04:20.845036  8397 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:20.845079 14730 raft_consensus.cc:740] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:20.845691  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:20.783039 14691 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:20.855875  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:20.856536  8330 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:04:20.834461  8251 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 24.
I20260709 13:04:20.857132  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:20.817860 14822 raft_consensus.cc:697] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 23 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:20.850760  8369 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:20.839990 14774 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:20.861891  8398 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:04:20.870483 14708 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:20.832206 14887 raft_consensus.cc:697] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 19 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:20.824127 14867 consensus_queue.cc:1059] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:04:20.862733  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:20.891346  8251 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:04:20.863161  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:20.892335  8326 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
W20260709 13:04:20.892355 14836 log.cc:927] Time spent T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 1.143s	user 0.004s	sys 0.008s
I20260709 13:04:20.834990 14819 consensus_queue.cc:1243] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:04:20.866822  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:20.893901  8401 raft_consensus.cc:2393] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 15: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:20.894034  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2879) took 1965 ms. Trace:
I20260709 13:04:20.894456  8332 rpcz_store.cc:276] 0709 13:04:18.928752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.928908 (+   156us) service_pool.cc:224] Handling call
0709 13:04:18.950346 (+ 21438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:18.974608 (+ 24262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:18.974642 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:18.996337 (+ 21695us) log.cc:844] Serialized 26 byte log entry
0709 13:04:19.245692 (+249355us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:19.245758 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.245846 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:20.745655 (+1499809us) spinlock_profiling.cc:243] Waited 8.01 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:04:20.893762 (+148107us) raft_consensus.cc:1685] finished
0709 13:04:20.893782 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.893979 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23909,"raft.queue_time_us":26908,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":9914,"spinlock_wait_cycles":18417920,"thread_start_us":210867,"threads_started":1,"wal-append.queue_time_us":625106}
I20260709 13:04:20.803741 14866 consensus_queue.cc:1059] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:04:20.899628  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:20.900259  9002 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 24.
I20260709 13:04:20.830816 14889 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:20.825171 14450 consensus_queue.cc:1059] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:04:20.886000  8253 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:20.887631 14902 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:20.889459 14869 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:20.855069 14874 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.873507 14805 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:20.847985 14877 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:20.890388 14861 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:20.815904 14779 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:20.830035 14491 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:04:20.892115 14753 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:20.830562 14826 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:20.903411  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 101501us
I20260709 13:04:20.907058  8369 leader_election.cc:400] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15.
I20260709 13:04:20.849926  8222 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:20.908305  8252 raft_consensus.cc:1240] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:20.924656  8369 leader_election.cc:403] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:20.912603  8401 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:20.913195  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:20.880285 14475 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.035s
I20260709 13:04:20.918334 14691 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:20.918030 14822 consensus_queue.cc:237] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:20.918780 14874 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:20.896510 14762 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:20.921502  8254 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:20.926489  8369 leader_election.cc:400] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 18. Current term is 19.
I20260709 13:04:20.917054 14877 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:20.930786  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:20.914989 14805 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:20.931892  8249 raft_consensus.cc:1240] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:20.949308  8252 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:20.928885  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 79023.2us
I20260709 13:04:20.935992  8369 leader_election.cc:403] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:04:20.939476 14874 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:20.929306 14798 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15.
I20260709 13:04:20.933115 14866 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:20.941583 14887 consensus_queue.cc:237] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:20.943600 14427 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:20.931500 14870 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:20.947459 14779 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:20.924842 14786 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:20.962652  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:20.982550  8250 raft_consensus.cc:2393] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 16: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:20.957980 14805 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:20.970506 14820 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:20.985435 14820 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:20.971508 14837 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 18. Current term is 19.
I20260709 13:04:20.972040  8401 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:46846
I20260709 13:04:20.978869  8253 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:20.969063 14826 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:20.936373 14878 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.988204  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:20.990355  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:20.994434  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:20.994601  8248 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:04:20.997875  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:20.998468  8328 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:04:20.999159  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:20.960752 14783 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:21.001567  9004 raft_consensus.cc:1240] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:21.003804  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:21.004423  9001 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:20.991997 14779 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.006541  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:21.007392  8327 raft_consensus.cc:2368] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 22. Current term is 23.
I20260709 13:04:21.008508  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:21.013127  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:21.013065  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:21.013854  8250 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:04:20.974695 14870 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:21.017057  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:21.019837  8326 raft_consensus.cc:2393] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 15: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:21.023250  8252 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 23.
I20260709 13:04:21.028718 14758 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:21.029856 14907 log.cc:927] Time spent T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 13:04:21.005812 14898 consensus_queue.cc:1059] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:04:21.030059  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:20.932343 14730 consensus_queue.cc:260] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:21.035745  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:21.042544  8401 raft_consensus.cc:606] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:21.042843  9001 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:21.043064  8401 raft_consensus.cc:614] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:21.044216  8298 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:21.034211 14877 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:21.038486  8252 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 23. Current term is 24. Ops: []
I20260709 13:04:21.020809 14870 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:21.032670 14823 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:20.979079 14866 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:21.004667 14878 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:21.016453 14779 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:21.055397 14826 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:21.081928  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:21.085484  8252 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:04:21.092945 14866 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:21.099107  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:21.103976  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:21.112358  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:21.116513 14875 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:21.119665 14875 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:21.122081  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:21.130841 14769 raft_consensus.cc:3055] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:21.131510 14769 raft_consensus.cc:740] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:21.137903 14730 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:21.143076 14779 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:21.144042 14878 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:21.145336 14427 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:21.148116  8332 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:21.152693 14570 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:21.094272 14826 raft_consensus.cc:697] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:21.163271 14826 consensus_queue.cc:237] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:21.170768  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:21.171578  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:21.172224  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:21.172159  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:21.172627  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:21.173753  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:21.193163  8254 raft_consensus.cc:2368] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15.
I20260709 13:04:21.174394  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:21.193524  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:21.193866  8256 raft_consensus.cc:2393] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 14: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:21.194219  8257 raft_consensus.cc:2393] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 20: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:21.175933  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:21.177778  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:21.097885 14775 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:21.204908  8255 raft_consensus.cc:2410] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:04:21.180506 14805 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:21.181821  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:21.184065  8297 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:21.102586 14840 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:04:21.233994  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 50012.8us
I20260709 13:04:21.188272 14854 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:21.189990 14774 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:21.175051  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:21.206621  8397 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18.
I20260709 13:04:21.132654 14769 consensus_queue.cc:260] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:21.209698 14868 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.210769 14775 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:21.211724 14796 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:21.211874  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:21.212630  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:21.205090 14758 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:21.271019  8328 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 13:04:21.218382  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:21.272390  8254 raft_consensus.cc:2393] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 18: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:21.190768 14874 raft_consensus.cc:3055] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:21.219792 14798 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:21.224867  8223 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:21.227140  8220 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:21.225625  8974 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:21.249648  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:21.283692  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 58163.3us
I20260709 13:04:21.253770  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:21.285588  8974 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:21.253660  8222 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:21.255205  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:21.287967  8332 raft_consensus.cc:2393] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 24: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:21.267551  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:21.272519  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:21.217015 14883 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:21.277202 14796 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:21.293167 14883 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:21.293277 14796 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:21.294018  9007 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:21.295248  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:21.296491 14805 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:21.298899 14774 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:21.299525 14775 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:21.300375  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:21.300625  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:21.301237  8323 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 21: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:21.301548 14908 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:21.302210  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:21.278622  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:21.303184  8398 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:21.303633  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:21.304164  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:21.255590 14868 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:21.304826  8398 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:21.305873  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:21.250661 14840 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:21.306645 14868 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:21.308244 14840 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:21.298017  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:21.302285 14908 raft_consensus.cc:697] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:21.309320  8330 raft_consensus.cc:2393] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 17: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:21.289011 14798 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:21.308573 14854 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:21.248948 14730 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:21.249799  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:21.310249  8398 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:21.221158 14870 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:21.206141 14795 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:21.328410  8401 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:21.330112  8330 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 20. Current term is 21. Ops: []
W20260709 13:04:21.318603  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 93809.4us
I20260709 13:04:21.332957  8223 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:21.278095 14874 raft_consensus.cc:740] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:21.339881 14870 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:21.341559 14908 consensus_queue.cc:237] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:21.342314 14795 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:21.345660  8399 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:04:21.351704  8398 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:21.287824 14758 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:21.359601  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 106018us
W20260709 13:04:21.359815  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 132800us
I20260709 13:04:21.287919 14871 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:21.316232  8297 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:21.374624  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:04:21.405414  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 89252.3us
I20260709 13:04:21.406239  8297 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:21.334506 14877 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:21.406371  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:21.389566  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:21.411650  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:21.408088  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:21.412371  8400 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:21.414474  8220 leader_election.cc:400] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 22. Current term is 23.
I20260709 13:04:21.414790  8220 leader_election.cc:403] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 13:04:21.421113  8400 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:21.422253  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:21.423825  8400 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:04:21.424825  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:21.364984 14887 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:21.426174  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:21.430647  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 97899.1us
I20260709 13:04:21.425464  8400 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:21.427541  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:21.361639 14730 raft_consensus.cc:697] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:21.412789 14854 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:21.450937 14854 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:21.451357 14854 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:21.451174 14855 raft_consensus.cc:3055] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:21.452224 14855 raft_consensus.cc:740] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:21.453351  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:21.454332 14932 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.454851 14932 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:21.435357  8220 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:21.442598  8401 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:04:21.427145 14754 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:21.390117 14775 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:21.453673 14855 consensus_queue.cc:260] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:21.437876 14753 consensus_meta.cc:287] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.513s	user 0.005s	sys 0.006s
I20260709 13:04:21.455099  8398 raft_consensus.cc:2393] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:21.467006 14805 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:21.500072 14805 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:04:21.466714  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 54891.2us
I20260709 13:04:21.492805 14902 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 22. Current term is 23.
I20260709 13:04:21.457068  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:21.501885  8222 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:21.502276 14664 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.503058 14887 raft_consensus.cc:697] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 18 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:21.503288  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 68007.3us
I20260709 13:04:21.537470  8220 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:21.503945 14857 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:21.504163 14887 consensus_queue.cc:237] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:21.504915 14889 raft_consensus.cc:3055] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:21.502866 14664 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:21.501248 14805 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:21.506249 14758 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:21.551496  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:21.465639 14869 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:21.450966 14730 consensus_queue.cc:237] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:21.457121 14932 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:21.510735 14795 consensus_queue.cc:1059] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:04:21.466912 14914 log.cc:927] Time spent T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.480s	user 0.003s	sys 0.000s
I20260709 13:04:21.476197 14798 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:21.504272 14786 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:21.514271  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:21.365589 14875 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:21.525324  9004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 1391) took 1097 ms. Trace:
I20260709 13:04:21.576606  9004 rpcz_store.cc:276] 0709 13:04:20.427571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.427775 (+   204us) service_pool.cc:224] Handling call
0709 13:04:20.428091 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.004276 (+576185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.525058 (+520782us) spinlock_profiling.cc:243] Waited 573 ms on lock 0x7b540069ebd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:21.525263 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1318291200}
W20260709 13:04:21.499311  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 67670.6us
I20260709 13:04:21.578903  8223 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:21.502875  8222 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:21.540493  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:21.544755  8400 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
W20260709 13:04:21.581229  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 78430.5us
I20260709 13:04:21.581749  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:21.546125  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:21.457245 14754 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:21.506259 14837 consensus_queue.cc:1059] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:21.559314  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:21.547252 14889 raft_consensus.cc:740] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:21.563480 14758 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:21.574507  8297 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:21.645925  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 71522.6us
I20260709 13:04:21.584077  8398 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:04:21.585856  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:21.647315  8403 raft_consensus.cc:2393] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 20: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:21.587364  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:21.648227  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:21.649019  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:21.589840  9005 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:04:21.573122 14795 raft_consensus.cc:3055] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:21.535246 14775 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:21.654175 14775 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:21.543197 14783 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:21.583782 14869 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:21.586143 14857 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:21.656430 14857 raft_consensus.cc:697] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:21.499424 14874 consensus_queue.cc:260] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:21.585171 14774 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:21.657936 14880 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:21.580374  8223 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:21.589666 14754 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:21.586457 14786 raft_consensus.cc:697] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:21.630568 14927 log.cc:927] Time spent T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:04:21.648231  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:21.650748  8400 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
W20260709 13:04:21.660662  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 80385us
I20260709 13:04:21.649418  8297 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:21.662756  8223 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:21.638568 14889 consensus_queue.cc:260] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:21.664026  8403 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 24: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:21.663450  8297 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:21.661160  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:21.662071  9009 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:04:21.665588  8223 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:21.665000  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:21.661010  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 144468us
I20260709 13:04:21.666451  8403 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
I20260709 13:04:21.667651  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:21.667761  8974 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:21.670661  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:21.672631  9005 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:21.673199  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:21.673995  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:21.721489  9005 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:04:21.674743  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:21.675236 14570 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:21.676086  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:21.664638 14783 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:21.691879  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:21.733701 14783 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.734248 14783 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:21.734334  8400 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 24.
I20260709 13:04:21.734755  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:21.735949  8404 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18.
I20260709 13:04:21.658181 14874 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:21.740821 14798 raft_consensus.cc:3055] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:21.659943 14880 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:21.741643 14798 raft_consensus.cc:740] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:21.741822 14880 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:21.742646 14798 consensus_queue.cc:260] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:21.743858 14798 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:21.744537  8399 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:21.671931 14753 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:21.745972 14753 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:21.746459 14753 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:21.651336 14795 raft_consensus.cc:740] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:21.749934  8402 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:21.750203  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 1033) took 1398 ms. Trace:
I20260709 13:04:21.733265 14570 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:21.750535  8325 rpcz_store.cc:276] 0709 13:04:20.351661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.351894 (+   233us) service_pool.cc:224] Handling call
0709 13:04:20.352447 (+   553us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:20.728048 (+375601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.749950 (+1021902us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b540011cbd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:21.750145 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":862139776}
I20260709 13:04:21.751389 14570 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:21.755280  8397 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:21.755993 14664 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:21.756966  8397 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:21.758180  8404 raft_consensus.cc:1240] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:21.759433  8405 raft_consensus.cc:1240] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:04:21.721781 14908 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:21.760097 14908 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.760516 14908 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:21.764077 14774 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:21.764771 14774 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:21.766363 14786 consensus_queue.cc:237] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:21.770066 14857 consensus_queue.cc:237] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:21.771427 14769 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:21.651587 14910 raft_consensus.cc:3055] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:21.778470  8297 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:21.779219  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:21.779304  9005 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 20. Current term is 21. Ops: [18.14-20.15]
I20260709 13:04:21.780754  9009 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:04:21.676596  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:21.656546 14854 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:21.795614  8404 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:21.797004  8403 raft_consensus.cc:1240] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:21.750447 14795 consensus_queue.cc:260] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:21.800489 14662 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:21.651861 14758 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:21.675001 14856 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:21.672191 14875 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:21.797684  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 255672us
I20260709 13:04:21.819479  8972 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:21.839936  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 50954.3us
W20260709 13:04:21.829085  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.823125  8406 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:04:21.827469  8397 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:21.843456  8403 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 18. Current term is 19. Ops: []
W20260709 13:04:21.729590 14928 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:04:21.844890  8372 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:21.803324 14889 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:21.846738  8398 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:21.815234 14662 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.849319 14662 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:21.849606 14708 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:21.850225 14708 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:21.850764 14708 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:21.851910 14783 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:21.815155 14820 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:21.821579 14826 consensus_queue.cc:1059] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:04:21.847013 14450 consensus_queue.cc:1059] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.006s
I20260709 13:04:21.867049  8403 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:04:21.847734  8372 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:21.782634 14910 raft_consensus.cc:740] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:21.862136  8397 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:04:21.876456  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:21.838653 14819 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:21.865180  8220 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:21.812582 14876 consensus_queue.cc:1059] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 12.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:04:21.854048 14475 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:21.844853 14775 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:21.822965 14621 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:04:21.903434 14621 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:21.903960 14621 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:21.836918 14664 consensus_queue.cc:1059] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:04:21.909168 14369 raft_consensus.cc:3055] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:21.909634 14369 raft_consensus.cc:740] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:21.910566 14369 consensus_queue.cc:260] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:21.911881 14369 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:21.915473  8222 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:21.898159 14450 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:21.916836 14450 raft_consensus.cc:697] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:21.918023 14450 consensus_queue.cc:237] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:21.870703 14784 consensus_queue.cc:1059] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:21.901115 14855 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:21.872327 14854 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:21.838184 14837 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:21.817754 14856 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.815784 14904 consensus_queue.cc:1059] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:04:21.891536  8397 raft_consensus.cc:1275] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:04:21.891477  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 43829.3us
I20260709 13:04:21.895182 14876 raft_consensus.cc:3055] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:21.895782 14882 raft_consensus.cc:3055] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:04:21.898262 14753 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:21.898242 14796 raft_consensus.cc:3055] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:21.899757 14758 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:21.917040  8222 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:21.921717 14560 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:04:21.954073  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 88867.3us
W20260709 13:04:21.984063  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 67109.3us
I20260709 13:04:21.949246  8398 raft_consensus.cc:2410] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:04:21.971329  8372 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:21.868150 14820 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:21.980053 14570 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:21.969640 14856 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:21.980628 14941 log.cc:927] Time spent T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:04:21.985249  8222 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:21.986307 14930 log.cc:927] Time spent T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.486s	user 0.003s	sys 0.000s
I20260709 13:04:21.960005 14902 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:21.901999 14775 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:22.001451 14758 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:21.993665 14875 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:21.894138 14819 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:21.972455 14876 raft_consensus.cc:740] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:21.981679 14795 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:22.035331  8220 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:22.013480  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:22.019842  8222 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:21.969002 14837 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:21.969192 14774 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:22.047827  8405 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:21.974467 14882 raft_consensus.cc:740] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:21.981567 14796 raft_consensus.cc:740] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:22.049278  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:22.056098  8220 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:22.062963  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:22.065855  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:22.057852 14898 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:22.069526  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:22.075968  8251 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 19: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:22.070367  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:22.077134  8254 raft_consensus.cc:2368] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 17. Current term is 18.
I20260709 13:04:22.071254  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:22.035506 14870 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:04:22.088389  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.971822 14708 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:22.054759 14875 raft_consensus.cc:697] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:22.095710 14819 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:22.096115 14875 consensus_queue.cc:237] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:22.058281 14869 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:22.099594 14869 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.100068 14869 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:22.100628 14854 raft_consensus.cc:3055] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:22.101063 14854 raft_consensus.cc:740] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:21.980746 14910 consensus_queue.cc:260] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:22.020388 14939 consensus_queue.cc:1059] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:04:22.094549 14876 consensus_queue.cc:260] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:22.102624  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:22.046578 14570 consensus_queue.cc:1059] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 13:04:22.064857 14621 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:22.010435 14880 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:22.117544  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 394511us
I20260709 13:04:22.106762 14874 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:22.131626  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:22.138088  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 293438us
I20260709 13:04:22.140236  8251 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:22.124066 14910 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:22.148080  8369 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:22.145977 14475 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:22.150252  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:22.160929  8254 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
W20260709 13:04:22.152326 14805 consensus_meta.cc:287] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.583s	user 0.011s	sys 0.000s
I20260709 13:04:22.116039 14754 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:22.139046 14876 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:22.173020  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:22.173273  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 353988us
I20260709 13:04:22.156057  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:22.156648  8405 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:04:22.155344  8297 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:22.158171  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.152664 14939 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:22.161762  9005 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:22.162111  8369 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:22.127291 14871 consensus_queue.cc:1059] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:04:22.166505 14908 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:22.181820  8297 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:22.150331 14621 consensus_queue.cc:1059] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:04:22.139194 14854 consensus_queue.cc:260] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:22.118768 14796 consensus_queue.cc:260] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:22.118640 14882 consensus_queue.cc:260] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:22.174686  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:22.189252  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:22.163595 14753 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:22.193243  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:22.196156  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:22.178161 14824 raft_consensus.cc:3055] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:04:22.245457  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 189398us
I20260709 13:04:22.198967  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:22.200035 14837 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:22.204118 14869 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:22.187879 14756 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 13:04:22.205690 14775 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:22.205933 14621 raft_consensus.cc:3055] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:22.206928 14475 raft_consensus.cc:3055] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:22.207234 14708 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:22.208262 14880 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:22.179278 14867 consensus_queue.cc:1059] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:04:22.210032 14796 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:22.211642  9008 raft_consensus.cc:1240] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:22.213830 14908 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:22.180691 14820 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:22.219947  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:22.308415  8397 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:22.221822 14822 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:04:22.222508  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 47890.6us
I20260709 13:04:22.228329  8404 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:22.230067  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:22.230721  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:22.234095  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:22.236414  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:22.237846  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:22.238022  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:22.169981 14805 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:22.240413 14754 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:22.240798  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:04:22.328653  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.241149 14882 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:22.241995 14622 raft_consensus.cc:3055] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:22.179607 14825 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:22.196141  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:22.255214  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.350656  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:22.265661  8223 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:22.269615  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:22.270056  8326 raft_consensus.cc:1240] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:22.277546  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:04:22.352982  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 87448.8us
I20260709 13:04:22.212738 14662 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:22.190155 14932 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:22.194664 14904 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:22.289922  8250 raft_consensus.cc:1240] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:22.290527  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:22.360785  8253 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
W20260709 13:04:22.292066  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.294337  8403 raft_consensus.cc:1240] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:22.299873  8405 raft_consensus.cc:1240] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:04:22.300495  8256 raft_consensus.cc:2393] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 19: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:04:22.302093  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 238074us
I20260709 13:04:22.303314  8327 raft_consensus.cc:1240] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 16. Current term is 17. Ops: []
W20260709 13:04:22.346927 14931 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.546s	user 0.004s	sys 0.000s
I20260709 13:04:22.243436 14824 raft_consensus.cc:740] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:22.350889 14822 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.035s
I20260709 13:04:22.268675 14889 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:22.324734  8332 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 17. Current term is 18. Ops: []
W20260709 13:04:22.331157  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.273944 14784 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:04:22.331692  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 277825us
I20260709 13:04:22.396358  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:22.397013  9008 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:22.334882  8972 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:22.282421 14854 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:22.403331 14664 raft_consensus.cc:3055] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:22.403789 14664 raft_consensus.cc:740] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:22.404649 14664 consensus_queue.cc:260] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:22.289297 14621 raft_consensus.cc:740] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:22.289186 14475 raft_consensus.cc:740] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:22.296063 14774 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:22.291263 14783 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:22.328202 14938 log.cc:927] Time spent T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
W20260709 13:04:22.353006  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 157000us
I20260709 13:04:22.357697  8400 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:22.310283  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:22.337797 14933 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:22.328490 14943 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:22.343186 14622 raft_consensus.cc:740] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:22.351809 14369 consensus_queue.cc:1059] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:04:22.384440 14662 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:22.386389 14824 consensus_queue.cc:260] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:22.388283  8254 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:04:22.345822 14935 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:22.385138  8331 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 24. Current term is 25. Ops: []
I20260709 13:04:22.346001 14893 raft_consensus.cc:3055] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:04:22.390267  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 48129.4us
I20260709 13:04:22.698320 14664 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:22.712755  8295 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:22.713948  8295 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:22.314741  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:22.346016 14825 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.331988 14855 consensus_queue.cc:1059] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:04:22.392428 14708 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:22.400352 14784 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:22.401108  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:04:22.401521  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 66766.5us
I20260709 13:04:22.405215 14950 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:22.198984 14871 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.117s
I20260709 13:04:22.408280  9000 raft_consensus.cc:1240] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:22.408702  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:22.408951 14889 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:22.409391  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:22.410104 14819 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:04:22.410482  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 45812.4us
I20260709 13:04:22.412958 14560 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.004s
I20260709 13:04:22.711309 14826 consensus_queue.cc:1059] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:04:22.732919 14950 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:22.738384 14950 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:22.414083 14870 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:22.739351 14870 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:22.414213 14877 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:22.417541 14910 consensus_queue.cc:1059] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:22.418448 14880 consensus_queue.cc:1059] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:04:22.410399  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 55793.1us
I20260709 13:04:22.411985 14856 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:22.420472 14960 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:22.421677 14954 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:22.423470  8327 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:04:22.426474  8332 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16. Ops: [15.14-15.14]
W20260709 13:04:22.428210  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:22.754343  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.435345  9008 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 25.
I20260709 13:04:22.733120 14825 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:22.452718  8404 raft_consensus.cc:1240] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:22.360641 14874 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:04:22.456672  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 48641.3us
I20260709 13:04:22.459004  8252 raft_consensus.cc:2955] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Committing config change with OpId 14.12: config changed from index 5 to 12, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:04:22.548015  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:22.630010  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:22.762431  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:22.762624  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:22.645673  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 233632us
I20260709 13:04:22.763320  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:22.606168 14869 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:22.591511  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2767) took 1377 ms. Trace:
I20260709 13:04:22.610862 14475 consensus_queue.cc:260] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:22.766357 14475 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:22.764763  8399 rpcz_store.cc:276] 0709 13:04:21.214512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.745335 (+530823us) service_pool.cc:224] Handling call
0709 13:04:21.780913 (+ 35578us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:21.845063 (+ 64150us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:21.845097 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:21.853302 (+  8205us) log.cc:844] Serialized 26 byte log entry
0709 13:04:22.287953 (+434651us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:22.288013 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.288114 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:22.580974 (+292860us) spinlock_profiling.cc:243] Waited 936 us on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:04:22.591228 (+ 10254us) raft_consensus.cc:1685] finished
0709 13:04:22.591252 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:22.591455 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58765,"raft.queue_time_us":59006,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":9774,"spinlock_wait_cycles":2153856,"thread_start_us":426775,"threads_started":1,"wal-append.queue_time_us":589933}
W20260709 13:04:22.591899  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.769508 14940 consensus_queue.cc:1059] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:04:22.618672 14957 log.cc:927] Time spent T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:04:22.770792 14940 consensus_queue.cc:1059] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 16.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:04:22.592751 14955 log.cc:927] Time spent T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:04:22.771602 14621 consensus_queue.cc:260] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:22.772038 14940 consensus_queue.cc:1059] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:04:22.672254  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.558239 14958 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:22.773648 14857 raft_consensus.cc:3055] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:22.748656 14961 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
I20260709 13:04:22.695727  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:22.699872  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3354) took 1722 ms. Trace:
I20260709 13:04:22.706454 14622 consensus_queue.cc:260] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:22.733124  8972 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:22.775269  8400 rpcz_store.cc:276] 0709 13:04:20.977502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.736245 (+758743us) service_pool.cc:224] Handling call
0709 13:04:21.736497 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:22.691337 (+954840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.699624 (+  8287us) spinlock_profiling.cc:243] Waited 621 ms on lock 0x7b54000c8d54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:22.699810 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1428037888}
I20260709 13:04:22.711349 14893 raft_consensus.cc:740] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:22.776264 14622 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:22.776883 14893 consensus_queue.cc:260] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:22.778306 14893 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:22.778419  8972 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:22.779940 14297 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:22.780308 14297 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:22.780725 14297 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:22.786566  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 90903.5us
I20260709 13:04:22.773150 14940 raft_consensus.cc:3055] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:22.789862  8369 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:22.791491 14940 raft_consensus.cc:740] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:22.755019 14560 consensus_queue.cc:1048] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
W20260709 13:04:22.795507  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.733507 14787 consensus_queue.cc:1048] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
W20260709 13:04:22.715287 14876 consensus_meta.cc:287] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.538s	user 0.012s	sys 0.001s
I20260709 13:04:22.714823 14854 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:22.816802  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:22.816939 14854 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:22.817413 14854 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:22.745142 14819 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:22.823446  9006 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:04:22.824040 14819 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:22.758445 14960 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:22.828552 14824 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:22.828698 14960 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:22.807870 14967 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:22.817454  8325 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:22.702880 14943 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:22.750088  8223 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:22.727985 14902 consensus_meta.cc:287] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.685s	user 0.007s	sys 0.005s
I20260709 13:04:22.715643 14662 raft_consensus.cc:3055] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:22.714144 14820 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:22.430665  8397 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 16.
I20260709 13:04:22.451156  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:22.756242 14786 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.003s
I20260709 13:04:22.757102 14856 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:22.757300 14904 raft_consensus.cc:3055] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:22.757668 14954 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:22.605726 14951 raft_consensus.cc:3055] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:22.763700  8403 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:22.764871  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:22.590171 14805 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:22.765410  8401 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:04:22.772797 14621 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:22.774144 14857 raft_consensus.cc:740] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:22.773870 14958 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
W20260709 13:04:22.594161 14956 log.cc:927] Time spent T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:04:22.788727 14796 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:22.799785  8332 raft_consensus.cc:1240] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:22.838471 14902 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:22.839918 14868 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:22.815649 14887 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.800413  8331 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:22.801071 14889 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:22.785662 14874 raft_consensus.cc:3055] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:22.744912 14877 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:22.806133  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:22.813506 14870 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:22.778324 14882 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:22.747939 14570 consensus_queue.cc:1059] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:04:22.813376  8399 raft_consensus.cc:1275] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:22.737279  8295 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:22.814031  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:22.902941  8400 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:22.833190  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 83192.4us
I20260709 13:04:22.836853  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:22.834898  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:22.904146  9008 raft_consensus.cc:2393] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 19: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:22.904484  8223 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:22.854108  9005 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:04:22.854024  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:22.905615  8397 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:22.844434  8974 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:04:22.906840  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 62491.7us
I20260709 13:04:22.860069  8255 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:22.874603  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:22.882095  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:22.909199  8298 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:22.850247  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.806990  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:22.854849 14784 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:22.862566 14969 log.cc:927] Time spent T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:04:22.815526  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:22.903199  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:22.838954 14662 raft_consensus.cc:740] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:22.792994 14940 consensus_queue.cc:260] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:22.859263 14932 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:22.857064 14958 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:22.864104 14943 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:22.846165 14904 raft_consensus.cc:740] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:22.940209  8220 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:22.847877 14954 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:22.857596 14756 raft_consensus.cc:3055] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:22.949719 14880 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.965137 14756 raft_consensus.cc:740] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:22.965651 14880 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:22.965996 14898 consensus_queue.cc:1059] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:22.840399 14967 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:22.968103 14880 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:22.968451 14967 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:22.968739  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 56731.5us
I20260709 13:04:22.969298 14954 consensus_queue.cc:1048] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:22.969617  8297 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:22.973116 14960 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:22.973744 14960 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:22.973343 14904 consensus_queue.cc:260] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:22.974236 14960 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:22.974640 14943 consensus_queue.cc:1059] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:22.975589 14943 raft_consensus.cc:3055] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:22.975950 14943 raft_consensus.cc:740] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:22.962774 14855 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:22.977999 14855 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.978418 14855 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:22.971103  8297 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:22.962966 14787 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:22.979933 14754 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:22.980026 14787 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:22.980647 14787 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:22.855273 14857 consensus_queue.cc:260] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:22.982635 14857 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:22.944628  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 129188us
I20260709 13:04:22.944988  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:22.888278 14874 raft_consensus.cc:740] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:22.985294 14874 consensus_queue.cc:260] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:22.986641 14874 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:22.946535  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 209304us
I20260709 13:04:22.947191  8399 raft_consensus.cc:2393] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 19: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:22.989950  8295 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:22.991115 14958 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:22.991863 14958 raft_consensus.cc:697] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 24 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:22.992354 14898 raft_consensus.cc:3055] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:22.992798 14898 raft_consensus.cc:740] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:22.992854 14958 consensus_queue.cc:237] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:22.993999  8369 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:22.948902  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:22.903729 14774 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.996258 14774 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:22.903476 14882 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:22.997501 14882 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:22.998790  8327 raft_consensus.cc:1275] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:23.000453  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:22.971175 14954 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:23.002012 14954 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:23.001893 14662 consensus_queue.cc:260] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:23.002561 14954 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:23.003575 14662 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:23.003739  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:23.004498  8324 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:22.848050 14951 raft_consensus.cc:740] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:22.840029 14819 consensus_queue.cc:1059] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 13:04:23.006043 14898 consensus_queue.cc:260] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:23.006420 14882 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:23.007108  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:23.007432 14898 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:23.007792  8402 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:04:22.976819 14943 consensus_queue.cc:260] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:23.009577 14943 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:22.938454 14871 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:23.010638 14871 raft_consensus.cc:697] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:23.013052 14880 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:23.013723 14880 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:22.861954 14887 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:22.848806 14950 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:23.016175 14967 consensus_queue.cc:1048] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:04:23.016832  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.018334 14950 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:23.019343  8297 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:22.941975 14820 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:23.020330 14820 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:23.021642  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:22.954896  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 50485.3us
I20260709 13:04:22.991184  8295 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:23.036664  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:23.037310  9005 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:23.037866  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:23.039897  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:23.040339  9009 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 13:04:23.041124  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:23.041894  9006 raft_consensus.cc:2393] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 21: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:22.945525 14870 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:23.047185  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 86037.3us
I20260709 13:04:22.950910 14935 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:23.031388  8220 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:23.046766 14819 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:23.054428 14877 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:23.054625 14819 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:23.054816 14950 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:04:23.055110 14819 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:23.055279 14758 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:22.950466 14786 consensus_queue.cc:1059] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.105s
I20260709 13:04:22.960383 14971 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.109s
I20260709 13:04:23.057020 14960 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:23.051271 14953 log.cc:927] Time spent T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:04:23.062227  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 2602) took 1014 ms. Trace:
I20260709 13:04:23.062887  8255 rpcz_store.cc:276] 0709 13:04:22.048206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.048523 (+   317us) service_pool.cc:224] Handling call
0709 13:04:22.054186 (+  5663us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:22.936250 (+882064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.061984 (+125734us) spinlock_profiling.cc:243] Waited 806 ms on lock 0x7b540016f154. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:23.062167 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1852916992}
I20260709 13:04:23.064154 14960 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:23.063330 14819 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:23.066787 14902 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:23.067319 14902 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:23.068689 14819 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:23.069840  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 75933.2us
W20260709 13:04:23.070607  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.070590  8369 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:23.071863  8369 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:23.073037  8369 leader_election.cc:400] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 17. Current term is 18.
I20260709 13:04:23.073382  8400 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
I20260709 13:04:23.073657 14935 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:23.074129 14856 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:04:23.074283  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 83177.8us
I20260709 13:04:23.074452 14935 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:23.069295 14902 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:23.074784 14856 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:23.076028  8295 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:23.078095 14787 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:23.078233 14621 consensus_queue.cc:1059] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:23.079015 14910 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:23.079746 14910 raft_consensus.cc:697] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:23.080209 14940 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:23.081715 14369 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:23.082322 14369 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:23.082757 14787 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:23.084534 14369 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:23.085284 14826 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:23.048533 14570 consensus_queue.cc:1059] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:04:23.077201 14856 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:23.086596 14570 raft_consensus.cc:3055] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:23.087103 14570 raft_consensus.cc:740] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:23.075230 14868 raft_consensus.cc:3055] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:23.088069 14868 raft_consensus.cc:740] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:23.088553 14774 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:23.087844 14570 consensus_queue.cc:260] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:23.085822 14826 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:23.089999 14826 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:23.048055  8298 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:23.091677  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:23.079082  8295 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:23.064155  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:23.092154  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 44194.9us
I20260709 13:04:23.093390  8298 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:23.093954  9006 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:23.089576 14570 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:22.997009 14854 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:23.097833  8248 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:04:23.098846  8369 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:23.080783 14910 consensus_queue.cc:237] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:23.108855  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:23.111738  8295 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:23.094538 14856 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:23.096143 14622 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:23.115674  8297 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:23.089035 14868 consensus_queue.cc:260] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:23.103756 14754 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:04:23.140786  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.141273 14622 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.141719  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:23.141813 14622 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:23.142613  8332 raft_consensus.cc:2393] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 18: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:23.094166 14297 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:23.162179  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:23.142918 14820 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:23.162758 14854 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:23.107401 14880 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:23.163223 14297 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:23.163784 14621 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:23.164081  9000 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:23.113747 14774 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:23.167379 14774 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:23.167979 14774 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:23.114090 14889 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:23.143589 14935 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.170079  8327 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:04:23.170176 14935 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:23.171597  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:23.172185  8327 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 24.
I20260709 13:04:23.172521 14935 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:23.144032 14369 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:23.144721 14887 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:23.144816  8222 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:23.055127 14805 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.060s
I20260709 13:04:23.145031 14786 consensus_queue.cc:1059] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:23.146760 14756 consensus_queue.cc:260] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:23.139252 14708 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.224687  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:23.151878  8323 raft_consensus.cc:1240] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13. Ops: []
W20260709 13:04:23.152722  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.234583  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:23.154474  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:23.132068 14932 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:23.173383  8327 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:04:23.181070  8397 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 13:04:23.237779  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.238859  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:23.239565  8397 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:23.239408  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:23.240672  8402 raft_consensus.cc:2393] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 20: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:23.176393  8405 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 19. Current term is 20. Ops: [19.12-19.12]
W20260709 13:04:23.241513  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.177155  9000 raft_consensus.cc:1240] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:23.178054  8406 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:23.243129  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:23.180164  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:23.177314  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:23.243995  8257 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:04:23.245085  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:23.182699 14935 consensus_queue.cc:1059] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:23.183684 14369 raft_consensus.cc:697] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:23.184746  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:23.110773 14753 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:23.105507 14758 consensus_queue.cc:1059] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.074s
I20260709 13:04:23.177353 14898 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:23.264490  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:04:23.185408 14972 log.cc:927] Time spent T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:04:23.188512 14826 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:23.198697 14825 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:23.202275  9005 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18.
I20260709 13:04:23.145180 14622 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:23.247797 14868 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:23.208046  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:23.231060  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:23.205714  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:23.233404  9006 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:04:23.252478  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:23.253132  8404 raft_consensus.cc:1275] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:23.283914  8323 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
I20260709 13:04:23.200618 14875 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:23.294100  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:23.257333  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:23.294665  8253 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:04:23.295109  9000 raft_consensus.cc:1813] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Handling vote request from an unknown peer ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:23.295339  8255 raft_consensus.cc:2410] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:04:23.258193  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:23.295811  9000 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 15: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:23.295964  8327 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:23.183853 14855 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:23.258144  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:23.300037  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:23.301257  8331 raft_consensus.cc:2393] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 19: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:23.176146 14882 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:23.263226  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:23.263322  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:23.262300  8972 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:23.306998  8254 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:23.270143  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.184711 14960 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 13:04:23.273500  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:23.271304  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:04:23.308163  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:23.308756  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.310211  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:23.276865  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 209455us
I20260709 13:04:23.277093  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:04:23.277503  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.223498  8400 raft_consensus.cc:1240] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:04:23.279534  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:23.280220  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:23.280789  8397 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:23.285863  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:23.314639  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:23.292330  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.264879 14904 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:23.329066  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:23.329411  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:23.330811  8330 raft_consensus.cc:2368] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 18. Current term is 19.
I20260709 13:04:23.329918  8323 raft_consensus.cc:2393] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 18: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:23.157025 14951 consensus_queue.cc:260] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:23.247256 14708 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:23.299685  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:23.357331  8327 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
I20260709 13:04:23.237789 14857 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.359110  8220 leader_election.cc:400] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 18. Current term is 19.
I20260709 13:04:23.359643  8220 leader_election.cc:403] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:23.315485  8406 raft_consensus.cc:2393] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 20: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:23.316064  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:23.263743 14753 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:23.365335  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:23.306452 14622 raft_consensus.cc:3055] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:23.365594  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:23.352540 14940 raft_consensus.cc:3055] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:23.356637  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:23.138242 14871 consensus_queue.cc:237] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:23.361372  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 45299.7us
I20260709 13:04:23.375027  8254 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
I20260709 13:04:23.327661  9008 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:04:23.342204  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:23.370002  8328 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:04:23.373270  8327 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:23.384658  8406 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:04:23.387990  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:04:23.372728 14969 log.cc:927] Time spent T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:04:23.397110  8332 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:23.373041 14950 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:23.378681 14940 raft_consensus.cc:740] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:23.399537  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:23.399869  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:23.407045  8327 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
I20260709 13:04:23.408713  8323 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:04:23.398280  8254 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
W20260709 13:04:23.409915  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 115538us
W20260709 13:04:23.403980  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 141778us
I20260709 13:04:23.373637 14622 raft_consensus.cc:740] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:23.366971 14979 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:04:23.405778  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:23.402611  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:23.389521 14951 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:23.406265  8369 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:23.405148  8147 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45448 (request call id 287) took 12114 ms (client timeout 15000 ms). Trace:
I20260709 13:04:23.410990  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:23.420336  8254 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 21: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:23.421424  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:23.370218 14874 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:04:23.422309  8330 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:23.423437  8329 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 24 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 23. Current term is 24. Ops: [23.18-23.18]
I20260709 13:04:23.423409  8298 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:23.424212  8254 raft_consensus.cc:2393] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 13: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:23.333608 14369 consensus_queue.cc:237] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:23.373174 14857 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:23.405813 14950 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:04:23.422662 14976 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:04:23.430945  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:23.445827  8255 raft_consensus.cc:2393] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 13: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:23.446182  8297 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:23.431941  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:23.436307  8298 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:23.437880  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:23.407114 14893 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.426111  8972 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:23.454092  8972 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:23.458057 14882 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:23.458971 14882 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:23.459545 14882 raft_consensus.cc:697] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 15 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:23.430174  8220 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:23.462306 14870 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:23.396790 14805 consensus_queue.cc:1059] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 13:04:23.443403  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:23.463083 14870 raft_consensus.cc:697] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:23.463696  8257 raft_consensus.cc:2368] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15.
I20260709 13:04:23.464912  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:23.464502 14870 consensus_queue.cc:237] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:23.466090 14887 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.466279  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:23.451565  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:23.466907 14887 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:23.470729 14887 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:23.425972 14877 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.473806 14877 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:23.476714 14877 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:23.460701 14882 consensus_queue.cc:237] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:23.484529  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:23.425055 14958 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:23.451323  8297 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:23.414368  8147 rpcz_store.cc:269] 0709 13:04:11.289455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.305890 (+ 16435us) service_pool.cc:224] Handling call
0709 13:04:23.400785 (+12094895us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:18.694005 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:18.755088 (+ 61083us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:18.755143 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:04:18.755167 (+    24us) tablet.cc:662] Decoding operations
0709 13:04:18.856241 (+101074us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:18.856359 (+   118us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:18.856386 (+    27us) tablet.cc:685] Acquiring locks for 35 operations
0709 13:04:18.915304 (+ 58918us) tablet.cc:701] Row locks acquired
0709 13:04:18.915366 (+    62us) write_op.cc:260] PREPARE: finished
0709 13:04:18.915562 (+   196us) write_op.cc:270] Start()
0709 13:04:18.915722 (+   160us) write_op.cc:276] Timestamp: P: 1783602258915488 usec, L: 0
0709 13:04:18.915766 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 13:04:18.916603 (+   837us) log.cc:844] Serialized 13524 byte log entry
0709 13:04:20.675156 (+1758553us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:04:20.917034 (+241878us) op_driver.cc:464] REPLICATION: finished
0709 13:04:21.093595 (+176561us) write_op.cc:301] APPLY: starting
0709 13:04:21.102904 (+  9309us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:21.638884 (+535980us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:21.638934 (+    50us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:22.432446 (+793512us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:22.433358 (+   912us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=70,key_file_lookups=70,delta_file_lookups=70,mrs_lookups=0
0709 13:04:22.433499 (+   141us) write_op.cc:312] APPLY: finished
0709 13:04:22.434433 (+   934us) log.cc:844] Serialized 300 byte log entry
0709 13:04:22.580206 (+145773us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:22.580568 (+   362us) write_op.cc:454] Released schema lock
0709 13:04:22.580680 (+   112us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:22.581226 (+   546us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:22.581269 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":321146,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":136099,"apply.run_cpu_time_us":110443,"apply.run_wall_time_us":1499780,"cfile_cache_hit":142,"cfile_cache_hit_bytes":150497,"num_ops":35,"prepare.queue_time_us":319155,"prepare.run_cpu_time_us":18421,"prepare.run_wall_time_us":670781,"raft.queue_time_us":95033,"raft.run_cpu_time_us":55439,"raft.run_wall_time_us":77226,"replication_time_us":2001073,"spinlock_wait_cycles":26488576,"thread_start_us":829483,"threads_started":5,"wal-append.queue_time_us":1208624,"wal-append.run_cpu_time_us":18859,"wal-append.run_wall_time_us":1974228}]]}
W20260709 13:04:23.511451  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 75214.1us
I20260709 13:04:23.408896 14560 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:04:23.515465  8298 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:23.512254 14662 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:23.439229 14822 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.077s
I20260709 13:04:23.517133 14708 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:23.513201 14824 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:23.439534 14954 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:23.531322  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:23.514513 14956 log.cc:927] Time spent T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:04:23.444079 14950 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:23.546703  8324 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 18. Current term is 19. Ops: []
W20260709 13:04:23.547997  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 96765.2us
I20260709 13:04:23.564977  8297 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:23.549999  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 143818us
I20260709 13:04:23.444859 14876 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:23.552875  8223 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:23.556315  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.452133 14758 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:23.570274 14758 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:23.571666  8223 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:23.572496 14758 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:23.532680  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:23.573596 14902 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:23.574242 14902 consensus_queue.cc:1059] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:04:23.575028 14980 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:04:23.575449 14758 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.576736 14570 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:23.576057 14758 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:23.577306 14570 raft_consensus.cc:697] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 18 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:23.578356 14570 consensus_queue.cc:237] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:23.579016 14758 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:23.555696 14708 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:23.569176 14876 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:23.580891  8325 raft_consensus.cc:1240] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:23.581840 14876 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:23.582544 14876 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:04:23.584120 14889 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:23.584800 14889 raft_consensus.cc:697] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:04:23.586001 14977 log.cc:927] Time spent T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
I20260709 13:04:23.585886 14889 consensus_queue.cc:237] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:23.521539  8151 catalog_manager.cc:5697] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:23.589643 14622 consensus_queue.cc:260] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:23.594319 14855 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:23.599763 14935 raft_consensus.cc:3055] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:23.600306 14935 raft_consensus.cc:740] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:23.601192 14935 consensus_queue.cc:260] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:23.601747  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 164195us
I20260709 13:04:23.601840 14753 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:23.602574 14826 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:23.440347 14786 consensus_queue.cc:1059] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.114s
W20260709 13:04:23.607148  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.596220  8151 catalog_manager.cc:5697] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 8 to 11, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 11 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:23.559831  8248 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
W20260709 13:04:23.521343 14943 consensus_meta.cc:287] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.511s	user 0.007s	sys 0.005s
I20260709 13:04:23.584131 14876 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 13:04:23.556990 14664 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:23.616923  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:23.522131 14825 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:23.618418  8400 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:23.620263  8403 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:23.616977  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:23.625967  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 168904us
I20260709 13:04:23.632161  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:23.452853 14960 consensus_queue.cc:1048] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.113s
W20260709 13:04:23.604256 14976 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:04:23.616225  8151 catalog_manager.cc:5697] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:23.632954  8331 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:23.562978 14805 raft_consensus.cc:3055] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:23.616200 14857 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:23.647645  8369 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:23.649868  8254 raft_consensus.cc:2410] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:04:23.641536  8250 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:04:23.512598 14893 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:23.643554  8972 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:23.645614  8332 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 16.
I20260709 13:04:23.579079 14954 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:23.619256 14664 raft_consensus.cc:697] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:23.652732  8400 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:23.617193 14753 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:23.649204  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:23.651140  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:23.508322 14940 consensus_queue.cc:260] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:23.665930  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:23.666850  8250 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
I20260709 13:04:23.668876  8400 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:23.620630 14560 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:04:23.664475  8297 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:23.643638 14805 raft_consensus.cc:740] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:23.673291  8297 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:23.672145  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:23.680068  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.674961 14708 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:23.675081  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:23.660131 14958 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:23.658241 14981 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:23.678572  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:23.687948  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:23.704622  8331 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 24.
I20260709 13:04:23.705987  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:23.688748 14297 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:23.563807 14986 consensus_queue.cc:1059] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.107s
I20260709 13:04:23.680187 14856 raft_consensus.cc:2764] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:04:23.686335 14708 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:23.669898 14857 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.681324 14987 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:04:23.682512 14753 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:23.729712  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:23.711396 14990 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:23.740985  8250 raft_consensus.cc:2393] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 18: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:23.736363  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:23.714852 14756 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:23.737322  8369 leader_election.cc:400] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15.
I20260709 13:04:23.746562  8400 raft_consensus.cc:1275] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:23.721500 14708 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:23.751932  8369 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:23.719770 14857 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:23.704802 14786 consensus_queue.cc:1059] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
W20260709 13:04:23.756291  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 112823us
I20260709 13:04:23.755802  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:23.762046  8972 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:23.748795 14990 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:23.712267 14958 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:23.763082  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:23.806097  8250 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:04:23.764402 14856 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:04:23.769348  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:23.769912 14857 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:23.762997 14758 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:23.755507 14753 raft_consensus.cc:697] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:23.710860 14664 consensus_queue.cc:237] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:23.815078 14954 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.819222 14954 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:23.821108 14996 log.cc:927] Time spent T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:04:23.822029 14857 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.822855 14857 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:23.823557  8257 raft_consensus.cc:1240] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:23.826534 14857 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:23.827479  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:23.828177  8405 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:04:23.833151  8223 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:23.834590  8223 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:23.835963 14857 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:23.836719 14857 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:23.842617  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 80657.7us
W20260709 13:04:23.845103  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.847061 14871 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:23.847626 14871 raft_consensus.cc:697] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 15 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:23.848456 14871 consensus_queue.cc:237] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:23.853868  8327 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:23.855111 14960 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.855679 14960 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:23.805962 14990 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:23.811714  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:23.862741  8250 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:04:23.759636 14989 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:04:23.853188 14932 consensus_meta.cc:287] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.606s	user 0.006s	sys 0.003s
I20260709 13:04:23.774761  8151 catalog_manager.cc:5697] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 10 to 11, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 11 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:23.859458  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:23.886607  9007 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:04:23.860731  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:23.863034  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:23.888648  9006 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:04:23.865588  9003 raft_consensus.cc:1240] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 13:04:23.776743 14979 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 18. Current term is 19.
I20260709 13:04:23.829111  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:23.743187 14805 consensus_queue.cc:260] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:23.945973  8405 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:23.977916  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:23.978539  8249 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:23.945590 14960 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:23.943248  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:23.813145 14758 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:23.945050  8972 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.800s (SENT) [suppressed 3 similar messages]
W20260709 13:04:23.983142 14977 log.cc:927] Time spent T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:04:23.946821 14932 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:23.943861 14855 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.988842 14954 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:24.005648 14758 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:23.977167 14822 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:24.024312 14951 consensus_meta.cc:287] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.602s	user 0.002s	sys 0.006s
I20260709 13:04:24.004413 14910 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:23.938323  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.056231  8251 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:24.090219  8326 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 13:04:24.099470  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.086817 14822 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:24.066802  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.992542 14893 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:24.069835  8404 raft_consensus.cc:1240] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:23.947618 14950 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:24.116700  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:24.117084  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 3201) took 1315 ms. Trace:
I20260709 13:04:24.117826  9006 raft_consensus.cc:2393] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 19: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
W20260709 13:04:24.118729  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 46047.9us
W20260709 13:04:24.119462  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:24.120112 15013 log.cc:927] Time spent T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:04:24.118052  9009 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 24.
I20260709 13:04:24.121567  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:24.122575  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:24.124876 14880 consensus_queue.cc:1059] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:24.121501  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:24.126515  9007 raft_consensus.cc:2393] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 16: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:24.128337  8249 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
W20260709 13:04:24.065644  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 120621us
W20260709 13:04:24.131713  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.102824  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:24.110134  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.117591  8326 rpcz_store.cc:276] 0709 13:04:22.802002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.802682 (+   680us) service_pool.cc:224] Handling call
0709 13:04:22.803292 (+   610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:24.090768 (+1287476us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.116753 (+ 25985us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b540020ec54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:24.117015 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2958744832}
I20260709 13:04:24.136766  8372 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:24.068507 14855 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:24.142007 14297 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.025380 14753 consensus_queue.cc:237] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:24.142554 14297 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:24.151754  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 49070.6us
I20260709 13:04:24.152891  8369 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:24.143381  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:24.154055  8249 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:24.161057 14971 raft_consensus.cc:3055] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 24 LEADER]: Stepping down as leader of term 24
W20260709 13:04:24.161978 15003 log.cc:927] Time spent T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:04:24.162073 14971 raft_consensus.cc:740] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:24.163455 14971 consensus_queue.cc:260] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:24.126914 14989 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:24.161033 15007 consensus_queue.cc:1059] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.005s
I20260709 13:04:24.165310 14971 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:04:24.168387 14662 consensus_meta.cc:287] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.617s	user 0.008s	sys 0.000s
I20260709 13:04:24.141862 14876 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.177902 14876 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:24.180141 14876 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:24.182446  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:24.183615  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:24.168883 14662 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:24.164960 14989 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:24.092144 14910 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:24.199514 14933 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:04:24.233632  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 82385.7us
I20260709 13:04:24.200598 14875 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:24.142777 14951 raft_consensus.cc:3055] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:24.202769 14971 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.249742 15009 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.249888 14971 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:24.250373 15009 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:24.252198 14971 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:24.253165 15009 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:24.234378 14933 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.207753 14855 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:24.208230 15013 log.cc:927] Time spent T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:04:24.208765 14904 consensus_meta.cc:287] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.867s	user 0.012s	sys 0.000s
I20260709 13:04:24.119484 14805 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:24.217625  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2913) took 1101 ms. Trace:
I20260709 13:04:24.318516  8324 rpcz_store.cc:276] 0709 13:04:23.115804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.116005 (+   201us) service_pool.cc:224] Handling call
0709 13:04:23.528875 (+412870us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:23.563122 (+ 34247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.217383 (+654261us) spinlock_profiling.cc:243] Waited 324 ms on lock 0x7b540037c650. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:24.217564 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":746246016}
I20260709 13:04:24.219961  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:24.218737  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.321857  8222 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:24.323058  8222 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:24.222591  8223 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:24.223583  8251 raft_consensus.cc:1240] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:04:24.325031  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2675) took 1043 ms. Trace:
I20260709 13:04:24.325519  8251 rpcz_store.cc:276] 0709 13:04:23.281475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.197012 (+915537us) service_pool.cc:224] Handling call
0709 13:04:24.223281 (+ 26269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:24.324633 (+101352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.324956 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:24.235460 14297 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:24.237008  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:24.327260  8406 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:04:24.238601  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:04:24.216516 14868 consensus_meta.cc:287] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.918s	user 0.012s	sys 0.000s
W20260709 13:04:24.254947  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 102182us
I20260709 13:04:24.329283  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:24.204418 14774 consensus_meta.cc:287] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 1.036s	user 0.013s	sys 0.001s
I20260709 13:04:24.258231  8248 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:24.160845 14932 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:24.265095  8249 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
W20260709 13:04:24.330669  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 108186us
I20260709 13:04:24.245208 14875 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:24.338456  8324 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15. Ops: []
W20260709 13:04:24.345504  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 225375us
I20260709 13:04:24.341835  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:24.345600  8256 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:24.245108 14951 raft_consensus.cc:740] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:24.350785  9006 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 13:04:24.264480 15008 raft_consensus.cc:3055] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:24.264075 14933 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:24.264361 15007 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.281862 14981 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:24.366852  8249 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:04:24.367409  8249 pending_rounds.cc:85] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 11
I20260709 13:04:24.367641  8249 pending_rounds.cc:107] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:04:24.367879  8249 raft_consensus.cc:2889] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:24.238516 14708 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:24.342634 14961 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:24.342365 14874 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:24.348364 14971 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:24.383082  8222 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:24.385557 14971 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.386628 14971 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:24.389168 14882 raft_consensus.cc:3055] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:24.389750 14882 raft_consensus.cc:740] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:24.390897 14882 consensus_queue.cc:260] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:24.392289 14882 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:24.393409  8249 raft_consensus.cc:1240] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 18 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:24.360069 15008 raft_consensus.cc:740] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:24.398379 15008 consensus_queue.cc:260] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:24.399963 15008 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:24.407521  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:24.408250  8253 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 24.
I20260709 13:04:24.410557 14910 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:24.414543  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2915) took 1276 ms. Trace:
I20260709 13:04:24.415006  8325 rpcz_store.cc:276] 0709 13:04:23.138042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.141813 (+  3771us) service_pool.cc:224] Handling call
0709 13:04:23.142093 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:23.581340 (+439247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.414312 (+832972us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b540014cb54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:24.414483 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1008750976}
I20260709 13:04:24.366561 14756 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:24.416308 14756 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:24.418509 14756 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:24.420182  8254 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 22. Current term is 23. Ops: [22.11-22.11]
I20260709 13:04:24.420544 14708 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:24.355574 14904 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.409271  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:24.421137  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2677) took 1118 ms. Trace:
I20260709 13:04:24.423983  8402 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:24.427479  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:24.429265  8254 rpcz_store.cc:276] 0709 13:04:23.303077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.419420 (+1116343us) service_pool.cc:224] Handling call
0709 13:04:24.419874 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:24.420795 (+   921us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.421065 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:24.431267  8399 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:24.347023 14774 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:24.371196 14981 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.414582  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:24.421197  8403 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:24.439524  8401 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:24.370910 14985 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:24.382086 14874 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.444422 14874 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:24.447602  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 1043) took 1307 ms. Trace:
I20260709 13:04:24.448228  8328 rpcz_store.cc:276] 0709 13:04:23.140278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.142764 (+  2486us) service_pool.cc:224] Handling call
0709 13:04:23.143101 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:23.378872 (+235771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.447150 (+1068278us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b5400365454. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:24.447511 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":521190784}
I20260709 13:04:24.449631 14981 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:24.452461 14774 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:24.378916 15007 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:24.453118 14774 consensus_queue.cc:1059] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:24.443652 14985 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:24.455034 15007 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:24.379544 14961 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.455919 14985 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:24.456182 14961 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:24.421270 14708 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:24.423273 14951 consensus_queue.cc:260] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:24.458671 14951 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:24.433801  8253 raft_consensus.cc:2410] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 26 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 13:04:24.460523 14990 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:24.471637 14990 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:24.423012 14758 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:24.472179 14875 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:24.455933 15007 raft_consensus.cc:3055] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:24.474923  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:24.475716  8249 raft_consensus.cc:2410] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:04:24.423169 14887 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:24.433521  8222 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:04:24.494011  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 60573.2us
W20260709 13:04:24.497960  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.473829 15007 raft_consensus.cc:740] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:04:24.512595  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:24.513983  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.530658  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 2921) took 1277 ms. Trace:
I20260709 13:04:24.531280  8323 rpcz_store.cc:276] 0709 13:04:23.253417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.432673 (+179256us) service_pool.cc:224] Handling call
0709 13:04:23.442919 (+ 10246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:23.528993 (+ 86074us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:23.529040 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:23.529098 (+    58us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:23.529145 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.530315 (+1001170us) spinlock_profiling.cc:243] Waited 85.8 ms on lock 0x7b54003f0cd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:24.530397 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:24.530579 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":197374720}
I20260709 13:04:24.527243 14758 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:24.477800 15002 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.487145 14887 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.486349 14987 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:24.476994 14475 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 13:04:24.432396 14904 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:24.513022 14898 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:04:24.609122  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 249425us
I20260709 13:04:24.562966 14887 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:24.610868  8297 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:24.563279 14987 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.594488 14898 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:24.607038 14991 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:04:24.475412 14971 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:24.618013  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:24.602330 14753 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:24.562361 15002 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:24.624452  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:24.629556  8254 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 21: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:24.633450  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:24.626564  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:24.590130 14874 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:24.616678 14987 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:24.637235  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:24.645917  8399 raft_consensus.cc:1275] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:04:24.633495 14875 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:24.623270 14898 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:24.640561 14904 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:24.641533 15019 log.cc:927] Time spent T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:04:24.619719 14664 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:24.613821 14708 consensus_queue.cc:1059] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:04:24.636847 14753 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:24.642598 15016 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.644404  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:24.645010  8397 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 15. (term mismatch)
I20260709 13:04:24.687117  8397 pending_rounds.cc:85] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 14
I20260709 13:04:24.687177  8257 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:24.687521  8397 pending_rounds.cc:107] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 17.15
I20260709 13:04:24.687803  8397 raft_consensus.cc:2889] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:24.648105  8254 raft_consensus.cc:2393] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 15: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:04:24.651162  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.694837  8398 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:04:24.695499 14753 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:24.652881  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:24.659584 15008 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:24.698457 15008 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:24.700555 15008 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:24.689365  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:24.701855 14664 raft_consensus.cc:697] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:24.702226  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:24.703305  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:24.704224  8254 raft_consensus.cc:2393] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 21: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:24.703253 14664 consensus_queue.cc:237] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:24.704985  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:24.706040 15017 log.cc:927] Time spent T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:04:24.658512  8369 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:24.706930  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 48488.6us
I20260709 13:04:24.709203  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:24.709570 14889 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.710248 14796 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:24.710536 15002 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:24.711329 14825 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:24.711884 14825 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:24.713581 15008 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:24.713799  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:24.714418  8332 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:04:24.714869  8326 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:24.715471 14758 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.715689 14951 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.715996 14758 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:24.716245 14951 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:24.718225  8327 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 24.
I20260709 13:04:24.718415 14758 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:24.718812 14951 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:24.665688  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:24.719434 15026 log.cc:927] Time spent T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:04:24.720557  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:24.721035 15016 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:24.722074  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:24.723134  8253 raft_consensus.cc:2410] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:04:24.723383 15016 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:24.659222  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.726437 14958 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:24.727241  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:24.641831 14971 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:24.727876  8325 raft_consensus.cc:2376] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 16. Re-sending same reply.
I20260709 13:04:24.680974  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:24.728709  8400 raft_consensus.cc:2393] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 20: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:24.710057 14889 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:24.662459  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:24.729739  8405 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
W20260709 13:04:24.730473  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.611481 15007 consensus_queue.cc:260] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:24.730969 14889 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:24.731313 15007 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:24.674031  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:24.734683  8404 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
I20260709 13:04:24.724149  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:24.738824  8253 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:04:24.758765  8325 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:04:24.739218 15000 consensus_queue.cc:1059] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:04:24.739943 14825 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:24.741344  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:24.762348  8329 raft_consensus.cc:2410] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:04:24.742057  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:24.742291  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:24.763448  8326 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 25.
I20260709 13:04:24.764019  8325 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 13:04:24.764660 15025 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:24.743453  8295 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:24.714196 15008 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:24.754920  8251 raft_consensus.cc:1275] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:24.741210 14935 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:24.772657  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:24.730700  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:24.774569  8251 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:24.627440 14933 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:24.771472 14825 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:24.764829 14822 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:24.718822 14874 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:24.767968 15016 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:24.788761  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 243641us
W20260709 13:04:24.822361  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 55322.5us
I20260709 13:04:24.825619  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:24.826316  8405 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:24.770635  8295 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:24.831678  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 61128us
W20260709 13:04:24.832080  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:24.789896 15031 log.cc:927] Time spent T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:04:24.832408  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:24.833745  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:24.759771  8253 raft_consensus.cc:1240] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:24.835539  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:24.840183  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:24.840634  9005 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:24.836141 15000 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:24.836166  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:24.837119  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:24.837224  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:24.845304  9000 raft_consensus.cc:2410] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:04:24.837854  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:24.845968  8398 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:24.838562  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:24.846805  8400 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:24.847150  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.838812  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:24.851339  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:24.855708  9005 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 25.
I20260709 13:04:24.844403  8404 raft_consensus.cc:2410] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:04:24.819643 14904 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.848424  8297 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:24.861749  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:24.866114  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.830411 14825 raft_consensus.cc:697] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:24.769567 15025 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:24.883805  8372 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:24.860848  9008 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:04:24.871157  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:24.884397  8220 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:24.844257 14993 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:24.839023  8295 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:24.862641  8222 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:24.879227  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:24.796221 14753 consensus_queue.cc:1048] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 13:04:24.841476 14954 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:24.886690  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:24.894834  9001 raft_consensus.cc:2393] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 21: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:24.887403  8372 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:24.865216  8297 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:24.875357  8400 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:24.892467  8257 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 22.
I20260709 13:04:24.857559 15024 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.899125  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:24.830282 15016 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:24.901136  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 62202.3us
I20260709 13:04:24.902145  8295 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:24.901738  8297 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:24.900471  8372 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:24.877017 14856 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:24.862389 14875 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:04:24.910224  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.903298 14889 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:24.926851  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:24.858167 14958 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:04:24.933300  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:24.896821 14570 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:24.936355  8295 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:24.935145  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:24.920068  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:24.943856  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:24.944267  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:24.945176  8325 raft_consensus.cc:2410] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 24 because replica has last-logged OpId of term: 22 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 10.
I20260709 13:04:24.947057  8295 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:24.946029  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 57933.1us
I20260709 13:04:24.949622  8220 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:24.903648 14758 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:24.966295  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:24.884027 14904 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:24.968552  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:24.968956  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 125190us
I20260709 13:04:24.970042  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:24.884218 14981 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:24.905376 15035 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:24.900471 14993 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:24.911106 14985 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:24.908330 14951 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:25.003125  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:24.995358  8330 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:24.915278 14954 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:24.911792 14856 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.010160  8329 raft_consensus.cc:2393] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 23: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:24.858901 15000 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:24.977802  8974 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:25.022246  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 44577.5us
I20260709 13:04:24.896207 15036 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.083s
I20260709 13:04:24.943123 14756 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:04:25.042721  8399 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:04:24.986733 14662 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:04:24.759866 14796 consensus_queue.cc:1175] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8c42259d7c4b463a9f485f42c320c81b" responder_term: 19 status { last_received { term: 18 index: 12 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 18. Current term is 19. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:04:24.937482 14875 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:25.023908 14874 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:04:25.062686  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:25.050432  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 150059us
I20260709 13:04:25.002710 14889 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:25.005122 14708 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Leader election won for term 25
W20260709 13:04:25.053762  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 85834.5us
W20260709 13:04:25.055827  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.024192 14954 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.982419 15035 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.016513 14951 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.026005 14856 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:24.903513 15024 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:25.047017 14985 raft_consensus.cc:697] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 22 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:25.090687 14622 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:25.091017 15053 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:25.091665 15053 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:25.095678 15000 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:25.098500 14904 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:25.098937  8406 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:04:25.067878 15008 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:25.066488 14874 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.061648 14662 raft_consensus.cc:697] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:25.065326 14805 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:25.078531 14954 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:25.108834  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:25.085925 15035 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:25.112391  8404 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:04:25.029913 14987 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:25.109833  8331 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:04:25.128253  8331 raft_consensus.cc:606] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
W20260709 13:04:25.118577  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.083685 14951 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:25.086803 15034 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.089248 14933 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:25.128672  8331 raft_consensus.cc:614] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:04:25.128531  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 181665us
I20260709 13:04:25.068783 14868 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:04:25.101997 15023 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.110908 14874 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:25.106575 14935 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:25.136240 14783 raft_consensus.cc:3055] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:25.148190  8295 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:25.101996 14887 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:25.054500 14825 consensus_queue.cc:237] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:25.119911 15001 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:25.133085 14664 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:25.150254  8220 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:25.150405  9009 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 24. Current term is 25. Ops: []
I20260709 13:04:25.123616 14958 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.230821  8254 raft_consensus.cc:2393] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 15: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:25.197507  8295 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:04:25.216903  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 193809us
W20260709 13:04:25.228268  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 78102.3us
I20260709 13:04:25.143615 14783 raft_consensus.cc:740] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:25.196929 15033 log.cc:927] Time spent T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.295s	user 0.001s	sys 0.000s
I20260709 13:04:25.232991  8220 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:25.144158 15053 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:25.235929  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 84806.4us
I20260709 13:04:25.196648 14753 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:25.232623  8974 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:25.214279 14708 raft_consensus.cc:697] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 25 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:25.238790  8220 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:25.241991  8295 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:04:25.232841 15038 log.cc:927] Time spent T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
W20260709 13:04:25.234200 15027 log.cc:927] Time spent T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
I20260709 13:04:25.149581 14935 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:25.244328  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:25.142226 15034 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:25.228785 15025 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:25.247718  8295 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:25.230928 14758 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:25.250207  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:25.253369  8400 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 22.
I20260709 13:04:25.152377 14662 consensus_queue.cc:237] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:25.245522 15043 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
I20260709 13:04:25.235980 15016 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:25.256057  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:25.259135  9003 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:25.241299 14904 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:25.244622 15051 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:04:25.255784 14898 consensus_meta.cc:287] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.577s	user 0.010s	sys 0.000s
I20260709 13:04:25.146157 15023 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:25.235662 14958 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:25.256688 15060 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:25.273746  8298 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:25.259372 15036 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:25.271631  8297 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:25.228600 14664 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:25.259510 14981 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:25.233340 14985 consensus_queue.cc:237] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:25.281497  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:25.265791 15057 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:25.288100  8400 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:04:25.270514 14904 raft_consensus.cc:697] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:25.275806 15016 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:25.291802  9007 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 15: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:25.366573 15024 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:25.283142 14875 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:25.273232 14783 consensus_queue.cc:260] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:25.371300 15067 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.371419  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:25.392457  9003 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:04:25.271190 15059 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:25.260803 14758 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:25.297981 15048 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
W20260709 13:04:25.375469  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 512877us
I20260709 13:04:25.274621 14898 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:25.419829  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 168321us
I20260709 13:04:25.289696 15060 raft_consensus.cc:697] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:25.290937 15062 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:25.436383  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 164856us
W20260709 13:04:25.438611  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 190929us
I20260709 13:04:25.430846  8222 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:25.374513 14933 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:25.375171 15051 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.388823 14756 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:25.440768 15059 raft_consensus.cc:697] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:04:25.454452  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.451936  8295 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:25.470813  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:25.482703  8399 raft_consensus.cc:2393] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 17: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:25.484162  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:25.485972  8328 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:04:25.490804  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:04:25.499611  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 51912.3us
I20260709 13:04:25.452148 14987 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 25 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:25.507643 15071 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:25.437048 15001 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.479698 14989 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:25.376725 14708 consensus_queue.cc:237] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:25.499604 15008 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.480090 14933 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:25.423050 15067 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:25.504938 14756 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:25.503659 14887 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:25.515230 15071 raft_consensus.cc:697] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:25.554522  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 275819us
W20260709 13:04:25.444975 14882 consensus_meta.cc:287] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 1.052s	user 0.021s	sys 0.000s
W20260709 13:04:25.561224  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.568351 14887 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:25.468782 15062 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:25.512919 14875 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.568995 14887 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:25.563345 14756 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:25.581136  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 129335us
I20260709 13:04:25.583025  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:25.589025  8405 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:04:25.592047  8405 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:04:25.595227  8974 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:25.554965 15008 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:25.584759  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3411) took 1159 ms. Trace:
I20260709 13:04:25.601672  8397 rpcz_store.cc:276] 0709 13:04:24.425233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.425387 (+   154us) service_pool.cc:224] Handling call
0709 13:04:24.425732 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:24.688343 (+262611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.584499 (+896156us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b54003a20d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:25.584697 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":503368576}
I20260709 13:04:25.564531 14935 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:25.605453 15069 log.cc:927] Time spent T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:04:25.606398 14887 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:25.607033 14898 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:25.512317 15001 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:25.609223 15001 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:25.560384 15060 consensus_queue.cc:237] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:25.529289 15070 consensus_queue.cc:1059] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 13:04:25.587126  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 1132) took 2374 ms. Trace:
I20260709 13:04:25.494279 15051 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:25.587144  8399 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
W20260709 13:04:25.574991 14961 consensus_meta.cc:287] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.118s	user 0.012s	sys 0.012s
I20260709 13:04:25.600198  8295 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:25.603211  8328 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7c15354a2a84f868942919f011af448"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "8c42259d7c4b463a9f485f42c320c81b"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:04:25.570202 15071 consensus_queue.cc:237] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:25.586639  8256 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2676) took 2299 ms (client timeout 1876 ms). Trace:
I20260709 13:04:25.574554 15077 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:25.580106 15034 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:25.586663 14958 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:25.587347 14756 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:25.615319  8401 rpcz_store.cc:276] 0709 13:04:23.206095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.259759 (+ 53664us) service_pool.cc:224] Handling call
0709 13:04:23.260099 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:24.441905 (+1181806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.580416 (+1138511us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54001e7654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:25.580686 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2711957888}
I20260709 13:04:25.626943  8295 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:25.617137  8256 rpcz_store.cc:269] 0709 13:04:23.302586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.393802 (+ 91216us) service_pool.cc:224] Handling call
0709 13:04:25.581140 (+2187338us) spinlock_profiling.cc:243] Waited 932 ms on lock 0x7b5400406854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:25.581291 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2144711936}
I20260709 13:04:25.550789 14904 consensus_queue.cc:237] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:25.617048  8328 raft_consensus.cc:606] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Received request to transfer leadership to TS 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:25.630745  8328 raft_consensus.cc:614] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:25.585901 14875 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:25.637409 15081 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.638337 15081 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:25.634465 15001 consensus_queue.cc:1059] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.005s
I20260709 13:04:25.640239 15001 raft_consensus.cc:3055] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:25.640699 15001 raft_consensus.cc:740] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:25.645682  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 3401) took 2039 ms. Trace:
I20260709 13:04:25.646101  8402 rpcz_store.cc:276] 0709 13:04:23.606452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.606586 (+   134us) service_pool.cc:224] Handling call
0709 13:04:24.341631 (+735045us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b5800170798. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:04:25.645521 (+1303890us) spinlock_profiling.cc:243] Waited 77.6 ms on lock 0x7b540018fe54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:25.645622 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182011520}
I20260709 13:04:25.507659 15059 consensus_queue.cc:237] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:25.622751 14664 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:25.642606 15081 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:25.646435 14958 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:25.665087  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:25.669170 15081 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:25.669170  9008 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 25: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:25.621321 14961 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:25.654997 14758 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:25.694396 14887 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.695107 14887 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:25.700919  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 1132) took 1854 ms. Trace:
I20260709 13:04:25.701520  8248 rpcz_store.cc:276] 0709 13:04:23.846413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.332931 (+486518us) service_pool.cc:224] Handling call
0709 13:04:24.363926 (+ 30995us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:24.364373 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:24.364412 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:24.364967 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 13:04:25.654841 (+1289874us) spinlock_profiling.cc:243] Waited 32 ms on lock 0x7b18004819c0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e0d1 00007f8828779a0f
0709 13:04:25.697679 (+ 42838us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:25.697720 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.697787 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:25.697829 (+    42us) raft_consensus.cc:1685] finished
0709 13:04:25.697839 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:25.700836 (+  2997us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":843,"raft.queue_time_us":2196,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":8692,"spinlock_wait_cycles":73541504,"thread_start_us":42639,"threads_started":1,"wal-append.queue_time_us":1322133}
W20260709 13:04:25.709352  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 122369us
I20260709 13:04:25.712344  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:25.698352 14943 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.719750 14887 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:25.738394 15023 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:25.720418 14898 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:25.723800 15080 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.724886 15067 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:25.716434 15072 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.699219 14985 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.725696 14933 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:25.727124 15046 log.cc:927] Time spent T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.654s	user 0.001s	sys 0.007s
I20260709 13:04:25.769323  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:25.770179  9003 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
W20260709 13:04:25.707062 15068 log.cc:927] Time spent T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.306s	user 0.000s	sys 0.001s
I20260709 13:04:25.699307 15082 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.699071 14882 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:25.691597 15081 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.026s
I20260709 13:04:25.745565  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:25.654901 15001 consensus_queue.cc:260] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:25.717747 14758 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:25.745442 14951 consensus_meta.cc:287] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.601s	user 0.008s	sys 0.004s
W20260709 13:04:25.772292  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 126487us
W20260709 13:04:25.767669 14753 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.014s	sys 0.028s
W20260709 13:04:25.800863 14753 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.015s	sys 0.028s
I20260709 13:04:25.801226 14753 raft_consensus.cc:697] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:25.801849  8295 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:25.802333 14753 consensus_queue.cc:237] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:25.803645  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:25.804095  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.804186  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:25.768080 14887 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.793566 15008 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:25.783685 15035 consensus_meta.cc:287] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.645s	user 0.008s	sys 0.004s
I20260709 13:04:25.745824 14943 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:25.809886  8972 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:25.767336 15062 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:25.816614  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:25.823439  8972 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:25.794775 14986 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.060s	user 0.016s	sys 0.013s
I20260709 13:04:25.826577  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:25.749761 15080 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:25.827008  8972 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:25.826579  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:25.828624  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:25.829912  8250 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 20: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:25.830935 15093 consensus_queue.cc:1059] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:25.831918 15093 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:25.832206  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:25.832270 15093 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:25.832548 15086 raft_consensus.cc:3055] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:25.769979 14985 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:25.834537  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:25.821753 15001 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:25.838775  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:25.766800 15072 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:25.826442 14986 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.097s	user 0.017s	sys 0.013s
I20260709 13:04:25.837332 15093 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:25.870625 15085 log.cc:927] Time spent T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 13:04:25.870879 14985 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:25.872162 15083 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.840066  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:25.872831 15083 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:25.846573  8256 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 23.
I20260709 13:04:25.826321 14987 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:25.874856 14987 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.875351 14987 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:25.876456 15070 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.877058 15070 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:25.878990  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:25.879626 15070 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:25.829772 15034 raft_consensus.cc:2764] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Leader election decision vote started in defunct term 24: won
I20260709 13:04:25.882536  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:25.875514 15083 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:25.892437  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 1056) took 1185 ms. Trace:
I20260709 13:04:25.894524  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 1142) took 1052 ms. Trace:
I20260709 13:04:25.895174  8404 rpcz_store.cc:276] 0709 13:04:24.842237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.866157 (+ 23920us) service_pool.cc:224] Handling call
0709 13:04:24.866440 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:25.119896 (+253456us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.894083 (+774187us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b54002689d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:25.894349 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":564626176}
I20260709 13:04:25.902558 15051 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:25.786180 15082 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:25.841974 15086 raft_consensus.cc:740] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:25.848531 15057 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:25.806238 14758 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:25.848340 14875 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:25.812229 14951 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:25.820073 15035 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:25.843918 14958 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 13:04:25.904458 14979 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:25.909883 14822 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:25.924376  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.925292  8326 raft_consensus.cc:2393] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 21: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:04:25.925494  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.817600 14887 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:25.925912  8248 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:25.893087  8323 rpcz_store.cc:276] 0709 13:04:24.707084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.709593 (+  2509us) service_pool.cc:224] Handling call
0709 13:04:24.709913 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:24.731725 (+ 21812us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:24.731757 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:24.732214 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 13:04:24.829535 (+ 97321us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:04:24.829583 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.891789 (+1062206us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b54000906d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:25.891896 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:25.891970 (+    74us) raft_consensus.cc:1685] finished
0709 13:04:25.891991 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:25.892346 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1228,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"spinlock_wait_cycles":49403264,"thread_start_us":96837,"threads_started":1,"wal-append.queue_time_us":162036}
W20260709 13:04:25.969070  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.912580 15034 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:04:25.925896 14979 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:25.971752 15092 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:04:25.914234 15057 raft_consensus.cc:697] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:25.918090 14961 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:25.908842 15039 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
W20260709 13:04:25.993135  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 156507us
I20260709 13:04:25.973240 14875 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:25.973208 15051 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.973011 14870 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:25.977171 15087 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:25.993745 14960 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:04:25.999256 14954 consensus_meta.cc:287] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.867s	user 0.006s	sys 0.007s
I20260709 13:04:25.951159 14822 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:26.016489  8256 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:04:25.994890 14961 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:26.034449 15072 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:26.009680 14979 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:26.036446  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:26.039634  8256 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 22: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:26.042033 14951 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:26.042374 15035 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:26.044378 15057 consensus_queue.cc:237] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:26.047252  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:26.007710 14875 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:26.006647 15051 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:26.009603 14870 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:26.047184 14887 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:26.038386 14943 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:26.026274 14960 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:26.026945 14954 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:26.031132 14961 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:26.002499 15086 consensus_queue.cc:260] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.14, Last appended by leader: 14, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:26.061801 15082 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:26.068239 14993 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:26.070261 15036 consensus_meta.cc:287] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.786s	user 0.012s	sys 0.000s
I20260709 13:04:26.070734 15036 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:26.071134 14875 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:26.072124 15084 log.cc:927] Time spent T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.232s	user 0.002s	sys 0.000s
I20260709 13:04:26.073073 14870 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:26.074780 14987 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:26.075809 14874 consensus_meta.cc:287] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.926s	user 0.013s	sys 0.000s
I20260709 13:04:26.075501 14756 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:26.077015 14822 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:26.077193  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 3224) took 1158 ms. Trace:
I20260709 13:04:26.086138  8326 rpcz_store.cc:276] 0709 13:04:24.919283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.919421 (+   138us) service_pool.cc:224] Handling call
0709 13:04:26.076996 (+1157575us) spinlock_profiling.cc:243] Waited 981 ms on lock 0x7b540011eed4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:26.077132 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2257202944}
W20260709 13:04:26.077777 14856 consensus_meta.cc:287] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.970s	user 0.015s	sys 0.007s
I20260709 13:04:26.081749  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:26.082019  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:26.079912  8324 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 20: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:26.013440 15087 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:26.058657 14758 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:26.098604  8323 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:04:26.092439  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:26.078274 14986 raft_consensus.cc:3055] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:26.071169 15051 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:26.100024  8326 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:04:26.109375  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:26.113576  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:26.113031 15080 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:26.091876 15086 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:26.116074  8251 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:26.114966 14475 consensus_queue.cc:1059] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 13:04:26.118609  9009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 1159) took 1283 ms. Trace:
I20260709 13:04:26.119071  9009 rpcz_store.cc:276] 0709 13:04:24.835180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.835286 (+   106us) service_pool.cc:224] Handling call
0709 13:04:24.835589 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:25.227756 (+392167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.118338 (+890582us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b54005cf8d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:26.118549 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":715207040}
I20260709 13:04:26.124210  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:26.085731 14874 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:26.128383 14369 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:26.129247 15087 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:26.129415 14756 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:26.149648 14369 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:26.150630 15016 consensus_meta.cc:287] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.774s	user 0.011s	sys 0.001s
I20260709 13:04:26.151085 15016 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:26.100833 14856 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:26.152598  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:26.153239  8257 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 16.
W20260709 13:04:26.140364  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 1.16513e+06us
I20260709 13:04:26.140974 15093 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:26.122750 15101 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:26.142582 14805 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.143751  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:26.144263  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:26.144306  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:26.170095  8369 connection.cc:256] RPC call timeout handler was delayed by -1.1397290329999805s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21150000000000002s for a response
I20260709 13:04:26.184224  8253 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 22: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:26.184060  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:26.174468  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:26.185180  8257 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:04:26.185770  9003 raft_consensus.cc:2393] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 19: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:26.107460 14986 raft_consensus.cc:740] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:26.188125  8251 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:04:26.191358  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:26.191661  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:26.191996  8404 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
I20260709 13:04:26.193287  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:26.194520  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:26.194754  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:26.195982  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:26.196502  8397 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:04:26.198920  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:26.199697  8398 raft_consensus.cc:2393] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 21: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:26.200425  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:26.201092  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:26.201387  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:26.202298  8323 raft_consensus.cc:2410] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:04:26.202337  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:26.203150  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:26.203225  8326 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 25.
I20260709 13:04:26.205191  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:26.211236  8332 raft_consensus.cc:2393] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 15: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:26.205775  8404 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:26.216617  8397 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 24.
I20260709 13:04:26.171329 14758 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:26.153092 15006 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:26.222375  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:26.183293 15101 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.225579  8402 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 21.
I20260709 13:04:26.206770 14708 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.212168 14951 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:26.206009 14932 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:26.221695 14992 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.226826 14369 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:26.235723 15093 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:26.235924 14960 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:26.236230 14932 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:26.236932 15075 log.cc:927] Time spent T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.365s	user 0.001s	sys 0.000s
I20260709 13:04:26.237717 14954 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:26.227205 14985 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:26.189764 14805 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:26.238569 14987 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:26.240477 14951 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:26.244024 14961 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:26.247495 14951 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:26.247632 14932 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:26.248158 14664 raft_consensus.cc:3055] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:26.227320 14758 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:26.248615 14664 raft_consensus.cc:740] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:26.248812 14805 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:26.234579 14708 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:26.249578 14664 consensus_queue.cc:260] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:26.250219 14783 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:26.250762 14664 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:26.251076 14874 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:26.251209 14708 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:26.229420 15006 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:26.219441 14990 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:26.229250 15062 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:26.229738 15101 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:26.243635 14856 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:26.254145 14990 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:26.253685 15062 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:26.256517 15062 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:26.252061 14985 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:26.258523 15032 log.cc:927] Time spent T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 1.521s	user 0.000s	sys 0.016s
I20260709 13:04:26.259264 14985 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:26.259503 15006 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:26.242338 14992 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:26.262014 14992 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:26.262722 14990 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:26.284726 15101 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:26.359246  8248 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:51220
I20260709 13:04:26.361230  8248 raft_consensus.cc:606] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:04:26.364988  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:04:26.364810  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:26.366389  8250 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:04:26.367211  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:26.368480  8248 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:04:26.369972  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:26.375938  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:26.386703  8251 raft_consensus.cc:2393] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 17: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:26.386540  8297 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:26.387993  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:26.398099  8251 raft_consensus.cc:2393] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 14: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:26.400166  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:26.409852  8251 raft_consensus.cc:2393] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 23: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:26.415417  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:26.418818  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 1144) took 1146 ms. Trace:
I20260709 13:04:26.427425  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:26.286973 15101 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:26.442708 14990 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.443928 14990 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:26.450452  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2834) took 1346 ms. Trace:
I20260709 13:04:26.250252 14758 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:26.459961 14990 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:26.460598  8398 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:04:26.420749  8400 rpcz_store.cc:276] 0709 13:04:25.272518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.272723 (+   205us) service_pool.cc:224] Handling call
0709 13:04:25.280142 (+  7419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:25.293481 (+ 13339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.418139 (+1124658us) spinlock_profiling.cc:243] Waited 7.63 ms on lock 0x7b540019b9d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:26.418537 (+   398us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17555712}
I20260709 13:04:26.437999  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:26.475845  8251 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:26.476868  8248 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:04:26.474399 14987 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 13:04:26.479810  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:26.480804  8248 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:04:26.469362  8372 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:26.451037  8403 rpcz_store.cc:276] 0709 13:04:25.103677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.103879 (+   202us) service_pool.cc:224] Handling call
0709 13:04:25.104470 (+   591us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:25.126288 (+ 21818us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:25.126341 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:25.127022 (+   681us) log.cc:844] Serialized 26 byte log entry
0709 13:04:25.690992 (+563970us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:25.691052 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.691149 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:26.236389 (+545240us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:04:26.449610 (+213221us) raft_consensus.cc:1685] finished
0709 13:04:26.449664 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.450212 (+   548us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9406,"raft.queue_time_us":11877,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":7413,"spinlock_wait_cycles":32931584,"thread_start_us":476204,"threads_started":1,"wal-append.queue_time_us":724262}
I20260709 13:04:26.486017  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 1139) took 1848 ms. Trace:
I20260709 13:04:26.486696  8406 rpcz_store.cc:276] 0709 13:04:24.637134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.637350 (+   216us) service_pool.cc:224] Handling call
0709 13:04:24.637649 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:25.099442 (+461793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.441284 (+1341842us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b54000fc3d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:26.485746 (+ 44462us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1060057344}
I20260709 13:04:26.494462 14758 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:26.497332  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:26.503576  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:26.497464  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:26.509819  8372 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:26.510095  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:26.514123 14990 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:26.515045 14990 raft_consensus.cc:697] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:26.520924  8372 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:26.530354  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 3225) took 1610 ms. Trace:
I20260709 13:04:26.530926  8332 rpcz_store.cc:276] 0709 13:04:24.919614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.919727 (+   113us) service_pool.cc:224] Handling call
0709 13:04:26.529841 (+1610114us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54003c9954. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:26.530090 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2921293568}
I20260709 13:04:26.534111  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:26.533234  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:26.536052  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:26.537483  8254 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2688) took 1887 ms (client timeout 2077 ms). Trace:
W20260709 13:04:26.538025  8254 rpcz_store.cc:269] 0709 13:04:24.650228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.704757 (+ 54529us) service_pool.cc:224] Handling call
0709 13:04:26.537017 (+1832260us) spinlock_profiling.cc:243] Waited 509 ms on lock 0x7b540033ead4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:26.537269 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1170922240}
I20260709 13:04:26.535987  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:26.534186  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:26.539479  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:26.523882  8257 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 25.
I20260709 13:04:26.505403  8402 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:26.540524  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:26.501711  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:26.551532 14950 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 24 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:26.551717 14992 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.516811 14990 consensus_queue.cc:237] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:26.551743  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:26.546923  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:26.551844  8324 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 3227) took 1626 ms (client timeout 1906 ms). Trace:
I20260709 13:04:26.549278  8249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49220 (request call id 1136) took 2134 ms. Trace:
I20260709 13:04:26.558233  8403 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
I20260709 13:04:26.557977  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:26.554142  8324 rpcz_store.cc:269] 0709 13:04:24.925983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.926535 (+   552us) service_pool.cc:224] Handling call
0709 13:04:26.547312 (+1620777us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540037a354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:26.551666 (+  4354us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2622856960}
I20260709 13:04:26.556559  8249 rpcz_store.cc:276] 0709 13:04:24.402986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.414286 (+ 11300us) service_pool.cc:224] Handling call
0709 13:04:26.537016 (+2122730us) spinlock_profiling.cc:243] Waited 34.2 ms on lock 0x7b54003b1d54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:26.537269 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":78582656}
I20260709 13:04:26.560386  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:26.563671  8254 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:26.552950 14992 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:26.567799  8248 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 11.
I20260709 13:04:26.571780 14805 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.572255 14856 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:26.572844 14856 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.572553 14805 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:26.569378 15062 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:26.575929  9008 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:04:26.578449  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:26.573290 14856 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:26.583365  8249 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:04:26.577898 15062 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:26.565308  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 344238us
I20260709 13:04:26.589864  8250 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:04:26.587250 15062 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:26.595242  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:26.595338  8297 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:26.595538 15059 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:26.596046 15086 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.596138 14805 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:26.596663  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:26.589291  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:26.567070 14950 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:04:26.596990  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:26.600539  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:26.602933  8329 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:26.604038 15086 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:26.600557  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:26.607261  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:26.608454  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:26.610926  8399 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:26.616487  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:26.620458  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:26.621502  9009 raft_consensus.cc:2393] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 17: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:26.621762  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:26.622430  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:26.623208  9007 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 25.
I20260709 13:04:26.623503  8256 raft_consensus.cc:2393] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 17: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:26.627964  8331 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
W20260709 13:04:26.641321  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 46196us
I20260709 13:04:26.654942  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:26.655526  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:26.655942  9008 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:26.656965  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:26.657830 14805 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.658665 14805 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:26.660917 14805 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:26.661914  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:26.662747  8327 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:26.670482 15006 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:26.671051  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:26.671253 15006 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:26.671792  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:26.673694 15093 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:26.674202 15093 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.674633 15093 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:26.678704  8406 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:26.685698  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:26.687203  8402 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
I20260709 13:04:26.689079  8251 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 24.
I20260709 13:04:26.691653  8248 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:04:26.692581  8329 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 25.
I20260709 13:04:26.690886  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2680) took 3283 ms. Trace:
I20260709 13:04:26.693740  8254 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:04:26.679771 15006 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:26.698066  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:26.687849 15086 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:26.701087  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:26.701721  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:26.696240  8297 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:26.704573  8331 raft_consensus.cc:2368] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 23. Current term is 24.
I20260709 13:04:26.705747  8400 raft_consensus.cc:2410] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:04:26.679512 15059 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:26.693176  8252 rpcz_store.cc:276] 0709 13:04:23.305346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.134762 (+2829416us) service_pool.cc:224] Handling call
0709 13:04:26.135154 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.135422 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:04:26.204727 (+ 69305us) log.cc:844] Serialized 14 byte log entry
0709 13:04:26.564811 (+360084us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:26.570146 (+  5335us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:26.582405 (+ 12259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.587588 (+  5183us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.588853 (+  1265us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13958,"prepare.queue_time_us":318481,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":72,"raft.queue_time_us":10198,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":199,"thread_start_us":192792,"threads_started":2,"wal-append.queue_time_us":350372}
I20260709 13:04:26.717829  8295 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:26.718937  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 68817us
I20260709 13:04:26.721750  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:26.722653 15086 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:26.735435  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:26.738175  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:26.738668  8330 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 25: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:26.742239  8297 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:26.744374  8325 raft_consensus.cc:2410] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
W20260709 13:04:26.744728  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 132047us
I20260709 13:04:26.747432 15058 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:26.739128 15086 raft_consensus.cc:697] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 24 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:26.740873 14805 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:26.754421  8331 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:04:26.760149  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:26.756170 15058 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:26.772070 15058 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.752183 14856 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:26.784164  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:26.785743  8331 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:04:26.769877 14805 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:26.783877 15058 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:26.800868  8324 raft_consensus.cc:2393] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 21: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:26.802678  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:26.806383  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:26.811307 15101 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:26.812065 15101 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:26.804529  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:26.814586  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 104748us
I20260709 13:04:26.790071 15086 consensus_queue.cc:237] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:26.815246  8974 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:26.825784  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:26.834561  9001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 1310) took 1598 ms. Trace:
I20260709 13:04:26.835144  9001 rpcz_store.cc:276] 0709 13:04:25.236440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.236601 (+   161us) service_pool.cc:224] Handling call
0709 13:04:25.236909 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:25.285987 (+ 49078us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:25.286035 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:25.286534 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 13:04:25.693866 (+407332us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:25.693924 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.833832 (+1139908us) spinlock_profiling.cc:243] Waited 29.1 ms on lock 0x7b5400598dd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:26.833910 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:26.833994 (+    84us) raft_consensus.cc:1685] finished
0709 13:04:26.834006 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.834349 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23999,"raft.queue_time_us":19734,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":78520,"spinlock_wait_cycles":66831104,"thread_start_us":317869,"threads_started":1,"wal-append.queue_time_us":319864,"wal-append.run_cpu_time_us":2000,"wal-append.run_wall_time_us":1007302}
I20260709 13:04:26.837982  8295 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:26.840054  8295 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:26.841063 14992 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:26.863334  8329 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:04:26.810783 15059 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:26.862937  8295 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:26.871106  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:26.874262  8327 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 22.
W20260709 13:04:26.877849  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 78049.2us
I20260709 13:04:26.804363 14986 consensus_queue.cc:260] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:26.892987  8974 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:26.895876  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:26.900383 14871 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:04:26.891626 15128 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:26.894304 15093 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:26.902042 15101 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:26.891809 14805 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:26.903692 14856 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:26.904012 15093 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:26.905194  8369 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:26.905361 14958 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:26.906241 14882 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:26.906868  8369 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:26.905236  8330 raft_consensus.cc:2393] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 17: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:26.910544 14990 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:26.911306 14990 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:26.911741 14990 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:26.912127 14990 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:26.906911 14940 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:26.889400 14664 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:26.889567 15011 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:26.929522  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:26.933320  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 90119.3us
I20260709 13:04:26.906884 14882 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:26.946075  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:26.909595 15054 consensus_queue.cc:1059] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.055s
I20260709 13:04:26.901196 15128 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.948263 15128 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:26.949496  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:26.950196  8404 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 13.
I20260709 13:04:26.951076  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:26.951884  8404 raft_consensus.cc:2393] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 18: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:26.952720  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:26.953503  8401 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 24.
I20260709 13:04:26.954591  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:26.955482  8405 raft_consensus.cc:2393] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 21: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:26.956907  8397 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:26.958400  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:26.959123  8400 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18.
I20260709 13:04:26.960155 14987 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:26.960752 14987 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:26.963594 14987 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:26.958582 14950 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:26.947486  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:04:26.958402  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 79837.2us
I20260709 13:04:26.971375  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:26.974288  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:26.937189  8295 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:26.987591  8399 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:04:26.990610  8251 raft_consensus.cc:1275] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:04:26.992688  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 74324.1us
I20260709 13:04:26.994572  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:26.995889  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:26.998190  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:26.998096  8332 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:04:26.991675 14664 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.009100  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:27.011595  8256 raft_consensus.cc:2393] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 15: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:27.012111  8332 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:04:27.013330  8974 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:27.015692  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:26.953109 15062 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:27.027006  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:27.032544  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:27.032792  8372 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:27.033466  8400 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:04:27.033764  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:27.034598  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:27.033780 14868 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.035327  8397 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18.
I20260709 13:04:27.035791 14986 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:27.035609 14868 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:27.036701 15062 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:27.037671  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:27.039209  8402 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:27.042028  8254 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 20: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:27.044999  8401 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:27.042973  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:27.044960 14950 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:27.047706  8298 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:27.048223 14868 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:27.048967  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 61840.5us
I20260709 13:04:27.049845  8298 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:27.050722 14856 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:27.051496 15119 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.330s	user 0.000s	sys 0.004s
I20260709 13:04:27.049742 15093 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:27.054452  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:27.054762  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:27.055244  8256 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:04:27.034087 15066 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
W20260709 13:04:27.058336 15124 log.cc:927] Time spent T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:04:27.047770 14950 raft_consensus.cc:697] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:27.082638  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:27.086436  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:27.086649  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:27.089211  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:27.091431  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 154410us
I20260709 13:04:27.093714 15058 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:27.097141  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.097831 15066 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.097918  8223 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:27.098515 15066 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:27.099481 15086 consensus_queue.cc:1059] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.011s
I20260709 13:04:27.100679 14950 consensus_queue.cc:237] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:27.083381 15128 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:27.103845 15128 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:27.105077  8252 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 18
W20260709 13:04:27.088145  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 348026us
I20260709 13:04:27.112941  8401 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
I20260709 13:04:27.114243 15089 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:27.110405  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:27.119447  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:27.119783  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:27.115623  8295 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:27.120710  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:27.117504  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:27.123903  8399 raft_consensus.cc:2393] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 15: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:27.131004  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:27.125200  8248 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:04:27.131639  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:27.104707 15062 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:27.134665  8397 raft_consensus.cc:2393] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 19: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:27.135684  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:27.137452  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 116443us
I20260709 13:04:27.131771  8250 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:04:27.139187  9001 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 11.
I20260709 13:04:27.144326  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:27.144055  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:27.076534 14664 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:27.145155  9005 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
I20260709 13:04:27.147940  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:27.153801  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 44168.1us
I20260709 13:04:27.156795 15011 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:27.158394 14664 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:27.158782  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:27.159538  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 44023.5us
I20260709 13:04:27.159724  9005 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:04:27.160496  8295 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:27.117107 14856 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:04:27.117146 14940 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:04:27.162276 14662 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:27.163283 14940 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:27.150110 15086 consensus_queue.cc:1243] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 23.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 13:04:27.163846 14940 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:27.164434 14940 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:27.117790 14992 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:27.163354  9009 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:04:27.171099  8400 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:04:27.172549  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:27.138289 15062 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:27.172744 14967 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.154784  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:27.174057  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:27.176295  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.192480  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:27.193253  8397 raft_consensus.cc:1275] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:04:27.194988  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 97272.4us
I20260709 13:04:27.197638  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:27.199679  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:27.202316  8399 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:04:27.202629 14992 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:27.208370  9008 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 16: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:27.209514  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:27.188839 15062 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:27.209715  8398 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:27.205047 15058 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:27.194463 14758 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:04:27.224620  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 121483us
I20260709 13:04:27.207064 14992 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:27.235810  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:27.237265 14958 raft_consensus.cc:3055] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:04:27.233256 14805 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:27.247061 15066 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:27.256245  8398 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 16: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:27.257319  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:27.260964  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:27.261508  8369 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:27.274088  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:27.271651  9003 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 21.
W20260709 13:04:27.279726  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 49900.2us
I20260709 13:04:27.281023  8399 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:27.279109  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:27.283066  8252 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:04:27.283352  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:27.284543  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:27.199536 14967 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:27.244999 14990 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:27.291239  8403 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 22: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:27.256083 14805 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:27.293475  8332 raft_consensus.cc:2368] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 21. Current term is 22.
I20260709 13:04:27.254709 14958 raft_consensus.cc:740] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:27.293284  8297 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:27.288192  8298 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:27.224270 14882 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:27.299263  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:27.325228  8406 consensus_meta.cc:287] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.631s	user 0.010s	sys 0.002s
I20260709 13:04:27.325770  8406 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:27.323009  8372 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:27.328348  8372 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:27.329814 15006 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:27.330219 15006 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:27.330555 15006 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:27.270874 14758 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:27.328491  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 135276us
I20260709 13:04:27.318125  8401 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:27.316883  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:27.294814 14992 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:04:27.356915  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 58226.5us
I20260709 13:04:27.277318 14664 consensus_queue.cc:1059] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 13:04:27.360594  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:27.357942  8406 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:04:27.358560  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 85950.8us
I20260709 13:04:27.361851  8255 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
I20260709 13:04:27.368788  9000 raft_consensus.cc:2368] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 17. Current term is 18.
I20260709 13:04:27.369697  8252 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:04:27.302259 14805 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:27.363336  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:27.436033  9007 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:27.384155 14940 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:27.395773 14990 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:27.397984 14758 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:27.410900 15086 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:27.416479  8222 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:27.419575  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:27.422343 15136 log.cc:927] Time spent T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:04:27.422745  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:04:27.423549 15135 log.cc:927] Time spent T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
W20260709 13:04:27.424278  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 164867us
W20260709 13:04:27.424533  8255 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 2613) took 4029 ms (client timeout 2115 ms). Trace:
W20260709 13:04:27.434237  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 56909.3us
I20260709 13:04:27.371439  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:27.448361  9008 raft_consensus.cc:1813] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Handling vote request from an unknown peer 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:27.449254  8329 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 13:04:27.448061  8255 rpcz_store.cc:269] 0709 13:04:23.394735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.446645 (+ 51910us) service_pool.cc:224] Handling call
0709 13:04:27.424235 (+3977590us) spinlock_profiling.cc:243] Waited 3.75 s on lock 0x7b540034c6d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:27.424422 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8631873920}
I20260709 13:04:27.356945 14992 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:04:27.443964  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 119458us
I20260709 13:04:27.446338 14664 consensus_queue.cc:1243] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer 8c42259d7c4b463a9f485f42c320c81b log is divergent from this leader: its last log entry 19.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:04:27.462374 15011 raft_consensus.cc:3055] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:27.462774 15011 raft_consensus.cc:740] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:27.463661 15011 consensus_queue.cc:260] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:27.465060 15011 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:27.466955 14958 consensus_queue.cc:260] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:27.468305 14958 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:27.470822 14940 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:27.443511 15086 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:27.471906 15086 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:27.473577 15062 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:27.475725 15062 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:27.443897  8330 raft_consensus.cc:2376] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e64761f43e4240319c41117cf886d22f in term 25. Re-sending same reply.
W20260709 13:04:27.453603  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 77008.3us
I20260709 13:04:27.453408 14758 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:27.479614 14758 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:27.479991 14758 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:27.481709 14825 consensus_queue.cc:1048] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:04:27.453478  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 80600.5us
I20260709 13:04:27.483774  8298 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:27.480010 15066 raft_consensus.cc:993] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b: : Instructing follower e64761f43e4240319c41117cf886d22f to start an election
I20260709 13:04:27.485270 15139 raft_consensus.cc:1081] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Signalling peer e64761f43e4240319c41117cf886d22f to start an election
I20260709 13:04:27.446357 14950 consensus_queue.cc:1059] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 13:04:27.486353 14950 raft_consensus.cc:3055] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:27.486724 14950 raft_consensus.cc:740] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:27.455984  9008 raft_consensus.cc:2368] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 24. Current term is 25.
I20260709 13:04:27.485302 14856 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:27.487387 14967 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:27.487674 14856 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:27.459105  8222 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:27.490903  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 130618us
I20260709 13:04:27.492897  9007 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:04:27.442292  8253 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:27.487638 14950 consensus_queue.cc:260] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:27.506933  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 66214.3us
I20260709 13:04:27.509205  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:27.509601  9007 raft_consensus.cc:1240] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:27.509936  8330 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:27.510051  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 51072.7us
I20260709 13:04:27.511071  8222 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:27.451431 15058 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:27.519690  8222 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:27.491267 14992 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:27.524055  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 241091us
I20260709 13:04:27.509411 14950 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:27.526381  9008 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:04:27.526297  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:27.527091  8255 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:27.502219 14856 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:27.529024  8253 raft_consensus.cc:1275] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:27.529785  8369 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:27.538389  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:27.538637  8222 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:27.538753  8150 catalog_manager.cc:5697] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 12 to 16, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 16 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:27.540273 15138 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:27.541765 15066 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:27.542461 14898 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:27.543490  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:27.544090  8400 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:04:27.545222  8254 raft_consensus.cc:1275] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:27.545680  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:27.547055  8256 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:04:27.547444 15132 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:27.548010  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:27.548898  8251 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 22: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:27.554219  8252 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 16.
I20260709 13:04:27.565562 14940 raft_consensus.cc:2764] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:04:27.593889  9000 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:04:27.596227 14882 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:27.597357  8329 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:27.598795  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:27.599493  8397 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:27.601987  8220 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:27.520093 15058 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:27.603240 15058 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:27.604842 15138 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:27.605417 15138 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:27.605830 15138 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:27.608180 15006 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:27.610402 15006 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:27.610590  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 102580us
I20260709 13:04:27.611002 15006 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:27.614168  8329 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:04:27.618263  8330 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
I20260709 13:04:27.621415  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:27.611325  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:04:27.623776  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 91375.5us
I20260709 13:04:27.540716  8150 catalog_manager.cc:5697] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:27.624549  8297 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:27.522158 14868 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:27.635092 14758 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:27.623486 14882 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.642052 14882 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:27.644165 14882 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:27.632833 15138 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:27.626626  8223 leader_election.cc:400] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 17. Current term is 18.
W20260709 13:04:27.637949  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 106608us
I20260709 13:04:27.641330 14662 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:27.646853  8223 leader_election.cc:403] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:27.640728  8297 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:27.648139  8399 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:27.650163  8398 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:27.611351 15006 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:27.657956  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:27.660238  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 123600us
I20260709 13:04:27.661069  8223 leader_election.cc:400] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 24. Current term is 25.
I20260709 13:04:27.661499  8223 leader_election.cc:403] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:27.671612  8297 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:27.656574 14990 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:27.610423 14825 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:04:27.673445  8223 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:27.677868  8297 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:27.659209 15006 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:27.676834 15139 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:04:27.657932 14664 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:27.726310  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 46649.2us
I20260709 13:04:27.731297  8245 tablet_service.cc:1558] Processing DeleteTablet for tablet 81804f59e3124d27998ea80fc1c36663 with delete_type TABLET_DATA_TOMBSTONED (TS 8c42259d7c4b463a9f485f42c320c81b not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:51208
I20260709 13:04:27.731554  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:27.742285  8369 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:27.743945  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:27.746860 15062 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:27.679778 14990 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:27.747992 14990 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:27.750463 14758 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:27.751354 14758 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:27.752177 14758 consensus_queue.cc:1243] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer 8c42259d7c4b463a9f485f42c320c81b log is divergent from this leader: its last log entry 19.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:04:27.752045 14992 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:27.753219  8400 raft_consensus.cc:1240] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:27.755515 15132 consensus_queue.cc:1059] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:27.756573 15132 raft_consensus.cc:3055] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:27.756402 14856 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:27.757040 15132 raft_consensus.cc:740] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:27.736826 14805 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:27.679559 14868 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 17. Current term is 18.
I20260709 13:04:27.681813 14662 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:27.672155 15058 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:27.780956  9007 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:27.794351 14940 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:27.800164 15132 consensus_queue.cc:260] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:27.801127  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:27.801873  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:27.802594 14958 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:27.794775 14662 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:27.794883 15011 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:27.804519 14805 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:27.871445  8325 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 16.
I20260709 13:04:27.873489  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:27.874396  9000 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:27.809644  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 136322us
I20260709 13:04:27.791834 15138 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 26 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:27.878760  8332 raft_consensus.cc:1240] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:27.817612 14753 consensus_queue.cc:1048] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.141s
I20260709 13:04:27.819388  8972 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:27.823130 15034 consensus_queue.cc:1059] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.144s
W20260709 13:04:27.833968  8151 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45434 (request call id 283) took 62550 ms (client timeout 15000 ms). Trace:
I20260709 13:04:27.757697 15086 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:27.681236 15093 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.100s
I20260709 13:04:27.795686 14825 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:27.875067 14662 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:27.793243 14758 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:27.880059  8223 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:27.811969 14856 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:27.866597  8397 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
I20260709 13:04:27.816341  8150 catalog_manager.cc:5697] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 12 to 13, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 13 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:27.875239 14940 raft_consensus.cc:697] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:27.897382 14805 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:27.919685  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 379834us
I20260709 13:04:27.919744  8974 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:27.940373  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:27.941251  9007 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:27.872228 14958 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:27.904161 14882 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:27.906809 14758 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:27.916885 14898 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:27.889349 15011 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:27.889655 14868 consensus_queue.cc:1059] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.040s
W20260709 13:04:27.950675  8372 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.017s	sys 0.045s
I20260709 13:04:27.894778 15066 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.034s
W20260709 13:04:27.988030  8372 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.017s	sys 0.045s
I20260709 13:04:27.921330 14992 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:27.948645  8222 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:04:27.994184  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 174928us
W20260709 13:04:27.988545  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 660241us
I20260709 13:04:27.982620 14950 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:28.011065 14950 raft_consensus.cc:697] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:28.012274 14950 consensus_queue.cc:237] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:27.945026 14882 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:27.983783 14758 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:27.987987 14805 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:28.037405  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 93323.7us
I20260709 13:04:28.039019 14805 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:28.040314  8222 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:27.992934 15066 consensus_queue.cc:1243] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:04:28.052217 14990 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:28.054580 14868 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:28.055101 14868 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:28.055507 14868 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:28.031601 14882 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:28.062872 15130 raft_consensus.cc:3055] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:28.063314 15130 raft_consensus.cc:740] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:28.064227 15130 consensus_queue.cc:260] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:28.070174 15058 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:28.073536 14805 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:04:27.891458  8151 rpcz_store.cc:269] 0709 13:03:25.283560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.505990 (+222430us) service_pool.cc:224] Handling call
0709 13:04:27.833675 (+62327685us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:24.264068 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:24.395338 (+131270us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:24.395379 (+    41us) write_op.cc:435] Acquired schema lock
0709 13:04:24.395399 (+    20us) tablet.cc:662] Decoding operations
0709 13:04:24.396067 (+   668us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:24.396152 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:24.396185 (+    33us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:04:24.396592 (+   407us) tablet.cc:701] Row locks acquired
0709 13:04:24.396612 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:04:24.396774 (+   162us) write_op.cc:270] Start()
0709 13:04:24.396904 (+   130us) write_op.cc:276] Timestamp: P: 1783602264396723 usec, L: 0
0709 13:04:24.396928 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:04:24.413717 (+ 16789us) log.cc:844] Serialized 1672 byte log entry
0709 13:04:24.756095 (+342378us) spinlock_profiling.cc:243] Waited 34.6 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:04:25.070297 (+314202us) op_driver.cc:464] REPLICATION: finished
0709 13:04:25.234901 (+164604us) write_op.cc:301] APPLY: starting
0709 13:04:25.238838 (+  3937us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:25.368867 (+130029us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:25.368893 (+    26us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:25.570016 (+201123us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:25.570239 (+   223us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 13:04:25.570298 (+    59us) write_op.cc:312] APPLY: finished
0709 13:04:25.619414 (+ 49116us) log.cc:844] Serialized 49 byte log entry
0709 13:04:25.626731 (+  7317us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:25.634848 (+  8117us) write_op.cc:454] Released schema lock
0709 13:04:25.641717 (+  6869us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:25.661847 (+ 20130us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:25.665845 (+  3998us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":79636,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":146734,"apply.run_cpu_time_us":10385,"apply.run_wall_time_us":423728,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20855,"num_ops":4,"prepare.queue_time_us":168624,"prepare.run_cpu_time_us":11943,"prepare.run_wall_time_us":374290,"raft.queue_time_us":211187,"raft.run_cpu_time_us":6275,"raft.run_wall_time_us":140639,"replication_time_us":630850,"spinlock_wait_cycles":79630720,"thread_start_us":510724,"threads_started":4,"wal-append.queue_time_us":306679,"wal-append.run_cpu_time_us":9293,"wal-append.run_wall_time_us":1953738}]]}
I20260709 13:04:28.082178  9007 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 26.
I20260709 13:04:28.086925  8295 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:28.096236  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:28.101495  9000 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 22.
I20260709 13:04:28.106092  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:28.085336 15130 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:28.110730  8298 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:28.393585  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 282923us
W20260709 13:04:28.203804  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 529656us
W20260709 13:04:28.225803  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 481883us
I20260709 13:04:28.202083  8150 catalog_manager.cc:5697] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:04:28.307691  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 312481us
I20260709 13:04:28.162369 14940 consensus_queue.cc:237] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:28.070238 14967 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:28.401264  8150 catalog_manager.cc:5697] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:28.226610  8329 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.006s	sys 0.022s
W20260709 13:04:28.406483  8329 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.009s	sys 0.024s
W20260709 13:04:28.254220  8404 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.011s	sys 0.011s
W20260709 13:04:28.284418 15062 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.011s	sys 0.017s
I20260709 13:04:28.102332 15058 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.098934 15093 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:28.386611 14987 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:28.400153  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:28.400835  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 316619us
I20260709 13:04:28.395347  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:28.401353  8252 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
W20260709 13:04:28.403431  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 363160us
I20260709 13:04:28.121212 14825 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:04:28.407632  8404 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.014s	sys 0.013s
W20260709 13:04:28.406266  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 319448us
I20260709 13:04:28.411864  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:28.383849 15132 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:28.397851  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2855) took 1295 ms. Trace:
I20260709 13:04:28.427145 14868 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:28.429538  8250 raft_consensus.cc:1275] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:04:28.412799 14753 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:04:28.430631  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.431371 14958 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:28.431272 15058 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:28.405901 14967 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:28.434329  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:28.439895  8402 rpcz_store.cc:276] 0709 13:04:27.102700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.152369 (+ 49669us) service_pool.cc:224] Handling call
0709 13:04:27.167275 (+ 14906us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.219061 (+ 51786us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:27.615188 (+396127us) log.cc:844] Serialized 14 byte log entry
0709 13:04:28.380657 (+765469us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:28.380707 (+    50us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:28.380749 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.397624 (+ 16875us) spinlock_profiling.cc:243] Waited 39.5 ms on lock 0x7b54003e7954. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:28.397670 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.397794 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":221514,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":26,"raft.queue_time_us":49485,"raft.run_cpu_time_us":49,"raft.run_wall_time_us":49,"spinlock_wait_cycles":90908544,"thread_start_us":986658,"threads_started":3}
W20260709 13:04:28.412663 15062 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.011s	sys 0.020s
I20260709 13:04:28.397696 14898 raft_consensus.cc:697] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:28.456117  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 1164) took 1228 ms. Trace:
I20260709 13:04:28.466022  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:28.463198  8150 catalog_manager.cc:5697] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 9 to 11, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 11 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:28.472469  8253 raft_consensus.cc:1240] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:04:28.481182  8222 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:28.482867  9003 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 21. Current term is 22. Ops: [18.14-21.16]
I20260709 13:04:28.432433 15093 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:28.463658  8404 rpcz_store.cc:276] 0709 13:04:27.227752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.228847 (+  1095us) service_pool.cc:224] Handling call
0709 13:04:27.258800 (+ 29953us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.259017 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:27.680276 (+421259us) log.cc:844] Serialized 14 byte log entry
0709 13:04:28.455732 (+775456us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:28.455797 (+    65us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:28.455846 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.455921 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.456052 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41151,"prepare.queue_time_us":196278,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":35351,"thread_start_us":1140096,"threads_started":2}
W20260709 13:04:28.486815  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 89953.5us
I20260709 13:04:28.430325 15155 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:04:28.463485 14967 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:28.457262 15086 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:28.493037  8297 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:04:28.502873  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 107595us
I20260709 13:04:28.502986  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:04:28.504159 15150 log.cc:927] Time spent T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:04:28.505860 15093 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:28.500200 15006 consensus_meta.cc:287] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.771s	user 0.008s	sys 0.012s
I20260709 13:04:28.509701  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:28.432500 14662 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:28.532562 14662 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:28.463850 15062 raft_consensus.cc:697] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 18 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:28.533882 14898 consensus_queue.cc:237] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:28.535012 15062 consensus_queue.cc:237] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:28.465840 14753 raft_consensus.cc:697] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 22 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:28.515947  8369 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:28.518142 14990 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:28.518806  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:28.544812  9007 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:28.521996  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:28.523078  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:04:28.523816 15151 log.cc:927] Time spent T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:04:28.525951 14882 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:28.551347  8369 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:28.528635 14868 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:28.500923 14992 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:28.530517 15006 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:28.523829 15058 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:28.538794  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:28.552135 14805 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:28.554600  9005 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:04:28.538775 15066 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:04:28.537982  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 45093.7us
I20260709 13:04:28.543478 15159 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:28.543668 14664 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:28.516356 15011 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:28.566213  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:28.571380  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:28.572556  8332 raft_consensus.cc:2410] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:04:28.513907 15161 consensus_queue.cc:1048] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:04:28.564217  9001 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:28.569617  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 52871.2us
I20260709 13:04:28.574340  8972 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:04:28.576346  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 110219us
W20260709 13:04:28.576399  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 140909us
I20260709 13:04:28.577350  8222 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:28.572722  8329 raft_consensus.cc:2410] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:04:28.577456  8295 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:28.578773 14825 raft_consensus.cc:697] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:28.579782 14825 consensus_queue.cc:237] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:28.563149 15146 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:28.622959  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:28.625876  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 48625.6us
I20260709 13:04:28.567013 15138 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.622041 15066 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:28.622610 14758 consensus_meta.cc:287] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.581s	user 0.015s	sys 0.004s
I20260709 13:04:28.654086 14758 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:28.630956 15006 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:28.630831  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:28.633774  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:28.634850  8332 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:04:28.656584 15138 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:28.658192  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:28.658953  8332 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:28.567731 14805 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:28.662552 14882 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:28.663513 14805 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:28.638337  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:28.664733  8323 raft_consensus.cc:2393] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 20: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:28.665954  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:28.666949  8323 raft_consensus.cc:2368] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 17. Current term is 18.
I20260709 13:04:28.638484  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:28.669665 14753 consensus_queue.cc:237] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:28.670626  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:28.672168  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:28.638825  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:28.639428 14990 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:28.674922  8328 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:04:28.675017 14990 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:28.639740  8369 connection.cc:256] RPC call timeout handler was delayed by -1.3129592450000018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29350000000000004s for a response
W20260709 13:04:28.676587  8369 connection.cc:256] RPC call timeout handler was delayed by -1.1137450859999944s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3046s for a response
I20260709 13:04:28.640347  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:28.656417  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:28.628844 15058 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:04:28.646747 15158 log.cc:927] Time spent T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:04:28.742019  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:28.621929 14987 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:28.678776  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:28.707612  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:28.571535 15155 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:28.695544  8222 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:28.677975  8325 raft_consensus.cc:2376] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e64761f43e4240319c41117cf886d22f in term 22. Re-sending same reply.
I20260709 13:04:28.613921 15171 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:28.731760  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:28.731874  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:28.619725 15034 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:28.734674  9009 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:04:28.754050  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:28.754455  8256 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:04:28.756054  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:28.756212  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:28.757025  8249 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:28.757309  8250 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 16: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
W20260709 13:04:28.650033 15139 consensus_meta.cc:287] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.869s	user 0.008s	sys 0.012s
I20260709 13:04:28.677497  8330 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
W20260709 13:04:28.677726  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 53846.8us
I20260709 13:04:28.744254  9000 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 14.
I20260709 13:04:28.731567  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:28.764523  8295 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:28.750874  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:28.574594 14992 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:28.656008 15165 raft_consensus.cc:3055] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:28.583405 15159 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:28.583602 14664 raft_consensus.cc:697] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:28.750721  8328 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:28.656457 14758 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:28.758772  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:28.758993  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:28.759567  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:28.759965  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2723) took 1215 ms. Trace:
I20260709 13:04:28.766961  9007 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 22.
I20260709 13:04:28.777403  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2716) took 1243 ms. Trace:
I20260709 13:04:28.796056  8252 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:28.751752  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 44237.6us
I20260709 13:04:28.768103  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:28.794301  8298 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:28.768932  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:28.770119  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:28.794301  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:28.797784  8402 raft_consensus.cc:2368] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 17. Current term is 18.
I20260709 13:04:28.706413 14882 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:28.773888 14805 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:28.798738  8256 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:28.784912  9001 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:04:28.800788  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:28.796096  8255 rpcz_store.cc:276] 0709 13:04:27.534014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.534139 (+   125us) service_pool.cc:224] Handling call
0709 13:04:27.542300 (+  8161us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:27.639279 (+ 96979us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:27.639323 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:27.639871 (+   548us) log.cc:844] Serialized 26 byte log entry
0709 13:04:27.998917 (+359046us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:28.433756 (+434839us) log.cc:844] Serialized 14 byte log entry
0709 13:04:28.775601 (+341845us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b1800403d80. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88283664bc 00007f88256b448b 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256ce455 00007f88287b6060 00007f88287c8716 00007f88287d9287
0709 13:04:28.775737 (+   136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.777041 (+  1304us) spinlock_profiling.cc:243] Waited 96.6 ms on lock 0x7b5400309154. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:28.777097 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:28.777158 (+    61us) raft_consensus.cc:1685] finished
0709 13:04:28.777199 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.777341 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45550,"prepare.queue_time_us":58942,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":124372,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":7656,"spinlock_wait_cycles":254396672,"thread_start_us":912624,"threads_started":3,"wal-append.queue_time_us":749614}
I20260709 13:04:28.834921  8404 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:28.732076 14958 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
W20260709 13:04:28.833604  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 51176.2us
I20260709 13:04:28.837667  8405 raft_consensus.cc:1240] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:28.837898  8397 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:28.838284  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:28.647207 15157 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.148s
I20260709 13:04:28.846594  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:28.801652  8400 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:04:28.800993  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:28.840186  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:28.769141 14992 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:28.768261 14871 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.853495  8220 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:28.842278  8326 raft_consensus.cc:1813] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Handling vote request from an unknown peer 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:28.797752  8257 rpcz_store.cc:276] 0709 13:04:27.536163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.536263 (+   100us) service_pool.cc:224] Handling call
0709 13:04:27.536592 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:27.549867 (+ 13275us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:27.549901 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:27.550459 (+   558us) log.cc:844] Serialized 26 byte log entry
0709 13:04:27.922205 (+371746us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:28.384527 (+462322us) log.cc:844] Serialized 14 byte log entry
0709 13:04:28.397220 (+ 12693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.397292 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:28.751618 (+354326us) raft_consensus.cc:1685] finished
0709 13:04:28.751641 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.751762 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39209,"prepare.queue_time_us":135416,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":156971,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":13922,"thread_start_us":527808,"threads_started":2,"wal-append.queue_time_us":570929}
I20260709 13:04:28.794181 15058 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:28.780645 14987 raft_consensus.cc:697] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:28.852958  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:28.853137  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:28.863677  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:28.794713 15139 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 24. Current term is 25.
I20260709 13:04:28.778002 15130 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:28.868167  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 241490us
I20260709 13:04:28.778119 15034 raft_consensus.cc:697] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:28.780792 15178 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.884708 15178 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:28.886689  8248 pending_rounds.cc:85] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 11
I20260709 13:04:28.887295  8248 pending_rounds.cc:107] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 13:04:28.887662  8248 raft_consensus.cc:2889] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:28.862388 15011 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:04:28.867610 15148 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:28.890051  8328 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:04:28.867856 14662 consensus_queue.cc:1048] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:04:28.890729 14967 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:28.893792 15138 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:28.854368 14992 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:28.861852  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:28.922688  8257 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:28.839648 14868 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:28.863304  8326 raft_consensus.cc:2368] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 24. Current term is 25.
I20260709 13:04:28.926677  8397 raft_consensus.cc:1275] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:28.777602 15165 raft_consensus.cc:740] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:28.927733  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 144926us
I20260709 13:04:28.928632 15165 consensus_queue.cc:260] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:28.890112 15148 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:28.870779  8323 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 22: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:28.929818 14958 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.929968 15165 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:28.930492 14958 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:28.932303  8332 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 26.
I20260709 13:04:28.844507 14856 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.841909 15086 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:28.933220  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 79839.4us
I20260709 13:04:28.933120 14856 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:28.872530  8324 raft_consensus.cc:1275] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:04:28.873055  8406 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 13:04:28.930099 15148 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:28.939838  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:28.875264 15181 consensus_queue.cc:1048] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 13:04:28.876279 15179 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.842149 14805 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:28.878111 15168 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:28.862622 14871 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:28.878971 14664 consensus_queue.cc:237] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:28.836261 14882 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:28.883999 15164 log.cc:927] Time spent T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.234s	user 0.000s	sys 0.001s
I20260709 13:04:28.858865  8331 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 22 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:28.898092  9008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 1175) took 1155 ms. Trace:
I20260709 13:04:28.795050 14898 consensus_queue.cc:1059] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.082s
I20260709 13:04:28.906363  8249 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:04:28.913681  8252 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:04:28.915295 15142 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:28.851225 15171 raft_consensus.cc:697] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:28.920186 14990 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:28.928922  8369 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:28.869014  8222 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:28.844301 15161 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:28.935573  8251 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:28.795027 15155 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:28.933684 15086 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:28.988786  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 214633us
I20260709 13:04:28.941608  8256 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
W20260709 13:04:28.944383  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 100848us
I20260709 13:04:28.945555  8397 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:04:28.945045  9004 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:04:28.951552  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:28.985541  8372 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:28.958674  8405 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:28.969807  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:28.934978 15178 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:28.991873  8400 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:28.976254  8974 leader_election.cc:400] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 17. Current term is 18.
I20260709 13:04:28.992707  8974 leader_election.cc:403] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:04:28.993723  8404 raft_consensus.cc:1275] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:04:28.980974  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:28.969945 14882 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:28.971223  9008 rpcz_store.cc:276] 0709 13:04:27.742345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.744836 (+  2491us) service_pool.cc:224] Handling call
0709 13:04:27.745177 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:27.745594 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:27.745624 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:27.746160 (+   536us) log.cc:844] Serialized 26 byte log entry
0709 13:04:28.381061 (+634901us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:28.381099 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.381156 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:28.897859 (+516703us) raft_consensus.cc:1685] finished
0709 13:04:28.897882 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.898026 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":146,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"thread_start_us":480123,"threads_started":1,"wal-append.queue_time_us":714087}
I20260709 13:04:28.898830 15130 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:28.989168  9007 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:04:28.874315  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:28.991154  8406 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 21.
I20260709 13:04:28.995050  8401 raft_consensus.cc:2393] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 20: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:28.997119  8220 leader_election.cc:400] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 17. Current term is 18.
I20260709 13:04:29.000590  8372 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:29.010948  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:29.012750  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 143830us
I20260709 13:04:28.989363 14898 consensus_queue.cc:1243] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:04:28.987339 15161 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:28.990211 14958 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:28.995903 14868 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:29.028926  8222 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:29.030810  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:29.031561  8249 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:28.999550 14967 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:29.037374  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 108559us
I20260709 13:04:29.038086  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:29.038702  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:29.013823 15157 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:29.040315  8327 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 13:04:29.026180 15185 log.cc:927] Time spent T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:04:29.049290 15086 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:29.053306 15157 raft_consensus.cc:697] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 22 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:29.054566 15190 consensus_queue.cc:1059] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:04:29.054941 14805 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:29.054446 15157 consensus_queue.cc:237] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:29.055541 15190 raft_consensus.cc:3055] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:29.055945 15190 raft_consensus.cc:740] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:29.043092 14898 raft_consensus.cc:2764] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:04:29.066215  8251 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 12.
I20260709 13:04:29.027029 15161 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:28.972398 15179 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:29.023527 14882 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 17. Current term is 18.
I20260709 13:04:29.014570 14990 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:29.041100  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:29.043318  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:29.076640  8249 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:04:29.013998 15168 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:29.047113  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:29.041000  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:29.071359  8257 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
I20260709 13:04:29.034482 15011 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.073923  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:29.074806  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:29.030592 14868 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:29.080899  8248 raft_consensus.cc:1813] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Handling vote request from an unknown peer ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:29.083281  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:29.083341  9007 raft_consensus.cc:2410] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:04:29.056795 15190 consensus_queue.cc:260] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:29.081107  8222 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:29.090453  8257 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:04:29.092021  8253 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 21.
I20260709 13:04:29.092494  8250 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:29.098099  8248 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 16: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
W20260709 13:04:29.099035  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.099048 15173 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:29.099797 15173 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:29.101881 15192 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.102677 15192 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:29.103210 15173 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:29.106112  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:29.107558  8249 raft_consensus.cc:2410] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:04:29.115479 15190 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:29.117628 14856 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:29.021621 14987 consensus_queue.cc:237] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:29.122596 15192 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:29.093351 15011 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:29.127264 15179 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:29.092767 14868 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:29.136005  8331 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 25.
W20260709 13:04:29.137584  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 91534.4us
I20260709 13:04:29.143352  8250 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
I20260709 13:04:29.087598 14990 raft_consensus.cc:697] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:04:29.146943  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 72414.3us
I20260709 13:04:29.146951  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:29.149585  8222 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:29.110185 14758 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:29.078186 15148 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:29.036828 15034 consensus_queue.cc:237] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:29.147456 14898 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:29.160419 15195 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.162144 15148 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:29.163173  8252 raft_consensus.cc:1240] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:04:29.160492  8369 leader_election.cc:400] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 23. Current term is 24.
I20260709 13:04:29.168767  8369 leader_election.cc:403] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:29.169453 15191 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:29.169695 14758 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 23. Current term is 24.
I20260709 13:04:29.170970  8369 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:29.172837  8369 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:29.175390 15157 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:29.177822 15058 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:29.178694 15058 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:29.177945  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:29.182334  8254 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:29.186851 15011 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:29.191690  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:29.194809 14990 consensus_queue.cc:237] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:29.168521 14992 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:29.199681 14992 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:29.200508 15177 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:29.203635 15156 tablet_replica.cc:333] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:04:29.203220 14868 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:29.206367 15139 raft_consensus.cc:3055] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:29.206897 15139 raft_consensus.cc:740] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:29.207827 15139 consensus_queue.cc:260] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:29.210227  8402 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:04:29.216802  8297 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:29.219816  8297 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:29.209170 15139 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:29.235948  8255 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
I20260709 13:04:29.240470  8405 raft_consensus.cc:1275] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:04:29.243659  8297 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:29.250278 15178 consensus_queue.cc:1059] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:29.218726 15138 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:04:29.252823  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 106241us
W20260709 13:04:29.239665  8220 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.145s (SENT) [suppressed 8 similar messages]
I20260709 13:04:29.258742  8402 raft_consensus.cc:1240] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:04:29.260573  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:29.237141 15190 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:29.223268 14898 raft_consensus.cc:697] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 22 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:29.264063  8406 raft_consensus.cc:1275] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:04:29.237672 15181 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:29.243594 14758 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:29.238955 15130 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:29.201267 15177 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:29.274593  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:29.274645  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:29.277729  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:29.279043  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 204311us
I20260709 13:04:29.243007 15155 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:29.285045  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:29.236989 14825 consensus_queue.cc:1059] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:04:29.282277  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:29.282436  8972 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:29.170032 15191 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:29.280988  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:29.236779 14958 consensus_queue.cc:1059] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 13:04:29.287232  8972 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:29.286772  8403 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:29.188436 15195 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:29.252099 15086 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:04:29.291944  8399 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:04:29.278354 14967 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.144877 15171 consensus_queue.cc:237] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:29.279517 15011 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:29.313920 15011 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:29.296443 14898 consensus_queue.cc:237] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:29.314381 15058 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:29.283731  8402 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:39308
I20260709 13:04:29.316519  8402 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:29.316951  8402 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:29.317901 15066 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:29.301882 15174 consensus_queue.cc:1048] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:04:29.323071  8295 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:29.283345 15138 raft_consensus.cc:697] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:29.285166 14992 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:29.294539 15130 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:29.294152 14868 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:29.336800  8295 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:29.341548  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:29.343086  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:29.343866  8397 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:29.314383 15011 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:29.344233  8295 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:29.330941 14958 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:29.356546  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:29.357726  8257 raft_consensus.cc:2368] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 17. Current term is 18.
I20260709 13:04:29.308346 14987 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:29.360908  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:29.348500 14986 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:29.349368 15138 consensus_queue.cc:237] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:29.614207  8248 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:04:29.350792 15159 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:29.350874 15195 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:29.351275 15174 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:29.623208 15174 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.623787 15174 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:29.352137  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:29.365450  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.340315  8402 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:29.429545  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:29.438179 14871 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:29.509102 14992 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:29.651815  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:29.422106  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 160445us
I20260709 13:04:29.530874  8372 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:29.439363 14825 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:29.426666  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:29.655383  8252 raft_consensus.cc:2410] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 26 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 18.
I20260709 13:04:29.331547 14967 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:29.339944 15058 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:04:29.564615  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 62249.2us
I20260709 13:04:29.415694  8222 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:29.418514 15093 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:29.423415  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:29.407347  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:29.426939 15155 raft_consensus.cc:3055] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:29.432926  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:29.578873 14662 consensus_queue.cc:1059] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 13:04:29.449795  8253 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:04:29.441382  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:29.667237  8404 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:29.667534  8400 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:29.667704 14898 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:29.667932  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:29.668243 14898 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.668629 14898 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:29.506316  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:29.485556  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:29.446249  8257 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:04:29.433547  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:29.429275  8397 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:04:29.439849 15142 consensus_queue.cc:1048] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 13:04:29.415100  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:29.415941  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:29.676697  8251 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 23.
I20260709 13:04:29.572160  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:29.412839  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:29.334578 14950 consensus_queue.cc:1059] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 13:04:29.506211 14868 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:29.416999 15191 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:29.663926 15058 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:04:29.516901 15202 log.cc:927] Time spent T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:04:29.339183 15130 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:29.369813 14958 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:29.352034 15177 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:29.649014  8398 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
W20260709 13:04:29.558729  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 315176us
I20260709 13:04:29.704213  8297 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:04:29.655880  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 125143us
I20260709 13:04:29.657248  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:29.658752  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:29.660605  9008 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 25.
I20260709 13:04:29.413743 15157 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:29.663329  8298 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:29.663754  8402 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:29.553359 14753 consensus_queue.cc:1059] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.101s
W20260709 13:04:29.665639  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 250008us
I20260709 13:04:29.571547 15086 consensus_queue.cc:1059] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
W20260709 13:04:29.677966  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.679085  8401 raft_consensus.cc:2393] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 19: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:29.658023 14825 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.728600 14825 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:29.728749 15139 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.729547 15139 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:29.730393 15203 log.cc:927] Time spent T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc: Append to log took a long time: real 0.312s	user 0.007s	sys 0.000s
I20260709 13:04:29.732560 15139 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:29.707516 15130 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:29.743664 15011 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:29.745565 15210 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.746294 15210 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:29.748490 15210 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:29.686777  8404 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:04:29.659953 15159 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:29.695030  8397 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:04:29.518553  8330 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:29.669747 15093 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:29.662420 15155 raft_consensus.cc:740] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:29.705587  8399 raft_consensus.cc:2393] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 20: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:29.716305 14992 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:29.708945  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 45754.7us
I20260709 13:04:29.718766  8406 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
W20260709 13:04:29.749764 15161 consensus_meta.cc:287] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.658s	user 0.008s	sys 0.004s
I20260709 13:04:29.750917  8405 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:29.753301 14967 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:29.852967  8298 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:29.754804 15174 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:29.681063  8403 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 22 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:29.757467 14958 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:29.765216  9008 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 13:04:29.770908 15209 log.cc:927] Time spent T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.364s	user 0.001s	sys 0.000s
I20260709 13:04:29.896348 15161 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:29.828781  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 124686us
I20260709 13:04:29.760641 15130 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:29.702399 15058 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:29.819900 14987 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.852658 15159 raft_consensus.cc:697] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 18 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:04:29.909849  9004 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56948 (request call id 1456) took 1408 ms (client timeout 1610 ms). Trace:
I20260709 13:04:29.850750 15178 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:29.912038  8405 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 26.
I20260709 13:04:29.907964  8297 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:29.918141  9004 rpcz_store.cc:269] 0709 13:04:28.495875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.521573 (+ 25698us) service_pool.cc:224] Handling call
0709 13:04:29.904411 (+1382838us) spinlock_profiling.cc:243] Waited 371 ms on lock 0x7b54004c2f54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:29.904518 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":852473600}
I20260709 13:04:29.918727 15011 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:29.922459  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 69623.7us
I20260709 13:04:29.853909 14950 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.065s
I20260709 13:04:29.929312  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:04:29.929513  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 272369us
I20260709 13:04:29.854074 14898 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:29.926375 15216 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:29.942582  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:29.940440  8298 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:29.946324  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:29.947009  8328 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 23.
I20260709 13:04:29.934676 15178 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:29.922266 14992 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.020s
I20260709 13:04:29.919777 14987 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:29.957185  8372 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:29.935295 14967 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:29.966091  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:29.973006  8331 raft_consensus.cc:2410] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 26 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 18.
I20260709 13:04:29.942078 15217 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:29.947288 15216 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:29.946111 15161 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:29.960037 14758 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.920470 15155 consensus_queue.cc:260] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:29.943885 15011 consensus_queue.cc:1048] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:04:29.944866 15210 consensus_queue.cc:1059] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:04:30.025154  8369 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:04:30.040644  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 80054.1us
W20260709 13:04:30.040547  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 111762us
I20260709 13:04:30.035497  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2737) took 1023 ms. Trace:
I20260709 13:04:29.982693 15216 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:30.044694  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 96304.8us
I20260709 13:04:29.978420 15223 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:30.042152  8253 rpcz_store.cc:276] 0709 13:04:29.011865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.106229 (+ 94364us) service_pool.cc:224] Handling call
0709 13:04:29.449515 (+343286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:29.666637 (+217122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.034944 (+368307us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b540026cd50. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:30.035397 (+   453us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":724867456}
I20260709 13:04:30.051549  8298 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:30.027683 15190 raft_consensus.cc:3055] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:30.027000 15217 raft_consensus.cc:3055] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:30.025241 15208 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:30.061582  8369 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:30.044240 15161 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.026223 15224 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.044360 15157 raft_consensus.cc:2764] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:04:30.060834  8220 leader_election.cc:400] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 24. Current term is 25.
I20260709 13:04:29.991448 14758 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:29.992081 14967 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:30.059352 15190 raft_consensus.cc:740] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:30.074977  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:30.076344  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:30.093082  8327 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:30.084805  8324 raft_consensus.cc:1240] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 14. Current term is 15. Ops: []
W20260709 13:04:30.094502 15220 log.cc:927] Time spent T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:04:30.091253 15222 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.095616 15222 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:30.084911 15093 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:29.981011 15159 consensus_queue.cc:237] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:30.100358 15212 log.cc:927] Time spent T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.688s	user 0.002s	sys 0.000s
I20260709 13:04:30.061651 15148 consensus_queue.cc:1059] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:04:30.101193 15148 raft_consensus.cc:3055] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:30.101672 15148 raft_consensus.cc:740] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:30.102303 14987 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:30.102473 15148 consensus_queue.cc:260] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:30.103705 15148 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:30.086333 15177 consensus_queue.cc:1048] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:30.086594  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:30.086503  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:30.105441  8328 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
I20260709 13:04:30.088538 15224 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:30.071038 15215 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:30.103623  8369 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:30.109146 15215 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.109607 15215 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:30.109762 14898 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:30.060600 15217 raft_consensus.cc:740] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:30.110157  8369 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:30.110884 14967 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:30.111079 15217 consensus_queue.cc:260] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:30.112531 15217 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:30.075268 15161 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:30.073617 15130 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:30.073895 14992 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:30.114773 14992 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:30.115527 14992 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:30.115840  8369 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:30.118180 15190 consensus_queue.cc:260] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:30.115888 14992 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:30.108320 15224 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:30.119385 14992 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:30.119673 15190 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:30.120764 14856 raft_consensus.cc:3055] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:30.121178 14856 raft_consensus.cc:740] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:30.122749 14990 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:30.123286 14990 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:30.122826 14856 consensus_queue.cc:260] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:30.124584 14856 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:30.108392 14825 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:30.123711 14990 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:30.133657  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:30.134347  8332 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:04:30.135250  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:30.135959  8332 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 22: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:04:30.131194  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 86871.8us
I20260709 13:04:30.137871  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:30.121122  8369 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:30.139228  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:30.143848  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:30.146451  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:30.151186  8972 leader_election.cc:400] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 17. Current term is 18.
W20260709 13:04:30.201781  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 84677.1us
I20260709 13:04:30.202759  8369 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:30.204209  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:30.206763  8972 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:30.209767  8325 raft_consensus.cc:2368] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 21. Current term is 22.
I20260709 13:04:30.210397  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:30.213263  8327 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:30.212267  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:30.217185  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:30.114799 15161 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:30.132458 15222 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:30.230718  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 150706us
I20260709 13:04:30.233373  8220 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:30.234653  8220 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:30.234577 14868 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:30.235666 15142 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:30.236308  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:30.236865  8326 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:30.235297 15139 consensus_queue.cc:1059] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:30.237849  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:30.238595  8327 raft_consensus.cc:2376] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 26. Re-sending same reply.
I20260709 13:04:30.222097 15132 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:30.244133  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:30.245544  8327 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:30.248749  8220 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:30.252583  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:30.253897  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:30.254431  8327 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:30.257592 14758 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:30.258176 14758 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:30.242282 15157 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:30.267271  8326 raft_consensus.cc:2468] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:04:30.268653  8972 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:30.269601 15233 raft_consensus.cc:2804] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:30.270197 15233 raft_consensus.cc:697] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 17 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:30.270138  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:30.271425 15233 consensus_queue.cc:237] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:30.273169 15216 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:30.274417  8331 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:30.236817 14882 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:30.278124  8327 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:30.235349 14868 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:30.245733 15132 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.241910 15142 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:30.251654 15139 consensus_queue.cc:1243] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer ff7a6fe533e94c1bae555b4cae427728 log is divergent from this leader: its last log entry 13.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:04:30.247241 15181 raft_consensus.cc:3055] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:04:30.278594 15195 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.000s	sys 0.025s
I20260709 13:04:30.296813  8327 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:04:30.293313  8220 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:30.298373  8220 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:30.286725 15066 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:30.302806  8326 raft_consensus.cc:1275] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:04:30.303187  8326 pending_rounds.cc:85] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 8
I20260709 13:04:30.303411  8326 pending_rounds.cc:107] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:04:30.303720  8326 raft_consensus.cc:2889] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:30.304653  8220 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:30.301582  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 98926.2us
I20260709 13:04:30.286232 14868 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:30.284052 15178 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:30.286082 15132 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:04:30.297506 15195 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.001s	sys 0.025s
I20260709 13:04:30.296133 15181 raft_consensus.cc:740] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:30.294720 15142 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:30.321206 15207 log.cc:927] Time spent T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.816s	user 0.002s	sys 0.004s
I20260709 13:04:30.322572 15173 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:30.302759 15066 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:30.333892 15066 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:30.334704 15177 raft_consensus.cc:3055] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:30.335120 15177 raft_consensus.cc:740] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:30.335913 15177 consensus_queue.cc:260] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:30.337076 15177 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:30.309371 14898 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:30.309794 15139 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:04:30.299947 15223 raft_consensus.cc:697] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:30.325726 15216 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:30.329805 15138 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.017s	sys 0.032s
I20260709 13:04:30.345077  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:30.354713 15245 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:30.355309  8369 leader_election.cc:400] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 21. Current term is 22.
I20260709 13:04:30.355722  8369 leader_election.cc:403] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:04:30.356178 15245 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 21. Current term is 22.
I20260709 13:04:30.357620 15238 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:30.358150 15238 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.358515 15238 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:04:30.351866 15138 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.018s	sys 0.032s
I20260709 13:04:30.367877 15157 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:30.374311 15066 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:30.353652 14825 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:30.386035  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.389812 15215 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:30.392112 15215 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:30.374161 15058 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:30.376027 14992 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:30.377641 15181 consensus_queue.cc:260] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:30.381932 15148 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:30.384194 14868 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:30.397766  8328 raft_consensus.cc:2368] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 25. Current term is 26.
I20260709 13:04:30.425282  8369 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:30.502803  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 80491.6us
I20260709 13:04:30.503356 14868 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:30.504091 14868 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:30.504196  8369 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:30.504673 14868 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:30.497750 15163 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.499446 15066 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:30.352401 15139 raft_consensus.cc:697] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 26 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:30.598605 15139 consensus_queue.cc:237] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:30.602615 15156 raft_consensus.cc:2243] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.606552  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:30.607399  8402 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
I20260709 13:04:30.607851  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:30.608455  8397 raft_consensus.cc:1813] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Handling vote request from an unknown peer ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:30.608933  8397 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 16: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:30.520689  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:30.609977  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:30.610695  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:30.611294  8325 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 13:04:30.613015 15230 log.cc:927] Time spent T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.513s	user 0.000s	sys 0.002s
I20260709 13:04:30.614547  8325 raft_consensus.cc:1240] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 17. Current term is 18. Ops: []
W20260709 13:04:30.541049  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.535102  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:30.505867 15243 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:30.617911 15243 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:30.618251 15243 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:30.618633 15243 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:30.619499 15155 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:30.553193 15128 raft_consensus.cc:3055] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:30.621626 15128 raft_consensus.cc:740] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:30.622750 15128 consensus_queue.cc:260] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:30.623196 15163 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:30.624401 15128 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:30.517805  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:30.627573  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:30.627904  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.510506 15181 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:30.520494 15058 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:30.549180 15215 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:30.527341 15237 log.cc:927] Time spent T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.294s	user 0.001s	sys 0.000s
I20260709 13:04:30.643172  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:30.535785 14825 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.507479 14992 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:30.515965 15211 log.cc:927] Time spent T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 1.110s	user 0.006s	sys 0.005s
W20260709 13:04:30.535122  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.660760  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:30.516773  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3466) took 1164 ms. Trace:
I20260709 13:04:30.536463 14662 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:30.523674 14882 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:04:30.535287 15242 log.cc:927] Time spent T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:04:30.509509 15157 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:04:30.589250  8220 leader_election.cc:336] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.563s (SENT)
I20260709 13:04:30.667239  8220 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:30.608620  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:30.634292  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 130189us
I20260709 13:04:30.668406  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:04:30.636826  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.519037  8323 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:30.640540  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:30.500643 15148 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:30.645725  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:30.651935  8399 raft_consensus.cc:1240] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:04:30.630896 14967 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.661404  8400 rpcz_store.cc:276] 0709 13:04:29.352372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.354394 (+  2022us) service_pool.cc:224] Handling call
0709 13:04:29.405950 (+ 51556us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.667945 (+261995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.516496 (+848551us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b54002be654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:30.516703 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":598256896}
I20260709 13:04:30.661106  8972 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:30.669397  8369 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:30.671380  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:30.655754 15138 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.678740  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:30.661820 15159 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.016s	sys 0.028s
I20260709 13:04:30.652899 14898 raft_consensus.cc:697] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:30.684024  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.685045  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:30.666875 14882 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:30.690688  8400 raft_consensus.cc:1240] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:04:30.686782  8404 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:04:30.663761 15034 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:30.669533 14662 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:30.688830 15190 consensus_meta.cc:287] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.569s	user 0.008s	sys 0.004s
I20260709 13:04:30.689131 15205 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:04:30.684947 15159 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.016s	sys 0.028s
I20260709 13:04:30.673714 15163 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:30.651489 15215 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:30.708933 15148 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:30.709930  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:30.710075  8222 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:30.710559  8331 raft_consensus.cc:1240] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:30.713575 14992 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:30.713837 15157 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.714087 14992 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.714514 15157 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:30.714558 14992 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:30.700285 15054 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:30.702816 15205 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:30.720793 15205 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:30.722734 14958 raft_consensus.cc:3055] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:30.723187 14958 raft_consensus.cc:740] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:30.724155 14958 consensus_queue.cc:260] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:30.681722 14967 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:30.726598 14992 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:30.727375 14967 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:30.700767 15034 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:30.729853  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:30.728515 14662 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:30.680770 15218 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:30.659644 14825 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:30.703334  8329 raft_consensus.cc:2410] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:04:30.689554 15156 raft_consensus.cc:2272] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.730901  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:30.732645  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:30.699541  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:30.725629 14958 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:30.736754  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:30.691768 15245 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:30.747274  8331 raft_consensus.cc:1240] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:30.716763 15157 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:30.743978  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 44484.8us
I20260709 13:04:30.754992  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 1087) took 1716 ms. Trace:
I20260709 13:04:30.755352  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:30.756151  8325 raft_consensus.cc:2393] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 22: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:30.734264 15148 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:30.762636 14871 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:30.762861 15148 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:30.755360  8330 rpcz_store.cc:276] 0709 13:04:29.030478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.032757 (+  2279us) service_pool.cc:224] Handling call
0709 13:04:29.042854 (+ 10097us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.820569 (+777715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.746599 (+926030us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b540037c154. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:30.746813 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1073537408}
I20260709 13:04:30.766342 15156 ts_tablet_manager.cc:1916] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:30.766472  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:30.762701 14898 consensus_queue.cc:237] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:30.743700 15171 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:30.690223 15138 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:30.769654 14950 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:04:30.770809 15205 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:30.771508 15138 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:30.744764 15218 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:30.772967 15205 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:30.774035 15218 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:30.774506  8332 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:04:30.668205 15223 consensus_queue.cc:237] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:30.775695 14825 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:30.759262 15066 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.765230 15148 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:30.782379 15066 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:30.783025 15174 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:30.784030  8405 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 3457) took 1511 ms (client timeout 1579 ms). Trace:
W20260709 13:04:30.784420  8405 rpcz_store.cc:269] 0709 13:04:29.272836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.274049 (+  1213us) service_pool.cc:224] Handling call
0709 13:04:30.783839 (+1509790us) spinlock_profiling.cc:243] Waited 470 ms on lock 0x7b54001cc854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:30.783954 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1080223616}
I20260709 13:04:30.752104 15054 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:30.787936  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:30.745589 15159 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.763206 14868 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:30.789265 15159 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:30.791546 15239 log.cc:927] Time spent T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.557s	user 0.005s	sys 0.021s
I20260709 13:04:30.814551  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:30.801553  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:30.811411  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:30.794322  8404 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:30.738794 15247 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:30.806252  9000 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:04:30.791608 15159 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:30.845774 14986 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:30.759016 14992 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:30.851136  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:30.814030  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:30.843043  8222 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:30.850268 14986 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:30.853868  9009 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 17 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17. Ops: []
W20260709 13:04:30.831601  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 242400us
I20260709 13:04:30.858542  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:30.861624  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.774981 15034 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:30.742857 15132 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:30.866210  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:30.892037  8404 raft_consensus.cc:2393] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 20: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:30.814922 15245 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:30.799628 14987 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:30.866971  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.868628  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:30.785308 15066 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:30.800675 15215 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:30.870967 15171 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:30.871824  8222 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:30.905905 15142 consensus_meta.cc:287] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.563s	user 0.008s	sys 0.001s
I20260709 13:04:30.916983  8397 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:30.917484  9007 raft_consensus.cc:2393] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 24: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:30.877871  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:30.841037 15247 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:04:30.909940 15217 consensus_meta.cc:287] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.797s	user 0.004s	sys 0.007s
I20260709 13:04:30.880445  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:30.852843 15205 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.930228 15205 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:30.931070  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:30.932334 15205 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:30.932483  8401 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:04:30.938519 14986 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:30.940659 15253 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:04:30.940946  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 69211.8us
I20260709 13:04:30.922982 15142 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:30.943406  8222 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:30.944754 15142 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:30.845556 14868 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:30.945761 15142 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:30.945538 14868 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:30.946283 15142 raft_consensus.cc:697] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:30.947242 15142 consensus_queue.cc:237] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:30.947851 14868 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:30.948491 14868 consensus_queue.cc:1059] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:04:30.801615 14950 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.069s
I20260709 13:04:30.885779  9005 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:30.889736  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:30.886072 15054 consensus_queue.cc:1059] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
W20260709 13:04:30.903683 15195 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.008s	sys 0.005s
I20260709 13:04:30.911590  8399 raft_consensus.cc:1240] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:04:30.918500  9001 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:30.878813  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:30.920746  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:30.921494  9004 raft_consensus.cc:1240] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:30.961706  8405 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:30.923039  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:30.962504  8397 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:30.923285  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:30.941315 15253 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:30.865979 14990 consensus_meta.cc:287] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.709s	user 0.009s	sys 0.001s
I20260709 13:04:30.942286 15171 raft_consensus.cc:697] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:30.902443  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:30.951519  8328 raft_consensus.cc:2410] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:04:30.949188 14868 consensus_queue.cc:1243] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 22.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 13:04:30.964015  8404 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:04:30.888835 15159 consensus_queue.cc:1059] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:04:30.922552 15247 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:30.910518 14967 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:04:30.970803  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:30.981222  9001 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 26.
I20260709 13:04:30.976024 15243 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:30.977355  9000 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 13:04:30.985841  9000 pending_rounds.cc:85] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: Aborting all ops after (but not including) 10
I20260709 13:04:30.978251  8403 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:30.986089  9000 pending_rounds.cc:107] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 13:04:30.943436 15249 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:30.982832  8298 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:30.965583 15195 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.009s	sys 0.006s
I20260709 13:04:30.983219  8405 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:04:30.986353  9000 raft_consensus.cc:2889] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:30.986477  8397 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
W20260709 13:04:30.981259  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 251473us
I20260709 13:04:30.990051  8398 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:04:30.914054 14987 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:31.004779  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:31.006075  8404 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
I20260709 13:04:31.006297  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:31.023072  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:31.024106  8402 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:04:30.991063 15249 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:31.026093 15249 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:31.019939 14958 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:04:31.027042 15132 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 24 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:31.027335 14958 raft_consensus.cc:697] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 26 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:31.028352  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:31.029071  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:31.029469  8405 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:04:30.950973 14882 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:31.028291  8406 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:31.028291  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:30.969233 15217 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 13:04:31.030578  8398 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:04:31.035597  9007 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:04:31.036032  8402 raft_consensus.cc:2393] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 22: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:31.028529  8369 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:31.040802  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:31.044405  8401 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:31.045827 14958 consensus_queue.cc:237] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:31.046823  8403 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 22.
I20260709 13:04:31.047727  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:31.048276  8403 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:04:31.049757  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:31.051898 14882 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 26 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:30.970696 14990 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:31.052718 15251 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:31.053807 15171 consensus_queue.cc:237] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:31.057816  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:31.058799 14990 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:31.059329 15163 raft_consensus.cc:3055] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:31.059729 15163 raft_consensus.cc:740] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:31.060526 15163 consensus_queue.cc:260] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:31.061970  8403 raft_consensus.cc:1240] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:31.062695  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:04:31.024935 15238 consensus_meta.cc:287] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.664s	user 0.011s	sys 0.000s
I20260709 13:04:31.022487 15159 consensus_queue.cc:1243] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 17.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:04:31.070106  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:31.075399  8405 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 22.
I20260709 13:04:31.076709  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:31.058879 14825 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:04:31.077742  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:31.077865 15243 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:31.078783  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 50347.3us
I20260709 13:04:31.139557  8369 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:31.004359 15157 consensus_queue.cc:1059] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.071s
I20260709 13:04:31.147593  8398 raft_consensus.cc:2410] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
W20260709 13:04:31.136338  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 55309.2us
I20260709 13:04:31.153592  8297 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:31.140802  8974 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:31.158812  8297 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:31.074640 15217 consensus_queue.cc:1243] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:04:31.162424  8253 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:04:31.160033  8297 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:31.157074  8257 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:04:31.164283  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 263481us
I20260709 13:04:31.167860  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:31.169087  8369 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:31.141637  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 263525us
I20260709 13:04:31.164599 15174 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:31.174003 14662 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:31.174578 14992 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:31.175688  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:31.176966  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.161901 15218 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:31.174579 14662 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:31.178490 14662 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:31.179090 15256 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:31.162475 14986 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:31.183235 14986 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:31.183451  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:31.184351  8402 raft_consensus.cc:2393] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 20: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:31.136871 14882 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.185935 14758 raft_consensus.cc:3055] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:31.186369 14882 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:31.186381 14758 raft_consensus.cc:740] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:31.188583 14882 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:31.188174 14758 consensus_queue.cc:260] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:31.191432 14758 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:31.183705 14986 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:31.194255  9008 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 17 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:31.193081  8297 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:31.189175 14882 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.198776 15157 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:04:31.166265 15177 consensus_meta.cc:287] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.829s	user 0.010s	sys 0.001s
I20260709 13:04:31.200161  8297 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:31.202934 15173 consensus_queue.cc:1059] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:31.202850 15066 consensus_queue.cc:1059] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:04:31.203348 15128 consensus_meta.cc:287] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.578s	user 0.011s	sys 0.003s
I20260709 13:04:31.165539 15233 raft_consensus.cc:3055] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:31.211550 15233 raft_consensus.cc:740] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:31.211807 15066 raft_consensus.cc:3055] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:31.212270 15066 raft_consensus.cc:740] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:31.212422 15233 consensus_queue.cc:260] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:31.213161 15066 consensus_queue.cc:260] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:31.214341  8401 raft_consensus.cc:3055] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:31.214715  8401 raft_consensus.cc:740] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:31.214656 15066 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:31.215595  8401 consensus_queue.cc:260] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:31.216872  8401 raft_consensus.cc:3060] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:31.136813 15238 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:31.219285 15238 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:31.219712  8251 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:31.221868 15233 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:31.225559 14987 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:31.199630 15215 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:31.199664 14950 consensus_queue.cc:1059] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 13:04:31.208274  8220 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:31.210637 15253 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:31.210794  8328 raft_consensus.cc:1275] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:04:31.210968 15173 raft_consensus.cc:3055] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:31.230121 15173 raft_consensus.cc:740] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:31.230549 15253 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:31.231045 15173 consensus_queue.cc:260] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:31.232336 15173 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:31.211218  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.217320 15195 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:31.234342 15155 consensus_meta.cc:287] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.614s	user 0.011s	sys 0.002s
I20260709 13:04:31.234612  8220 leader_election.cc:400] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 pre-election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 25. Current term is 26.
I20260709 13:04:31.235024  8220 leader_election.cc:403] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:31.235309  8324 raft_consensus.cc:2410] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:04:31.235635 15138 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Advancing to term 26
I20260709 13:04:31.236347  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 3277) took 1105 ms. Trace:
I20260709 13:04:31.236726  8324 rpcz_store.cc:276] 0709 13:04:30.130576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.130704 (+   128us) service_pool.cc:224] Handling call
0709 13:04:31.236161 (+1105457us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540037c654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:31.236296 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2507829504}
I20260709 13:04:31.228184 14950 raft_consensus.cc:3055] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:31.237735 14950 raft_consensus.cc:740] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:04:31.239030 15181 consensus_meta.cc:287] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.601s	user 0.005s	sys 0.005s
I20260709 13:04:31.238785 14950 consensus_queue.cc:260] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:31.239892  8326 raft_consensus.cc:2410] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:04:31.240464 14950 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:31.158365 15132 consensus_queue.cc:1059] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.051s
I20260709 13:04:31.242285 15132 raft_consensus.cc:3055] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:31.242710 15132 raft_consensus.cc:740] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:31.243772 15132 consensus_queue.cc:260] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:31.245401 15132 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:31.199478 15174 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:04:31.209017 14882 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:31.248243 14882 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:31.250499  9008 pending_rounds.cc:85] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e: Aborting all ops after (but not including) 13
I20260709 13:04:31.250865  9008 pending_rounds.cc:107] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 13:04:31.251117  9008 raft_consensus.cc:2889] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:31.255429 15174 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.256040 15174 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:31.258296 15174 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:31.260157  8401 raft_consensus.cc:1275] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:04:31.260749 15245 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 13:04:31.262317  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:31.263031  9001 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:31.264156  8298 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:31.264854  8257 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 23. Current term is 24. Ops: []
I20260709 13:04:31.265267 15181 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:31.265738 15181 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:31.266101 15181 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:31.266610  8257 raft_consensus.cc:1240] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:31.269801  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:31.270519  8251 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:31.195833  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:31.272914  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:31.272965 14662 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:31.274751  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:31.225044 15157 raft_consensus.cc:697] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:31.275974  8326 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:31.276772  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:31.276538 15157 consensus_queue.cc:237] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:31.275368  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:31.277737 15247 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:31.278514  8329 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:31.279788 15247 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:31.269801  8223 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:31.286852 15173 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:31.288051  8329 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 26.
W20260709 13:04:31.295166  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 60717.5us
I20260709 13:04:31.295961  8223 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:31.297473  8223 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:31.297806  8326 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 24.
I20260709 13:04:31.288827  8251 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
W20260709 13:04:31.289795  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 94051.4us
I20260709 13:04:31.245227 15249 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:31.301334  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:31.306432  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 361512us
I20260709 13:04:31.309747  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:31.312986  8329 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
I20260709 13:04:31.305868  8974 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:31.316061  8297 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:31.316458  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:31.317204  8329 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:04:31.320497  8223 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:31.322093  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:31.315981  8972 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:31.320542  8325 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:31.295571 15245 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:31.326179  8223 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:31.327404  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:31.306201 15247 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:31.303529 15174 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.308171 15215 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:31.328449  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:31.337389  8222 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:31.338474  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:31.340029  8326 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 13:04:31.328536  8297 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:31.317679 14992 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:31.343318  8325 raft_consensus.cc:1240] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:31.318222 15128 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:31.327891  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:31.326653 15264 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:31.350111  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:31.333416  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:31.348980  8256 raft_consensus.cc:1240] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:31.342474  8297 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:31.376237  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:31.377205  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:31.378330  8405 raft_consensus.cc:1240] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:31.378839  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:31.328064 15173 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:31.336475 15139 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:04:31.334131 15054 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:04:31.402356  8324 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:31.453871  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:31.325675 14871 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 13:04:31.402603  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:31.454805  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:31.456307  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:31.462574  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:31.465673  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:31.405071  8400 raft_consensus.cc:1275] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:31.342295 15245 raft_consensus.cc:697] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:31.337894 15238 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.135s
I20260709 13:04:31.341714 15156 ts_tablet_manager.cc:1929] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 18.10
I20260709 13:04:31.341594 15148 consensus_queue.cc:1059] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.166s
I20260709 13:04:31.343184 15174 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:31.402817  8403 raft_consensus.cc:2393] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 18: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:04:31.561993  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:31.418111  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 75707.1us
I20260709 13:04:31.566568  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:31.566722  8297 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:31.339635 14958 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:31.480494  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:31.569046  8252 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:04:31.576114  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:31.576810  8252 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:31.568068  8256 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:31.371951 15247 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:31.328147 15266 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:31.401623 15195 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.517639  8249 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 13:04:31.602578  8249 pending_rounds.cc:85] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 15
I20260709 13:04:31.602988  8249 pending_rounds.cc:107] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 13:04:31.603334  8249 raft_consensus.cc:2889] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:31.605460  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:31.372020 15225 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 1.254s	user 0.011s	sys 0.041s
I20260709 13:04:31.379786 14825 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:31.596719 15054 raft_consensus.cc:697] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 26 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:31.590574 15247 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:31.380466 14990 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:31.572304  8254 raft_consensus.cc:2410] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 15 because replica has last-logged OpId of term: 14 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 14.
I20260709 13:04:31.406399 14882 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:31.571890 15171 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.013s	sys 0.009s
I20260709 13:04:31.605732  9001 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:04:31.568006  8297 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:31.595758 15034 consensus_queue.cc:1048] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:04:31.503114 14898 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.371636 14986 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:31.583537 15218 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:31.405679 15128 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:31.406598 14967 consensus_queue.cc:1059] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.201s
I20260709 13:04:31.401908 15138 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 25. Current term is 26.
W20260709 13:04:31.608685 15225 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.516s	user 0.015s	sys 0.051s
I20260709 13:04:31.596328 14958 raft_consensus.cc:697] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:31.608518 15156 log.cc:1199] T 81804f59e3124d27998ea80fc1c36663 P 8c42259d7c4b463a9f485f42c320c81b: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-0-root/wals/81804f59e3124d27998ea80fc1c36663
I20260709 13:04:31.583343 15252 consensus_queue.cc:1059] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
W20260709 13:04:31.678180  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 110246us
I20260709 13:04:31.470511 15215 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:31.804903 15215 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:31.805392 15215 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:31.806066 15278 log.cc:927] Time spent T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 13:04:31.806393 14958 consensus_queue.cc:237] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:31.679056  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 351489us
I20260709 13:04:31.809145  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:31.679325  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:31.810607  8251 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 26.
I20260709 13:04:31.613191 14825 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:31.624704 15247 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:31.689687  8256 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:31.696740  8252 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 22.
W20260709 13:04:31.706549  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:31.678234 15171 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.014s	sys 0.009s
I20260709 13:04:31.609946 15195 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:31.720477  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 215551us
I20260709 13:04:31.768929  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:31.830905  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:31.683943 15217 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:31.678295 14662 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:31.774634  9000 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:04:31.782313  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:31.719082 15249 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:31.705964 15218 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:04:31.793452  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:31.795356  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.797873  8297 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:31.791414 14992 raft_consensus.cc:697] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:04:31.800732 15275 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.236s	user 0.000s	sys 0.000s
I20260709 13:04:31.802151 15156 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51208 (request call id 0) took 4144 ms. Trace:
W20260709 13:04:31.803256 15271 log.cc:927] Time spent T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.188s	user 0.000s	sys 0.000s
I20260709 13:04:31.803516 15128 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:31.816856  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.827407  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:31.682230 14882 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:31.685865 14898 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:31.828197  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:31.832178  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:31.876847  8256 raft_consensus.cc:2393] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 19: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:31.825935 14825 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:31.783911 15258 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
W20260709 13:04:31.866277  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 68467.9us
W20260709 13:04:31.882606  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 413354us
I20260709 13:04:31.887005  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:31.848580 15238 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.849756 15217 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:31.864004 15218 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:31.892756  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:31.891659  8369 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:31.867144 15132 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:31.912194  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:31.902915  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:31.913003  8252 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:31.904208  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 115382us
I20260709 13:04:31.907276  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:31.906862  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:31.904752  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 89178.8us
I20260709 13:04:31.916220  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:31.916254  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:31.921403  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: true dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:31.921447  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:31.922574  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:31.922552  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:31.923828  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:31.923826  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:31.925063  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:31.925104  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:31.926385  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:31.926602  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:31.927299  8255 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 23: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:31.927904  8256 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:31.928351  8255 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:04:31.932520  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:31.933238  8255 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
I20260709 13:04:31.899163 15163 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:31.939275 14990 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.939556  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:04:31.939651  8248 consensus_meta.cc:287] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.557s	user 0.009s	sys 0.003s
I20260709 13:04:31.940361  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:31.907390 15217 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:31.935803  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:31.936642  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 106558us
I20260709 13:04:31.942839  8255 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:31.940744  8248 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:04:31.935580  8256 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:31.939870 14990 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:31.892453 15128 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:04:31.944486  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 1106) took 1127 ms. Trace:
I20260709 13:04:32.038218  8331 rpcz_store.cc:276] 0709 13:04:30.817124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.822419 (+  5295us) service_pool.cc:224] Handling call
0709 13:04:30.846549 (+ 24130us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:30.846974 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:30.847013 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:30.869332 (+ 22319us) log.cc:844] Serialized 26 byte log entry
0709 13:04:31.294029 (+424697us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:31.294078 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.294149 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:31.944272 (+650123us) raft_consensus.cc:1685] finished
0709 13:04:31.944291 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:31.944420 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4982,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":16123,"thread_start_us":374425,"threads_started":1,"wal-append.queue_time_us":624483}
I20260709 13:04:31.944491  9005 raft_consensus.cc:1275] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:04:31.955137  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:31.957617  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:31.911944 15159 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:04:32.018086  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 127208us
I20260709 13:04:32.175781  8251 raft_consensus.cc:2393] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 24: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:32.092752  8222 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:32.184528  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 91863.7us
I20260709 13:04:32.141908 15034 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.868104 15156 rpcz_store.cc:276] 0709 13:04:27.682486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.730983 (+ 48497us) service_pool.cc:224] Handling call
0709 13:04:28.400484 (+669501us) ts_tablet_manager.cc:1151] Deleting tablet 81804f59e3124d27998ea80fc1c36663
0709 13:04:28.400582 (+    98us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:04:29.203318 (+802736us) spinlock_profiling.cc:243] Waited 534 ms on lock 0x7b54001301d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c6ff9 00007f882d7d4482 00007f882d8052d0 00007f882d7eb83d 00007f882d7eb7e9 00007f882d7eb779 00007f882d7eb741 00007f882d7ea96d 000000000038f354 000000000038f199
0709 13:04:31.203332 (+2000014us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:31.333708 (+130376us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:31.802049 (+468341us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1417598,"spinlock_wait_cycles":1227690880,"tablet-delete.queue_time_us":668246,"thread_start_us":379282,"threads_started":1}
I20260709 13:04:32.185343  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:32.202445 15290 log.cc:927] Time spent T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:04:32.206035 14990 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:32.133378 15215 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:32.145401  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 217238us
W20260709 13:04:31.961371 14758 consensus_meta.cc:287] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.769s	user 0.011s	sys 0.005s
W20260709 13:04:31.964437 15284 log.cc:927] Time spent T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:04:31.959378 14986 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:31.942299 14950 consensus_meta.cc:287] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.701s	user 0.014s	sys 0.000s
I20260709 13:04:32.150703  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:31.913532 15155 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.910533 15238 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:32.147972  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:31.722898 15245 consensus_queue.cc:237] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:32.124460  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:32.008601  8252 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:31.832538 15054 consensus_queue.cc:237] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:31.963472 14868 raft_consensus.cc:3055] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:31.956674 14662 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:04:32.171489  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.011979  8254 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 22.
I20260709 13:04:31.927191 15253 raft_consensus.cc:3055] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:32.162580  8405 raft_consensus.cc:1275] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:04:31.952961 15181 consensus_meta.cc:287] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.686s	user 0.009s	sys 0.001s
I20260709 13:04:32.091703 15174 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:32.133945 15139 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.172923 15302 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.184713  8402 raft_consensus.cc:1240] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:04:31.945699 15256 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:32.107501  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:32.191001  8222 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:32.233343  8256 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:32.232900  8248 raft_consensus.cc:2410] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 23 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:04:32.232627  8398 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:32.233203  8253 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:04:32.091427 15299 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.144s
W20260709 13:04:32.214494 15279 log.cc:927] Time spent T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.362s	user 0.002s	sys 0.000s
I20260709 13:04:32.242482  8223 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:32.243922 15253 raft_consensus.cc:740] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:32.234416 14868 raft_consensus.cc:740] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:32.247634 15309 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.195770 15157 raft_consensus.cc:3055] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:04:32.254297  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 63382.4us
I20260709 13:04:32.270728  8249 raft_consensus.cc:2410] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:04:32.188715 15148 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:32.208762 15215 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:32.211603 14882 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:32.192579 15269 log.cc:927] Time spent T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.512s	user 0.000s	sys 0.001s
I20260709 13:04:32.266496  8972 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:32.280434  8223 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:32.253283 15256 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:32.194339 15142 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 13:04:32.252693 15181 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:32.268635 15174 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:32.290753  8328 raft_consensus.cc:2368] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 20. Current term is 21.
I20260709 13:04:32.212559 14898 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:32.288005  8972 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:32.244854 14990 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:32.143988 14992 consensus_queue.cc:237] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:32.293205  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:32.264457 15171 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:32.302166 14990 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:32.303643 15238 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:32.290040 15253 consensus_queue.cc:260] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:32.304426 14990 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:32.304765 15253 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:32.268442 15302 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:32.306903 15302 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:32.263379 15139 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:32.309940 15139 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:32.249392 15155 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:32.279078 15294 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:04:32.313704 15155 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:32.265166 15221 consensus_queue.cc:1059] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.049s
I20260709 13:04:32.285336 15249 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:32.274850 14986 consensus_queue.cc:1059] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:04:32.187175 15034 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:32.298516  9007 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 26.
I20260709 13:04:32.323244  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 1178) took 1055 ms. Trace:
I20260709 13:04:32.323818  8257 rpcz_store.cc:276] 0709 13:04:31.267607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.267709 (+   102us) service_pool.cc:224] Handling call
0709 13:04:31.268003 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:31.268333 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:31.268361 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:31.268819 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 13:04:31.621531 (+352712us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:31.621642 (+   111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.621738 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:32.322924 (+701186us) raft_consensus.cc:1685] finished
0709 13:04:32.322954 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.323132 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"thread_start_us":352352,"threads_started":1,"wal-append.queue_time_us":596594}
W20260709 13:04:32.294550  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 175744us
I20260709 13:04:32.330691  8295 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:32.279734 15157 raft_consensus.cc:740] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:04:32.340209  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.290684 15256 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:32.294947 15128 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:04:32.360472  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 184305us
I20260709 13:04:32.290328 15174 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:32.252638 15289 consensus_queue.cc:1059] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 13:04:32.329025 15215 consensus_queue.cc:1059] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 13:04:32.287190 15309 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:32.327632 14882 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:32.400264  8295 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:32.410125  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 502647us
W20260709 13:04:32.485196  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 191919us
I20260709 13:04:32.430193  9000 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 26 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 25. Current term is 26. Ops: []
I20260709 13:04:32.415510  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:32.428823  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.496515  8255 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 16.
W20260709 13:04:32.487748  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 207587us
I20260709 13:04:32.497915  8223 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:32.328032 15304 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.327867 14898 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:32.360206 15308 log.cc:927] Time spent T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.174s	user 0.006s	sys 0.000s
I20260709 13:04:32.501451  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:32.499825  8406 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:32.503576  8223 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:32.453838 15277 log.cc:927] Time spent T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.602s	user 0.002s	sys 0.001s
I20260709 13:04:32.446352 15307 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.069s
I20260709 13:04:32.408547 15132 raft_consensus.cc:697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 24 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:32.360085 15195 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:32.522168  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:32.513281 15304 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:32.560707  8402 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:32.406147 15128 raft_consensus.cc:697] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:32.494807 14662 raft_consensus.cc:697] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 22 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:04:32.557325  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.456796 15296 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:32.426611 14871 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:32.441623 15305 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:32.430830 15306 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:04:32.558841  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 160657us
W20260709 13:04:32.559531  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 447914us
I20260709 13:04:32.559978  8398 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:04:32.556898  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:32.554169  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 50735.6us
I20260709 13:04:32.332819 14825 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:32.603494  8222 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:32.517920 15217 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:32.602200  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 45366.4us
I20260709 13:04:32.604713  8222 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:32.608467  8369 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:32.605623  8223 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:32.492540 15157 consensus_queue.cc:260] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:32.616600  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:32.615660  8255 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:04:32.620364  8972 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:32.566905  8295 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:32.561784 15315 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.562366 14898 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:32.623255  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:32.624585  8369 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:32.615598 15306 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:32.627130 15252 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:32.631423  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:32.632292  8331 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 26.
I20260709 13:04:32.634263  8324 raft_consensus.cc:1240] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:32.565490 15296 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:32.637552  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:32.639518  9004 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 13:04:32.639567  8972 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:32.607452 15266 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:32.653100 15303 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.653800 15303 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:32.656383 15303 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:32.657064 15303 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:32.658149 15303 raft_consensus.cc:697] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:32.659297 15303 consensus_queue.cc:237] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:32.614713 14967 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:32.614413 14990 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:32.718619  8402 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 13:04:32.695695  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:32.726576  9001 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:04:32.625021 15139 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:32.696605  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 57122.4us
I20260709 13:04:32.588395 14871 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:32.647668 15296 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:32.648698 15159 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.622849 15294 consensus_queue.cc:1059] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.106s
I20260709 13:04:32.781659  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:32.649309 15258 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.638352 14758 consensus_queue.cc:1059] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.096s
I20260709 13:04:32.588722 15305 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:32.664286 15034 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:32.665594 15266 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.791680 15288 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:32.726636 14990 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.784657 15307 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:04:32.795132  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 228318us
W20260709 13:04:32.793215  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.776266 14967 raft_consensus.cc:697] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:32.790710 15195 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:32.784269 15157 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:32.645941 15221 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:32.784731 15289 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:04:32.804953 15163 consensus_meta.cc:287] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.826s	user 0.010s	sys 0.002s
I20260709 13:04:32.828999 15313 raft_consensus.cc:3055] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:32.829653 15313 raft_consensus.cc:740] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:32.831034 15305 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:32.830783 15313 consensus_queue.cc:260] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:32.712581 14882 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:32.734700 14898 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.070s
I20260709 13:04:32.665867 14958 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.819764  8295 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:32.827023 14871 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:32.820525 15289 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:32.725865 15132 consensus_queue.cc:237] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:32.813777 15294 consensus_queue.cc:1243] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer ff7a6fe533e94c1bae555b4cae427728 log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:04:32.797210 15159 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:32.638136 15315 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:32.807269 15252 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.809101 15266 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:32.804571 15318 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:32.811693 15288 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.775466 15128 consensus_queue.cc:237] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:32.815022 14990 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:32.792011 15321 consensus_queue.cc:1048] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.332s
I20260709 13:04:32.832280 15312 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:32.866902  8295 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:32.835875 14987 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.904392 15306 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:32.729748 14662 consensus_queue.cc:237] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:32.842212 15313 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:32.872936  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.873132  8223 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:32.961263  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:32.982298  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 200751us
I20260709 13:04:32.842618 15034 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:32.804574 15258 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:32.838630 15174 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:32.861507 15289 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:32.832603 15268 consensus_queue.cc:1059] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.315s
I20260709 13:04:32.896549 15304 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:32.897975 15329 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:32.901540 15338 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:32.901968 15310 log.cc:927] Time spent T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.324s	user 0.000s	sys 0.001s
I20260709 13:04:32.819414 15307 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:32.902786 15334 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:04:32.903954 15324 log.cc:927] Time spent T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 13:04:32.904457 14987 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:32.989507 15321 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:32.906104 15322 log.cc:927] Time spent T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 13:04:32.906646 15323 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:32.907712 15282 log.cc:927] Time spent T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.954s	user 0.001s	sys 0.006s
I20260709 13:04:32.909739 15311 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:32.910665 15283 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.720s	user 0.004s	sys 0.003s
I20260709 13:04:32.911859 15315 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:32.913976 15054 raft_consensus.cc:3055] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:32.917652 15243 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:32.918301 15259 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:04:33.043200  8249 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2769) took 1269 ms (client timeout 1516 ms). Trace:
I20260709 13:04:32.885963 15253 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 13:04:32.885488 15288 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:32.834573 15195 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:32.931295  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:04:33.046577  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.046765  8399 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:32.932760  8295 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:32.933840  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 27 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:33.048955  8402 raft_consensus.cc:2410] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 27 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 13:04:32.846127 14882 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:32.984843  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 113101us
I20260709 13:04:32.985456  8403 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
W20260709 13:04:33.048959  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 116274us
I20260709 13:04:33.050808  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:33.051261  8295 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:33.051465  8403 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:33.052349  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:33.049803  8223 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:33.004760  8972 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:32.934641 14825 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:32.979658 15217 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:32.877741 15252 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:33.074384  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:32.852733 14958 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:33.025568  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:33.079879  8404 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
I20260709 13:04:33.080365  8253 raft_consensus.cc:1275] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:04:33.033308  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:32.957858 15303 consensus_queue.cc:1059] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:04:32.919250 15301 raft_consensus.cc:3055] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:04:33.043866  8249 rpcz_store.cc:269] 0709 13:04:31.774312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.827957 (+ 53645us) service_pool.cc:224] Handling call
0709 13:04:33.042910 (+1214953us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b54000762d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:33.043115 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":906129664}
I20260709 13:04:33.026090 15309 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:33.065538 15328 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:33.057075  8256 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:04:33.057396  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:33.068971 15174 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:32.930361 14967 consensus_queue.cc:237] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:33.069240  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 64579.9us
I20260709 13:04:33.016127 15338 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:32.852190  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 27 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:33.084358  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 51117.6us
I20260709 13:04:33.084630  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:33.101452  8250 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:33.101548  8369 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:33.088518  8974 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:33.093219  8329 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:33.094660  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:33.049371 15054 raft_consensus.cc:740] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:33.055594  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:33.065193 15272 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.064656 14898 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:33.058364  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:33.108484  8254 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 24.
I20260709 13:04:33.108233  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 27 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:33.107427  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:33.109406  8329 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 26.
I20260709 13:04:33.078950 15332 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:33.110145  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:04:33.097254 15247 consensus_meta.cc:287] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.248s	user 0.014s	sys 0.000s
I20260709 13:04:33.113184  8399 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:04:33.054329 15243 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:33.095183 15315 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:33.112768  8256 raft_consensus.cc:2376] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18. Re-sending same reply.
I20260709 13:04:33.132875  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:33.116164  8402 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:04:33.026587 15334 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:33.145013  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.142410  9008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 1201) took 1892 ms. Trace:
I20260709 13:04:33.103677 15205 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.108240 14898 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.078227 15329 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.045s
I20260709 13:04:33.113495 15328 raft_consensus.cc:697] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 22 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:33.120270  8974 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:33.095360 15301 raft_consensus.cc:740] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:33.064695 15259 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:33.125118 15337 log.cc:927] Time spent T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 13:04:33.125801 15331 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:33.184962 15331 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:33.187170 15331 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:33.131098 15256 consensus_meta.cc:287] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.733s	user 0.011s	sys 0.000s
I20260709 13:04:33.038156 15323 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:33.191238 15054 consensus_queue.cc:260] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:33.193953 15054 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:33.195086 15323 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:33.138190  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:33.197829  8400 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:04:33.198762  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:33.199378  8400 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:33.148341  9008 rpcz_store.cc:276] 0709 13:04:31.249731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.249863 (+   132us) service_pool.cc:224] Handling call
0709 13:04:31.250252 (+   389us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:04:31.252018 (+  1766us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:31.252057 (+    39us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:04:31.252778 (+   721us) log.cc:844] Serialized 78 byte log entry
0709 13:04:31.680569 (+427791us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:31.768966 (+ 88397us) log.cc:844] Serialized 14 byte log entry
0709 13:04:31.769411 (+   445us) log.cc:844] Serialized 14 byte log entry
0709 13:04:31.772108 (+  2697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.772211 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:32.189869 (+417658us) spinlock_profiling.cc:243] Waited 20.6 ms on lock 0x7b180048aa20. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:04:32.791927 (+602058us) spinlock_profiling.cc:243] Waited 2.13 ms on lock 0x7b180048aa20. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:04:32.909853 (+117926us) spinlock_profiling.cc:243] Waited 4.14 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:04:33.142153 (+232300us) raft_consensus.cc:1685] finished
0709 13:04:33.142176 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:33.142334 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18113,"prepare.queue_time_us":127822,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":98,"raft.queue_time_us":38773,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":9179,"spinlock_wait_cycles":61844352,"thread_start_us":422820,"threads_started":1,"wal-append.queue_time_us":575758}
I20260709 13:04:33.117344  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:33.167505  8972 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:33.114883 14871 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:33.202426 14871 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.202952 14871 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:33.204205 15338 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:33.119179  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:33.205699  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:33.206326  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:33.122292 15218 consensus_meta.cc:287] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.193s	user 0.007s	sys 0.007s
I20260709 13:04:33.207180 15218 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:33.112308 15332 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:33.209198 15332 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:33.114594 15174 raft_consensus.cc:697] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:33.116397  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:33.177947  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 57772.9us
I20260709 13:04:33.176614 14898 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:33.179576  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:33.182145  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:33.114883 15272 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:33.182579  8328 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 1113) took 1910 ms (client timeout 2031 ms). Trace:
W20260709 13:04:33.260802  8328 rpcz_store.cc:269] 0709 13:04:31.272374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.272694 (+   320us) service_pool.cc:224] Handling call
0709 13:04:33.182334 (+1909640us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540014cb54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:33.182510 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2338979584}
I20260709 13:04:33.184026 15334 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:33.261970 15171 consensus_meta.cc:287] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.947s	user 0.011s	sys 0.000s
I20260709 13:04:33.262468 15171 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:33.262889  8326 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 13:04:33.250867 15340 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:33.132999 15247 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:32.924954  8139 catalog_manager.cc:5028] TS 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): tablet 81804f59e3124d27998ea80fc1c36663 (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
W20260709 13:04:33.264417  8139 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 339612us
I20260709 13:04:33.265658  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:33.266319  8398 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 20.
I20260709 13:04:33.267136 14958 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:33.267902 15307 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:33.268540 15258 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:33.184187 15034 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:33.269531 15034 raft_consensus.cc:697] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:33.253834 15301 consensus_queue.cc:260] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:33.270807 15301 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:33.270541 15034 consensus_queue.cc:237] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:33.272087 15272 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:33.209791 15256 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:33.125831 14987 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:33.174541 15205 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:33.275794 15256 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:33.202394  8254 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
I20260709 13:04:33.250437 15195 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:33.276999  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:33.277638  8254 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:04:33.277653 15205 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:33.175119  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:33.279469  8254 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 23. Current term is 24. Ops: []
I20260709 13:04:33.279765  8256 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:33.205109 15328 consensus_queue.cc:237] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:33.172443 15159 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:33.117751  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:33.280864  8252 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
W20260709 13:04:33.281239 15343 log.cc:927] Time spent T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:04:33.282860  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:33.284374  8974 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:33.258062  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:33.285145  8400 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 21.
I20260709 13:04:33.285460  8255 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:04:33.258725 14868 consensus_queue.cc:260] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:33.259850 15174 consensus_queue.cc:237] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:33.182777 15325 log.cc:927] Time spent T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.269s	user 0.000s	sys 0.001s
I20260709 13:04:33.298704 15264 raft_consensus.cc:3055] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:04:33.299152 15264 raft_consensus.cc:740] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:33.300051 15264 consensus_queue.cc:260] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:33.301119 15264 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:33.290205 15252 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:33.277112  9000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 1483) took 1286 ms. Trace:
W20260709 13:04:33.253980  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.180255 14990 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:33.309521  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:33.310500  8400 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:04:33.286615  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:04:33.235625  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.313349 14871 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:33.315753 15128 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:33.316457 15128 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:33.221879  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:33.293531  8327 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
W20260709 13:04:33.232355  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 64996.7us
W20260709 13:04:33.296370  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.318948  9001 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18.
I20260709 13:04:33.302878  9000 rpcz_store.cc:276] 0709 13:04:31.990964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.991163 (+   199us) service_pool.cc:224] Handling call
0709 13:04:31.991574 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:32.501469 (+509895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.276893 (+775424us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b54007198d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:33.277055 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1008067968}
I20260709 13:04:33.296576  8332 raft_consensus.cc:2393] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 15: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:33.320932 15128 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:33.318395 14871 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:33.322420  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:33.327059  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:33.328629  9000 raft_consensus.cc:2368] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 19. Current term is 20.
I20260709 13:04:33.254977 15243 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:33.333220  8403 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 12.
W20260709 13:04:33.342844  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.266340 15259 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:33.295651 15299 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:33.349225  8399 raft_consensus.cc:2393] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 16: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:33.198701 15288 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:33.360476  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:33.362659  9001 raft_consensus.cc:2410] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:04:33.370409  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:33.371286  8255 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 22.
I20260709 13:04:33.373034  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:33.373970  8255 raft_consensus.cc:2393] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 24: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:33.375245  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:33.379086  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:33.358388 15299 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.409899 15299 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:33.359797 14868 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:33.418233 14990 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:33.377974  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:33.425819  8257 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
W20260709 13:04:33.434065  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:33.427140 15266 consensus_meta.cc:287] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.541s	user 0.005s	sys 0.005s
W20260709 13:04:33.452893  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 165457us
I20260709 13:04:33.454380  8295 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:33.457693 15313 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:33.458312 15288 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:33.458240 15313 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.463398  8295 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:33.464304 15313 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:33.456725 15311 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:33.447467 14882 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:33.544873  8372 leader_election.cc:400] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 19. Current term is 20.
I20260709 13:04:33.469159 15328 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:33.545377  8372 leader_election.cc:403] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:33.545543 15328 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.546026 15328 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:33.546664 15258 consensus_queue.cc:1059] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:33.547463 15258 consensus_queue.cc:1243] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 26.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:04:33.547976 15258 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:04:33.551503  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:33.454450 15266 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:33.486769 15295 log.cc:927] Time spent T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 1.318s	user 0.005s	sys 0.000s
W20260709 13:04:33.487902  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.494437  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:33.556480  9004 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:04:33.495164  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:33.495837  9006 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:04:33.498203  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:33.498626 15313 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:33.500255 14898 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:33.501540 15289 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:33.507568  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:33.508284 15296 consensus_meta.cc:287] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.712s	user 0.008s	sys 0.004s
I20260709 13:04:33.508832 15247 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:33.510334  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:33.467497  8295 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:33.513460  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:33.516551  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 415101us
I20260709 13:04:33.579111  8253 raft_consensus.cc:1813] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Handling vote request from an unknown peer ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:33.578979  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 111568us
I20260709 13:04:33.516680  8401 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:33.579399  8369 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:33.517134 15345 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:33.517848 15218 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:33.521780  8974 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:33.523886 14758 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:33.524744 14662 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:33.558154 15155 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.600775  8401 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:04:33.533454 15309 consensus_queue.cc:1059] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:04:33.544987 15356 log.cc:927] Time spent T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:04:33.552675  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:33.560411 15334 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:33.614158  9004 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 21.
W20260709 13:04:33.554755  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.559896  8332 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:04:33.553387  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:04:33.567880  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 247238us
I20260709 13:04:33.615597  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:33.616143  8332 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:04:33.573132  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:33.576887  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:33.617336  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:33.617231  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:33.580991  8223 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:33.582219  8253 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 17: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:33.582849  8369 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:33.570338 15311 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.524839  9009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56940 (request call id 1347) took 1586 ms. Trace:
W20260709 13:04:33.597256  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 75594us
I20260709 13:04:33.598596  8250 raft_consensus.cc:2368] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 15. Current term is 16.
I20260709 13:04:33.599426  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:33.621136  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:33.621182  8223 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:33.603771  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:33.476538 15332 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:33.561378 14868 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.621390  9009 rpcz_store.cc:276] 0709 13:04:31.938616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.939288 (+   672us) service_pool.cc:224] Handling call
0709 13:04:33.524667 (+1585379us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54006de2d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:33.524786 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3146581888}
I20260709 13:04:33.624428  8252 raft_consensus.cc:1240] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:04:33.609293  8297 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:33.624745  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:33.619211  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:33.619365  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:33.610234  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:33.620114  8325 raft_consensus.cc:2393] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 24: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:33.578208 15163 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.621888  8326 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:33.583227 14967 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:33.454555  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 365946us
I20260709 13:04:33.591504  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:33.595567 14825 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:33.595009 15288 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:33.645679  8369 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:33.604367 15298 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:33.634635  8329 raft_consensus.cc:1240] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:33.650676  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:33.640463  8223 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:33.645036  9006 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 18. Current term is 19. Ops: []
W20260709 13:04:33.631065 15354 log.cc:927] Time spent T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
W20260709 13:04:33.638276 15358 log.cc:927] Time spent T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
I20260709 13:04:33.635170  8325 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:33.732329  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:33.732591  8325 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 22. Current term is 23. Ops: []
W20260709 13:04:33.733404  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 90149.8us
I20260709 13:04:33.734186  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:33.734714  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:04:33.653126  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 43947.7us
I20260709 13:04:33.596215 15171 consensus_queue.cc:1059] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:04:33.656164  9000 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:33.620241 15259 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:33.659029  8327 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 13:04:33.742641  8326 raft_consensus.cc:1275] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:04:33.742827  8329 raft_consensus.cc:1275] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:04:33.650098  8369 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:33.632462 15318 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:33.743170  8329 pending_rounds.cc:85] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 7
W20260709 13:04:33.743957  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 93922.9us
I20260709 13:04:33.763090  8329 pending_rounds.cc:107] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:04:33.763437  8329 raft_consensus.cc:2889] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:33.764047  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 114696us
I20260709 13:04:33.764546  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:33.741040  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:33.742385  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 87891.4us
W20260709 13:04:33.765499  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 125113us
I20260709 13:04:33.737703  8297 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:33.766897  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:33.645881 14825 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.649005 15142 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:33.603482 14662 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:33.770817  8399 raft_consensus.cc:2393] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 22: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:33.618817 15155 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:33.783790 15155 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:33.784667  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:33.613680 14758 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:33.779179 15195 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:33.785777  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:33.786489 15195 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.786924 15195 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:33.787029 14758 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:33.785318  9009 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 24.
I20260709 13:04:33.772689  8972 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:33.650694 15345 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:33.654031 15298 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.651899 15218 raft_consensus.cc:3055] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:33.652037 15301 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.126s
I20260709 13:04:33.651883 15338 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:33.770269  8369 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:33.763029 15318 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:33.776686  8298 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:33.774999  9007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 1489) took 1448 ms. Trace:
I20260709 13:04:33.734906 14958 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:33.817313  9007 rpcz_store.cc:276] 0709 13:04:32.326191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.326373 (+   182us) service_pool.cc:224] Handling call
0709 13:04:32.326872 (+   499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:32.493593 (+166721us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:32.493644 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:32.508567 (+ 14923us) log.cc:844] Serialized 26 byte log entry
0709 13:04:32.691864 (+183297us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:32.691927 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.768105 (+1076178us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54004c2f54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:33.768150 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:33.768193 (+    43us) raft_consensus.cc:1685] finished
0709 13:04:33.768209 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:33.774937 (+  6728us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6672,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":5399,"spinlock_wait_cycles":382119680,"thread_start_us":89079,"threads_started":1,"wal-append.queue_time_us":281836}
I20260709 13:04:33.629664 15334 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:33.646539 15163 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:33.646220 14868 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:33.801745 15298 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:33.624228 15349 consensus_queue.cc:1059] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.151s
W20260709 13:04:33.797185  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.596133 15296 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:33.768896 15132 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:33.633512 15311 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:33.776549 14825 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:33.824720 15345 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:33.777602 15268 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:33.649912 14967 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:33.850798 15317 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.054s
I20260709 13:04:33.851475 15317 consensus_queue.cc:1243] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 23.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 13:04:33.852072  8255 raft_consensus.cc:1240] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:33.818974  8298 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:33.821231  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 51037.9us
I20260709 13:04:33.731539 15299 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:33.816747  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:33.764788 15288 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:33.799069 15034 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:33.867327  8328 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:33.830655  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:33.801602 15218 raft_consensus.cc:740] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:33.868615  8253 raft_consensus.cc:2393] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 22: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:33.802587 15247 raft_consensus.cc:3055] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:33.820778 15318 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:33.821879 15174 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:33.850279 15352 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:04:33.852479 15327 log.cc:927] Time spent T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:04:33.887346  8324 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:33.864501  8399 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 21: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:33.859043  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:33.865767  8323 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18.
I20260709 13:04:33.867301  8369 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:33.841454 15132 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:33.869037 14990 consensus_queue.cc:1048] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:04:33.860636 15259 consensus_queue.cc:1059] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:04:33.867754 15034 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.875842 15311 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:33.879412 15328 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:33.843036 15054 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:33.866634 14958 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:33.894814  8323 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:33.894644  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:33.896035  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:33.898664  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:33.900188  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:33.902189  8403 raft_consensus.cc:2393] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 19: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:33.902961  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:33.911688  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:33.898030 15288 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:33.936206 15288 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:33.936659 15288 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:33.937187  8403 raft_consensus.cc:1275] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:04:33.938898  8403 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:04:33.935753  8324 raft_consensus.cc:2393] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 19: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:33.914414  8369 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:33.871774 15247 raft_consensus.cc:740] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:33.921553  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.942553 15163 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:33.860499 14758 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:04:33.944032 15296 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:33.926546  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:33.945369  8327 raft_consensus.cc:2393] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 18: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:33.927438  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:33.866303 15245 consensus_queue.cc:1059] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:04:33.947225 15245 raft_consensus.cc:3055] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:33.947732 15245 raft_consensus.cc:740] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:33.912623 15289 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:33.948813 15245 consensus_queue.cc:260] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:33.950618 15245 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:33.898304 15132 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:33.931272 15328 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:33.871601 15304 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 13:04:33.953226  8331 raft_consensus.cc:2410] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
W20260709 13:04:33.956610  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 45500.5us
I20260709 13:04:33.912834 15174 raft_consensus.cc:697] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 21 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:33.958729  8250 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:04:33.959769 15174 consensus_queue.cc:237] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:33.921855 15034 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:33.922556 15268 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.968022  8253 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:04:33.968676  8972 leader_election.cc:400] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 15. Current term is 16.
I20260709 13:04:33.969146  8972 leader_election.cc:403] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:04:33.973687  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 197618us
I20260709 13:04:33.974325  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:33.974912  8250 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:33.933931 14958 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:33.977546  8253 raft_consensus.cc:1275] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:04:33.974617  8297 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:33.978111  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:33.982264  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:33.982306  8297 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:33.983079  8249 raft_consensus.cc:2393] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 18: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:33.983608  8297 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:33.978490  8256 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 17. (term mismatch)
I20260709 13:04:33.979216  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:33.984251  8256 pending_rounds.cc:85] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 16
I20260709 13:04:33.983825  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:33.984426  8252 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:33.985085  8256 pending_rounds.cc:107] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 21.17
I20260709 13:04:33.985349  8256 raft_consensus.cc:2889] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:33.962504 15181 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:33.963605 15338 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:04:33.986057  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:33.990742  8297 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:33.986426  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:33.981158  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:33.991715  8253 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:04:33.992532  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:33.993829  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:33.994916  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:33.995574  8249 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 22.
I20260709 13:04:33.997653  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:33.998271  8254 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:04:33.998499  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:34.001560  8250 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
I20260709 13:04:33.952344 15054 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:34.000478 15317 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:33.987742 15296 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:33.976352 15333 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:04:34.007037 15054 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:34.007834 15258 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 19. Current term is 20.
I20260709 13:04:33.973387 15268 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:34.010130 15299 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:34.004091  9001 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 26 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 25. Current term is 26. Ops: []
I20260709 13:04:33.999197  8249 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:04:34.004859 15138 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.990334 15307 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:34.009148 15195 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:33.987213 14958 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:34.012615  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:34.024802  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2908) took 1038 ms. Trace:
I20260709 13:04:34.024955  8323 raft_consensus.cc:2393] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 16: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:34.003053  8369 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:33.935637 15218 consensus_queue.cc:260] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:34.030462  8406 raft_consensus.cc:3055] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:04:33.973237 15289 consensus_queue.cc:1059] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:04:33.979038 15316 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:04:33.994128 15338 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:34.014106  8254 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:04:34.017833 15138 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:34.019106  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:34.030885  8406 raft_consensus.cc:740] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 25 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:34.055438  8406 consensus_queue.cc:260] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:34.030761 15218 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:34.020332  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:34.020970  8256 raft_consensus.cc:1275] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:04:33.942662 15247 consensus_queue.cc:260] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:34.020993  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:33.997354 15285 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.031s
I20260709 13:04:34.029501 15334 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:34.011708 15243 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.018s
W20260709 13:04:34.088300 15374 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 13:04:34.088972  8326 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 21: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:34.089629 15159 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:34.090142 15159 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.090426  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:34.091068 15312 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.091147  8251 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:34.091674 15312 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:34.092269  8325 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 13:04:34.094175 15344 consensus_queue.cc:1059] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 13:04:34.056725  8406 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:34.003643 14990 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 13:04:34.046993  8400 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 20.
I20260709 13:04:34.058244  8252 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:04:34.019906 15195 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:34.098692  8254 raft_consensus.cc:1240] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17. Ops: [15.14-16.15]
I20260709 13:04:34.090570 15159 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:33.999163 15311 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:34.033869  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 1189) took 1169 ms. Trace:
I20260709 13:04:34.047473  8399 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:04:34.086443  8402 rpcz_store.cc:276] 0709 13:04:32.986168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.054268 (+ 68100us) service_pool.cc:224] Handling call
0709 13:04:33.115921 (+ 61653us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:33.147150 (+ 31229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:34.024609 (+877459us) spinlock_profiling.cc:243] Waited 61.3 ms on lock 0x7b5400267fd0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:34.024744 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":140924160}
W20260709 13:04:34.078517  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 75505.9us
I20260709 13:04:34.142537  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:34.012619 15307 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:34.035835 15282 log.cc:927] Time spent T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.993s	user 0.000s	sys 0.016s
I20260709 13:04:34.057667 15233 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.026191 14662 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:34.032035 15268 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:34.041213 14882 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:34.057121 15139 consensus_queue.cc:1059] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.097s
I20260709 13:04:34.056454 14825 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:34.163651 15243 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:34.144423 15163 consensus_queue.cc:1059] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
W20260709 13:04:34.164526 15376 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:04:34.097756 15382 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.043047 15288 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:34.060496 15338 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:34.155570 15307 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:34.145460  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:34.104645 15334 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.149192 15247 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:34.188889  8406 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
W20260709 13:04:34.189139  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 43863.9us
I20260709 13:04:34.191020  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 1227) took 1225 ms. Trace:
I20260709 13:04:34.192993  8369 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:34.181391 15333 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 15. Current term is 16.
I20260709 13:04:34.193440  8406 rpcz_store.cc:276] 0709 13:04:32.965719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.966834 (+  1115us) service_pool.cc:224] Handling call
0709 13:04:32.967241 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:34.030365 (+1063124us) spinlock_profiling.cc:243] Waited 324 ms on lock 0x7b54000dc120. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f8828735dc0 00007f88287c01c0 00007f88287c1231 00007f88287bdd8f 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c
0709 13:04:34.190740 (+160375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:34.190975 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":746131712}
I20260709 13:04:34.182044 15243 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.186719 15312 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:34.196633 15243 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:34.193645 15294 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 26 current term: 26 is higher.
I20260709 13:04:34.205446 15344 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:34.208056 15132 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:34.208424 15034 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:34.205214 15298 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:34.211436 15305 raft_consensus.cc:3055] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:34.211843 15305 raft_consensus.cc:740] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:34.212621 15305 consensus_queue.cc:260] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:34.213297  8369 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:34.213858 14868 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:34.196137  8405 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 24. Current term is 25. Ops: []
I20260709 13:04:34.213917 15305 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:34.187673 15256 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.008s	sys 0.021s
I20260709 13:04:34.216135  8369 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:34.214376 15258 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:34.217415  8369 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:34.176540 15233 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:34.216645 15256 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.008s	sys 0.022s
I20260709 13:04:34.191560 14990 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:34.213528 15132 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:34.194031 15138 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:34.193382 15334 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:34.217717 15304 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:34.220031 15259 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:34.230053 14825 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:34.251988 15334 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:34.254083 15375 log.cc:927] Time spent T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.369s	user 0.003s	sys 0.000s
I20260709 13:04:34.230305 15288 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:34.228339 14990 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.255613 14990 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:34.231305 15318 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:34.259263  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:34.259799 15318 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:34.190343 15382 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:34.260476  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:34.261058  8324 raft_consensus.cc:2376] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18. Re-sending same reply.
I20260709 13:04:34.238116 14958 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:34.261992  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:34.241691 15233 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:34.263494  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:34.237159 15298 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:34.264047  8324 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:34.264220 14958 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:34.249363 15344 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:34.249521 15258 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.269927 15382 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:34.269112 15258 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:34.246503 15034 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:34.205204 15243 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:34.276203  8324 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:04:34.283620  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:34.284360 15243 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:34.282446  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2911) took 1295 ms. Trace:
I20260709 13:04:34.286840  8404 rpcz_store.cc:276] 0709 13:04:32.987137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.080579 (+ 93442us) service_pool.cc:224] Handling call
0709 13:04:33.099696 (+ 19117us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:33.175756 (+ 76060us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:33.175787 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:33.175829 (+    42us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:33.175860 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:34.282170 (+1106310us) spinlock_profiling.cc:243] Waited 75.9 ms on lock 0x7b54003e7954. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:34.282222 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:34.282385 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174508800}
I20260709 13:04:34.292405 14990 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:34.284832  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:34.280478 15338 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:34.294093  8324 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:34.275024 15258 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:34.286535 15159 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:34.303854  8324 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
I20260709 13:04:34.305158  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:34.307318 15159 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:34.311486  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:34.308826 15258 consensus_queue.cc:1059] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 13:04:34.315631 15307 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:34.318881 15338 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:34.319466  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:34.323707 15258 raft_consensus.cc:3055] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:34.324474 15258 raft_consensus.cc:740] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:34.308523 14990 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:34.325361 15258 consensus_queue.cc:260] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:34.326731 15258 raft_consensus.cc:3060] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:34.321470  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:34.321657  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.329792 15307 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:34.332034  8324 raft_consensus.cc:1275] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:04:34.332510  8324 pending_rounds.cc:85] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 8
W20260709 13:04:34.332628  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.332847  8324 pending_rounds.cc:107] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:04:34.333276  8324 raft_consensus.cc:2889] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:34.333211 15238 raft_consensus.cc:3055] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:34.333916 15238 raft_consensus.cc:740] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:34.335930 15268 raft_consensus.cc:3055] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:34.336412 15268 raft_consensus.cc:740] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:34.337407 15268 consensus_queue.cc:260] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:34.343683 14898 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:34.345979  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:34.346313  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.352283  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:34.353504  8256 raft_consensus.cc:2393] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 19: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:34.353885 15256 raft_consensus.cc:3055] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:34.354367 15256 raft_consensus.cc:740] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:34.354641  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:34.355484  8256 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:34.355379 15256 consensus_queue.cc:260] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:34.356961 15256 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:34.335240 15238 consensus_queue.cc:260] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:34.360041 15238 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:34.336321 15299 raft_consensus.cc:3055] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:34.364662 15299 raft_consensus.cc:740] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:34.365710 15299 consensus_queue.cc:260] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:34.361835  8329 raft_consensus.cc:3055] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:34.367126  8329 raft_consensus.cc:740] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:34.367990  8329 consensus_queue.cc:260] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:34.369097  8329 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:34.387380 15382 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.388082 15382 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:34.390465 15382 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:34.391114 15382 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.391590 15382 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:34.394181 15382 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:34.399735  8256 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 25.
I20260709 13:04:34.402405  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:34.403668  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:34.404203  8256 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:34.435775 15256 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:34.436434 15256 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:34.438620 15256 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:34.439571 15034 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.440132 15034 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:34.440616  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:34.441462  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:34.464481 14950 consensus_queue.cc:1059] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:34.465821 15306 raft_consensus.cc:3055] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:34.466208 15306 raft_consensus.cc:740] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:34.467038 15306 consensus_queue.cc:260] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:34.468209 15306 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:34.475000  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:34.476527  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:34.478550  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:34.480484  9008 pending_rounds.cc:85] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: Aborting all ops after (but not including) 17
I20260709 13:04:34.480858  9008 pending_rounds.cc:107] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: Aborting uncommitted NO_OP operation due to leader change: 23.18
I20260709 13:04:34.481114  9008 raft_consensus.cc:2889] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:34.484637 15256 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:34.477214  8257 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:34.501288 15369 log.cc:927] Time spent T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.958s	user 0.000s	sys 0.011s
W20260709 13:04:34.504432  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.505149  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3509) took 1422 ms. Trace:
I20260709 13:04:34.505647  8398 rpcz_store.cc:276] 0709 13:04:33.082506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.268716 (+186210us) service_pool.cc:224] Handling call
0709 13:04:33.269001 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:33.518137 (+249136us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:33.518166 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:33.518687 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 13:04:33.554468 (+ 35781us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:33.554505 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.554566 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:34.500584 (+946018us) spinlock_profiling.cc:243] Waited 31.4 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:04:34.504850 (+  4266us) raft_consensus.cc:1685] finished
0709 13:04:34.504874 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:34.505086 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":248769,"raft.queue_time_us":249961,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":78,"spinlock_wait_cycles":72267136,"thread_start_us":35417,"threads_started":1,"wal-append.queue_time_us":23839}
I20260709 13:04:34.518329 15313 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:34.557097  9001 raft_consensus.cc:1240] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:34.560104 15294 consensus_queue.cc:1059] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:34.584880  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:34.594781  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:34.596030  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:34.596621  8250 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:34.601171  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.602403 15332 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:34.604918  8332 raft_consensus.cc:2368] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 20. Current term is 21.
W20260709 13:04:34.608100  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 508666us
I20260709 13:04:34.612721  8253 raft_consensus.cc:2410] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:04:34.616027  8253 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:04:34.619349  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 1130) took 1336 ms. Trace:
I20260709 13:04:34.619843  8328 rpcz_store.cc:276] 0709 13:04:33.282866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.283285 (+   419us) service_pool.cc:224] Handling call
0709 13:04:33.304160 (+ 20875us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:33.869673 (+565513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:34.619118 (+749445us) spinlock_profiling.cc:243] Waited 563 ms on lock 0x7b540037d054. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:34.619276 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1294706432}
I20260709 13:04:34.622054 15266 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:34.645550  8248 rpcz_store.cc:276] 0709 13:04:32.864134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.864308 (+   174us) service_pool.cc:224] Handling call
0709 13:04:32.864746 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:33.647900 (+783154us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:33.647933 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:33.647972 (+    39us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:33.648004 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:34.033632 (+385628us) spinlock_profiling.cc:243] Waited 783 ms on lock 0x7b540009ecd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:34.033680 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:34.033809 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1800641408}
I20260709 13:04:34.648826 15266 consensus_queue.cc:1059] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:34.650269 14898 raft_consensus.cc:3055] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:34.650763 14898 raft_consensus.cc:740] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:34.651677 14898 consensus_queue.cc:260] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:34.655112  8251 consensus_meta.cc:287] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.563s	user 0.014s	sys 0.005s
I20260709 13:04:34.655608  8251 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
W20260709 13:04:34.666390  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.673168  8248 raft_consensus.cc:1240] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:04:34.478786  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:35.162391  9000 raft_consensus.cc:2368] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19.
I20260709 13:04:34.674660  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:35.164284  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:35.164935  8330 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:04:34.677042  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:35.165776  8324 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:34.677561  8331 raft_consensus.cc:1240] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:34.683687  8256 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 23.
I20260709 13:04:34.684422  8250 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
W20260709 13:04:34.689481  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.172034 15313 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.172611  8297 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:34.689939  8220 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:34.691058  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:34.691650  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:34.692643  8326 raft_consensus.cc:1275] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:04:34.693361  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:04:35.173840  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 483981us
W20260709 13:04:34.748176 15249 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.833s	user 0.006s	sys 0.013s
I20260709 13:04:34.751642 15329 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:34.759208  8257 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:04:34.763221  8329 raft_consensus.cc:1275] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:04:35.177990  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 3330) took 1076 ms. Trace:
I20260709 13:04:34.774557 15266 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:35.178268  8329 rpcz_store.cc:276] 0709 13:04:34.101908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.361096 (+259188us) service_pool.cc:224] Handling call
0709 13:04:34.361474 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:35.177719 (+816245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:35.177947 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:34.780601 15252 raft_consensus.cc:3055] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:34.781070 14868 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:34.794593 15268 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:35.173383  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:34.819773  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.823485  8323 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:34.822829  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 3321) took 1227 ms. Trace:
I20260709 13:04:34.868425 15384 consensus_queue.cc:1048] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:34.879480  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 1197) took 1299 ms. Trace:
I20260709 13:04:34.893198 15034 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:34.933624 15321 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:34.973323  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.975127  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:04:35.177420 15249 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.283s	user 0.007s	sys 0.014s
I20260709 13:04:34.976717 15256 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:04:35.005757 15306 consensus_meta.cc:287] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.537s	user 0.006s	sys 0.018s
W20260709 13:04:35.020623 15394 log.cc:927] Time spent T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.370s	user 0.002s	sys 0.000s
I20260709 13:04:35.023486 14898 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:35.226089 15256 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:35.228516 15256 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:35.034054  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.038520  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:35.230300  9004 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 24.
I20260709 13:04:35.039320 14967 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:35.052040 15264 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:35.052640  8974 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:04:35.232445  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 179880us
I20260709 13:04:35.054888 15335 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.055425 15298 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:35.063254 15299 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:35.210588 15252 raft_consensus.cc:740] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:35.248715 15252 consensus_queue.cc:260] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:35.066318 15352 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:35.251716 15352 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:35.254025 15352 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:35.067360  9009 raft_consensus.cc:1240] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 12. Current term is 13. Ops: []
W20260709 13:04:35.069962 15393 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.418s	user 0.000s	sys 0.002s
W20260709 13:04:35.094624 15392 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.205s	user 0.003s	sys 0.000s
I20260709 13:04:35.096334  8295 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:35.255357 15335 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:35.101794 15294 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:35.256693 15294 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:35.103578 15382 consensus_queue.cc:1048] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:35.258661 15294 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:35.119215 15247 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:35.123996 15132 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:35.259524 15247 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:35.131899  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:35.260633  9001 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:35.148880 15272 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:35.261628 15247 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:35.261559 15272 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:35.263648 15272 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:35.152422  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 673924us
W20260709 13:04:35.153640 15397 log.cc:927] Time spent T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.263s	user 0.003s	sys 0.000s
W20260709 13:04:35.264575  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.264957  8369 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:35.265645 15128 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:35.266172 15128 raft_consensus.cc:697] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 20 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:35.266268  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:35.267164 15128 consensus_queue.cc:237] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:35.267683  8369 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:34.743757  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 27 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:35.174548  8220 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:04:35.270700  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 96222.3us
I20260709 13:04:35.271679  8220 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:35.272868  8220 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:35.176674  8297 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:35.182034 15217 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:04:34.842515  8972 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:35.202960  8330 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
W20260709 13:04:35.274506  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 432067us
I20260709 13:04:35.274961  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:35.275234  8972 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:35.275611  8330 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 20.
I20260709 13:04:35.276501  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:35.201036  8324 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 27.
I20260709 13:04:35.277362 15339 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:35.277750  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:35.278023 15339 raft_consensus.cc:697] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:35.278435  8324 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:35.279039 15339 consensus_queue.cc:237] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:35.203109  8332 rpcz_store.cc:276] 0709 13:04:33.594980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.624524 (+ 29544us) service_pool.cc:224] Handling call
0709 13:04:34.822588 (+1198064us) spinlock_profiling.cc:243] Waited 979 ms on lock 0x7b540037c654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:34.822766 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2252165376}
I20260709 13:04:35.202968  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:35.202464 14868 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:35.209568  8255 rpcz_store.cc:276] 0709 13:04:33.579766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.579878 (+   112us) service_pool.cc:224] Handling call
0709 13:04:33.608019 (+ 28141us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:33.852459 (+244440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:34.879125 (+1026666us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b5400406d54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:34.879355 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":560846720}
I20260709 13:04:35.172586 15313 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:35.210980  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:34.976976  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.213982  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:35.292790  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:35.293530  8405 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 12.
I20260709 13:04:35.294490  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:35.295256  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:35.295624  8405 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:04:35.182936  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:35.216224  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:35.216814  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:35.296558  8329 raft_consensus.cc:2368] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13.
I20260709 13:04:35.218464  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:35.219095  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:35.297123  8404 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:35.219914  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.297613  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 27 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:35.177289 15296 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:35.302031  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:35.220711  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:35.220795  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:35.201769 15329 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:35.220897  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:35.222865  9002 raft_consensus.cc:2393] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 19: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:35.222570 15321 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:35.340224 15352 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:35.234732  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:35.239611 15264 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:35.240017  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 3004) took 1327 ms. Trace:
I20260709 13:04:35.240582  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:04:35.367303  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 271059us
I20260709 13:04:35.241376  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:35.241788  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:35.368510  9007 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:04:35.243356  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:35.369791  8328 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:04:35.269645 15307 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:35.270766 14967 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:35.282572  8972 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:35.367525  8323 rpcz_store.cc:276] 0709 13:04:33.912148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.034815 (+122667us) service_pool.cc:224] Handling call
0709 13:04:34.823291 (+788476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:35.203060 (+379769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:35.239797 (+ 36737us) spinlock_profiling.cc:243] Waited 788 ms on lock 0x7b540037c650. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:35.239954 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1812531968}
I20260709 13:04:35.282611  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:35.283432  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:35.368088  9006 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:04:35.372167  9001 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
W20260709 13:04:35.372709  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 90224.6us
I20260709 13:04:35.373458  9004 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:35.373560  8972 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:35.374410  9000 raft_consensus.cc:2393] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 21: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:35.284322  8220 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:35.285557  8223 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:35.286342  8324 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 21.
I20260709 13:04:35.287519  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:35.249763 15252 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:35.215624  8406 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:04:35.217682 14758 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:35.296211  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:35.314894  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:35.379806  8405 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:35.290867 15272 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:04:35.299968  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 61602.3us
I20260709 13:04:35.300593 15401 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:04:35.301060  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 124466us
I20260709 13:04:35.301137 15403 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:35.302500  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:35.302850  8399 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:35.342742 15256 consensus_queue.cc:1059] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:04:35.277616 15306 raft_consensus.cc:2804] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:35.292332 15288 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:35.350317  8327 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:04:35.348429  8330 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:35.349412  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:35.313244  8329 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:04:35.302306 14950 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:35.414490  8297 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:35.370865 15352 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:35.299494 15294 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:35.435146 15294 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.435849 15217 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:35.437608 15400 log.cc:927] Time spent T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.137s	user 0.002s	sys 0.003s
I20260709 13:04:35.386972 15132 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:35.348817  8151 catalog_manager.cc:5697] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 17 to 20, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 20 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:04:35.361697  8402 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:35.362882  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:35.339929 15335 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:35.292985 15247 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:35.488617 15247 raft_consensus.cc:697] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:35.489658 15247 consensus_queue.cc:237] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:35.492815 15335 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:35.177536 15318 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:35.493463 15335 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:35.366184  8325 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 13:04:35.495136  8325 pending_rounds.cc:85] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 10
I20260709 13:04:35.495409  8325 pending_rounds.cc:107] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:04:35.495683  8325 raft_consensus.cc:2889] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:35.397804 14967 raft_consensus.cc:697] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 18 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:35.396716 15401 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:35.378815  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:35.381139 14758 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:35.382004 15298 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:04:35.382434  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.382611 15264 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:35.302101 15296 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:35.387240  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 101839us
I20260709 13:04:35.512813  8223 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:35.513618 15298 raft_consensus.cc:697] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:35.513806 15318 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:04:35.389333  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 105077us
I20260709 13:04:35.392712  8404 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 21.
I20260709 13:04:35.393129  8398 raft_consensus.cc:2468] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:04:35.521890  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:35.522538  8398 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:35.395149  9003 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:04:35.412817  8329 raft_consensus.cc:606] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:04:35.498601  8151 catalog_manager.cc:5697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 16 to 18, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 18 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:35.527242  8329 raft_consensus.cc:614] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:35.412722  8397 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
I20260709 13:04:35.425801 14950 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:35.435781 15294 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:35.469090  9005 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:04:35.353582  8401 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
I20260709 13:04:35.406297 15268 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:35.406893 15272 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:35.368168  8295 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:35.417073 15352 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:35.514323  8223 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:35.388556  8372 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:04:35.572674  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 204588us
W20260709 13:04:35.572811  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 184363us
I20260709 13:04:35.573791  8372 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:35.573915  8295 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:35.538862  8151 catalog_manager.cc:5697] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 9 to 19, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 19 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:35.551208  8402 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
W20260709 13:04:35.552417  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 166608us
I20260709 13:04:35.576154  8298 leader_election.cc:400] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19.
I20260709 13:04:35.576464  8298 leader_election.cc:403] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:35.576938  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:35.566421  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 187097us
I20260709 13:04:35.567830 15410 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:35.588076 15410 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:35.588915 15382 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.589095 14758 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:35.506269 15132 raft_consensus.cc:697] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 13 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:35.506529 15401 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:35.538358 15318 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.483671 15307 consensus_queue.cc:1048] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:04:35.578488  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:35.627645  8406 raft_consensus.cc:2410] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:04:35.578591  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:35.580606  8151 catalog_manager.cc:5697] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 18 to 19, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 19 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:35.628825  8151 catalog_manager.cc:5697] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 11 to 17. New cstate: current_term: 17 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:35.631181  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:35.633555  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:35.635185  8330 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 20.
W20260709 13:04:35.569625  8972 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.565s (SENT) [suppressed 14 similar messages]
I20260709 13:04:35.582300  8398 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
W20260709 13:04:35.568130  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 281768us
I20260709 13:04:35.586072  8402 raft_consensus.cc:2393] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 19: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:35.599861 15171 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:35.608518 15253 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:35.631916  8403 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:35.599627 15411 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:35.616005 15403 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:35.497677 15288 consensus_queue.cc:1048] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:04:35.607836  9008 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
I20260709 13:04:35.609412 15408 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:35.599910 15384 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:35.589730 14868 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:35.599112  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 84906.3us
I20260709 13:04:35.598212 15268 raft_consensus.cc:697] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 22 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:35.597770 15249 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:35.713670 15249 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:35.714078 15268 consensus_queue.cc:237] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:35.715969 15249 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:35.716487 15249 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.594849 15294 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:35.600613 15266 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.579466 15272 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.599012 14950 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:35.582314  8974 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:35.591270  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:35.753618  8401 raft_consensus.cc:2368] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19.
W20260709 13:04:35.726835  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.625013  9005 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
I20260709 13:04:35.563108  8405 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:35.634819  8151 catalog_manager.cc:5697] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 17 to 19, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 19 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:35.756870  8405 raft_consensus.cc:1275] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:04:35.756652  8151 catalog_manager.cc:5697] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:04:35.757974  8151 catalog_manager.cc:5697] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:04:35.759100  8151 catalog_manager.cc:5697] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 17 to 18, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 18 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:04:35.590166 15410 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:35.647049  9009 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 24.
I20260709 13:04:35.589553 15382 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:35.638229  8399 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 13.
W20260709 13:04:35.688169  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 49791.3us
I20260709 13:04:35.636147 14758 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:35.695350  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:35.759418  8405 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:35.762285 15382 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:35.763581  8403 raft_consensus.cc:2410] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 27 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
W20260709 13:04:35.696324  8150 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45434 (request call id 284) took 44953 ms (client timeout 15000 ms). Trace:
I20260709 13:04:35.697698 15411 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:35.698091 15306 raft_consensus.cc:697] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 17 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:35.699756  8245 tablet_service.cc:1558] Processing DeleteTablet for tablet 81804f59e3124d27998ea80fc1c36663 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 6 (current committed config index is 11)) from {username='slave'} at 127.0.0.1:51208
I20260709 13:04:35.763906 15410 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.764814 15410 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:35.765822  8403 raft_consensus.cc:1240] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:04:35.766254  8405 raft_consensus.cc:1275] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:04:35.767686 15411 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:35.763770  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.708582  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:35.770185  8406 raft_consensus.cc:2410] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:04:35.771392  8406 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 26. Current term is 27. Ops: []
I20260709 13:04:35.630596 15318 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:35.629184 15256 consensus_queue.cc:1243] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Peer ff7a6fe533e94c1bae555b4cae427728 log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:04:35.760632  8151 catalog_manager.cc:5697] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 17 to 18, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 18 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:35.772421  8405 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:35.720525  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:35.752194  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:35.733196  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.688452  8397 raft_consensus.cc:2393] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 23: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:35.716964 15249 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:35.553707 14898 consensus_queue.cc:1048] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 13:04:35.724781  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:35.761672  8399 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:04:35.602586 15329 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:35.764974 15306 consensus_queue.cc:237] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:35.700094  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 126238us
I20260709 13:04:35.805811  8295 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:35.789968  8220 leader_election.cc:336] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.664s (SENT)
I20260709 13:04:35.806550  8220 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:35.790699  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:35.807464  8330 raft_consensus.cc:2393] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 16: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:35.790778  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:35.791543  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:35.792368  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 210117us
I20260709 13:04:35.808102  8329 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
I20260709 13:04:35.808626  8974 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:35.793886  8401 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 22. Current term is 23. Ops: []
W20260709 13:04:35.794312  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.783017  8151 catalog_manager.cc:5697] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 14 to 15, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 15 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:35.810664  8974 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:04:35.817142  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.697223 15171 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:35.797574 14758 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:35.842492 15424 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.900933 14758 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:35.901105 15424 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:35.902923 15405 raft_consensus.cc:3055] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:35.903419 15405 raft_consensus.cc:740] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:35.903653 15424 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:35.904824 15299 consensus_meta.cc:287] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.660s	user 0.009s	sys 0.004s
I20260709 13:04:35.629812 15298 consensus_queue.cc:237] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:35.842819  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:35.707646 15253 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:35.914485  8353 debug-util.cc:398] Leaking SignalData structure 0x7b08000cdd00 after lost signal to thread 7167
W20260709 13:04:35.915659  8353 debug-util.cc:398] Leaking SignalData structure 0x7b0800167fe0 after lost signal to thread 8189
I20260709 13:04:35.916539 15253 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:35.916991  8353 debug-util.cc:398] Leaking SignalData structure 0x7b08002f1e80 after lost signal to thread 8281
I20260709 13:04:35.717854 15266 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:35.919288 15266 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:35.694108 14967 consensus_queue.cc:237] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:35.927167  8405 raft_consensus.cc:1275] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:04:35.753472 15272 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:04:35.862142  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 51660.9us
I20260709 13:04:35.793316 15403 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:35.788002 15313 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:35.721246 15384 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.767657 15410 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:35.864567  8151 catalog_manager.cc:5697] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 20 to 24. New cstate: current_term: 24 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:35.799197 15408 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.800287 15294 consensus_queue.cc:1059] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.144s
I20260709 13:04:35.904340 15405 consensus_queue.cc:260] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:35.909723  8326 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
W20260709 13:04:35.845815  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 55892.5us
I20260709 13:04:35.804317 15332 raft_consensus.cc:2764] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 13:04:35.913249  8328 consensus_meta.cc:287] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.538s	user 0.007s	sys 0.014s
I20260709 13:04:36.006963  8325 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 15.
I20260709 13:04:35.925367 15307 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:35.814409  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:04:35.936255  8353 debug-util.cc:398] Leaking SignalData structure 0x7b0800330e20 after lost signal to thread 8356
W20260709 13:04:36.013381  8353 debug-util.cc:398] Leaking SignalData structure 0x7b0800330e40 after lost signal to thread 8357
W20260709 13:04:36.014611  8353 debug-util.cc:398] Leaking SignalData structure 0x7b080032e220 after lost signal to thread 8430
W20260709 13:04:36.015919  8353 debug-util.cc:398] Leaking SignalData structure 0x7b080034af00 after lost signal to thread 9034
W20260709 13:04:36.016228  8353 debug-util.cc:398] Leaking SignalData structure 0x7b0800314d60 after lost signal to thread 9035
I20260709 13:04:35.938767  8400 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
I20260709 13:04:35.864158 15321 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:35.979768  8974 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:36.020335  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:35.858628 15238 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:35.984812  8297 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.008s	sys 0.002s
W20260709 13:04:36.021807  8297 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.009s	sys 0.002s
W20260709 13:04:36.022594  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 608164us
I20260709 13:04:35.813381 15335 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:35.763988  8150 rpcz_store.cc:269] 0709 13:03:50.742893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.744252 (+  1359us) service_pool.cc:224] Handling call
0709 13:03:51.164828 (+420576us) spinlock_profiling.cc:243] Waited 34 ms on lock 0x7b5400200490. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f882569a0b2 00007f882e613a89 00007f882e5abda6 00007f882e57d088 00007f882e67b37e 00007f882cb3c2a4 00007f882cb3c232 00007f882cb3c161 00007f882cb3c0dc
0709 13:03:51.546292 (+381464us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b5400200490. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699e9c 00007f882569a0f6 00007f882e613a89 00007f882e5abda6 00007f882e57d088 00007f882e67b37e 00007f882cb3c2a4 00007f882cb3c232 00007f882cb3c161 00007f882cb3c0dc
0709 13:04:35.696222 (+44149930us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:28.990031 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:29.050547 (+ 60516us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:29.050605 (+    58us) write_op.cc:435] Acquired schema lock
0709 13:04:29.050624 (+    19us) tablet.cc:662] Decoding operations
0709 13:04:29.051530 (+   906us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:29.051640 (+   110us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:29.051664 (+    24us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:04:29.052261 (+   597us) tablet.cc:701] Row locks acquired
0709 13:04:29.052306 (+    45us) write_op.cc:260] PREPARE: finished
0709 13:04:29.052481 (+   175us) write_op.cc:270] Start()
0709 13:04:29.052621 (+   140us) write_op.cc:276] Timestamp: P: 1783602269052413 usec, L: 0
0709 13:04:29.052658 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 13:04:29.086541 (+ 33883us) log.cc:844] Serialized 2416 byte log entry
0709 13:04:29.448964 (+362423us) spinlock_profiling.cc:243] Waited 26.8 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:04:30.066228 (+617264us) op_driver.cc:464] REPLICATION: finished
0709 13:04:30.787382 (+721154us) spinlock_profiling.cc:243] Waited 39 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f88256b40ac 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882c1f4ef0
0709 13:04:31.797626 (+1010244us) write_op.cc:301] APPLY: starting
0709 13:04:31.827788 (+ 30162us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:32.176817 (+349029us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:32.176908 (+    91us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:32.449355 (+272447us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:32.449721 (+   366us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=12,mrs_lookups=0
0709 13:04:32.449781 (+    60us) write_op.cc:312] APPLY: finished
0709 13:04:32.590103 (+140322us) log.cc:844] Serialized 65 byte log entry
0709 13:04:32.787564 (+197461us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:32.792206 (+  4642us) write_op.cc:454] Released schema lock
0709 13:04:32.807963 (+ 15757us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:32.849039 (+ 41076us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:32.850576 (+  1537us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":373703,"reported_tablets":50,"spinlock_wait_cycles":115445888,"child_traces":[["op",{"apply.queue_time_us":1644227,"apply.run_cpu_time_us":27608,"apply.run_wall_time_us":1057305,"cfile_cache_hit":26,"cfile_cache_hit_bytes":29219,"num_ops":6,"prepare.queue_time_us":389452,"prepare.run_cpu_time_us":36914,"prepare.run_wall_time_us":328434,"raft.queue_time_us":250365,"raft.run_cpu_time_us":71858,"raft.run_wall_time_us":1805821,"replication_time_us":987834,"spinlock_wait_cycles":151452928,"thread_start_us":2130832,"threads_started":5,"wal-append.queue_time_us":469847,"wal-append.run_cpu_time_us":27956,"wal-append.run_wall_time_us":2919359}]]}
I20260709 13:04:35.845006 15382 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:04:35.992993  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 423375us
I20260709 13:04:35.989591 15414 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:35.897624  8255 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:51220
I20260709 13:04:36.038388  8255 raft_consensus.cc:606] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:04:36.038823  8255 raft_consensus.cc:614] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:35.845081 15407 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:35.895396 14950 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:35.990808  8151 catalog_manager.cc:5697] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:35.897296 15252 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:35.896278 15256 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.008934  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:35.912550 15422 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:36.010572  8328 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
I20260709 13:04:35.880275  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:36.012207  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:36.019315  8353 debug-util.cc:398] Leaking SignalData structure 0x7b08003137a0 after lost signal to thread 15328
I20260709 13:04:35.864305 15128 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.144s
I20260709 13:04:35.998785 15384 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:36.032893  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:35.993640  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.033840 15264 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:36.033928 15238 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:36.048108 15407 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.104926  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:36.105273  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:36.106299  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:36.053555 15405 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:36.107064  8248 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 22.
I20260709 13:04:36.041666  8297 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:36.054073  8298 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:36.053741  8151 catalog_manager.cc:5697] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 16 to 17, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 17 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:36.037021 15414 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:36.058755  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:36.059168  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:36.119534  9004 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:04:36.059716  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:36.062655  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 182510us
W20260709 13:04:36.062929  8353 debug-util.cc:398] Leaking SignalData structure 0x7b080034dc20 after lost signal to thread 15379
I20260709 13:04:36.121485  8256 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:04:36.122468  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:36.030045 15294 consensus_queue.cc:1243] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer ff7a6fe533e94c1bae555b4cae427728 log is divergent from this leader: its last log entry 19.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:04:35.859833 15132 consensus_queue.cc:237] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:36.034403 15382 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:36.092748 15217 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19.
I20260709 13:04:36.095322 14950 raft_consensus.cc:697] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 21 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:36.004882 15408 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:36.139532  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 85550.5us
I20260709 13:04:36.191864  8298 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:36.133679  8369 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:36.136574  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:36.135671  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:36.132647  9008 raft_consensus.cc:2410] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:04:36.143167  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:36.136296  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:36.049175 14758 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:36.038760 15409 consensus_queue.cc:1059] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.082s
I20260709 13:04:36.120725 14898 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.120322 15407 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:36.181953  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:36.180933  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:36.225389  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 91785.5us
I20260709 13:04:36.225639  8325 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:04:36.138693 15288 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:04:36.120332 15431 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.118250  8151 catalog_manager.cc:5697] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:36.091778 15256 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:36.095224 15307 consensus_queue.cc:1059] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 24.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.048s
W20260709 13:04:36.199028  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 157453us
I20260709 13:04:36.244612  8297 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:36.223589  8254 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:36.223905  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:36.098266 15432 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:36.224476  8249 raft_consensus.cc:1240] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:36.247923  8222 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:36.226171  8369 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:36.226519  8398 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:04:36.227741  8256 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:36.232671  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:36.135227 15268 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:36.236742  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:36.241700  8255 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 24. Current term is 25. Ops: []
I20260709 13:04:36.238857  8151 catalog_manager.cc:5697] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 14 to 16, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 16 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:04:36.220216 15393 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:04:36.251422  8223 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:36.245519  8253 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
I20260709 13:04:36.136559 15252 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:36.139199 15294 consensus_queue.cc:1048] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.097s
I20260709 13:04:36.133275 15438 consensus_queue.cc:1059] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.141s
I20260709 13:04:36.264071  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:36.341722  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:04:36.341905  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 90553.8us
I20260709 13:04:36.297374  8151 catalog_manager.cc:5697] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 14 to 15, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 15 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:36.345355  8253 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:36.260670  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:36.346416  8254 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:36.346211  8151 catalog_manager.cc:5697] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 16 to 18, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 18 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:04:36.260102  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:36.347621  8398 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
I20260709 13:04:36.348788  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:36.319990  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:36.349596  8398 raft_consensus.cc:2393] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 19: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:36.345340  8256 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:36.346076  8397 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:36.264354  8405 raft_consensus.cc:2393] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 20: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:36.238121 15425 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:36.296772  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:36.243438 15426 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:36.356310  8151 catalog_manager.cc:5697] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
W20260709 13:04:36.304025  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 112288us
I20260709 13:04:36.196892 15434 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:36.366420  8151 catalog_manager.cc:5697] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 11 to 19, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 19 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:36.354254  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:36.365489  8298 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:36.371061  8151 catalog_manager.cc:5697] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 21 to 23, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 23 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:36.376170  8399 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
W20260709 13:04:36.356441 15411 consensus_meta.cc:287] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.588s	user 0.010s	sys 0.001s
I20260709 13:04:36.318018 15266 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:36.255756 15423 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:36.316934 15249 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:36.239573 14898 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:36.256052 15419 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Leader election won for term 25
W20260709 13:04:36.372830 15420 log.cc:927] Time spent T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.392s	user 0.000s	sys 0.001s
I20260709 13:04:36.356653 15321 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:36.258006 15432 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:04:36.439767 15435 log.cc:927] Time spent T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.183s	user 0.000s	sys 0.000s
I20260709 13:04:36.441159  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 1253) took 1074 ms. Trace:
I20260709 13:04:36.377822  8256 raft_consensus.cc:2410] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
W20260709 13:04:36.362305  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 136213us
I20260709 13:04:36.441597  8403 rpcz_store.cc:276] 0709 13:04:35.366712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.766595 (+399883us) service_pool.cc:224] Handling call
0709 13:04:35.766860 (+   265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:35.816531 (+ 49671us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:35.816564 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:35.858111 (+ 41547us) log.cc:844] Serialized 26 byte log entry
0709 13:04:35.989949 (+131838us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:35.990112 (+   163us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:35.990583 (+   471us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:36.440936 (+450353us) raft_consensus.cc:1685] finished
0709 13:04:36.440965 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:36.441099 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26017,"raft.queue_time_us":43481,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":44258,"thread_start_us":175924,"threads_started":2,"wal-append.queue_time_us":277948}
I20260709 13:04:36.442806  8369 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:36.362426  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 117896us
W20260709 13:04:36.443318 15352 consensus_meta.cc:287] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.850s	user 0.009s	sys 0.001s
I20260709 13:04:36.443746 15352 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:36.444363  8297 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:36.392745  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:36.417819 15444 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.445613  8404 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:36.445719 15444 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:36.364077  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:36.447952  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:36.449069  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:36.449961  8249 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 17: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:36.450906  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:36.451196 15415 consensus_queue.cc:1059] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:36.452488 15415 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:36.418116  8151 catalog_manager.cc:5697] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 13 to 16, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 16 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:36.453671  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:36.396085  8254 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 21.
I20260709 13:04:36.401391  8253 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:36.455719 15272 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:36.456875  8255 raft_consensus.cc:2368] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 24. Current term is 25.
I20260709 13:04:36.458297  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:36.456707  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:36.459450 15382 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:36.459959 15410 raft_consensus.cc:3055] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:04:36.460626 15437 log.cc:927] Time spent T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.265s	user 0.000s	sys 0.001s
I20260709 13:04:36.462186  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:36.426551  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 70577us
I20260709 13:04:36.460310  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:36.463269  8253 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:36.449862 15352 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:36.465785  8151 catalog_manager.cc:5697] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 13 to 14, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 14 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:36.453133 15415 raft_consensus.cc:3055] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:36.246101 15431 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:36.471038  8222 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:36.402428 15171 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:36.361914  8397 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
I20260709 13:04:36.482121  8400 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:04:36.459156  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.492444  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:36.517815  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 73509.2us
I20260709 13:04:36.518522  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:36.416416 15426 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:36.526013  8253 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
W20260709 13:04:36.458513 15429 log.cc:927] Time spent T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.357s	user 0.001s	sys 0.000s
I20260709 13:04:36.415766 15423 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:36.401877 15434 raft_consensus.cc:697] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:36.428946 15321 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:36.531298  8403 raft_consensus.cc:2410] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 23 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:04:36.530710  8297 leader_election.cc:400] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19.
I20260709 13:04:36.570384  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:36.571036  8297 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:36.532580  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 61870.7us
I20260709 13:04:36.536366  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:36.530841  8256 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:04:36.460409 15410 raft_consensus.cc:740] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:36.541666  8253 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:51220
I20260709 13:04:36.437115 15432 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:36.465498 15296 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:36.396265 14950 consensus_queue.cc:237] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:36.599965  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:36.601166  8256 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: []
W20260709 13:04:36.546653  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 181241us
W20260709 13:04:36.554121  8372 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.004s	sys 0.004s
I20260709 13:04:36.377200 15425 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:36.555701  9003 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:04:36.362008 15268 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:36.528307 15288 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.518177 15415 raft_consensus.cc:740] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:36.400350 15411 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:36.622982  8402 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:36.623991 15411 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:36.536926 15352 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:04:36.529729 15447 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:36.560496 15426 raft_consensus.cc:697] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 24 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:36.626288 15447 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:36.627245 15426 consensus_queue.cc:237] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:36.629890 15434 consensus_queue.cc:237] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:36.630869 14898 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:36.632603 15414 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:36.633762 15256 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:36.634526 15440 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.224s	user 0.000s	sys 0.000s
W20260709 13:04:36.545243 15298 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.005s	sys 0.005s
I20260709 13:04:36.577833  8400 raft_consensus.cc:1275] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
W20260709 13:04:36.643594  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:36.605085  8372 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.004s	sys 0.004s
I20260709 13:04:36.605548  9004 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:04:36.556959 15294 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:36.546895 15452 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:36.576713 15321 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:36.614221 15296 consensus_queue.cc:1059] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 13:04:36.619030 15423 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:36.591449 15382 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:36.644789 15441 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:36.654021 15441 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:36.654397 15441 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:36.657820 15455 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:36.658021 15450 consensus_queue.cc:1059] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:36.658494 15455 raft_consensus.cc:697] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:36.659246 15451 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:36.659780 15451 raft_consensus.cc:697] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:36.659436 15455 consensus_queue.cc:237] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:36.660632 15451 consensus_queue.cc:237] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:36.662458 15448 consensus_queue.cc:1048] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:36.665889 15403 raft_consensus.cc:3055] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:36.666347 15403 raft_consensus.cc:740] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:36.667125 15403 consensus_queue.cc:260] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:36.668483 15447 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:36.670342 15268 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:36.699337 15264 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.007s	sys 0.024s
W20260709 13:04:36.649123 15298 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.005s	sys 0.005s
I20260709 13:04:36.673992 15296 raft_consensus.cc:3055] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:36.612597 15332 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.123s
I20260709 13:04:36.641563 15288 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:36.757961 15288 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:36.650918 15452 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:36.651315 15431 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:36.768630 15419 raft_consensus.cc:697] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 25 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:04:36.772434 15407 consensus_meta.cc:287] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.532s	user 0.005s	sys 0.005s
I20260709 13:04:36.771847 15419 consensus_queue.cc:237] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:36.772861 15407 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:36.750996 15408 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:36.907238 15408 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:36.908504 15408 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:36.817389  8251 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 22.
I20260709 13:04:36.924679 15408 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:36.819286  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 354358us
W20260709 13:04:36.822036  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 228464us
W20260709 13:04:36.822912  8248 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.003s	sys 0.008s
W20260709 13:04:36.926769  8248 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.004s	sys 0.008s
I20260709 13:04:36.579059  8250 raft_consensus.cc:1275] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:04:36.928432  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:36.576763  8253 raft_consensus.cc:606] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:04:36.929978  8253 raft_consensus.cc:614] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:36.677806 15410 consensus_queue.cc:260] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:36.609320  8256 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:36.931871 15410 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:36.829742  8139 catalog_manager.cc:5028] TS 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): tablet 81804f59e3124d27998ea80fc1c36663 (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:04:36.932140 15413 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:36.933202  8139 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 103699us
W20260709 13:04:36.844353  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 401474us
W20260709 13:04:36.870347  8972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.015s	sys 0.000s
W20260709 13:04:36.877933  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 286097us
I20260709 13:04:36.879598  9009 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:36.879911  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:36.882567 14967 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.754340 15296 raft_consensus.cc:740] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:36.713771 15272 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:36.749779 15264 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.007s	sys 0.024s
I20260709 13:04:36.672521 14758 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:36.889094 15313 raft_consensus.cc:3055] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 24 LEADER]: Stepping down as leader of term 24
W20260709 13:04:36.758886 15318 consensus_meta.cc:287] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.986s	user 0.008s	sys 0.007s
W20260709 13:04:36.945760  8972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.016s	sys 0.000s
I20260709 13:04:36.899606 15321 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:36.904021  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:36.608291  8249 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:36.909581 15238 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:36.910043 15401 consensus_meta.cc:287] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.276s	user 0.011s	sys 0.001s
I20260709 13:04:36.911185  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 1249) took 1677 ms. Trace:
I20260709 13:04:36.916631  8254 raft_consensus.cc:1275] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:04:36.918229  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:36.774783 15452 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:36.974251 15318 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:36.948483  8295 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:36.949689  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:36.978507 15401 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:36.979100  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 659174us
I20260709 13:04:36.979707  9008 raft_consensus.cc:2368] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 19. Current term is 20.
I20260709 13:04:36.980638  8972 leader_election.cc:400] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 12 pre-election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 12. Current term is 13.
I20260709 13:04:36.777323 15441 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:36.972558  8404 rpcz_store.cc:276] 0709 13:04:35.215361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.789053 (+573692us) service_pool.cc:224] Handling call
0709 13:04:35.842633 (+ 53580us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:36.446058 (+603425us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:36.892747 (+446689us) spinlock_profiling.cc:243] Waited 603 ms on lock 0x7b5400276854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:36.893211 (+   464us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1386320768}
I20260709 13:04:36.982074 15434 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:36.982769 15434 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.983333 15434 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:36.983953 15441 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:36.984988 15417 consensus_queue.cc:1059] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:04:36.985625  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 50232.9us
I20260709 13:04:36.952015 15448 consensus_queue.cc:1059] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 24.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:36.785028 15432 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:36.965155  9005 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:04:36.945855 15313 raft_consensus.cc:740] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:36.994933  8369 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:36.998270 15417 raft_consensus.cc:3055] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:36.998849 15417 raft_consensus.cc:740] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:36.999895  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:37.000299 15417 consensus_queue.cc:260] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:36.986883 15401 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:37.001973 15467 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:36.994982 14967 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:37.003338 15467 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:37.003799 15313 consensus_queue.cc:260] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:37.006618 14898 consensus_queue.cc:1059] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:37.008455 15417 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:37.010129  8252 raft_consensus.cc:2393] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 20: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:37.014600  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:37.015483 15321 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:37.015890  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 3527) took 1550 ms. Trace:
I20260709 13:04:37.016466  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:04:37.017781  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.017261 15296 consensus_queue.cc:260] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:37.018213  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.019188 15296 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:37.020368 15407 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:37.022076  8295 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:37.025168  8372 leader_election.cc:400] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 19. Current term is 20.
I20260709 13:04:37.025602  8372 leader_election.cc:403] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:04:37.016381  8402 rpcz_store.cc:276] 0709 13:04:35.465468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.694996 (+229528us) service_pool.cc:224] Handling call
0709 13:04:37.015403 (+1320407us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b54001cc354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:37.015628 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1923688704}
I20260709 13:04:37.026806  8372 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:37.026942 15288 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:37.027714 15431 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.028367 15431 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.024945  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:37.031122 15431 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:37.031478 15401 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:37.031880 15449 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:37.032545 15449 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:37.033689 15313 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:37.036005 15252 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:36.988724 15473 log.cc:927] Time spent T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
W20260709 13:04:37.039754  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 52773us
I20260709 13:04:37.033164 15449 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:37.039155  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:36.992486 15318 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:37.053982  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:37.054430  8249 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:37.055899 15432 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:37.057331  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:37.064213  8404 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:04:37.066308  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 55815.7us
I20260709 13:04:37.075242 15382 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.075649  8972 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:37.076040  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:37.077261 15339 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:37.079073 15339 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:37.080107 15444 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:37.080842 15252 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:04:37.069753  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 45387.1us
I20260709 13:04:37.081455 15252 raft_consensus.cc:697] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 23 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:37.081964  8295 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:37.082626 15252 consensus_queue.cc:237] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:37.084123  8295 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:37.084473 15439 log.cc:927] Time spent T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.626s	user 0.000s	sys 0.003s
I20260709 13:04:37.075846 15382 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:37.086547  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:37.087378 15425 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:36.953044 15413 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.069078 15401 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 13:04:37.090384 15415 consensus_queue.cc:260] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.094138 15471 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:37.094489 15415 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:37.094822 15471 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.098536  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:37.091744 15413 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:37.092065  8401 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.942s	user 0.006s	sys 0.019s
I20260709 13:04:37.092587 15410 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:37.106405 15426 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:37.107216  8401 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.069s	user 0.007s	sys 0.019s
I20260709 13:04:37.107905 15455 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:37.108310  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3536) took 1213 ms. Trace:
I20260709 13:04:37.108485 15455 raft_consensus.cc:697] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 21 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:37.109719 15455 consensus_queue.cc:237] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:37.111851 15403 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:37.110119 15410 raft_consensus.cc:697] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 19 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:37.114339 15410 consensus_queue.cc:237] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:37.108744  8401 rpcz_store.cc:276] 0709 13:04:35.894621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.895264 (+   643us) service_pool.cc:224] Handling call
0709 13:04:35.895597 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:35.895889 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:36.035857 (+139968us) log.cc:844] Serialized 14 byte log entry
0709 13:04:37.107852 (+1071995us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:37.107957 (+   105us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:37.108002 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.108094 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:37.108250 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":467073,"prepare.run_cpu_time_us":205,"prepare.run_wall_time_us":1464,"raft.queue_time_us":2837,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":119,"thread_start_us":1192092,"threads_started":2}
I20260709 13:04:37.119309  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:37.120281 15426 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:37.121081  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 69889.2us
I20260709 13:04:37.121138 15426 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:37.121837  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:37.122936  8297 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:37.123790 15426 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:37.124446 15426 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.988545 15264 raft_consensus.cc:697] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:37.125298  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:37.097251 15471 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:37.132848  9004 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:04:37.125392 15421 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:37.139122  9005 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:37.104266 15288 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 19. Current term is 20.
W20260709 13:04:37.149277  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 48669.8us
I20260709 13:04:37.148795  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 1220) took 1039 ms. Trace:
W20260709 13:04:37.154536  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 70616.4us
I20260709 13:04:37.134985 14967 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:37.156229 15449 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.139938 15434 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:37.160117 15384 consensus_meta.cc:287] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.020s	user 0.000s	sys 0.020s
I20260709 13:04:37.125017 15426 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:37.160614  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:37.161373 15477 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:04:37.162120  8401 raft_consensus.cc:2393] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 20: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:37.162461 15477 consensus_queue.cc:1243] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:04:37.163292 15426 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:37.160622 15384 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.166872  8257 raft_consensus.cc:2393] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 16: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:37.166759  8972 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:37.168166  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 78843.8us
I20260709 13:04:37.169459  8369 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:37.169786 15384 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:37.170876 15384 consensus_queue.cc:1048] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:37.172592 15288 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:37.169305  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49220 (request call id 1214) took 1068 ms. Trace:
I20260709 13:04:37.173175 15288 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.151589  8248 rpcz_store.cc:276] 0709 13:04:36.108454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.112373 (+  3919us) service_pool.cc:224] Handling call
0709 13:04:36.112685 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:36.112930 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 13:04:36.927528 (+814598us) log.cc:844] Serialized 14 byte log entry
0709 13:04:37.047284 (+119756us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:37.050759 (+  3475us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:04:37.105572 (+ 54813us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.134109 (+ 28537us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:37.147422 (+ 13313us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54797,"prepare.queue_time_us":478645,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":3418,"raft.queue_time_us":87106,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":8166,"thread_start_us":803010,"threads_started":2,"wal-append.queue_time_us":164593}
I20260709 13:04:37.173393  8257 rpcz_store.cc:276] 0709 13:04:36.101407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.105053 (+  3646us) service_pool.cc:224] Handling call
0709 13:04:37.169011 (+1063958us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400406d54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:37.169217 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2438935936}
W20260709 13:04:37.177627 15405 consensus_meta.cc:287] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 1.016s	user 0.000s	sys 0.028s
I20260709 13:04:37.178727  8222 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:37.178786  8406 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:37.180683  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 3541) took 1172 ms. Trace:
I20260709 13:04:36.988897  8256 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:51220
I20260709 13:04:37.182041  8256 raft_consensus.cc:606] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:04:37.182476  8256 raft_consensus.cc:614] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:37.182741 15405 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.183365 15405 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:37.184731  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.173727 15288 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:37.181102  8406 rpcz_store.cc:276] 0709 13:04:36.008762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.008892 (+   130us) service_pool.cc:224] Handling call
0709 13:04:37.180435 (+1171543us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54002be154. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:37.180613 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2566814080}
I20260709 13:04:37.186224 15264 consensus_queue.cc:237] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.187129 15384 consensus_queue.cc:1048] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:37.188448 15403 consensus_queue.cc:1059] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:04:37.133709 15425 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 13:04:37.159571 15434 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:37.169279 15272 consensus_queue.cc:1059] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 13:04:37.181243  8222 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:37.181012 14898 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:37.227515  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:04:37.234287 15409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.871s	user 0.014s	sys 0.029s
W20260709 13:04:37.256052  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 74947.1us
I20260709 13:04:37.258273  8256 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:51220
I20260709 13:04:37.267625  9009 raft_consensus.cc:2393] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 20: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:37.219151 15405 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:37.263561  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:04:37.257323 15409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.994s	user 0.015s	sys 0.030s
I20260709 13:04:37.275244  9008 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 15.
I20260709 13:04:37.229630 15417 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:37.268507  8256 raft_consensus.cc:606] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:04:37.280977  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:37.257850 15413 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 13:04:37.285615  8222 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:37.289724  8256 raft_consensus.cc:614] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:37.219615 15421 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:04:37.288486  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:37.278829 15384 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:37.313676 14898 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:37.314214 14898 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.314554 14898 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:37.310495 15384 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.316694 15384 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:37.317662  8400 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:04:37.324925  8372 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:37.325374 15428 raft_consensus.cc:3055] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:04:37.325805 15428 raft_consensus.cc:740] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:37.326179 15471 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:37.326766 15471 raft_consensus.cc:697] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:37.326951 15415 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.327908 15471 consensus_queue.cc:237] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.328737 15318 raft_consensus.cc:3055] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:37.330437 15449 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:37.331887  8397 raft_consensus.cc:1275] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:04:37.329110  9003 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:37.333662 15444 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:37.329169 15318 raft_consensus.cc:740] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:37.335464 15318 consensus_queue.cc:260] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:37.337015 15318 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:37.337174  8298 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:37.372170  9005 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 25.
W20260709 13:04:37.379424  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 212795us
I20260709 13:04:37.329566 15415 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:37.393713  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 56737us
I20260709 13:04:37.393872  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:37.395067 15329 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:37.395640 15329 raft_consensus.cc:697] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:37.365906 15288 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:37.413082  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:37.396812 15329 consensus_queue.cc:237] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.420465  8406 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 20.
W20260709 13:04:37.443991  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 57083.5us
I20260709 13:04:37.427837  8397 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:04:37.326980 15428 consensus_queue.cc:260] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:37.451728 15483 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:04:37.452206 15428 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:37.454073 15425 raft_consensus.cc:3055] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:37.454566 15425 raft_consensus.cc:740] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:37.456224 15034 raft_consensus.cc:3055] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:37.456660 15034 raft_consensus.cc:740] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:37.457777 15034 consensus_queue.cc:260] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.458153 15382 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:37.459882  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:37.460857 15382 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:37.461023  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.462127  8331 raft_consensus.cc:2410] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 23 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
W20260709 13:04:37.462591  8220 leader_election.cc:336] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.925s (SENT)
I20260709 13:04:37.463480 15451 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:37.463855 15438 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.899s	user 0.017s	sys 0.000s
W20260709 13:04:37.464229 15438 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.068s	user 0.017s	sys 0.000s
I20260709 13:04:37.464181 15451 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:37.464740 15171 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.466166  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:37.467106  8326 raft_consensus.cc:2393] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 24: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:37.468171  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:37.468611 15452 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:37.469705  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.470489  8329 raft_consensus.cc:2393] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 21: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:37.470918 15452 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:37.471359 15408 consensus_queue.cc:1059] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:37.471653 15307 consensus_queue.cc:1059] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.201s
I20260709 13:04:37.472391 15307 consensus_queue.cc:1243] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 18.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 13:04:37.474211  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:37.474978  8328 raft_consensus.cc:2376] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23. Re-sending same reply.
W20260709 13:04:37.457791  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.478732 15034 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:37.481292 15449 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:37.482023 15449 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:37.484225 15449 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:37.484422 15413 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:37.484228  8972 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:37.484882 15171 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.485595  8256 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:51220
I20260709 13:04:37.485947 15417 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:37.486241  8256 raft_consensus.cc:606] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 18 LEADER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:04:37.486416 15417 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.486898 15417 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:37.487479 15171 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:37.488075 15171 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.488358  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:37.488729 15171 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:37.484186  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:37.490000 15321 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:37.490957  8223 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:37.492275  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.493361 15415 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:37.494009 15451 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:37.485164  8329 raft_consensus.cc:1240] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:04:37.494259 15425 consensus_queue.cc:260] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.16, Last appended by leader: 16, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.497608 14967 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.498572  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.506821  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:04:37.499447  8972 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:37.497080 15417 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.500869 15288 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:37.425359 15435 log.cc:927] Time spent T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:04:37.405565 15252 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.502485  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:37.464243 15477 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:37.503937 15402 raft_consensus.cc:3055] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:37.506879  8327 raft_consensus.cc:1240] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:37.506959  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.509209  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.489290  8326 raft_consensus.cc:2393] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 16: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:37.511938  8222 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:37.522464  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.523614  8330 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:37.527825 15419 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:37.528496  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:37.538100  9003 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:04:37.533625 15409 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
W20260709 13:04:37.543589  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 44299.8us
I20260709 13:04:37.544070 15415 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:37.543748 14898 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:37.546842  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:37.550359  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 59578.3us
I20260709 13:04:37.551956  8372 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:37.558101  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.559350  8325 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:04:37.559506 15477 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.562448  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:37.562294 15419 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:37.539317 15384 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:37.563897 15477 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:37.564486 15419 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:37.559944 15403 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.565210 15384 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:37.565358 15403 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:37.566032  8397 raft_consensus.cc:1240] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 19 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:04:37.552004 15272 raft_consensus.cc:2764] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:04:37.567086  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:37.568221  8323 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:37.569348  9001 raft_consensus.cc:1275] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:37.569671 15415 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:37.569911  8331 raft_consensus.cc:2393] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 22: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:37.570144 15415 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:37.570541 15415 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:37.570935 15384 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:37.570989  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 59215.2us
I20260709 13:04:37.572458  8222 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:37.573280 15272 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:37.574702 14898 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:37.575037 15299 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:37.575314 15288 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:37.557592 15455 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:37.576301  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 48067.2us
I20260709 13:04:37.576083 15455 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.577853 15490 consensus_queue.cc:1048] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:37.578449 15455 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:37.533885 15413 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 13:04:37.578742 15410 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:37.579370 15410 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:37.579762 15410 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:37.565716 14967 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.583091  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:37.575215 15425 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:37.569075 15252 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.583410  8972 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:37.583912  8223 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:37.578485  8369 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:37.547881 15402 raft_consensus.cc:740] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:37.593963  8328 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 23.
I20260709 13:04:37.586930  8406 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:37.597680  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:37.602123  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:37.574656 15417 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:37.607106  8330 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
I20260709 13:04:37.608461  9001 raft_consensus.cc:2410] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 17 because replica has last-logged OpId of term: 14 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 14.
I20260709 13:04:37.611984  8332 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 26. Current term is 27. Ops: [24.15-26.16]
I20260709 13:04:37.598735 15288 raft_consensus.cc:697] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:37.612789 15272 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:04:37.613082  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.613708 15272 raft_consensus.cc:697] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 25 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:37.613996 15288 consensus_queue.cc:237] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.614873  8405 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:37.616307  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:37.610685  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 52313.3us
I20260709 13:04:37.617107  8400 raft_consensus.cc:2393] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 20: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:37.618278 14758 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:37.618754 14758 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:37.619135  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:04:37.619360  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.620018  8222 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:37.621044 15321 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.621743  8222 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:37.622871  8256 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 22. Current term is 23. Ops: []
W20260709 13:04:37.623170  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 53753.6us
I20260709 13:04:37.624030 15034 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:37.624553 15034 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:37.625018 15034 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:37.614082  8324 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 23: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:37.621622 15321 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.615125 15272 consensus_queue.cc:237] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.640703  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:37.641893  9003 raft_consensus.cc:2393] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 16: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:37.644204 15321 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:37.644483  8295 leader_election.cc:400] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 24. Current term is 25.
I20260709 13:04:37.645709  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:37.646701  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:37.647765  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:37.648789  8251 raft_consensus.cc:2410] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:04:37.649165 15438 consensus_queue.cc:1048] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:37.649856 15438 consensus_queue.cc:1243] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer 8c42259d7c4b463a9f485f42c320c81b log is divergent from this leader: its last log entry 19.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:04:37.650102  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:37.650533 15171 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:37.650717  8255 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 22.
I20260709 13:04:37.651480  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.652048  8254 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:04:37.652110  8328 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:04:37.645747  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:37.653194 15402 consensus_queue.cc:260] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.653735  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:37.654130  8332 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 21 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:04:37.596122 15403 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:37.654330  8325 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:37.619155 14758 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:37.655236  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:37.648617  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:37.654727 15402 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:37.645776  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:37.647971 15477 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.659860  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:37.646711  8252 raft_consensus.cc:2393] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 17: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:37.662403  8257 raft_consensus.cc:2368] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16.
I20260709 13:04:37.662760  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.664145  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.644830 15321 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:37.665272  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:37.665988  8256 raft_consensus.cc:2368] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 22. Current term is 23.
I20260709 13:04:37.666033  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:37.666599  8326 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 13:04:37.667750  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.667603 15477 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.671430 15427 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:37.671916 15427 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:37.672291 15427 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:37.673604  8323 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:04:37.674441  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:37.675140  8250 raft_consensus.cc:2368] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 19. Current term is 20.
I20260709 13:04:37.677353  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:37.677992 15449 consensus_queue.cc:1059] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:37.678340  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:37.685204  8397 raft_consensus.cc:2393] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 22: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:37.658484 15419 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.691273  8325 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
I20260709 13:04:37.692353  8295 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:37.696241  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:37.698074  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:37.703148  8400 raft_consensus.cc:2393] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 19: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:37.699738  8252 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
W20260709 13:04:37.704175  8220 leader_election.cc:336] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.131s (SENT)
I20260709 13:04:37.704074  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.706367  8974 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:37.704604  8220 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:37.710378  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:37.711328  8252 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:04:37.712431  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:37.717734 14967 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:37.696022 15419 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.720443 15419 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:37.721050 15419 raft_consensus.cc:3055] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:37.721576 15477 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:37.725773 15419 raft_consensus.cc:740] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:37.713279  8252 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 23.
I20260709 13:04:37.728539  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:37.736016  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:37.736901  8252 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 23.
I20260709 13:04:37.738441  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.739255  8323 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:04:37.740113  8328 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 13 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:04:37.740187  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.740899  8327 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
I20260709 13:04:37.741485  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.741864  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:37.743243 15490 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.743450  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.743844 15490 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:37.744324  8327 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:37.733505 15403 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:37.741945  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.750181  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.745390  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.750963  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:37.747468  8974 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:04:37.749552  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 57937us
I20260709 13:04:37.752673  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.726872 15419 consensus_queue.cc:260] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:37.748477  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.751668  8397 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 21.
I20260709 13:04:37.738044  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:37.749315  8248 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:37.751478  8327 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:37.747967  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.747830  8329 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:04:37.761473 15264 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:37.761956 15264 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.762329 15264 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:37.748042 14967 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:37.763705 15490 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:37.764518  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:37.765734  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:37.766441  9003 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:37.766868 15034 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.768520  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:37.768908 15034 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:37.770759  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:37.771708  8298 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:37.772002  8251 raft_consensus.cc:2393] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 17: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:37.776465  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:37.747323 15409 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:37.780921 15409 raft_consensus.cc:697] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 20 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:37.782200 15409 consensus_queue.cc:237] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.782903  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:37.784173  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:37.784688  8329 raft_consensus.cc:1275] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:04:37.785691  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:37.786422  8329 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:37.749501 15410 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:37.790127  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.765353  8249 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:37.773123  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:37.785640  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:37.791354  8332 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:37.746184 15252 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:37.792505  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:37.793205  8251 raft_consensus.cc:2393] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 20: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:37.746487 15403 raft_consensus.cc:697] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:37.794800  8330 raft_consensus.cc:2393] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 21: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:37.797102  8325 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:04:37.797230  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:37.799579  8252 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:04:37.773202 15490 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:37.770555 15427 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.812019  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.819398  8330 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 25.
I20260709 13:04:37.820801 15455 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.821870 15455 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.823719  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:37.825124 15455 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:37.801821 14758 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:37.827574 14758 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:37.879822  8255 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 18: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:37.827828  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:37.828498  8330 raft_consensus.cc:2393] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 20: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:37.845602  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:37.858946  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.815598 15490 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.885663 15490 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:37.886627 15264 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.838541 15384 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:37.883813  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.891661  8252 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:37.893771  8325 raft_consensus.cc:2393] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 21: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:37.894764  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.896036  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.896835  8325 raft_consensus.cc:2410] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 23 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 13:04:37.899467  8250 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:37.904207 15490 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.905063  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.905292  8256 raft_consensus.cc:1275] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:04:37.907876  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:37.908480  8248 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:04:37.905464  8332 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:04:37.909262  8329 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:37.905890  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:37.910784  8254 raft_consensus.cc:2393] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 20: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:37.906904  8257 raft_consensus.cc:1275] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:04:37.918939  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:04:37.919514  8220 leader_election.cc:336] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 2.396s (SENT)
I20260709 13:04:37.919968  8251 raft_consensus.cc:2393] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 16: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:37.920089  8220 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:37.921886 15403 consensus_queue.cc:237] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:37.923547 15490 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:37.923695  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:37.924279  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:37.924474  9001 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:37.923834 15384 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:04:37.928437 15264 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:37.907292  8249 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
I20260709 13:04:37.932575  9001 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
W20260709 13:04:37.880316  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 196205us
I20260709 13:04:37.930589 15410 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:37.933709  8298 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:37.944162  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.945039  8332 raft_consensus.cc:2376] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23. Re-sending same reply.
I20260709 13:04:37.949920  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.950738  8329 raft_consensus.cc:2376] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17. Re-sending same reply.
I20260709 13:04:37.951903  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:37.952826  8330 raft_consensus.cc:2393] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 20: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:37.996404  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:04:37.954983 15492 log.cc:927] Time spent T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.413s	user 0.000s	sys 0.005s
W20260709 13:04:38.007022  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 48263.7us
I20260709 13:04:38.008049  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:38.008677  8250 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:38.010816  8220 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:37.995294 15438 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:38.018918  8295 leader_election.cc:400] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 22. Current term is 23.
I20260709 13:04:38.019371  8295 leader_election.cc:403] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 13:04:38.023515  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:38.025501  8325 raft_consensus.cc:2410] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 19 because replica has last-logged OpId of term: 17 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 15.
I20260709 13:04:38.025532  8295 leader_election.cc:400] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 15 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16.
I20260709 13:04:38.027215  8295 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:38.019927 15264 consensus_queue.cc:1048] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 13:04:38.024977 15477 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 22. Current term is 23.
I20260709 13:04:38.021854 15455 consensus_queue.cc:1059] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.012s
I20260709 13:04:38.037107 15425 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:04:38.043617 15497 log.cc:927] Time spent T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc: Append to log took a long time: real 0.111s	user 0.003s	sys 0.000s
I20260709 13:04:38.043711 15425 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:38.044298 15425 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:38.045464 15417 consensus_queue.cc:1059] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.004s
I20260709 13:04:38.047788  9001 raft_consensus.cc:1240] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:38.048327 15318 consensus_queue.cc:1059] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:38.049309 15318 raft_consensus.cc:3055] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:38.049647 15318 raft_consensus.cc:740] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:38.050482 15318 consensus_queue.cc:260] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:38.051687 15318 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:38.052999  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:38.052912 15490 raft_consensus.cc:3055] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:04:38.061367  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 406014us
I20260709 13:04:38.063855  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:38.086982  8323 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:38.030761 15415 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:38.097813  8220 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:38.033829 15339 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:04:38.062896 15490 raft_consensus.cc:740] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:38.106118  9002 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 22. Current term is 23. Ops: []
W20260709 13:04:38.108129  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 49655.9us
I20260709 13:04:38.108639 15384 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:38.111413 15427 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:38.113042 15419 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:38.116812  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:38.122709 15402 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.124881  9001 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 24. Current term is 25. Ops: []
I20260709 13:04:38.126495  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:38.130251 15413 raft_consensus.cc:3055] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:38.054919 15455 consensus_queue.cc:1059] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:04:38.136382  9009 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:38.137809 15415 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:38.104821 15471 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.141654 15405 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.142043 14758 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:38.154685  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 532990us
I20260709 13:04:38.158681  8369 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:38.150219 15455 raft_consensus.cc:3055] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:04:38.154803 15288 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.015s	sys 0.027s
I20260709 13:04:38.170820 14758 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:38.150782 15413 raft_consensus.cc:740] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:04:38.181349 15288 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.015s	sys 0.028s
I20260709 13:04:38.190914 14758 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:38.173619 15455 raft_consensus.cc:740] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:38.153391 15402 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:38.202941  8220 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:38.183597 15449 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:38.195802  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:38.195647 15413 consensus_queue.cc:260] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:38.198818  8402 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:04:38.216517  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:38.201764 15252 consensus_queue.cc:1059] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:04:38.202477  8369 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:38.193143 14758 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:38.158195 15405 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:38.224242 14758 raft_consensus.cc:697] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:38.158447 15471 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:38.232122  8402 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:04:38.226428 15288 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.232957 15288 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:38.233664 15507 raft_consensus.cc:3055] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:38.234071 15507 raft_consensus.cc:740] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:38.235292 15288 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:38.235112 15507 consensus_queue.cc:260] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:38.178812 15329 consensus_queue.cc:1059] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.037s
I20260709 13:04:38.236284 14758 consensus_queue.cc:237] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:38.236497 15507 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:38.240672 15318 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:38.241537 15428 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:38.242187  9004 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:38.242408 15428 consensus_queue.cc:1243] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer ff7a6fe533e94c1bae555b4cae427728 log is divergent from this leader: its last log entry 13.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:04:38.239646 15417 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.243403 15417 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:38.250360 15410 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:38.241770  8220 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:38.253067  8220 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:38.253633 15384 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:38.254043 15318 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:38.254207  8220 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:38.254491 15318 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.254913 15318 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:38.255059 15384 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:38.255559 15384 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.255916 15384 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:38.258102  8220 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:38.250558  8369 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:38.260172  8369 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:38.261055 15451 consensus_queue.cc:1059] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:04:38.261823 15410 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:38.262411 15410 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:38.262797 15410 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:38.268774  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:38.240355  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:38.281191  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:38.220834 15449 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.285996  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:38.289597  8250 raft_consensus.cc:2468] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:04:38.295953  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:38.296101  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:38.296738  8255 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:04:38.303580  8323 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
I20260709 13:04:38.303709  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:38.304562  8330 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 14. (term mismatch)
I20260709 13:04:38.303309  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:04:38.309206  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 51307.8us
I20260709 13:04:38.309602  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:38.305079  8330 pending_rounds.cc:85] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 13
I20260709 13:04:38.319077  8249 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 13:04:38.286715 15329 raft_consensus.cc:3055] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:04:38.286535 15449 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:38.323028  8329 raft_consensus.cc:1275] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 13:04:38.255188 15509 consensus_queue.cc:1059] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 13:04:38.310308  8330 pending_rounds.cc:107] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 22.14
I20260709 13:04:38.304653 15252 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:38.326304  8252 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:04:38.327805  8329 pending_rounds.cc:85] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 11
I20260709 13:04:38.373196  8329 pending_rounds.cc:107] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 13:04:38.373519  8329 raft_consensus.cc:2889] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:38.367048  8254 raft_consensus.cc:1275] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:04:38.325490  8220 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:38.267333 15425 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:38.375416  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 348352us
I20260709 13:04:38.306397 15272 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:38.320649 14967 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:38.370461  8330 raft_consensus.cc:2889] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:38.388924  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:38.396535  8255 raft_consensus.cc:1240] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:38.324100 15329 raft_consensus.cc:740] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:38.320027 15438 raft_consensus.cc:697] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:04:38.404378  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 217191us
W20260709 13:04:38.406169  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 80817.1us
I20260709 13:04:38.302991 15455 consensus_queue.cc:260] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:38.409108  8220 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:38.406390 15329 consensus_queue.cc:260] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:38.366328 15417 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:38.410633 15471 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:38.411366 15329 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:38.413848  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:38.416090 15272 consensus_queue.cc:1048] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:38.379344 15516 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:38.420790  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:38.421260  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:38.424387  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:38.427055  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:38.446636  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:38.453680  8254 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:04:38.454850  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:38.455562  9005 raft_consensus.cc:1240] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:38.498349  8297 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:38.425949 15288 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:38.585920  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:38.509961  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:38.618467  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:38.432618 15508 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:38.568164  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:38.509291  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:38.426256 15428 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:04:38.652379  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 244536us
I20260709 13:04:38.568353  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:38.648581  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 239535us
I20260709 13:04:38.641923  8398 pending_rounds.cc:85] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 10
I20260709 13:04:38.660107  8398 pending_rounds.cc:107] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:04:38.660547  8398 raft_consensus.cc:2889] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:38.507457  8369 leader_election.cc:400] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 19. Current term is 20.
I20260709 13:04:38.554883 14758 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:38.667198 14758 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:38.667717 14758 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:38.594908  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:38.665490  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:38.559754  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:38.630852  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:38.529285 15487 log.cc:927] Time spent T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:04:38.670914  8254 raft_consensus.cc:2393] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 23: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:38.669893  8405 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:38.670738  8329 raft_consensus.cc:2376] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19. Re-sending same reply.
I20260709 13:04:38.596266  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:38.628322 15535 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.500872 15252 raft_consensus.cc:697] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:38.680429  8220 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:38.505854 15417 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:38.670524  8326 raft_consensus.cc:2376] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 21. Re-sending same reply.
I20260709 13:04:38.704061  8254 pending_rounds.cc:85] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b: Aborting all ops after (but not including) 11
I20260709 13:04:38.706339  8254 pending_rounds.cc:107] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 13:04:38.707082  8254 raft_consensus.cc:2889] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:38.530912 15509 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.554756 15425 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:38.555647 15402 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:38.712265  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:38.559185 15452 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:38.659116 15428 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.591355 15403 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:38.716324 15519 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.185s	user 0.000s	sys 0.000s
I20260709 13:04:38.716028  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:38.595261 15405 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:38.730855  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:38.696882 15502 log.cc:927] Time spent T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:04:38.433000 15516 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:38.733415  8257 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:38.729611  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:38.748504  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 246158us
I20260709 13:04:38.649379 15488 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.065s
W20260709 13:04:38.750941  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:38.657461 15299 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.072s
I20260709 13:04:38.807142  8330 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:04:38.799896  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:38.799075  8324 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 16.
W20260709 13:04:38.807966  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 380675us
I20260709 13:04:38.808993  8327 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:04:38.758164 15318 consensus_meta.cc:287] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.502s	user 0.016s	sys 0.000s
I20260709 13:04:38.737346 15539 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.755380 15428 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:38.699802 15508 consensus_queue.cc:1048] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.121s
I20260709 13:04:38.565886 15438 consensus_queue.cc:237] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:38.799779 15527 log.cc:927] Time spent T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
W20260709 13:04:38.862571  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 193274us
I20260709 13:04:38.864032  8256 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:04:38.843209  8405 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 22.
I20260709 13:04:38.825332  8297 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:38.881812  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 220138us
I20260709 13:04:38.891178  8252 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 21.
I20260709 13:04:38.891718 14758 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:38.893491  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:38.896883  8257 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:38.898480  8251 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:38.906245  9001 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
W20260709 13:04:38.908561  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 108738us
I20260709 13:04:38.909674  8369 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:38.914922  8251 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:38.915391  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:38.705644 15535 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:38.963430  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 138229us
W20260709 13:04:38.964264  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 260841us
I20260709 13:04:38.791479 15417 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:38.764926 15403 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:38.791646 15509 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:38.971115  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:38.987236  8324 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 18 index: 16. (term mismatch)
W20260709 13:04:38.985409 15532 log.cc:927] Time spent T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:04:38.822631 15318 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:38.996448  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 86886us
I20260709 13:04:38.997255  8369 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:04:38.970407 15502 log.cc:927] Time spent T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:04:38.990497 14758 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:38.987669  8324 pending_rounds.cc:85] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 15
I20260709 13:04:39.007686  8324 pending_rounds.cc:107] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 17.16
I20260709 13:04:39.008021  8324 raft_consensus.cc:2889] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:38.985419 15520 log.cc:927] Time spent T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.326s	user 0.000s	sys 0.000s
I20260709 13:04:39.006036  8297 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:39.010195  8251 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 13:04:39.003183 15507 consensus_meta.cc:287] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.762s	user 0.007s	sys 0.004s
I20260709 13:04:39.016628  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:39.014638 14758 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:39.053968 14758 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:39.057063 15471 raft_consensus.cc:3055] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:39.057614 15471 raft_consensus.cc:740] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:39.057314 14758 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:39.058610 15471 consensus_queue.cc:260] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:39.059643 15471 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:39.059823 15547 log.cc:927] Time spent T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:04:39.061902  8222 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:38.901885 15539 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:39.025362  8399 raft_consensus.cc:1275] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:04:39.010643 15542 log.cc:927] Time spent T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:04:39.029042 15417 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:39.026669 15531 log.cc:927] Time spent T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
W20260709 13:04:39.006965 15384 consensus_meta.cc:287] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.751s	user 0.003s	sys 0.009s
I20260709 13:04:39.018016 15427 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:38.965896 15252 consensus_queue.cc:237] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:39.031845 15556 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:39.008999 15405 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:39.048319  8297 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:39.040125 15490 consensus_queue.cc:260] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:38.990188 15541 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:39.049029  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:39.083778 15299 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:39.108389  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 196834us
I20260709 13:04:39.111572  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:39.122726  8405 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:39.122792  8220 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:39.094272 14758 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:39.118911  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:39.083767 15425 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.083563 15477 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:39.119472  8972 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:39.085048 15529 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:39.133000  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:39.136601  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:39.100039 15417 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:39.141762  8295 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:39.146682 14758 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:39.148087  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:39.154912  8328 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.012s	sys 0.027s
W20260709 13:04:39.155292  8328 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.012s	sys 0.027s
I20260709 13:04:39.155020  8399 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
W20260709 13:04:39.160390  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 112189us
I20260709 13:04:39.166534  8324 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 20.
I20260709 13:04:39.143683 15528 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:39.144523 15508 raft_consensus.cc:2764] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:04:39.110523 15384 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:39.190641  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:39.191584  8399 raft_consensus.cc:2393] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 20: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:39.191747 15384 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:39.178592  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:39.193496  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:39.195240 15539 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:39.195920 15549 log.cc:927] Time spent T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:04:39.196971  8255 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 24: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:04:39.204140  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 47885.4us
I20260709 13:04:39.163383 15490 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:39.169193 15427 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:39.169463 15451 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:39.153052 15402 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:04:39.169309 15264 raft_consensus.cc:2804] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:39.145133 15529 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:39.269816 15535 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:39.272182 15529 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:39.221647 15509 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:39.265453  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:39.264354  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:39.275301  8398 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:39.227030  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:39.149235 15425 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:39.279632  8403 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
I20260709 13:04:39.228536  8400 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:04:39.234440 15318 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:39.239235  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:39.284689  8326 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 27: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:04:39.243619 15419 consensus_meta.cc:287] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 1.091s	user 0.010s	sys 0.003s
W20260709 13:04:39.245975  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 47780.2us
I20260709 13:04:39.195950 15477 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:39.251780 15410 consensus_meta.cc:287] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.966s	user 0.006s	sys 0.003s
I20260709 13:04:39.182173 15528 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:04:39.251849  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 46696us
I20260709 13:04:39.158077 15299 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:39.255738  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:39.299687  8406 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 27: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:39.257851 15455 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:39.257936 15403 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:39.216094 15548 consensus_queue.cc:1059] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.243s
I20260709 13:04:39.212689 15516 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:39.153242 15417 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:39.285745  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 122917us
I20260709 13:04:39.286998  8972 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:39.250293 15552 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.269641 15264 raft_consensus.cc:697] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:39.216869 15427 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.302675  8220 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:39.304951  8402 raft_consensus.cc:2410] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:04:39.307564  8404 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 23.
I20260709 13:04:39.308223  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:39.323971  8403 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:39.259320 15549 log.cc:927] Time spent T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:04:39.315902  8405 raft_consensus.cc:2410] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:04:39.313710  8297 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:04:39.307406 15329 consensus_meta.cc:287] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.892s	user 0.012s	sys 0.000s
I20260709 13:04:39.298079 15528 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:39.296373 15410 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:39.339998  8297 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:04:39.346275  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 349075us
I20260709 13:04:39.348627  8369 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:39.350844  8297 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:39.321712 15554 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:39.357822  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:39.334239 15427 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:39.363955  8222 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:39.358593  8406 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:04:39.356333  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:39.366070  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 63495.9us
I20260709 13:04:39.367264  8220 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:39.374434  8220 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:39.362694 15449 consensus_meta.cc:287] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.969s	user 0.012s	sys 0.000s
I20260709 13:04:39.378129  8222 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:39.381512  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 239879us
I20260709 13:04:39.354440 15548 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:39.357685 15318 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:39.357414 15471 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:39.353793 15516 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:39.387662  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:04:39.388156  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 70495.9us
I20260709 13:04:39.372921 15508 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:39.407941 15574 raft_consensus.cc:2764] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:04:39.409201 15557 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:39.410336 15583 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:39.410802 15583 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:39.411724  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:39.411808 15583 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:39.413347  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:39.417567  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:39.383122 15556 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:39.415688  8400 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:39.382965 15548 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.384855 15272 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:39.347786 15552 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:39.389714 15471 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:39.414812  8222 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:39.423228  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 72496.8us
I20260709 13:04:39.416011 15413 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:39.451398  8297 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:39.436100  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:39.437813 15299 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:39.458748  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:39.459239 15288 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:39.459741 15288 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:39.458776  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:39.440418  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:39.443205 14967 consensus_meta.cc:287] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.045s	user 0.011s	sys 0.000s
I20260709 13:04:39.457959  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:39.459383  8401 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:39.461783  8398 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
W20260709 13:04:39.442886  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:39.451524  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:39.471170 15577 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.471342  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:39.471796  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:39.471988  8399 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:39.471678 15577 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:39.472935  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:39.461575 15575 consensus_queue.cc:1059] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 13:04:39.474022 15577 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:39.429137 15556 raft_consensus.cc:697] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 23 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:39.478223 15556 consensus_queue.cc:237] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:39.480458 15403 raft_consensus.cc:3055] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:39.480893 15403 raft_consensus.cc:740] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:39.390549 15264 consensus_queue.cc:237] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:39.482252 15410 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:39.486045 15417 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:39.486399  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:39.487273  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:39.487977  8398 raft_consensus.cc:2393] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 20: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:39.488771  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:39.490983  8297 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:39.491413 15545 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:39.491727 15402 consensus_queue.cc:1048] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:39.492096 15545 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:39.492507 15545 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:39.492822 15545 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:39.426802 15516 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:39.496137  8404 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34462 (request call id 1277) took 1213 ms (client timeout 1563 ms). Trace:
I20260709 13:04:39.496711 15516 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:39.496554  8404 rpcz_store.cc:269] 0709 13:04:38.283224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.283371 (+   147us) service_pool.cc:224] Handling call
0709 13:04:39.495975 (+1212604us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b54001e7654. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:39.496078 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1749411200}
I20260709 13:04:39.452698 15471 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:39.384630 15449 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:39.497612 15575 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:39.499112 15449 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:39.499747  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 95876.5us
I20260709 13:04:39.499825  8297 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:39.501797  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:39.502060  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:39.502640 15570 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:39.502935  9005 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:04:39.502686  9009 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 26.
I20260709 13:04:39.510807  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:39.511598  8398 raft_consensus.cc:2410] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:04:39.512419  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:39.512987 15550 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.513603  8398 raft_consensus.cc:2410] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 23 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:04:39.513545 15550 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:39.514766  8398 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:04:39.515751 15550 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:39.516419 15550 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.424959 15548 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:39.516906 15550 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:39.517679 15425 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:39.518314  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 114166us
W20260709 13:04:39.518567  8220 connection.cc:256] RPC call timeout handler was delayed by -1.1174529009999787s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15980000000000003s for a response
I20260709 13:04:39.519039  8972 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:39.421949 15508 raft_consensus.cc:697] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:39.515882  8398 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:39.521670 15561 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.522367 15384 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.522192 15508 consensus_queue.cc:237] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:39.522881 15384 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:39.524837  8331 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 23.
I20260709 13:04:39.525098 15538 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.526749  8402 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 15.
I20260709 13:04:39.513140  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:39.533689  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:39.534030  8325 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:04:39.536897  8249 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 27.
W20260709 13:04:39.527617  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 57591.9us
I20260709 13:04:39.537835  8254 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 13:04:39.463804  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 49119.5us
I20260709 13:04:39.542922  8253 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:39.544498  8369 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:39.545769  8401 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 21.
I20260709 13:04:39.546917  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:39.548285  8406 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 24.
W20260709 13:04:39.550410  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:39.552021  8248 raft_consensus.cc:1275] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:04:39.553488  8404 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:39.547690  8402 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:04:39.558905  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:39.559558  8255 raft_consensus.cc:1240] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:39.556080  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:39.563795  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:39.565775  8369 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:39.534147 15452 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.577191  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:39.526522 14967 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:39.539446 15569 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 13:04:39.534951 15575 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:39.574152  8402 raft_consensus.cc:2393] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 19: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:39.579892  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:39.584803  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:39.545723 15417 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:39.550751 15252 raft_consensus.cc:2764] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 13:04:39.586040  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:39.589243  8974 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:39.587554  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:39.588012  8220 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:39.589133  8405 raft_consensus.cc:1240] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:39.562738 15516 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:04:39.580803 15428 consensus_meta.cc:287] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.674s	user 0.012s	sys 0.000s
I20260709 13:04:39.597481  8220 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:39.598321  8249 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
I20260709 13:04:39.590561  8399 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:04:39.540071 15299 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 13:04:39.539588 15419 consensus_queue.cc:1059] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 13:04:39.601816  9004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 1387) took 1319 ms. Trace:
I20260709 13:04:39.581843 15557 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.603304  9004 rpcz_store.cc:276] 0709 13:04:38.275115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.275257 (+   142us) service_pool.cc:224] Handling call
0709 13:04:38.275518 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:38.275709 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:04:38.625597 (+349888us) log.cc:844] Serialized 14 byte log entry
0709 13:04:38.656978 (+ 31381us) log.cc:844] Serialized 14 byte log entry
0709 13:04:39.474054 (+817076us) spinlock_profiling.cc:243] Waited 6.62 ms on lock 0x7b180045c940. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d1522 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:04:39.478631 (+  4577us) spinlock_profiling.cc:243] Waited 5.49 ms on lock 0x7b180045c940. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256ce455 00007f88287b6060 00007f88287c8716 00007f88287d9287 00007f88287d9208
0709 13:04:39.593947 (+115316us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:39.594021 (+    74us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:39.594094 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.594184 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:39.594331 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23840,"prepare.queue_time_us":147632,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":9084,"raft.queue_time_us":72228,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":25443,"spinlock_wait_cycles":27860736,"thread_start_us":425528,"threads_started":2}
I20260709 13:04:39.555048 15415 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 13:04:39.522233 15561 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:39.593629 15449 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:39.584838 15477 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:39.682053 15579 log.cc:927] Time spent T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
I20260709 13:04:39.703758  8222 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:39.612686  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:39.607486  8401 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:39.697082  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:39.595949 15516 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:39.545404 15570 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.053s
I20260709 13:04:39.606120 15428 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:39.660754 15509 raft_consensus.cc:3055] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:39.660488 15562 consensus_queue.cc:1059] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.084s
I20260709 13:04:39.663712 15537 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:39.665544 15425 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.666270 15524 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:39.669631 15575 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.694584  8325 pending_rounds.cc:85] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 13
W20260709 13:04:39.660537  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 73062.9us
I20260709 13:04:39.746415  8325 pending_rounds.cc:107] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 13.14
I20260709 13:04:39.704797  8405 raft_consensus.cc:2410] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:04:39.746832  8325 raft_consensus.cc:2889] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:39.697494  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:39.748469  8369 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:39.707341  8327 raft_consensus.cc:2410] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:04:39.708039  8403 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:39.701850  8297 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:39.602303 15488 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:04:39.611394  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 82036.8us
I20260709 13:04:39.652909 15264 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:39.611016 15551 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:39.583379 15452 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:39.726565 15329 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:39.543717 15538 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:39.592689 15417 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:39.700469 15405 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:39.710915 15516 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:39.681725 15553 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:39.680299 15272 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:39.765767 15410 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:39.677402 15552 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:39.726733 15509 raft_consensus.cc:740] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:39.679442 15384 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:39.754109 15402 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:39.780186  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:39.783419  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 79731.9us
I20260709 13:04:39.765844 15554 raft_consensus.cc:3055] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:39.711483 15477 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:39.798097 14758 consensus_queue.cc:1059] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:39.774991 15551 raft_consensus.cc:697] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 23 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:39.784550  8222 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:39.754418 15425 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:39.809408  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 212037us
I20260709 13:04:39.785457 15490 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:39.778481 15410 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:39.799700 15553 raft_consensus.cc:697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 19 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:39.789655 15570 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:39.781987 15550 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:39.822296 14758 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.834303 14758 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:39.835867 15425 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:39.836275 14758 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:39.824641 15509 consensus_queue.cc:260] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:39.776696 15419 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:04:39.837678 15509 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:39.839402 15584 log.cc:927] Time spent T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:04:39.802376 15577 consensus_queue.cc:1048] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:04:39.845821  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 1273) took 1660 ms. Trace:
I20260709 13:04:39.846014 15288 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:39.847159  8403 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:39.846390  8397 rpcz_store.cc:276] 0709 13:04:38.185059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.185183 (+   124us) service_pool.cc:224] Handling call
0709 13:04:38.194054 (+  8871us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:38.215622 (+ 21568us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:38.215654 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:38.240504 (+ 24850us) log.cc:844] Serialized 26 byte log entry
0709 13:04:39.313661 (+1073157us) spinlock_profiling.cc:243] Waited 1.6 ms on lock 0x7b1800357b20. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882569f2a6 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9
0709 13:04:39.446066 (+132405us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b1800357b20. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e0d1 00007f8828779a0f
0709 13:04:39.542642 (+ 96576us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:39.542679 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.542742 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:39.845516 (+302774us) raft_consensus.cc:1685] finished
0709 13:04:39.845545 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:39.845735 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":35659,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":19404,"spinlock_wait_cycles":56148480,"thread_start_us":96401,"threads_started":1,"wal-append.queue_time_us":1459931}
I20260709 13:04:39.848232 15427 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:39.788757 15554 raft_consensus.cc:740] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:39.785238 15329 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:39.852691 15329 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:39.817085 15537 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.856135 15561 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:39.787851 15405 raft_consensus.cc:697] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:39.825910 15471 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:39.857372 15288 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:39.857580 15405 consensus_queue.cc:237] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:39.859154 15471 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:39.829548 15560 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.596s	user 0.000s	sys 0.005s
I20260709 13:04:39.829954 15410 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:39.763979 15264 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:39.867617  8397 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:04:39.871061  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 1154) took 1256 ms. Trace:
I20260709 13:04:39.757161 15575 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:39.803625 15535 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.067s
I20260709 13:04:39.795581 15402 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:39.916306  8397 raft_consensus.cc:1275] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:39.805049 15569 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.101s
I20260709 13:04:39.821786 15490 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.660562 15557 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:39.921226  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 219487us
I20260709 13:04:39.926707  8297 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:39.921370  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 103953us
I20260709 13:04:39.749022 15524 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:39.871608  8332 rpcz_store.cc:276] 0709 13:04:38.605412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.605585 (+   173us) service_pool.cc:224] Handling call
0709 13:04:38.606034 (+   449us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:38.662714 (+ 56680us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:38.662767 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:38.668229 (+  5462us) log.cc:844] Serialized 52 byte log entry
0709 13:04:38.970481 (+302252us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:04:38.981167 (+ 10686us) log.cc:844] Serialized 14 byte log entry
0709 13:04:38.981674 (+   507us) log.cc:844] Serialized 14 byte log entry
0709 13:04:38.981896 (+   222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.982016 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:39.664820 (+682804us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:04:39.861458 (+196638us) raft_consensus.cc:1685] finished
0709 13:04:39.861483 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:39.861627 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27066,"prepare.queue_time_us":10785,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":73,"raft.queue_time_us":40522,"raft.run_cpu_time_us":414,"raft.run_wall_time_us":54831,"spinlock_wait_cycles":7140096,"thread_start_us":159202,"threads_started":1,"wal-append.queue_time_us":362743}
I20260709 13:04:39.928210  8222 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:39.906164 15419 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:39.905830 15587 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:39.854573 15537 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:39.830119 15551 consensus_queue.cc:237] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:39.941219  8297 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:39.943318  8397 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:39.944191  8248 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
I20260709 13:04:39.945006  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:39.928797 15490 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:39.926378 15471 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:39.978154  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 50070.5us
W20260709 13:04:39.979909  8400 consensus_meta.cc:287] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.554s	user 0.009s	sys 0.001s
I20260709 13:04:39.850322 15554 consensus_queue.cc:260] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:39.980507  8400 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
I20260709 13:04:39.861831 15403 consensus_queue.cc:260] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:39.831048 15553 consensus_queue.cc:237] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:39.982975  8297 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:39.979319  8222 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:39.982853  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:39.989197  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 208242us
I20260709 13:04:39.990846  8332 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
W20260709 13:04:39.990782  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 215350us
I20260709 13:04:39.996291  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:39.982719 15587 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:40.001273  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:40.005582  8332 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
I20260709 13:04:40.006768  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:40.007496  8332 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 27.
I20260709 13:04:40.008507  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:40.009158  8332 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:39.942328 15419 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:39.990236 15576 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:39.986249 15471 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:40.023347  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.005389 15554 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:40.003841 15272 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:40.003434 15428 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:40.033659  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:40.002978 15552 consensus_queue.cc:1059] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:04:40.047346 15477 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:40.048153 15477 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:40.048579 15477 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:40.048985 15477 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:40.053824 15557 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:40.058058 15596 log.cc:927] Time spent T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:04:40.030848 15403 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:40.070041  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.070289  8332 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:04:40.087630  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:40.031412 15598 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:40.088375  8332 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:40.095527 15537 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:40.031049 15576 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:40.096920  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:40.098114  9004 raft_consensus.cc:2410] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:04:40.098822 15524 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:40.099941 15575 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:40.076810 15528 consensus_meta.cc:287] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.716s	user 0.010s	sys 0.000s
I20260709 13:04:40.045058 15272 raft_consensus.cc:697] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 24 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:40.150278  8332 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 24.
I20260709 13:04:40.156037  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:40.159373  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:40.087718 15570 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:40.156898  8324 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:04:40.083791 15538 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:40.161559  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:40.162317  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 160479us
I20260709 13:04:40.165580  8222 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:40.153445 15587 raft_consensus.cc:3055] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:40.148276 15425 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:40.172320  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 189431us
I20260709 13:04:40.168666  8369 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:40.171031  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:40.175588  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.175680  8369 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:40.168371 15557 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:40.175452  8324 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:40.159256 15471 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:40.178203  8220 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:40.176527 15548 consensus_meta.cc:287] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.659s	user 0.005s	sys 0.006s
I20260709 13:04:40.156527 15598 raft_consensus.cc:697] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 21 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:40.152988 15528 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:40.224351  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:40.236382  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.171275 15264 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:40.169592 15587 raft_consensus.cc:740] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:40.299971 15583 consensus_meta.cc:287] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.887s	user 0.012s	sys 0.000s
I20260709 13:04:40.301548 15598 consensus_queue.cc:237] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:40.304301  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 126158us
I20260709 13:04:40.180148 15405 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:40.166386 15538 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 13:04:40.320134  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:40.344847  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 169242us
I20260709 13:04:40.346115  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:40.338670  8328 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:40.357896  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:40.358577  8324 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:40.360509  8369 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:40.325711 15608 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:40.367424  8369 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:40.351990 15413 consensus_meta.cc:287] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.907s	user 0.008s	sys 0.004s
I20260709 13:04:40.353523 15537 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:40.370756 14758 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:40.220716 15548 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:40.328528 15583 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:40.408097 15528 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:40.413725 15449 consensus_meta.cc:287] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.701s	user 0.007s	sys 0.004s
I20260709 13:04:40.381693 14967 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.361618 15272 consensus_queue.cc:237] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:40.389518  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.392503  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3565) took 1128 ms. Trace:
I20260709 13:04:40.378192 15537 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:40.395080  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3559) took 1178 ms. Trace:
I20260709 13:04:40.373592 15602 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:04:40.362251 15471 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:40.372583  8369 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:04:40.478104  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 105591us
I20260709 13:04:40.479432  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:40.378278  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:40.481106  8369 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:40.368623 15608 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.426548  8222 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:40.434355  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:40.482918  8397 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:04:40.377784 15562 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.494730 15562 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:40.436067  8326 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 22: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:40.438905  8402 rpcz_store.cc:276] 0709 13:04:39.263768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.591584 (+327816us) service_pool.cc:224] Handling call
0709 13:04:39.591919 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:39.763156 (+171237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:39.763188 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:39.763235 (+    47us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:39.763262 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.392239 (+628977us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b5400277c54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:40.392294 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.392439 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":393361664}
I20260709 13:04:40.496645  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:40.497772  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:40.498351  8402 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:40.499442  8402 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:40.500664  8402 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:40.441069  9002 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 13:04:40.501796 15594 log.cc:927] Time spent T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.339s	user 0.000s	sys 0.001s
I20260709 13:04:40.503300 15608 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:40.503587  8404 raft_consensus.cc:1240] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:40.504144 15583 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:40.466373 14967 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:40.505105  8404 raft_consensus.cc:1275] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:04:40.506769 14967 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:40.506953  8404 raft_consensus.cc:1275] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:40.441754  8329 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 24: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:40.443575 15548 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:40.445902 15591 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:40.448976 15545 consensus_meta.cc:287] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.956s	user 0.012s	sys 0.000s
I20260709 13:04:40.364125 15535 consensus_queue.cc:1048] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
W20260709 13:04:40.456645 15516 consensus_meta.cc:287] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.670s	user 0.009s	sys 0.001s
W20260709 13:04:40.465390 15605 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 13:04:40.466621  8406 rpcz_store.cc:276] 0709 13:04:39.217215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.583135 (+365920us) service_pool.cc:224] Handling call
0709 13:04:39.583456 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:39.657544 (+ 74088us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:39.657602 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:39.744438 (+ 86836us) log.cc:844] Serialized 26 byte log entry
0709 13:04:39.994733 (+250295us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:39.994772 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.994836 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:40.394877 (+400041us) raft_consensus.cc:1685] finished
0709 13:04:40.394897 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.395022 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3390,"raft.queue_time_us":63124,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":49600,"thread_start_us":212281,"threads_started":1,"wal-append.queue_time_us":406395}
W20260709 13:04:40.487890 15509 consensus_meta.cc:287] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.650s	user 0.008s	sys 0.000s
I20260709 13:04:40.489719 15405 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:40.490198 14758 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:04:40.492233  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 65798.4us
I20260709 13:04:40.492890  8399 raft_consensus.cc:2410] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:04:40.366415 15288 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:40.507318 14967 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:40.424871 15611 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:40.512182 15427 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 26 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:40.517122 15417 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:40.483963 15537 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:40.521008  8327 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 1159) took 1632 ms (client timeout 1594 ms). Trace:
W20260709 13:04:40.521361  8331 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41664 (request call id 1161) took 1525 ms (client timeout 1562 ms). Trace:
W20260709 13:04:40.535555  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 54570.9us
I20260709 13:04:40.572135  8369 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:40.544731 14758 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.574651 14758 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:40.545758  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 1288) took 1411 ms. Trace:
I20260709 13:04:40.548097  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:40.548609 15427 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:40.580402  8324 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:04:40.580157 15427 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:40.569152  8327 rpcz_store.cc:269] 0709 13:04:38.888015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.888179 (+   164us) service_pool.cc:224] Handling call
0709 13:04:40.520708 (+1632529us) spinlock_profiling.cc:243] Waited 791 ms on lock 0x7b540008a554. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:40.520844 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1819124096}
W20260709 13:04:40.569936  8331 rpcz_store.cc:269] 0709 13:04:38.995808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.995948 (+   140us) service_pool.cc:224] Handling call
0709 13:04:40.521198 (+1525250us) spinlock_profiling.cc:243] Waited 425 ms on lock 0x7b54003c8f54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:40.521291 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":976978176}
I20260709 13:04:40.547888  8222 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:40.554113  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:40.562181  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:40.584599  9000 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:40.563166  8402 raft_consensus.cc:1240] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:40.573577  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:40.589090  8397 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:04:40.575708  8398 rpcz_store.cc:276] 0709 13:04:39.134912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.523028 (+388116us) service_pool.cc:224] Handling call
0709 13:04:39.523308 (+   280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:39.558622 (+ 35314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:39.558669 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:39.573595 (+ 14926us) log.cc:844] Serialized 26 byte log entry
0709 13:04:39.989172 (+415577us) spinlock_profiling.cc:243] Waited 4.55 ms on lock 0x7b18003223c0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e0d1 00007f8828779a0f
0709 13:04:40.045898 (+ 56726us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:40.045939 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.046026 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:40.545379 (+499353us) raft_consensus.cc:1685] finished
0709 13:04:40.545413 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.545684 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31568,"raft.queue_time_us":29163,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":11632,"spinlock_wait_cycles":10476288,"thread_start_us":56519,"threads_started":1,"wal-append.queue_time_us":593052}
I20260709 13:04:40.588351  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:40.593899 15535 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:40.594897  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:40.595134  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:40.595705  8331 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:40.595975  9004 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
W20260709 13:04:40.517392 15490 consensus_meta.cc:287] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.526s	user 0.006s	sys 0.000s
I20260709 13:04:40.594434 15535 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.608837  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:40.548278 15611 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:40.535725 15602 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:04:40.544579 15405 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:40.519309 15561 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:40.610293  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:40.439442 15449 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:40.604581  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:40.681056  8327 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
W20260709 13:04:40.551753 15580 log.cc:927] Time spent T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 1.133s	user 0.012s	sys 0.000s
I20260709 13:04:40.631733 14758 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:40.651355  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:40.603485  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:40.684341  8331 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:40.653997  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:40.689352  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:40.598112 15410 consensus_meta.cc:287] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.691s	user 0.007s	sys 0.004s
I20260709 13:04:40.697615 14758 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:40.698750  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:40.610381 15318 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:40.612504 15573 log.cc:927] Time spent T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 1.123s	user 0.005s	sys 0.010s
I20260709 13:04:40.707705  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:40.680102 15602 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.708443  8327 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:40.713829  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:40.543671 15516 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:40.703498  8398 raft_consensus.cc:2393] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 16: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:40.600750 15288 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:40.597486 15509 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:40.682025 15451 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.705559  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:40.659678 15535 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:40.521641 15477 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:40.535247 15591 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:40.732501  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.684116 15405 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:40.548199 14967 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:40.760279 15288 raft_consensus.cc:697] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:40.542857 15545 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:40.770291 15509 raft_consensus.cc:697] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 24 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:40.704475 15471 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:40.778054  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:40.781847 15536 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.786096  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:40.799412  8397 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 22: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:40.800597  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:40.787122  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 1241) took 2063 ms. Trace:
I20260709 13:04:40.705055 15410 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:40.801249  8256 rpcz_store.cc:276] 0709 13:04:38.723318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.968263 (+244945us) service_pool.cc:224] Handling call
0709 13:04:38.968718 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:38.968992 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:38.969028 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:38.991415 (+ 22387us) log.cc:844] Serialized 26 byte log entry
0709 13:04:39.352049 (+360634us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:39.352091 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.352158 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:40.611714 (+1259556us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:04:40.786721 (+175007us) raft_consensus.cc:1685] finished
0709 13:04:40.786750 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.787028 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54383488,"thread_start_us":278059,"threads_started":1,"wal-append.queue_time_us":406171}
W20260709 13:04:40.804334 15595 log.cc:927] Time spent T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.751s	user 0.004s	sys 0.005s
W20260709 13:04:40.805778  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.789577  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:40.723698 15602 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:04:40.683262 15490 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:40.815100  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:40.797200  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:40.800725 15561 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:40.786886 15612 log.cc:927] Time spent T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.367s	user 0.001s	sys 0.000s
I20260709 13:04:40.817703  8324 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:40.723321 15318 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:40.819145 15561 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:40.819741  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2980) took 1226 ms. Trace:
I20260709 13:04:40.805862  8330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 3380) took 1115 ms. Trace:
I20260709 13:04:40.811376  8397 raft_consensus.cc:2393] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 24: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:40.820367  8400 rpcz_store.cc:276] 0709 13:04:39.593731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.983921 (+390190us) service_pool.cc:224] Handling call
0709 13:04:40.000219 (+ 16298us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:40.021165 (+ 20946us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:40.021215 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:40.022240 (+  1025us) log.cc:844] Serialized 52 byte log entry
0709 13:04:40.293618 (+271378us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:40.293661 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.293728 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:40.786219 (+492491us) spinlock_profiling.cc:243] Waited 85.5 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:04:40.819283 (+ 33064us) raft_consensus.cc:1685] finished
0709 13:04:40.819318 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.819649 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":256,"raft.queue_time_us":11131,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":6248,"spinlock_wait_cycles":196582144,"thread_start_us":215392,"threads_started":1,"wal-append.queue_time_us":308876}
I20260709 13:04:40.843390  8326 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:04:40.844453  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 3044) took 1051 ms. Trace:
I20260709 13:04:40.843506 15288 consensus_queue.cc:237] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:40.822455  8330 rpcz_store.cc:276] 0709 13:04:39.690785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.694682 (+  3897us) service_pool.cc:224] Handling call
0709 13:04:39.695025 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:39.695477 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:39.695513 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:39.696129 (+   616us) log.cc:844] Serialized 26 byte log entry
0709 13:04:39.943995 (+247866us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:39.944041 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.944104 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:40.803635 (+859531us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:04:40.805502 (+  1867us) raft_consensus.cc:1685] finished
0709 13:04:40.805531 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.805788 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1138,"raft.queue_time_us":9936,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":30908,"spinlock_wait_cycles":35882368,"thread_start_us":247435,"threads_started":1,"wal-append.queue_time_us":314660}
I20260709 13:04:40.761564 15451 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:40.844853  8326 rpcz_store.cc:276] 0709 13:04:39.793183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.828825 (+1035642us) service_pool.cc:224] Handling call
0709 13:04:40.829186 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.844152 (+ 14966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.844394 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:40.859395  8330 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:40.860842  8332 raft_consensus.cc:2393] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 20: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:40.814162 15471 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:40.875250  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.708101  8369 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:40.883898 14758 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:40.886487  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 28 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:40.897483  8330 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 18. (term mismatch)
I20260709 13:04:40.897931  8330 pending_rounds.cc:85] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 17
I20260709 13:04:40.898197  8330 pending_rounds.cc:107] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 24.18
W20260709 13:04:40.898240  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 190221us
I20260709 13:04:40.898710  8330 raft_consensus.cc:2889] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:40.902125  8330 raft_consensus.cc:1275] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:40.910036  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 28 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:40.911309  9004 raft_consensus.cc:2410] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 28 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 13:04:40.913168  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:40.914139  8400 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
I20260709 13:04:40.865790 15488 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:40.965277  8324 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 22.
I20260709 13:04:40.930171 14758 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:04:40.883477 15252 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:40.926206 15428 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.002s	sys 0.012s
W20260709 13:04:40.926363 15556 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.445s	user 0.000s	sys 0.027s
I20260709 13:04:40.979154  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:40.980063  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:40.981068  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:40.928074 15449 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:40.979822  8256 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
I20260709 13:04:40.954718  8331 raft_consensus.cc:2410] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 26 because replica has last-logged OpId of term: 25 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 12.
I20260709 13:04:40.980124  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:40.939590 15577 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.988696 14758 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.998191  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:40.998672  8331 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:04:40.996073  8330 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 25. Current term is 26. Ops: []
I20260709 13:04:41.000663  8323 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:41.000888  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:40.981698 15428 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.003s	sys 0.012s
I20260709 13:04:40.999081  8251 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 27.
I20260709 13:04:41.002547  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:40.971297 15581 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.484s	user 0.012s	sys 0.014s
I20260709 13:04:41.008446  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:41.008224 14758 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:41.008981  8257 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:40.983085 15556 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.502s	user 0.000s	sys 0.034s
I20260709 13:04:41.002059  8256 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
I20260709 13:04:41.004133  8397 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 13:04:41.015086  8397 pending_rounds.cc:85] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 13
I20260709 13:04:41.015405  8397 pending_rounds.cc:107] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 13:04:41.015678  8397 raft_consensus.cc:2889] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:41.004297  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.956210 15272 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.968586 15488 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:41.023566 15272 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:41.026394  8256 raft_consensus.cc:1240] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 25. Current term is 26. Ops: [25.13-25.13]
I20260709 13:04:41.028067 14758 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:41.029594 14758 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.006917  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:41.030361  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:04:41.030856 15578 log.cc:927] Time spent T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 1.550s	user 0.006s	sys 0.006s
I20260709 13:04:41.030967  8250 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
I20260709 13:04:41.031185  8248 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:40.963625 15318 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:41.010509  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:41.003196  8249 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:04:40.886400 15536 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:41.033097 15318 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:41.035135 15318 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:41.035676 15318 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:41.038641  8295 connection.cc:256] RPC call timeout handler was delayed by -1.0561038940001026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
I20260709 13:04:41.039944 15562 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:41.040767 15598 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.041518 15598 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:41.043686 15598 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:41.043903 15318 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:40.985399 15576 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:41.045584 15576 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.045615 14758 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:41.006382  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:41.046257 15576 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:41.048422 15576 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:41.021379  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:41.021642  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:41.050096  8255 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:41.050227  8253 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
W20260709 13:04:41.008198 15581 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.521s	user 0.013s	sys 0.015s
I20260709 13:04:40.995805 15449 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:41.029835 15619 log.cc:927] Time spent T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 13:04:41.045028  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 3383) took 1353 ms. Trace:
I20260709 13:04:40.975016 15410 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:41.051136  8251 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:04:41.051681  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:41.050135  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:41.058432 14758 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:41.058846  8250 raft_consensus.cc:2393] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 20: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:41.059158  8256 raft_consensus.cc:2410] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 20 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:04:41.054309  8327 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
W20260709 13:04:41.054236  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 506378us
I20260709 13:04:41.057129  8328 rpcz_store.cc:276] 0709 13:04:39.691799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.697575 (+  5776us) service_pool.cc:224] Handling call
0709 13:04:39.697959 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.360125 (+662166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.044795 (+684670us) spinlock_profiling.cc:243] Waited 640 ms on lock 0x7b5400090bd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:41.044968 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1473004288}
I20260709 13:04:40.999652 15452 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:41.064790  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:41.010362 15545 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:41.011345 15528 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.065685  8253 raft_consensus.cc:3060] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:41.018997 15403 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.068045  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:41.083248  8295 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:41.084137  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:41.084990  8398 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:04:41.085664  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:41.087001  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:41.055753 15562 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:41.088894 15561 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:40.981865 15252 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:41.081867 15452 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 13:04:41.014043 15577 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:41.082638 15591 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:41.076984 15611 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:41.087050 15618 log.cc:927] Time spent T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 13:04:41.091466 15490 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:41.092034 15537 consensus_meta.cc:287] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.536s	user 0.008s	sys 0.000s
W20260709 13:04:41.066342 15557 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.014s	sys 0.035s
I20260709 13:04:41.092916 15597 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:41.082056 15516 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:41.056631 15449 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:41.096159 15451 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:41.050478 15477 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:41.180932  8372 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:41.080652  8257 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 23.
I20260709 13:04:40.982311 15509 consensus_queue.cc:237] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:41.130887 14967 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:41.119906  8331 raft_consensus.cc:2410] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:04:41.121279  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:41.124414  8253 raft_consensus.cc:2468] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 22.
I20260709 13:04:41.070896 15427 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:41.114306 15455 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:41.113003 15410 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:41.239269 15574 consensus_queue.cc:1059] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:41.239440 15410 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:41.240814 15574 raft_consensus.cc:3055] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:41.241271 15574 raft_consensus.cc:740] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:41.241847 15410 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:41.242571 15574 consensus_queue.cc:260] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:41.244086 15574 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:41.245600 15611 consensus_queue.cc:1059] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:41.165812  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:41.248858  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:41.249948 15252 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:41.250058  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:41.251494  8248 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
I20260709 13:04:41.183657 15629 consensus_queue.cc:1059] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:41.184435  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:41.184527 15402 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:41.186012  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:41.189882  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 1238) took 2566 ms. Trace:
I20260709 13:04:41.099833  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:41.262813  9004 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 23.
I20260709 13:04:41.190070 15577 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:41.191287  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:41.191818 15591 consensus_queue.cc:1048] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:41.192235  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:41.193598  8329 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 3381) took 1502 ms (client timeout 1836 ms). Trace:
I20260709 13:04:41.195333  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:41.197158  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:41.197813 15536 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:41.119370 15438 consensus_queue.cc:1048] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 13:04:41.123366 15562 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:41.123634 15471 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:41.128539 15561 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:41.109050 15403 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:41.233040 15490 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.180156  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:41.179550  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 3039) took 2868 ms. Trace:
I20260709 13:04:41.206527  8255 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 24.
W20260709 13:04:41.161778 15557 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.836s	user 0.014s	sys 0.036s
I20260709 13:04:41.099452 15528 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:41.089570 15488 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:41.210974 14758 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:41.225229  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:41.204952  8369 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:41.227555  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:41.210101 15625 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.236006 15318 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:41.236374  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:41.236899 15581 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:41.237054  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:41.237548 15451 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:41.237759 15455 raft_consensus.cc:697] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:41.238502 15477 consensus_queue.cc:1048] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:41.242609 15535 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:41.247462 15425 raft_consensus.cc:3055] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:04:41.166762  8295 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.750s (SENT) [suppressed 10 similar messages]
I20260709 13:04:41.255355  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:04:41.209858 15608 consensus_meta.cc:287] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.699s	user 0.011s	sys 0.000s
I20260709 13:04:41.261010  8252 rpcz_store.cc:276] 0709 13:04:38.623613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.891868 (+268255us) service_pool.cc:224] Handling call
0709 13:04:38.892146 (+   278us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:38.993790 (+101644us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:38.993828 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:39.039603 (+ 45775us) log.cc:844] Serialized 52 byte log entry
0709 13:04:39.353734 (+314131us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:39.353773 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.353837 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:40.942293 (+1588456us) spinlock_profiling.cc:243] Waited 28.3 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:04:41.189504 (+247211us) raft_consensus.cc:1685] finished
0709 13:04:41.189529 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.189787 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":138997,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"spinlock_wait_cycles":64977152,"thread_start_us":400472,"threads_started":2,"wal-append.queue_time_us":380308}
W20260709 13:04:41.300209  8329 rpcz_store.cc:269] 0709 13:04:39.691114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.696764 (+  5650us) service_pool.cc:224] Handling call
0709 13:04:41.193358 (+1496594us) spinlock_profiling.cc:243] Waited 733 ms on lock 0x7b540037b754. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:41.193533 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1686458240}
W20260709 13:04:41.377758  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 172894us
I20260709 13:04:41.318773  8398 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:41.299851  8402 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 23.
W20260709 13:04:41.259315 15508 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.731s	user 0.028s	sys 0.049s
I20260709 13:04:41.326987  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:41.327925  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:41.165612 15537 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:41.235229  8220 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:41.201937  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:41.380540  8297 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:41.256234 15556 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.347327  8328 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 25.
I20260709 13:04:41.354354  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 1292) took 1825 ms. Trace:
I20260709 13:04:41.346315  8323 rpcz_store.cc:276] 0709 13:04:38.310698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.311229 (+   531us) service_pool.cc:224] Handling call
0709 13:04:38.311596 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:41.001072 (+2689476us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.179265 (+178193us) spinlock_profiling.cc:243] Waited 2.69 s on lock 0x7b54000587d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:41.179449 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6183961472}
W20260709 13:04:41.345584 15570 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.985s	user 0.013s	sys 0.020s
W20260709 13:04:41.356155  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 289280us
I20260709 13:04:41.267230 15609 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:41.366581  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:41.301986 15516 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:41.369694  8330 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
I20260709 13:04:41.328320 15577 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:41.405678  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:41.375380  8397 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:41.376489 14758 raft_consensus.cc:2804] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:41.382611  9005 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:04:41.355773 15557 raft_consensus.cc:697] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 22 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:41.385586  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:41.387739  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 338103us
I20260709 13:04:41.389508  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:41.387823  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:41.408367  8402 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:41.408694  8323 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 24: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:41.387233  8403 rpcz_store.cc:276] 0709 13:04:39.528624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.750050 (+221426us) service_pool.cc:224] Handling call
0709 13:04:39.750486 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:39.847773 (+ 97287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.353988 (+1506215us) spinlock_profiling.cc:243] Waited 96.5 ms on lock 0x7b5400267fd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:41.354251 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":221867392}
W20260709 13:04:41.390118  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 155023us
I20260709 13:04:41.409852  8220 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:41.394443  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 90761.3us
I20260709 13:04:41.395560  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:41.371733 15451 consensus_queue.cc:1243] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 20.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:04:41.374698 15425 raft_consensus.cc:740] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:04:41.382175 15508 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.857s	user 0.028s	sys 0.050s
I20260709 13:04:41.398031  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:41.399169  8222 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:41.391134 15570 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.035s	user 0.013s	sys 0.021s
I20260709 13:04:41.429198 15425 consensus_queue.cc:260] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:41.430644 15425 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:41.430861 15272 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:41.432439  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.432907  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 2978) took 1887 ms. Trace:
I20260709 13:04:41.436017  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 3048) took 1641 ms. Trace:
I20260709 13:04:41.436450  8332 rpcz_store.cc:276] 0709 13:04:39.794242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.653718 (+859476us) service_pool.cc:224] Handling call
0709 13:04:41.435784 (+782066us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b540014ffd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:41.435948 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":392683392}
I20260709 13:04:41.438702  8223 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:41.404593 15516 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:41.441773 15627 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:04:41.445448  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 46353.6us
I20260709 13:04:41.433310  8401 rpcz_store.cc:276] 0709 13:04:39.545503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.958673 (+413170us) service_pool.cc:224] Handling call
0709 13:04:39.984761 (+ 26088us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:40.074573 (+ 89812us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:40.074612 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:40.087180 (+ 12568us) log.cc:844] Serialized 52 byte log entry
0709 13:04:40.789850 (+702670us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b180042e1a0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e0d1 00007f8828779a0f
0709 13:04:40.942773 (+152923us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:40.983850 (+ 41077us) log.cc:844] Serialized 14 byte log entry
0709 13:04:41.206255 (+222405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.432576 (+226321us) spinlock_profiling.cc:243] Waited 81.2 ms on lock 0x7b540019bed4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:41.432627 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:41.432671 (+    44us) raft_consensus.cc:1685] finished
0709 13:04:41.432686 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.432850 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":41191,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":45845,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":6531,"spinlock_wait_cycles":213254144,"thread_start_us":374845,"threads_started":2,"wal-append.queue_time_us":988698}
I20260709 13:04:41.407339 14758 raft_consensus.cc:697] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 22 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:41.364301 15490 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:41.419157 15602 consensus_meta.cc:287] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.572s	user 0.010s	sys 0.001s
I20260709 13:04:41.346845 15561 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:41.390094 15597 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:04:41.426098  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.447317  8406 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 26.
I20260709 13:04:41.450961  8331 raft_consensus.cc:1240] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:41.406299  8328 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
W20260709 13:04:41.420357 15405 consensus_meta.cc:287] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.630s	user 0.016s	sys 0.005s
I20260709 13:04:41.366470 15625 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:41.398025 15644 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:41.444475 15562 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:41.461216  8403 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 13:04:41.475587  8403 pending_rounds.cc:85] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 13
I20260709 13:04:41.394979 15556 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:41.475857  8403 pending_rounds.cc:107] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 13:04:41.442610 15627 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:41.448143 14758 consensus_queue.cc:237] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:41.476141  8403 raft_consensus.cc:2889] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:41.461545  8402 raft_consensus.cc:2410] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:04:41.460255 15552 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:41.383359 15608 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:41.376681 15581 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:41.485965  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:41.468427  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:41.469130  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:41.469271  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:41.470395  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:41.487450  8253 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 24: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:41.471338  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:41.470516  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:41.479182  8398 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:41.488202  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:41.488488  9001 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:41.489625  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:41.489828  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:41.490688  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:41.404356 15553 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:04:41.462137 15557 consensus_queue.cc:237] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:41.481127  8397 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:41.485330  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:41.468232  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:41.488204  9003 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:41.498229  8372 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:41.498795  8248 raft_consensus.cc:2410] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:04:41.499830  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:41.500679  8250 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 23.
I20260709 13:04:41.490780  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:41.463092 15272 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:41.504688  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:41.505177  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 338421us
I20260709 13:04:41.505503  8257 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:41.463606 15597 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:41.509275  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 28 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:41.525454  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:41.525921  8329 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 24.
I20260709 13:04:41.526253  8250 raft_consensus.cc:2410] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 28 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 13:04:41.524533  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:41.402021 15609 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:41.417487 15577 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:41.526901  8253 raft_consensus.cc:2393] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 26: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:41.526868  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:41.482012 15552 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:41.528784  8254 raft_consensus.cc:1240] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:04:41.483191 15627 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:41.547212  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:41.549793  8257 raft_consensus.cc:1275] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:04:41.550024  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:41.508059 15252 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:41.547519  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:41.475445 15536 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.054s
W20260709 13:04:41.534301 15624 log.cc:927] Time spent T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc: Append to log took a long time: real 0.278s	user 0.000s	sys 0.001s
I20260709 13:04:41.482239 15405 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:41.529582 15597 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:04:41.574810  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 313163us
I20260709 13:04:41.473008 15602 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:41.544556 15552 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:41.482882 15455 consensus_queue.cc:237] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:41.618075  8252 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:04:41.619530  8257 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:41.609760  8974 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:04:41.611220  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.611909  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:41.624065  8248 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 26.
W20260709 13:04:41.615466  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.590063  8403 pending_rounds.cc:85] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 12
I20260709 13:04:41.627389  8403 pending_rounds.cc:107] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 13:04:41.627842  8403 raft_consensus.cc:2889] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:41.563268 15252 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:41.641053  9003 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:41.537583 15449 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
W20260709 13:04:41.609397 15384 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.562s	user 0.023s	sys 0.011s
I20260709 13:04:41.649947  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:41.618525 15612 log.cc:927] Time spent T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:04:41.591835 15516 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:41.652177  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:41.643642 15551 raft_consensus.cc:2764] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:04:41.656816 15537 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:41.672359 15621 log.cc:927] Time spent T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
I20260709 13:04:41.673590 15419 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:41.675110 15628 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.266s	user 0.001s	sys 0.001s
W20260709 13:04:41.966962 15622 log.cc:927] Time spent T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.739s	user 0.003s	sys 0.000s
I20260709 13:04:41.969063  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 3379) took 2278 ms. Trace:
I20260709 13:04:41.969730  9002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 1390) took 1409 ms. Trace:
W20260709 13:04:41.651192 15384 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.604s	user 0.024s	sys 0.011s
I20260709 13:04:41.656667 15535 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:41.970270  9002 rpcz_store.cc:276] 0709 13:04:40.560826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.582513 (+ 21687us) service_pool.cc:224] Handling call
0709 13:04:40.603602 (+ 21089us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:40.604063 (+   461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:40.604102 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:40.604670 (+   568us) log.cc:844] Serialized 26 byte log entry
0709 13:04:40.962940 (+358270us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:04:40.963010 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.964291 (+  1281us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:41.969530 (+1005239us) raft_consensus.cc:1685] finished
0709 13:04:41.969554 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.969676 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1311,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":100,"thread_start_us":357794,"threads_started":1,"wal-append.queue_time_us":651087}
I20260709 13:04:41.679294 15488 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:41.583940 15629 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:04:41.644613 15252 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:41.969480  8325 rpcz_store.cc:276] 0709 13:04:39.690417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.693799 (+  3382us) service_pool.cc:224] Handling call
0709 13:04:39.694294 (+   495us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:39.763499 (+ 69205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:39.763530 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:40.522312 (+758782us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b2c000d4468. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:04:40.522841 (+   529us) log.cc:844] Serialized 52 byte log entry
0709 13:04:40.982062 (+459221us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:41.007812 (+ 25750us) log.cc:844] Serialized 14 byte log entry
0709 13:04:41.013293 (+  5481us) log.cc:844] Serialized 14 byte log entry
0709 13:04:41.260415 (+247122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.260507 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:41.966356 (+705849us) spinlock_profiling.cc:243] Waited 4.61 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:04:41.968811 (+  2455us) raft_consensus.cc:1685] finished
0709 13:04:41.968834 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.968998 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5081,"prepare.queue_time_us":541781,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":84,"raft.queue_time_us":38616,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":10082,"spinlock_wait_cycles":64747008,"thread_start_us":705679,"threads_started":2,"wal-append.queue_time_us":596071}
W20260709 13:04:41.802852  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 355475us
I20260709 13:04:41.981475 15598 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:41.982281 15598 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:41.982880 15556 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:41.984551 15598 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:41.804134  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 337338us
I20260709 13:04:41.985569  8220 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:41.824000 15639 log.cc:927] Time spent T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.321s	user 0.001s	sys 0.001s
W20260709 13:04:41.832873 15648 log.cc:927] Time spent T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.197s	user 0.000s	sys 0.000s
W20260709 13:04:41.845822  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 211754us
I20260709 13:04:41.990331  8974 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:41.990901 15649 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:04:41.991593 15649 consensus_queue.cc:1059] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:41.991871 15488 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:41.992293 15488 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:41.992112 15669 consensus_queue.cc:1059] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:41.992774 15488 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:41.993301 15669 raft_consensus.cc:3055] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:41.993799 15669 raft_consensus.cc:740] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:04:41.800424  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 388573us
I20260709 13:04:41.898682 15581 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:41.899318 15602 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:41.900405 15405 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:41.676291 15582 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.996238 15582 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:41.674149 15647 consensus_queue.cc:1059] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.145s
I20260709 13:04:41.993052 15536 raft_consensus.cc:3055] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:42.002507 15536 raft_consensus.cc:740] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:42.003645 15536 consensus_queue.cc:260] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:42.005932  8220 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:42.007109  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:41.652503  8404 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 28: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:41.907071 15666 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:42.007794 15536 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:41.911748  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:42.007970 15666 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:42.008780  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 2988) took 1399 ms. Trace:
I20260709 13:04:41.995565  8972 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:42.010546 15666 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:42.010975 15551 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:42.011119 15666 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:42.011582 15666 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:42.011915 15666 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:42.008041  8330 raft_consensus.cc:2410] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 13:04:41.912786 15660 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:41.913475 15608 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:42.014237  8328 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:04:42.007570 15646 log.cc:927] Time spent T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.391s	user 0.001s	sys 0.008s
I20260709 13:04:41.913789  8326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 3060) took 1228 ms. Trace:
W20260709 13:04:41.913986 15654 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.371s	user 0.000s	sys 0.005s
I20260709 13:04:42.016400  8326 rpcz_store.cc:276] 0709 13:04:40.685692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.928248 (+242556us) service_pool.cc:224] Handling call
0709 13:04:40.971902 (+ 43654us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:40.982454 (+ 10552us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:40.982487 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:40.997763 (+ 15276us) log.cc:844] Serialized 26 byte log entry
0709 13:04:41.381783 (+384020us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:41.382669 (+   886us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.382742 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:41.913541 (+530799us) raft_consensus.cc:1685] finished
0709 13:04:41.913563 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.913735 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2402,"raft.queue_time_us":112,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":20948,"thread_start_us":320465,"threads_started":1,"wal-append.queue_time_us":883356}
I20260709 13:04:42.015513 15660 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:41.912351 15449 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:41.916976  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:41.917733 15609 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:42.018730  8298 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:42.018049 15669 consensus_queue.cc:260] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:42.020081 15669 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:41.913482 15662 log.cc:927] Time spent T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.320s	user 0.001s	sys 0.003s
I20260709 13:04:42.009208  8404 rpcz_store.cc:276] 0709 13:04:40.609052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.689265 (+ 80213us) service_pool.cc:224] Handling call
0709 13:04:42.008593 (+1319328us) spinlock_profiling.cc:243] Waited 763 ms on lock 0x7b54001cc854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:42.008726 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1754890112}
I20260709 13:04:41.918147 15490 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:41.922770 15587 consensus_queue.cc:260] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:41.935633 15574 consensus_meta.cc:287] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.691s	user 0.022s	sys 0.010s
W20260709 13:04:41.938885 15425 consensus_meta.cc:287] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.508s	user 0.006s	sys 0.006s
I20260709 13:04:41.939865 15561 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:41.940828 15577 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:41.615514 15569 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:41.677271 15651 log.cc:927] Time spent T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 13:04:42.006927  8220 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:41.910341  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:42.058842  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:42.022625  8327 raft_consensus.cc:1275] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:04:42.023000  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:42.023727  8402 raft_consensus.cc:1240] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:42.023869  8398 raft_consensus.cc:1240] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:42.061033  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:42.018211  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 2758) took 1005 ms. Trace:
I20260709 13:04:42.061420  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:42.062742  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 55920.7us
I20260709 13:04:42.062992  8402 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:42.064406  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:42.064625  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:42.020520 15329 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:42.054934  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:42.069123  8326 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 26.
I20260709 13:04:42.055102  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:42.058252  8404 raft_consensus.cc:1240] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:41.587311 15626 consensus_queue.cc:1059] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:04:41.937177 15597 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:42.042804  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:42.074575  8331 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:42.061254  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:42.025854  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:42.061619  8254 rpcz_store.cc:276] 0709 13:04:41.013633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.014995 (+  1362us) service_pool.cc:224] Handling call
0709 13:04:41.015418 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:41.541374 (+525956us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.017973 (+476599us) spinlock_profiling.cc:243] Waited 510 ms on lock 0x7b54003471d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:42.018156 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1173335296}
W20260709 13:04:42.052233 15428 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.002s	sys 0.011s
I20260709 13:04:42.068204  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:42.073303  8222 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:42.080698  8329 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:04:42.037350 15516 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.075943  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:42.083771  8404 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:42.087888  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:42.089023  8398 raft_consensus.cc:2410] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 19 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:04:42.068759 15329 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:42.038105 15653 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:04:42.092288  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 81768.2us
I20260709 13:04:42.092617  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:42.091673  8222 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:42.095261  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:42.099185  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:42.101217  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:42.104231  8327 raft_consensus.cc:2393] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 17: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:42.102856  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:42.104921  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:42.105304  8402 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:42.080395 15428 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.002s	sys 0.011s
I20260709 13:04:42.074265 15561 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:42.108315  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:42.075103 15318 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.106570 15665 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.079445 15528 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:42.112958  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:42.150933  8397 raft_consensus.cc:1813] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Handling vote request from an unknown peer ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:42.151379  8397 raft_consensus.cc:2368] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18.
I20260709 13:04:42.101977 15329 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:04:42.152879  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 61295.5us
I20260709 13:04:42.154171  8222 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:42.155763  8972 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:42.159029 15677 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:04:42.159142 15647 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:42.159719 15647 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:42.160076  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:42.160076 15647 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:42.160452  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 1170) took 1114 ms. Trace:
I20260709 13:04:42.161008  8332 rpcz_store.cc:276] 0709 13:04:41.046200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.437437 (+391237us) service_pool.cc:224] Handling call
0709 13:04:41.438430 (+   993us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:41.440869 (+  2439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:41.440907 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:41.441413 (+   506us) log.cc:844] Serialized 26 byte log entry
0709 13:04:41.803615 (+362202us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:41.803664 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.803741 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:42.160182 (+356441us) raft_consensus.cc:1685] finished
0709 13:04:42.160207 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.160380 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1542,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":133,"thread_start_us":340619,"threads_started":1,"wal-append.queue_time_us":596014}
I20260709 13:04:42.163020  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:42.163615  8332 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:42.164605  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:42.165125  8332 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:42.107756 15591 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:42.065399 15569 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:42.173651  8331 raft_consensus.cc:2468] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
I20260709 13:04:42.188366  8397 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:42.188298  8331 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:04:42.183585  8220 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:42.115731 15561 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:42.094318 15649 consensus_queue.cc:1059] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:04:42.114609 15582 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:42.168768 15627 consensus_meta.cc:287] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.605s	user 0.010s	sys 0.001s
I20260709 13:04:42.188362  8222 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:42.195443  9006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 1534) took 1013 ms. Trace:
W20260709 13:04:42.200805  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.185789  8329 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 28.
I20260709 13:04:42.097545 15516 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:42.207330  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:42.191192  8402 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:42.170046 15591 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:42.187522 15673 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:42.227916  8402 raft_consensus.cc:1240] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 18. Current term is 19. Ops: []
W20260709 13:04:42.234270  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.146586 15318 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:42.197912 15577 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.231534  8329 raft_consensus.cc:1240] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:42.231726  8222 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:42.198583 15561 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:42.117128 15653 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:42.210289  9006 rpcz_store.cc:276] 0709 13:04:41.181795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.190330 (+  8535us) service_pool.cc:224] Handling call
0709 13:04:41.190714 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:41.228464 (+ 37750us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:41.228498 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:41.238600 (+ 10102us) log.cc:844] Serialized 26 byte log entry
0709 13:04:41.384104 (+145504us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:41.384150 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.384239 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:42.195143 (+810904us) raft_consensus.cc:1685] finished
0709 13:04:42.195182 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.195366 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37325,"raft.queue_time_us":62528,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"thread_start_us":145014,"threads_started":1,"wal-append.queue_time_us":211630}
I20260709 13:04:42.209453 15384 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:04:42.223237 15552 consensus_meta.cc:287] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.583s	user 0.012s	sys 0.001s
I20260709 13:04:42.276371 15552 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:42.278916 15552 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:42.237668  8327 raft_consensus.cc:1240] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 13:04:42.243494  8402 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:04:42.247745 15516 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:42.248627 15536 consensus_queue.cc:1059] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:42.218977 15683 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 13:04:42.249728 15675 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:42.233618 15591 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:42.221029 15627 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:42.233230  8332 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 22.
W20260709 13:04:42.342971  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 284584us
I20260709 13:04:42.343775  8223 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:42.257249 15569 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:42.264384 15488 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:42.271391 15252 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:42.273890 15318 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:42.275731 15653 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:42.156291 15665 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:42.281919  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:42.283918 15551 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:42.287192 15647 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:42.393772  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:42.394552  9006 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
I20260709 13:04:42.214859 15626 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.075s
I20260709 13:04:42.343443 15673 consensus_queue.cc:1059] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:04:42.290786 15666 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:42.244973 15577 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:42.394577  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:42.399413  9005 raft_consensus.cc:2393] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 17: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:42.399679 15384 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:42.334898  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:42.400270 15384 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:42.400735  8327 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:42.403307  8327 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:04:42.392087  9002 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:04:42.392835  8329 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:42.393839  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:42.232271 15557 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:42.290024 15663 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:42.398633 15577 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:42.372749  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 217038us
I20260709 13:04:42.431227  8398 raft_consensus.cc:1240] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:42.431725  8972 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:42.438374  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 374694us
I20260709 13:04:42.441840  8329 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:42.449699  8248 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:04:42.374341 15681 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:42.493274 15641 log.cc:927] Time spent T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.843s	user 0.008s	sys 0.020s
W20260709 13:04:42.505929  9035 heartbeater.cc:646] Failed to heartbeat to 127.6.255.62:40337 (6 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.6.255.62:40337 timed out after 15.000s (SENT)
I20260709 13:04:42.507495  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 3061) took 1821 ms. Trace:
I20260709 13:04:42.507985  8324 rpcz_store.cc:276] 0709 13:04:40.686276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.966517 (+280241us) service_pool.cc:224] Handling call
0709 13:04:40.984170 (+ 17653us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:40.984522 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:40.984621 (+    99us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:40.985046 (+   425us) log.cc:844] Serialized 26 byte log entry
0709 13:04:41.164889 (+179843us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:41.164946 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.165103 (+   157us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:42.492503 (+1327400us) spinlock_profiling.cc:243] Waited 2.01 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:04:42.507150 (+ 14647us) raft_consensus.cc:1685] finished
0709 13:04:42.507173 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.507421 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16518,"raft.queue_time_us":13552,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":8814,"spinlock_wait_cycles":4619008,"thread_start_us":98707,"threads_started":1,"wal-append.queue_time_us":481340}
I20260709 13:04:42.495052  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:42.518774  8406 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 2992) took 1324 ms (client timeout 1655 ms). Trace:
W20260709 13:04:42.519222  8406 rpcz_store.cc:269] 0709 13:04:41.194311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.201695 (+  7384us) service_pool.cc:224] Handling call
0709 13:04:42.518584 (+1316889us) spinlock_profiling.cc:243] Waited 62.9 ms on lock 0x7b54001cbe54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:42.518715 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":144639872}
I20260709 13:04:42.520188  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:42.522110  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:42.523846  8298 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:42.525357 15637 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.526206 15625 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:42.526741  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:42.530062  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:42.530824  8406 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:42.532951  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:42.533839  8222 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:42.525970  8329 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 24.
W20260709 13:04:42.535255  8369 connection.cc:256] RPC call timeout handler was delayed by -1.0269969179998952s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1806s for a response
I20260709 13:04:42.527177  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:42.535413  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:42.535744  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:42.527988  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:42.536784  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 105193us
I20260709 13:04:42.536873  8406 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 20.
W20260709 13:04:42.535391  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.540153  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:42.426783 15653 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:42.552457  8406 raft_consensus.cc:2393] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:04:42.536033  8369 connection.cc:256] RPC call timeout handler was delayed by -1.0233969179998894s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18100000000000002s for a response
I20260709 13:04:42.537875  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:42.541390 15538 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:42.547243  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:42.440567 15663 raft_consensus.cc:697] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:42.548091  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:42.548846  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:42.561532  9006 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:04:42.551957  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:42.559077  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:42.559986  8406 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 13:04:42.547995 15669 consensus_meta.cc:287] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.527s	user 0.008s	sys 0.000s
I20260709 13:04:42.563362  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:42.564749  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:42.563015 15417 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:42.566452 15690 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:42.567741  8406 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:42.536178 15577 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:04:42.561311  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.555389 15455 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:42.572742 15417 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:42.576570 15685 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:42.575901  8972 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:42.577729 14758 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.560781 15384 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:42.557340 15538 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:42.578984  8406 raft_consensus.cc:1240] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:42.582775 14758 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:42.525933 15637 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:42.607234  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:42.577984 15569 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:42.577039 15665 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:42.583577 15551 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:42.582173 15417 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:42.577446 15405 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.564684 15582 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:04:42.581220 15455 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:42.582448 15685 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:42.539357 15681 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:42.612946 15569 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:42.579797 15653 consensus_queue.cc:1059] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.042s
W20260709 13:04:42.653502  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 309782us
I20260709 13:04:42.607686 15538 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:42.655090  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:42.658895  8223 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:42.659273  8406 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18.
I20260709 13:04:42.615523 15669 consensus_queue.cc:1048] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 13:04:42.651940 15690 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 13:04:42.688126  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 107503us
I20260709 13:04:42.614073 15384 consensus_queue.cc:1059] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:04:42.687201  8222 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:42.580451 15577 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:42.656533 15685 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:42.683899 15644 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.145s	user 0.029s	sys 0.074s
I20260709 13:04:42.696758  8369 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:42.657152 15516 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:04:42.686774 15694 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:42.652261 15455 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:42.706322  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 99160.5us
I20260709 13:04:42.707203  8369 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:42.697978 15644 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.165s	user 0.029s	sys 0.074s
I20260709 13:04:42.692607 15690 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:42.692766 15488 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:42.669869 15569 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:42.697577 15557 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:42.737772 15557 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:42.738176 15557 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:42.738765 15681 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:42.739010  8404 raft_consensus.cc:1240] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:42.739351 15681 consensus_queue.cc:1059] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 25.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:42.740152 15681 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:42.740948 15681 raft_consensus.cc:3055] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 25 LEADER]: Stepping down as leader of term 25
W20260709 13:04:42.716960  8972 connection.cc:256] RPC call timeout handler was delayed by -1.3096957370000837s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.209s for a response
I20260709 13:04:42.710747 15625 raft_consensus.cc:3055] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:42.742113 15625 raft_consensus.cc:740] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:42.742750 15637 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:42.718443 15690 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:42.742910 15625 consensus_queue.cc:260] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:42.720685  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:42.744508  9004 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:42.724406 15686 log.cc:927] Time spent T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:04:42.728113  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:42.748040  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 3003) took 1184 ms. Trace:
I20260709 13:04:42.713696 15644 raft_consensus.cc:697] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:42.748380  8403 rpcz_store.cc:276] 0709 13:04:41.563537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.563717 (+   180us) service_pool.cc:224] Handling call
0709 13:04:41.589781 (+ 26064us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:41.951655 (+361874us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:41.951705 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:41.952415 (+   710us) log.cc:844] Serialized 52 byte log entry
0709 13:04:42.374979 (+422564us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:42.375023 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.375174 (+   151us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:42.747839 (+372665us) raft_consensus.cc:1685] finished
0709 13:04:42.747864 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.747987 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77086,"raft.queue_time_us":293168,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":13927,"thread_start_us":293756,"threads_started":1,"wal-append.queue_time_us":517579}
I20260709 13:04:42.750211  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:42.750077 15644 consensus_queue.cc:237] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:42.750895  8403 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 28: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:04:42.725276 15687 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:04:42.708791 15449 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:42.754839 15449 raft_consensus.cc:697] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 19 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:42.755205 15597 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:42.755741 15449 consensus_queue.cc:237] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:42.756767  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 1258) took 1105 ms. Trace:
I20260709 13:04:42.647768 15663 consensus_queue.cc:237] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:42.757128  8253 rpcz_store.cc:276] 0709 13:04:41.651199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.651358 (+   159us) service_pool.cc:224] Handling call
0709 13:04:41.950649 (+299291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:41.962578 (+ 11929us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:41.962616 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:42.034035 (+ 71419us) log.cc:844] Serialized 26 byte log entry
0709 13:04:42.439251 (+405216us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:42.439310 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.439406 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:42.756568 (+317162us) raft_consensus.cc:1685] finished
0709 13:04:42.756587 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.756711 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11369,"raft.queue_time_us":11489,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"thread_start_us":360396,"threads_started":1,"wal-append.queue_time_us":524433}
I20260709 13:04:42.759078 14758 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:42.732738  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.760288 15438 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:42.760800 15438 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:42.762755  8295 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:42.766479 15575 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:42.716485 15252 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:42.769819  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 1296) took 1851 ms. Trace:
W20260709 13:04:42.771974  8399 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 2984) took 2164 ms (client timeout 2214 ms). Trace:
W20260709 13:04:42.772488  8399 rpcz_store.cc:269] 0709 13:04:40.607757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.609964 (+  2207us) service_pool.cc:224] Handling call
0709 13:04:42.771728 (+2161764us) spinlock_profiling.cc:243] Waited 715 ms on lock 0x7b54003160d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:42.771870 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1643927808}
I20260709 13:04:42.779837 15647 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:42.780493 15575 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.781095 15575 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:42.782289 15553 raft_consensus.cc:3055] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:42.782692 15553 raft_consensus.cc:740] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:42.783211 15575 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:42.783459 15553 consensus_queue.cc:260] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:42.784704 15553 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:42.786671 14967 raft_consensus.cc:3055] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:42.787228 14967 raft_consensus.cc:740] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:42.788288 14967 consensus_queue.cc:260] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:42.790773  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49220 (request call id 1259) took 1132 ms. Trace:
I20260709 13:04:42.791273  8248 rpcz_store.cc:276] 0709 13:04:41.658184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.908195 (+250011us) service_pool.cc:224] Handling call
0709 13:04:42.790555 (+882360us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b540009ecd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:42.790696 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":897122560}
I20260709 13:04:42.792071 15591 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:42.736757 15417 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:42.795007 15417 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 26 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:42.740206  8404 raft_consensus.cc:1240] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:42.770236  8400 rpcz_store.cc:276] 0709 13:04:40.918285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.918487 (+   202us) service_pool.cc:224] Handling call
0709 13:04:40.973940 (+ 55453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:41.127955 (+154015us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:41.127983 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:41.160547 (+ 32564us) log.cc:844] Serialized 26 byte log entry
0709 13:04:41.386469 (+225922us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:41.386525 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.769330 (+1382805us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b54001cc354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:42.769379 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:42.769461 (+    82us) raft_consensus.cc:1685] finished
0709 13:04:42.769483 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.769763 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":684,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":77,"spinlock_wait_cycles":353334016,"thread_start_us":130484,"threads_started":1,"wal-append.queue_time_us":250524}
I20260709 13:04:42.808180  9006 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 28.
I20260709 13:04:42.808615  8297 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:42.734297 15685 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:42.816100 14967 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:42.818145 15438 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:42.791638 15557 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:42.823421 15587 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:42.823342 15569 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:42.715094 15561 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:42.815989 15403 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:42.828524 15666 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:42.830293  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 67625.7us
I20260709 13:04:42.878257  8295 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:42.830859  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.829747 15627 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:42.741261 15681 raft_consensus.cc:740] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 25 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:42.846930  8220 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:42.842648 15545 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:42.843838  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 160729us
I20260709 13:04:42.891772  8223 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:42.862059  9004 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
I20260709 13:04:42.841244  8297 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:42.843811 15329 consensus_meta.cc:287] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.652s	user 0.013s	sys 0.000s
I20260709 13:04:42.649076 15405 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:42.895504  8372 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:42.907450  8295 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:42.918124  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 71297.6us
I20260709 13:04:42.897367  8974 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:42.919914  8220 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:42.844395 15575 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.066s
I20260709 13:04:42.922068  8223 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:42.864812 15616 consensus_queue.cc:1059] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
W20260709 13:04:42.926908  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 85657.4us
I20260709 13:04:42.931252  8297 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:42.867261 15417 consensus_queue.cc:1059] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:04:42.886981 15488 consensus_queue.cc:1059] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.038s
I20260709 13:04:42.936075  8297 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:42.924170  8295 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:42.922246 15438 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:42.945483  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:42.904702 15669 consensus_queue.cc:1059] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 13:04:42.924170 15318 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:42.924978 15471 raft_consensus.cc:2804] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:42.926275 15577 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:42.912416 15329 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:42.932999 15582 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:42.937587 15597 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:42.943948 15528 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:42.970293  8254 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:04:42.972935 15699 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.973572 15647 raft_consensus.cc:3055] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:04:42.938035 15704 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:42.974536  8253 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 21. Ops: []
I20260709 13:04:42.943614 15591 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:42.967901 15581 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:42.954972 15627 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:42.957836 15318 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:42.957187 15666 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:42.979422  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:43.010486  8254 raft_consensus.cc:2410] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 19 because replica has last-logged OpId of term: 18 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 15.
I20260709 13:04:42.979700  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:42.985134 15704 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:42.981568 15329 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:42.946489 15698 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:04:42.937225 15660 consensus_queue.cc:1059] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 13:04:43.014113 15660 raft_consensus.cc:3055] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:43.014587 15660 raft_consensus.cc:740] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:43.015655 15660 consensus_queue.cc:260] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:42.967345 15675 raft_consensus.cc:3055] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:43.017699 15675 raft_consensus.cc:740] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:43.018774 15675 consensus_queue.cc:260] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:43.020311 15675 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:43.021749 15697 log.cc:927] Time spent T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:04:42.968164 15629 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:43.025337 15561 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:43.026017  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.978654 15647 raft_consensus.cc:740] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:43.028782 15647 consensus_queue.cc:260] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:43.030356 15647 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:43.034431 15627 raft_consensus.cc:697] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:43.035866 15627 consensus_queue.cc:237] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:43.038256 15690 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:43.040704 15690 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 27 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:42.990100 15528 raft_consensus.cc:697] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:43.047025 15528 consensus_queue.cc:237] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:43.052260  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3594) took 1024 ms. Trace:
I20260709 13:04:42.994728  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:42.973095 15548 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:04:42.995479 15318 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:42.995955 15557 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:42.965909 15438 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:04:42.989094 15696 log.cc:927] Time spent T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.279s	user 0.001s	sys 0.001s
I20260709 13:04:43.001479 15405 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:42.956270 15681 consensus_queue.cc:260] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:43.006240 15569 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:43.007498 15570 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.011771 15704 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:42.981822  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:43.056434  8253 raft_consensus.cc:2393] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 25: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:43.052845  8402 rpcz_store.cc:276] 0709 13:04:42.028214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.234648 (+206434us) service_pool.cc:224] Handling call
0709 13:04:42.234931 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:42.295739 (+ 60808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.051923 (+756184us) spinlock_profiling.cc:243] Waited 6.21 ms on lock 0x7b540018fe54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:43.052166 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14292352}
I20260709 13:04:43.073369  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:42.973623 15699 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:42.968088 15471 raft_consensus.cc:697] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:42.995965 15685 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:43.067766 15660 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:43.081498  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:43.083395  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:43.083276  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:43.094552  8402 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:43.094933  8257 raft_consensus.cc:2393] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 20: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:43.095026  8254 raft_consensus.cc:2393] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 26: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:43.082361  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:43.082844  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:42.994531 15581 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:43.105762  8251 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:04:43.125353  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3596) took 1084 ms. Trace:
W20260709 13:04:43.092617  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 434809us
I20260709 13:04:43.107563  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:43.106738  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:43.068987 15438 raft_consensus.cc:697] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:43.126940  8253 raft_consensus.cc:1813] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Handling vote request from an unknown peer ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:43.127563  8253 raft_consensus.cc:2368] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18.
I20260709 13:04:43.125799  8401 rpcz_store.cc:276] 0709 13:04:42.052722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.240967 (+188245us) service_pool.cc:224] Handling call
0709 13:04:42.322288 (+ 81321us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:42.341918 (+ 19630us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:42.341973 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:42.375359 (+ 33386us) log.cc:844] Serialized 52 byte log entry
0709 13:04:42.654223 (+278864us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:43.124762 (+470539us) log.cc:844] Serialized 14 byte log entry
0709 13:04:43.125028 (+   266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.125112 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:43.125158 (+    46us) raft_consensus.cc:1685] finished
0709 13:04:43.125168 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:43.125302 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":273341,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":6817,"raft.queue_time_us":19222,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":24167,"thread_start_us":645433,"threads_started":2,"wal-append.queue_time_us":482871}
I20260709 13:04:43.152019  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:43.154974  8254 raft_consensus.cc:3055] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:43.158083  8254 raft_consensus.cc:740] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:43.159862  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:43.160624  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:43.094228 15681 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:43.161193  8401 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:43.151314  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:43.164927  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:04:43.167869  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.088979 15557 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.080s
I20260709 13:04:43.076855 15570 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:43.275276  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:04:43.327181  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.327836  9008 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:43.275022  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:43.244908  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:43.296062  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 355264us
I20260709 13:04:43.233148  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:43.219834  8397 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.007s	sys 0.033s
I20260709 13:04:43.293918  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:43.176620  8402 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
W20260709 13:04:43.295810  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 354016us
I20260709 13:04:43.250101  9004 raft_consensus.cc:1240] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:43.333937  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:43.330356  8397 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.011s	sys 0.034s
I20260709 13:04:43.339262  8402 raft_consensus.cc:2393] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 22: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:43.220098  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 3410) took 1149 ms. Trace:
I20260709 13:04:43.233722  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:43.148742  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 441575us
I20260709 13:04:43.329037  9007 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 28: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:43.340022  8332 rpcz_store.cc:276] 0709 13:04:42.071012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.343150 (+272138us) service_pool.cc:224] Handling call
0709 13:04:42.343494 (+   344us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:42.343840 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:42.343866 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:42.344367 (+   501us) log.cc:844] Serialized 52 byte log entry
0709 13:04:42.558746 (+214379us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:42.558809 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.558901 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:43.219856 (+660955us) raft_consensus.cc:1685] finished
0709 13:04:43.219877 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:43.220031 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":797,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":20412,"thread_start_us":153333,"threads_started":1,"wal-append.queue_time_us":280597}
I20260709 13:04:43.341558  9002 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 19: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:43.279037  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:04:43.222545  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:43.343997  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.194830  8250 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:43.276790  9005 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:43.267745  9006 raft_consensus.cc:1240] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:43.331764  8222 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:43.362651  9000 raft_consensus.cc:2393] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 20: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:43.363627  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:04:43.251151 15538 consensus_meta.cc:287] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.564s	user 0.012s	sys 0.000s
I20260709 13:04:43.208112  8254 consensus_queue.cc:260] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:43.348855  8297 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:43.368844  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:43.372278  8401 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
I20260709 13:04:43.381520  8369 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:43.384474  8223 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:43.439481  8401 raft_consensus.cc:1275] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:43.442131  8401 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:04:43.443017  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 94274.9us
I20260709 13:04:43.444522  8297 leader_election.cc:400] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18.
I20260709 13:04:43.444983  8297 leader_election.cc:403] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:43.450405  9002 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:04:43.440160  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:43.451313  8974 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:43.451583  9004 raft_consensus.cc:2393] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 20: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:43.441020  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:43.263882 15581 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:43.369688  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:43.452971  9007 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:43.367681  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:43.461412  8402 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 19: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:43.461784  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:43.464659  9008 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 24.
W20260709 13:04:43.458657  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 77309.2us
I20260709 13:04:43.468804  8369 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:43.463485  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:43.470669  8404 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:43.461828  8250 raft_consensus.cc:2393] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 17: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:04:43.474236  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 89909.8us
I20260709 13:04:43.226641 15699 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:43.474388  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:43.448731 15329 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:43.370186 15663 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:43.486912  9005 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 26. Current term is 27. Ops: []
I20260709 13:04:43.169559 15644 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.484006  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:43.488185  8250 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:04:43.454921 14967 consensus_meta.cc:287] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.615s	user 0.012s	sys 0.000s
I20260709 13:04:43.452875 15449 consensus_queue.cc:1048] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:04:43.455760 15616 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.453054 15577 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:04:43.519141  8223 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:43.332387 15569 consensus_queue.cc:1059] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.123s
I20260709 13:04:43.482976 15288 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:04:43.520884  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 69640.1us
I20260709 13:04:43.229236 15471 consensus_queue.cc:237] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:43.526350  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 75231.2us
I20260709 13:04:43.454659 15561 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:04:43.531175  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 86830.1us
I20260709 13:04:43.522063 15626 consensus_queue.cc:1059] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:04:43.556154 15625 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:43.569746 15561 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:43.570529 15608 raft_consensus.cc:3055] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:43.570914 15608 raft_consensus.cc:740] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:43.566247  8254 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:43.572184 15561 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:43.571882 15608 consensus_queue.cc:260] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:43.265645 15438 consensus_queue.cc:237] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:43.370981 15711 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.161s
I20260709 13:04:43.575757 15569 raft_consensus.cc:3055] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:43.576227 15569 raft_consensus.cc:740] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:43.572382  8297 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:43.577481 15569 consensus_queue.cc:260] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:43.578135 15581 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:04:43.578512 15581 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:43.578908 15581 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:43.581151  8399 raft_consensus.cc:1240] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:43.395720 15538 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:43.584954 15538 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:43.585624  8248 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 22. Current term is 24. Ops: []
W20260709 13:04:43.575809  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.529934 15644 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:43.535418 15577 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:43.535492 15616 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:43.611356  8223 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:04:43.594272  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.603811  8404 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:43.620595  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:43.485647 15508 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:43.609100  8332 raft_consensus.cc:1275] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 13:04:43.559836 15715 log.cc:927] Time spent T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.268s	user 0.001s	sys 0.000s
I20260709 13:04:43.641242  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:43.613159  9006 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:04:43.488909 15552 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:04:43.535351  8295 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.012s	sys 0.053s
I20260709 13:04:43.618148  8298 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:43.559231 15707 log.cc:927] Time spent T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:04:43.669338  8405 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:04:43.537715 15713 raft_consensus.cc:3060] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:04:43.517930 15536 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:43.623728  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:43.624550  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3591) took 1599 ms. Trace:
I20260709 13:04:43.556109 15663 raft_consensus.cc:697] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 22 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:43.672626  8398 rpcz_store.cc:276] 0709 13:04:42.025031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.190714 (+165683us) service_pool.cc:224] Handling call
0709 13:04:42.191043 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:42.431680 (+240637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.624310 (+1192630us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b5400067f54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:43.624494 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":551850368}
I20260709 13:04:43.626071  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:43.674180  8330 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:43.626425 15608 raft_consensus.cc:3060] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:43.675557  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:43.677052  8401 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:43.612941 15545 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:04:43.627647 15490 consensus_queue.cc:1059] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:43.628424  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:43.681973  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:43.682565  8324 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:43.682771  8327 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:43.561589 15570 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:43.631170  8222 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:43.698871  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 67788.7us
I20260709 13:04:43.699049 15545 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:43.699545 15545 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:43.699882 15545 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:43.699931  8222 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:04:43.631655 15553 consensus_meta.cc:287] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.847s	user 0.011s	sys 0.000s
I20260709 13:04:43.701735  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 1308) took 1181 ms. Trace:
I20260709 13:04:43.631908 15653 consensus_queue.cc:1048] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:43.702275  8406 rpcz_store.cc:276] 0709 13:04:42.520575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.686522 (+165947us) service_pool.cc:224] Handling call
0709 13:04:42.693980 (+  7458us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:42.694323 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:42.694353 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:42.694780 (+   427us) log.cc:844] Serialized 26 byte log entry
0709 13:04:43.059612 (+364832us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:43.059657 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.059748 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:43.701504 (+641756us) raft_consensus.cc:1685] finished
0709 13:04:43.701526 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:43.701679 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":940,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":3221,"thread_start_us":364512,"threads_started":1,"wal-append.queue_time_us":630538}
I20260709 13:04:43.632836 15561 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:43.704022  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:43.614182 15647 consensus_meta.cc:287] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.583s	user 0.005s	sys 0.007s
W20260709 13:04:43.613780 15704 consensus_meta.cc:287] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.525s	user 0.003s	sys 0.007s
I20260709 13:04:43.705688 15704 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:43.616040 15538 consensus_queue.cc:1059] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.091s
I20260709 13:04:43.706830  8330 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:43.707147 15538 raft_consensus.cc:3055] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:43.707508 15538 raft_consensus.cc:740] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:43.707746  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:43.708247 15704 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:43.708348  8330 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:43.705255  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:43.709308  8252 raft_consensus.cc:2410] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 24 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
W20260709 13:04:43.635556 15455 consensus_meta.cc:287] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.795s	user 0.006s	sys 0.003s
I20260709 13:04:43.712095  8406 raft_consensus.cc:2393] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 22: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:43.712250 15455 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:43.713268 15545 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:43.714744 15455 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:43.637573  8257 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:43.715620 15545 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:43.645323  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:04:43.647538  8295 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.012s	sys 0.053s
I20260709 13:04:43.727803  8329 raft_consensus.cc:2393] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 24: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:04:43.728026  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 803917us
I20260709 13:04:43.728820  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:43.729560  8329 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:43.622336  8323 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 28: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:43.669838  8250 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 28.
I20260709 13:04:43.736897  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:04:43.618257 15318 consensus_meta.cc:287] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.553s	user 0.010s	sys 0.000s
I20260709 13:04:43.533571 15706 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:43.678093  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:43.627496 15712 log.cc:927] Time spent T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:04:43.555398 15449 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:43.687657  8399 raft_consensus.cc:1240] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:43.691146  8254 raft_consensus.cc:1275] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:04:43.742770  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:43.743518  8254 raft_consensus.cc:2393] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 20: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:43.696234  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:43.697501  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.742182  8329 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 24.
I20260709 13:04:43.579675 15569 raft_consensus.cc:3060] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:43.709587 15674 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:43.634436 15644 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:43.718185  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:43.718416  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:43.638159  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:04:43.722062 15718 log.cc:927] Time spent T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.186s	user 0.000s	sys 0.000s
I20260709 13:04:43.702397 15553 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:43.755333  8329 raft_consensus.cc:1275] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:43.758193  8406 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:43.727134 15581 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:43.724195  8257 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
I20260709 13:04:43.755966  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:43.623011 15577 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:43.640650 15626 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:43.617381 15673 consensus_queue.cc:1059] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.131s
I20260709 13:04:43.744473  8295 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:43.701610 15699 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:04:43.767326  8972 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.025s	sys 0.045s
I20260709 13:04:43.740523 15318 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:43.746392  8326 raft_consensus.cc:2393] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 22: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:04:43.752846  8255 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.022s	sys 0.040s
W20260709 13:04:43.751644  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.587694  9003 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 25. Current term is 26. Ops: []
W20260709 13:04:43.770032  8255 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.023s	sys 0.040s
I20260709 13:04:43.760378  8254 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 20.
I20260709 13:04:43.766525  8295 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:43.755155 15554 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.037s	sys 0.042s
I20260709 13:04:43.745304 15288 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:04:43.767980  8972 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.025s	sys 0.045s
I20260709 13:04:43.741634 15591 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:04:43.772317  8250 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 19: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:04:43.754256 15556 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
W20260709 13:04:43.775367  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 663211us
I20260709 13:04:43.754681 15706 raft_consensus.cc:697] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:43.686786 15552 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:43.739058 15536 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:43.781826  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.783458  8327 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 25.
I20260709 13:04:43.785007  8249 raft_consensus.cc:2393] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 21: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:04:43.774077 15554 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.038s	sys 0.042s
I20260709 13:04:43.767241 15553 raft_consensus.cc:697] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:43.769941 15626 raft_consensus.cc:697] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 24 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:43.748111 15720 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
W20260709 13:04:43.782668 15675 consensus_meta.cc:287] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.725s	user 0.001s	sys 0.010s
W20260709 13:04:43.787815  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.902256  8220 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:43.848032  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:43.923651  8254 raft_consensus.cc:3060] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:43.934903  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 28 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:43.913385  8220 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:43.942811  9001 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 14.
I20260709 13:04:43.943910  8400 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 28: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:04:43.893522 15681 consensus_meta.cc:287] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.562s	user 0.009s	sys 0.000s
W20260709 13:04:43.945096  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.938818  8220 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:43.874490 15417 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.948395 15417 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:43.952855 15417 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:43.953733 15570 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:43.954778 15570 raft_consensus.cc:697] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 24 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:43.956060 15570 consensus_queue.cc:237] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:43.782449 15699 consensus_queue.cc:1059] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.160s
I20260709 13:04:43.776316 15288 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:43.953896  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 3009) took 1165 ms. Trace:
I20260709 13:04:43.965056  8400 rpcz_store.cc:276] 0709 13:04:42.788782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.827751 (+ 38969us) service_pool.cc:224] Handling call
0709 13:04:43.934692 (+1106941us) spinlock_profiling.cc:243] Waited 5.24 ms on lock 0x7b5800170798. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:04:43.953819 (+ 19127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12045056}
W20260709 13:04:43.968020  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.976903  8254 raft_consensus.cc:2468] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 22.
I20260709 13:04:43.980464  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:43.991922  8252 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:43.985015  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:43.987797  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3598) took 1934 ms. Trace:
I20260709 13:04:43.872906 15449 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:43.998152  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:43.996752  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:43.999353  8402 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
I20260709 13:04:44.000152  8400 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 20.
I20260709 13:04:44.002674  8406 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 21.
I20260709 13:04:43.993754  8397 rpcz_store.cc:276] 0709 13:04:42.053665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.549496 (+495831us) service_pool.cc:224] Handling call
0709 13:04:42.549881 (+   385us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:42.550317 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:42.550349 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:42.551112 (+   763us) log.cc:844] Serialized 52 byte log entry
0709 13:04:43.339207 (+788095us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:43.339245 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.339308 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:43.987572 (+648264us) raft_consensus.cc:1685] finished
0709 13:04:43.987592 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:43.987735 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":116,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"thread_start_us":787707,"threads_started":1,"wal-append.queue_time_us":818862}
W20260709 13:04:44.004309  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 56388.7us
I20260709 13:04:43.940308 15508 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:44.006803  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:43.960132 15647 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:44.009631  8400 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:43.909771 15673 consensus_queue.cc:1048] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.097s
I20260709 13:04:44.011366  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:44.012650  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:44.030905  8406 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:04:44.026824  8403 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.017s	sys 0.029s
W20260709 13:04:44.037812  8403 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.018s	sys 0.029s
I20260709 13:04:44.020843  8220 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:44.062924  8402 raft_consensus.cc:2393] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 20: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:44.001034 15681 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:44.015527 15528 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.008s	sys 0.027s
W20260709 13:04:44.113551  8369 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.009s	sys 0.011s
W20260709 13:04:44.129294  8369 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.009s	sys 0.011s
I20260709 13:04:44.129757  8252 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 21.
W20260709 13:04:44.130046  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 661303us
I20260709 13:04:43.786468 15663 consensus_queue.cc:237] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:44.124763  8406 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 24.
I20260709 13:04:44.131031  8972 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:44.131630  8369 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:44.008698 15629 consensus_queue.cc:1048] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.118s
I20260709 13:04:44.011255  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:44.030459 15660 consensus_meta.cc:287] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.907s	user 0.007s	sys 0.004s
I20260709 13:04:44.014195 15647 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:44.016453 15704 raft_consensus.cc:3055] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:44.135723 15553 consensus_queue.cc:237] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:44.166077 15704 raft_consensus.cc:740] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:44.167131 15704 consensus_queue.cc:260] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:44.168237 15704 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:44.140472 15626 consensus_queue.cc:237] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:44.171039  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:44.172253  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:44.172811  8250 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 16.
I20260709 13:04:44.174695 15552 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:44.175159  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2859) took 1205 ms. Trace:
I20260709 13:04:44.175472 15575 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:44.175561  8255 rpcz_store.cc:276] 0709 13:04:42.969792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.975315 (+  5523us) service_pool.cc:224] Handling call
0709 13:04:42.975660 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:42.993989 (+ 18329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:42.994036 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:43.066366 (+ 72330us) log.cc:844] Serialized 26 byte log entry
0709 13:04:43.259118 (+192752us) spinlock_profiling.cc:243] Waited 51.8 ms on lock 0x7b4800002dd8. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882d6e7aa1 00007f882d6deb37 00007f882d6de9ca 00007f882d7d626f 00007f882d7d5f37 00007f882d7ea55c 00007f882d7ea508 00007f882d7ea488 00007f882d7ea443 00007f882d7e96ec 00007f882c713a96
0709 13:04:43.770519 (+511401us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:43.770558 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.770617 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:44.174827 (+404210us) raft_consensus.cc:1685] finished
0709 13:04:44.174851 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:44.175095 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9399,"raft.queue_time_us":9713,"raft.run_cpu_time_us":2722,"raft.run_wall_time_us":268066,"spinlock_wait_cycles":119178112,"thread_start_us":673198,"threads_started":1,"wal-append.queue_time_us":1062569}
I20260709 13:04:44.156641 15508 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:44.178097  8249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2863) took 1198 ms. Trace:
I20260709 13:04:44.097222 15706 consensus_queue.cc:237] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:44.178524  8249 rpcz_store.cc:276] 0709 13:04:42.979601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.981543 (+  1942us) service_pool.cc:224] Handling call
0709 13:04:44.177884 (+1196341us) spinlock_profiling.cc:243] Waited 710 ms on lock 0x7b540034bcd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:44.178037 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1633314688}
I20260709 13:04:44.165169 15647 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:44.096210 15690 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.002s	user 0.017s	sys 0.018s
I20260709 13:04:44.185459  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:04:44.113850 15627 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.013s	sys 0.007s
I20260709 13:04:44.028625 15683 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:44.006724 15449 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:44.129119 15528 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.835s	user 0.008s	sys 0.028s
I20260709 13:04:44.159444  8369 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:44.163435 15554 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:04:44.143210 15438 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.019s	sys 0.014s
I20260709 13:04:44.182787  8250 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 24. Current term is 25. Ops: []
I20260709 13:04:44.183770  8223 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:44.134851  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:44.163460 15724 log.cc:927] Time spent T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
I20260709 13:04:44.160605 15711 raft_consensus.cc:2804] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:44.157918 15689 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:44.189778 15627 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.014s	sys 0.007s
I20260709 13:04:44.198487  8400 raft_consensus.cc:2393] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 21: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
W20260709 13:04:44.189168 15690 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.100s	user 0.018s	sys 0.018s
I20260709 13:04:44.219734  8400 raft_consensus.cc:1240] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:04:44.203337  8325 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:44.206854 15703 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:04:44.207415 15417 consensus_queue.cc:1059] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:04:44.216044  8253 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 20. Current term is 21. Ops: []
W20260709 13:04:44.211892 15625 consensus_meta.cc:287] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.641s	user 0.008s	sys 0.000s
I20260709 13:04:44.220031  9001 raft_consensus.cc:1240] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:44.138615 15681 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:44.220546  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 61173.6us
I20260709 13:04:44.193068 15575 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:44.196246 15438 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.019s	sys 0.015s
I20260709 13:04:44.226222  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:04:44.227216 15405 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.012s	sys 0.033s
I20260709 13:04:44.219521 15554 raft_consensus.cc:697] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 21 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:04:44.248011  8328 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.006s	sys 0.026s
W20260709 13:04:44.248358  8328 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.006s	sys 0.026s
W20260709 13:04:44.250445  8298 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.005s	sys 0.026s
W20260709 13:04:44.250881  8298 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.006s	sys 0.027s
I20260709 13:04:44.225592 15711 raft_consensus.cc:697] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 21 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:44.195233 15683 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:44.251447  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 633336us
W20260709 13:04:44.245049 15405 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.012s	sys 0.034s
W20260709 13:04:44.245255 15490 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.024s	sys 0.024s
I20260709 13:04:44.239156 15581 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:44.247927 15721 raft_consensus.cc:3055] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 24 LEADER]: Stepping down as leader of term 24
W20260709 13:04:44.274374  8972 leader_election.cc:336] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.562s (SENT)
I20260709 13:04:44.289842  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:44.280288  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:44.292096  8252 raft_consensus.cc:2393] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 19: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:44.292543  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:44.266669 15490 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.024s	sys 0.024s
W20260709 13:04:44.295426  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 274669us
I20260709 13:04:44.296623  8220 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:44.297093 15318 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:44.308286 15538 consensus_queue.cc:260] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:44.320519 15538 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:44.322000  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2855) took 1365 ms. Trace:
I20260709 13:04:44.322439  8248 rpcz_store.cc:276] 0709 13:04:42.956416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.956574 (+   158us) service_pool.cc:224] Handling call
0709 13:04:42.957990 (+  1416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:43.586060 (+628070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.321745 (+735685us) spinlock_profiling.cc:243] Waited 627 ms on lock 0x7b540030a054. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:44.321936 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1442977536}
I20260709 13:04:44.324250  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 1268) took 1007 ms. Trace:
I20260709 13:04:44.324611  8248 rpcz_store.cc:276] 0709 13:04:43.316344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.323418 (+1007074us) service_pool.cc:224] Handling call
0709 13:04:44.323760 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:44.323946 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:44.323969 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:44.324007 (+    38us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:44.324035 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.324095 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:44.324200 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:44.327515  8297 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:44.299817 15490 consensus_queue.cc:1059] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 13:04:44.301724 15562 raft_consensus.cc:3055] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:44.329450 15562 raft_consensus.cc:740] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:44.330344 15562 consensus_queue.cc:260] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:44.331699 15562 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:44.332593 15616 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:44.302986 15557 raft_consensus.cc:3055] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:44.333724 15557 raft_consensus.cc:740] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:44.310902  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 2771) took 1205 ms. Trace:
I20260709 13:04:44.334579  8253 rpcz_store.cc:276] 0709 13:04:43.105136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.149548 (+ 44412us) service_pool.cc:224] Handling call
0709 13:04:43.150005 (+   457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:44.226974 (+1076969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.310535 (+ 83561us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540034a8d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:44.310802 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2451110784}
I20260709 13:04:44.313145  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:44.313983  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:44.337518  9005 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:44.314100  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 2774) took 1156 ms. Trace:
W20260709 13:04:44.302979  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.315871  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:44.343231  8405 raft_consensus.cc:1240] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:44.341266  8257 rpcz_store.cc:276] 0709 13:04:43.158071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.164565 (+  6494us) service_pool.cc:224] Handling call
0709 13:04:44.313889 (+1149324us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b540034b7d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:44.314008 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":684961152}
I20260709 13:04:44.340793  8397 raft_consensus.cc:1275] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:04:44.224557  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:04:44.356194  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.362847  9005 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
W20260709 13:04:44.264096 15427 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.034s	sys 0.016s
I20260709 13:04:44.366281  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2857) took 1405 ms. Trace:
W20260709 13:04:44.367731  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:44.419521  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:44.420430  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:44.368170  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 48891us
I20260709 13:04:44.223855 15689 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:44.369851  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 73374.9us
I20260709 13:04:44.387202 15288 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:44.370793 15561 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:04:44.372035 15653 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.372499 15616 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.245234 15644 consensus_queue.cc:1059] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 13:04:44.382824  8399 raft_consensus.cc:3055] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:44.385224  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:44.289340 15403 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.022s	sys 0.034s
W20260709 13:04:44.499090  8251 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 2778) took 1278 ms (client timeout 1673 ms). Trace:
I20260709 13:04:44.292816 15721 raft_consensus.cc:740] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 24 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:44.364034 15536 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:44.421216  8972 leader_election.cc:336] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.580s (SENT)
I20260709 13:04:44.386231 15660 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.065s
I20260709 13:04:44.367197 15417 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:04:44.450958  8327 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.024s	sys 0.055s
I20260709 13:04:44.462359  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:44.519032  8326 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:44.519965  8327 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.024s	sys 0.055s
W20260709 13:04:44.490250  8404 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.735s	user 0.003s	sys 0.027s
I20260709 13:04:44.497385  8399 raft_consensus.cc:740] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:04:44.522240  8404 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.003s	sys 0.027s
W20260709 13:04:44.500351  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 370158us
I20260709 13:04:44.502228  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:44.524240  8332 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:44.524299  8222 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:44.530238  8222 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:44.503149  8972 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:44.419096  8256 rpcz_store.cc:276] 0709 13:04:42.961210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.969122 (+  7912us) service_pool.cc:224] Handling call
0709 13:04:42.982706 (+ 13584us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:42.987229 (+  4523us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:42.987270 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:42.987989 (+   719us) log.cc:844] Serialized 52 byte log entry
0709 13:04:43.458474 (+470485us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:43.458517 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.458585 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:44.365975 (+907390us) raft_consensus.cc:1685] finished
0709 13:04:44.366002 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:44.366208 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4061,"raft.queue_time_us":5153,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":54705,"thread_start_us":393133,"threads_started":1,"wal-append.queue_time_us":710834}
W20260709 13:04:44.499640  8251 rpcz_store.cc:269] 0709 13:04:43.220649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.366363 (+145714us) service_pool.cc:224] Handling call
0709 13:04:44.385038 (+1018675us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b5800002a98. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699a22 00007f882e5b2c09 00007f882d70cee0 00007f882d7d60ab 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:04:44.499008 (+113970us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3699456}
W20260709 13:04:44.413290 15471 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.014s	sys 0.013s
I20260709 13:04:44.521579  8399 consensus_queue.cc:260] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:44.556695  8399 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:44.509708  9003 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:44.502513 15403 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.917s	user 0.023s	sys 0.034s
W20260709 13:04:44.570061  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 148867us
I20260709 13:04:44.558787  8220 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:44.451184 15438 consensus_queue.cc:1059] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.088s
I20260709 13:04:44.386411 15722 consensus_queue.cc:1059] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.101s
I20260709 13:04:44.498493 15616 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:44.558068 15674 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:44.519487 15561 raft_consensus.cc:697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 25 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:44.602206  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.602756  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:44.562347 15471 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.927s	user 0.015s	sys 0.014s
I20260709 13:04:44.627509  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:44.406437 15554 consensus_queue.cc:237] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:44.449152 15427 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.034s	sys 0.017s
I20260709 13:04:44.504207 15653 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:44.580093 15690 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:44.601524 15644 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:44.564404 15591 consensus_queue.cc:1048] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 13:04:44.615374 15449 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:44.495514 15711 consensus_queue.cc:237] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:44.609905 15629 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:04:44.621155 15734 log.cc:927] Time spent T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.386s	user 0.000s	sys 0.001s
W20260709 13:04:44.643656  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.543624 15740 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:04:44.648427  9003 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
W20260709 13:04:44.638799  8295 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.867s	user 0.002s	sys 0.027s
I20260709 13:04:44.652334  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:44.623965 15627 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.624117 15508 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:44.627043 15750 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:44.640558 15681 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:44.670085  8295 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.899s	user 0.003s	sys 0.027s
I20260709 13:04:44.642735 15552 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:44.675014 15748 log.cc:927] Time spent T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:04:44.646543 15644 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:44.682683  8222 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:44.686856  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 920377us
I20260709 13:04:44.698978  8295 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:44.700505  8295 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:44.702003  8295 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:44.703529  8295 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:44.681205 15674 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.620599 15721 consensus_queue.cc:260] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:44.697299 15730 log.cc:927] Time spent T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.319s	user 0.000s	sys 0.001s
W20260709 13:04:44.716650 15729 log.cc:927] Time spent T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.306s	user 0.000s	sys 0.001s
W20260709 13:04:44.725543  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 303719us
I20260709 13:04:44.695812 15683 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:44.707646 15644 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:44.731352  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 3419) took 1121 ms. Trace:
I20260709 13:04:44.783195  8331 rpcz_store.cc:276] 0709 13:04:43.610199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.610314 (+   115us) service_pool.cc:224] Handling call
0709 13:04:43.610611 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:43.617662 (+  7051us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:43.617694 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:43.630137 (+ 12443us) log.cc:844] Serialized 26 byte log entry
0709 13:04:43.973979 (+343842us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b18003e42e0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e0d1 00007f8828779a0f
0709 13:04:44.187908 (+213929us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:44.187969 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.188052 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:44.730941 (+542889us) raft_consensus.cc:1685] finished
0709 13:04:44.730972 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:44.731273 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6689,"raft.queue_time_us":7201,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":43732,"spinlock_wait_cycles":28608896,"thread_start_us":213737,"threads_started":1,"wal-append.queue_time_us":865877}
I20260709 13:04:44.687429 15508 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:44.689097 15703 raft_consensus.cc:697] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 28 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:44.669818 15629 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:44.722771 15737 log.cc:927] Time spent T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:04:44.683228 15627 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:44.715914 15536 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:44.739408  8974 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:04:44.760812 15570 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.001s	sys 0.002s
I20260709 13:04:44.776057  8323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 3429) took 1135 ms. Trace:
I20260709 13:04:44.670742 15545 consensus_queue.cc:1059] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:04:44.696746 15552 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:44.799036 15681 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:44.737514 15736 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
W20260709 13:04:44.737018 15733 log.cc:927] Time spent T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
I20260709 13:04:44.691490 15750 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:44.723047 15318 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:44.797704  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.800140  8404 raft_consensus.cc:1275] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:04:44.809671  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 106217us
W20260709 13:04:44.748199 15455 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.004s	sys 0.031s
I20260709 13:04:44.834632  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 1274) took 1056 ms. Trace:
I20260709 13:04:44.745800 15721 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:04:44.843009  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 103697us
I20260709 13:04:44.947126  8248 rpcz_store.cc:276] 0709 13:04:43.778064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.330489 (+552425us) service_pool.cc:224] Handling call
0709 13:04:44.330746 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:44.330924 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:04:44.696914 (+365990us) log.cc:844] Serialized 14 byte log entry
0709 13:04:44.834272 (+137358us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:44.834320 (+    48us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:04:44.834359 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.834433 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:44.834567 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52734,"prepare.queue_time_us":354421,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":8605,"raft.queue_time_us":53744,"spinlock_wait_cycles":116368256,"thread_start_us":449847,"threads_started":2}
W20260709 13:04:44.839501  8369 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.002s	sys 0.008s
W20260709 13:04:44.949738  8369 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.002s	sys 0.008s
W20260709 13:04:44.950111  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 709119us
I20260709 13:04:44.852504  8326 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:44.844930 15490 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:44.840252  8323 rpcz_store.cc:276] 0709 13:04:43.640727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.736654 (+ 95927us) service_pool.cc:224] Handling call
0709 13:04:43.737021 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:43.737261 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 13:04:44.224877 (+487616us) log.cc:844] Serialized 14 byte log entry
0709 13:04:44.775531 (+550654us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:44.775610 (+    79us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:04:44.775676 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.775774 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:44.775964 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61940,"prepare.queue_time_us":727173,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":170722,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":15103,"thread_start_us":1019348,"threads_started":3,"wal-append.queue_time_us":611298}
W20260709 13:04:44.736567 15731 log.cc:927] Time spent T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
I20260709 13:04:44.899999 15663 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:44.840024 15570 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.001s	sys 0.002s
W20260709 13:04:44.984236  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 43738.4us
I20260709 13:04:44.986570  8295 leader_election.cc:400] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18.
I20260709 13:04:44.733925 15674 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:44.842113 15730 log.cc:927] Time spent T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:04:44.822141 15749 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:44.814785 15689 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:44.835799 15438 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:44.831650 15403 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:44.821931 15752 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:44.899374 15683 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:44.894661 15488 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.950829  8369 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:44.959857  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:04:44.929268 15746 log.cc:927] Time spent T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 13:04:44.708815 15561 consensus_queue.cc:237] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:44.956751  8298 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:45.042651 15653 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:45.081749 15545 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.094702 15545 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:45.096885 15545 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:44.930709 15626 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.753s	user 0.008s	sys 0.036s
W20260709 13:04:45.100124 15626 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.923s	user 0.009s	sys 0.036s
I20260709 13:04:44.966846  8399 raft_consensus.cc:1275] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:04:44.910321 15427 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:44.946882 15455 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.004s	sys 0.032s
I20260709 13:04:45.101679 15455 raft_consensus.cc:3055] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:45.101125 15427 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:45.102074 15455 raft_consensus.cc:740] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:45.102629  8399 raft_consensus.cc:1240] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:45.102928 15455 consensus_queue.cc:260] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:45.104089 15455 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:44.987699  8295 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:45.007834  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:45.105980  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 118365us
I20260709 13:04:45.106685  8295 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:44.947384 15616 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:44.933789 15781 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:45.107823  8295 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:45.107887 15781 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:45.110088 15781 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:45.110635 15781 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:45.112984 15781 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:44.910416 15681 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:45.114641 15552 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:45.115105 15681 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:45.091394 15787 log.cc:927] Time spent T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 13:04:45.093568  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3607) took 2561 ms. Trace:
I20260709 13:04:44.954190 15318 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:45.116485  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 159832us
I20260709 13:04:45.091212 15488 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:45.118045 15318 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:45.109220  8295 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:45.119575 15488 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:45.119776 15713 consensus_meta.cc:287] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 1.432s	user 0.009s	sys 0.003s
I20260709 13:04:45.120370 15713 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18.
I20260709 13:04:45.094039 15653 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.120652  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 28 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:45.121101 15653 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:45.121678  8248 raft_consensus.cc:2393] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 28: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:45.123536 15653 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:45.009776 15758 log.cc:927] Time spent T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:04:45.123991 15750 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:45.055820  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.986791 15556 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.078471 15683 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.124908 15703 consensus_queue.cc:237] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.16, Last appended by leader: 16, Current term: 28, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:45.125746 15683 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:45.125698 15556 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:45.128298 15556 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:45.124132 15782 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:45.134867 15683 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:45.135454  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:45.136369  8404 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 20.
I20260709 13:04:45.124025 15797 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:04:45.138795 15577 consensus_meta.cc:287] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 1.369s	user 0.011s	sys 0.000s
I20260709 13:04:45.139236  8298 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:45.139346 15577 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:45.133169 15647 consensus_meta.cc:287] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.940s	user 0.009s	sys 0.000s
W20260709 13:04:45.141229  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 190502us
I20260709 13:04:45.141614  8974 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:45.149235  9005 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:04:45.150950  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:45.151929  9001 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
I20260709 13:04:45.154677  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:45.155364  8253 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:45.140390 15740 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:04:45.156158  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:45.160796  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:45.161497  8248 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 16.
W20260709 13:04:45.156164  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.993012 15663 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:45.121239 15713 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:45.157104  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:45.155592 15786 log.cc:927] Time spent T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:04:45.121742 15780 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:04:45.186939  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:45.187763  8250 raft_consensus.cc:3060] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:45.161681  8298 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:45.169384  8402 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 26. Current term is 27. Ops: []
I20260709 13:04:45.170455  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:45.172646  8253 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 24: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:04:45.159022  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.174472  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3623) took 1166 ms. Trace:
I20260709 13:04:45.174783  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:45.160195 15762 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
W20260709 13:04:45.156092 15778 log.cc:927] Time spent T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
W20260709 13:04:45.156279 15792 log.cc:927] Time spent T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:04:45.216965  8248 raft_consensus.cc:1240] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 22 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:04:45.139277 15797 raft_consensus.cc:697] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:45.187060  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:45.218117  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 56518.1us
I20260709 13:04:45.218770  8369 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:45.188114  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:45.116478  8403 rpcz_store.cc:276] 0709 13:04:42.532489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.753266 (+220777us) service_pool.cc:224] Handling call
0709 13:04:42.832444 (+ 79178us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:42.832954 (+   510us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:43.168307 (+335353us) log.cc:844] Serialized 14 byte log entry
0709 13:04:44.039031 (+870724us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:44.109945 (+ 70914us) log.cc:844] Serialized 26 byte log entry
0709 13:04:44.110129 (+   184us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:44.110177 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.110266 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:45.093281 (+983015us) raft_consensus.cc:1685] finished
0709 13:04:45.093305 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:45.093474 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":134024,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":65028,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":27222,"spinlock_wait_cycles":2432,"thread_start_us":1022336,"threads_started":2,"wal-append.queue_time_us":1079668}
I20260709 13:04:45.189231  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:45.157814 15488 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:45.163384 15556 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:45.167958 15538 consensus_meta.cc:287] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.847s	user 0.009s	sys 0.002s
W20260709 13:04:45.171547 15575 consensus_meta.cc:287] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.926s	user 0.008s	sys 0.004s
I20260709 13:04:45.221771  8326 raft_consensus.cc:2368] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for earlier term 23. Current term is 24.
I20260709 13:04:45.229092  8248 raft_consensus.cc:1275] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:04:45.236414  8327 raft_consensus.cc:1240] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:04:45.238081  8327 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 26. Current term is 27. Ops: []
I20260709 13:04:45.239532  8327 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 24. Current term is 25. Ops: []
I20260709 13:04:45.233773  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:45.234071  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:45.189811  8405 rpcz_store.cc:276] 0709 13:04:44.008420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.344133 (+335713us) service_pool.cc:224] Handling call
0709 13:04:44.344506 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:44.344900 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:44.344940 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:44.629002 (+284062us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b2c000e1c48. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:04:44.645696 (+ 16694us) log.cc:844] Serialized 26 byte log entry
0709 13:04:44.750500 (+104804us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:44.750540 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.750619 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:45.173679 (+423060us) raft_consensus.cc:1685] finished
0709 13:04:45.173709 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:45.174394 (+   685us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":121081,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":7361,"spinlock_wait_cycles":44741376,"thread_start_us":68990,"threads_started":1,"wal-append.queue_time_us":172102}
I20260709 13:04:45.241245 15575 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:45.163573 15795 raft_consensus.cc:2804] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:45.226850 15488 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:45.218842 15562 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.008s	sys 0.000s
I20260709 13:04:45.247099  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:45.158095 15647 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:45.250411  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3624) took 1234 ms. Trace:
I20260709 13:04:45.234503  8369 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:45.254415  8323 raft_consensus.cc:1240] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:04:45.254362  8401 rpcz_store.cc:276] 0709 13:04:44.016184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.346073 (+329889us) service_pool.cc:224] Handling call
0709 13:04:44.346425 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:44.719797 (+373372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:44.719851 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:44.725844 (+  5993us) log.cc:844] Serialized 26 byte log entry
0709 13:04:44.912451 (+186607us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:44.913406 (+   955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.913591 (+   185us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:45.250183 (+336592us) raft_consensus.cc:1685] finished
0709 13:04:45.250204 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:45.250353 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":173785,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":20296,"thread_start_us":465040,"threads_started":2,"wal-append.queue_time_us":366739}
I20260709 13:04:45.245491  8250 raft_consensus.cc:2468] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:45.244876  8248 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:04:45.196676 15713 raft_consensus.cc:697] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:04:45.235935 15774 log.cc:927] Time spent T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:04:45.263098  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 1267) took 1947 ms. Trace:
I20260709 13:04:45.253221  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:45.233206 15704 consensus_meta.cc:287] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 1.065s	user 0.005s	sys 0.010s
W20260709 13:04:45.236193 15783 log.cc:927] Time spent T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:04:45.218623 15438 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:45.164350 15403 consensus_queue.cc:1059] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 13:04:45.248767 15488 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:45.261547  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:45.235852 15681 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:45.263614  8254 rpcz_store.cc:276] 0709 13:04:43.316079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.186538 (+870459us) service_pool.cc:224] Handling call
0709 13:04:44.186864 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:44.187104 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:44.187134 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:44.206343 (+ 19209us) log.cc:844] Serialized 26 byte log entry
0709 13:04:44.710868 (+504525us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:04:44.710906 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.710978 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:45.262854 (+551876us) raft_consensus.cc:1685] finished
0709 13:04:45.262880 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:45.263033 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":465241,"threads_started":1,"wal-append.queue_time_us":897641}
I20260709 13:04:45.265275  8252 raft_consensus.cc:2410] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 23 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
W20260709 13:04:45.262602  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 430405us
W20260709 13:04:45.248858 15562 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.008s	sys 0.000s
W20260709 13:04:45.253789 15755 log.cc:927] Time spent T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:04:45.267407  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:45.268999  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:04:45.269682  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.233078 15683 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:45.231447 15773 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:45.258661 15795 raft_consensus.cc:697] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 22 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:45.303164  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:45.275841  8369 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:45.243395 15772 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:45.310911  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:45.243081 15556 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:45.280463 15742 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:04:45.278578 15695 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.008s	sys 0.008s
I20260709 13:04:45.289448  8325 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:45.266803 15627 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:45.257678 15508 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:45.273870  8256 raft_consensus.cc:2368] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 22. Current term is 23.
I20260709 13:04:45.296571 15773 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:45.332392  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 3427) took 1693 ms. Trace:
I20260709 13:04:45.333907  8369 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:45.343165  8369 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:45.285212  8400 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.000s	sys 0.017s
W20260709 13:04:45.344229  8400 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.000s	sys 0.017s
I20260709 13:04:45.350138 15773 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:45.320104 15681 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:45.354156  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:45.332504 15695 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.008s	sys 0.008s
I20260709 13:04:45.370932  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:45.310328 15772 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:45.296871 15438 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:45.339056  8328 rpcz_store.cc:276] 0709 13:04:43.638780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.638906 (+   126us) service_pool.cc:224] Handling call
0709 13:04:43.639205 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:43.639526 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:44.248763 (+609237us) log.cc:844] Serialized 14 byte log entry
0709 13:04:44.831177 (+582414us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:44.841196 (+ 10019us) log.cc:844] Serialized 26 byte log entry
0709 13:04:44.841386 (+   190us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:44.841552 (+   166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.841635 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:45.332180 (+490545us) raft_consensus.cc:1685] finished
0709 13:04:45.332202 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:45.332328 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55678,"prepare.queue_time_us":648366,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":333519,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":71940,"thread_start_us":1016775,"threads_started":2,"wal-append.queue_time_us":685904}
W20260709 13:04:45.371222  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.319356 15403 consensus_queue.cc:1048] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:04:45.365105  8325 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 26.
I20260709 13:04:45.306962 15784 consensus_queue.cc:1048] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:04:45.381023  9001 raft_consensus.cc:1240] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 26. Current term is 27. Ops: []
I20260709 13:04:45.390565  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:45.334477 15449 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
W20260709 13:04:45.386464 15569 consensus_meta.cc:287] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 1.627s	user 0.016s	sys 0.003s
I20260709 13:04:45.369794 15674 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:45.430502 15782 consensus_queue.cc:1059] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 13:04:45.430939 15629 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:45.361626 15742 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:45.402115 15795 consensus_queue.cc:237] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:45.403061 15556 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:45.403635 15663 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:45.405596 15776 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:45.409466 15767 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:45.307241 15797 consensus_queue.cc:237] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:45.445719 15781 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:45.448148  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:45.448758  8329 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 20.
I20260709 13:04:45.449775  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:45.450080  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:45.450552  8254 raft_consensus.cc:2393] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 17: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:45.450783  8332 raft_consensus.cc:2393] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 17: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:45.444801 15776 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:45.417647 15713 consensus_queue.cc:237] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:45.451927  8250 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:04:45.452584 15581 raft_consensus.cc:3055] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:04:45.453038 15581 raft_consensus.cc:740] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:45.410436 15647 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:45.453420 15776 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:45.449775  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:45.387588 15769 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:45.396337 15577 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:45.450385  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:45.421659 15557 consensus_queue.cc:260] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:45.391129 15545 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.382897 15772 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:45.443480  9001 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
W20260709 13:04:45.465202  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 122156us
I20260709 13:04:45.450441 15773 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:45.471148 15773 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:45.441126 15706 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.472867  8369 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:45.473526  8326 raft_consensus.cc:2393] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 21: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:45.475000  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:45.454530 15690 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.477316  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:45.472380  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:45.500274  9000 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:45.502604  8369 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:45.454084 15581 consensus_queue.cc:260] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:45.504537 15767 raft_consensus.cc:2764] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader election decision vote started in defunct term 23: won
I20260709 13:04:45.505137 15577 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:45.505365 15767 raft_consensus.cc:2764] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election decision vote started in defunct term 20: won
I20260709 13:04:45.478677 15545 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:45.507236  8369 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:45.508234 15557 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:45.508354 15767 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:45.464494 15769 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:45.509773 15690 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:45.511507  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:45.512074 15644 consensus_meta.cc:287] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.696s	user 0.005s	sys 0.005s
I20260709 13:04:45.512663 15767 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:45.451964 15490 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:45.513813 15706 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:45.514061  8369 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:45.512504 15644 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:45.526823  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:45.538306  8369 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:45.535531  9001 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 22.
I20260709 13:04:45.500984 15773 consensus_queue.cc:1059] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
W20260709 13:04:45.517187 15721 consensus_meta.cc:287] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.555s	user 0.008s	sys 0.002s
W20260709 13:04:45.555016  8324 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.005s	sys 0.021s
W20260709 13:04:45.556056  8324 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.005s	sys 0.021s
I20260709 13:04:45.559841  8369 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:45.526175 15690 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:45.507244 15438 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:45.536422 15329 raft_consensus.cc:3055] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:45.561407  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:45.543855 15767 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:45.570313  8400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 3021) took 1752 ms. Trace:
I20260709 13:04:45.568885  8369 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:45.574626  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 126054us
W20260709 13:04:45.575928  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.570991  8400 rpcz_store.cc:276] 0709 13:04:43.818376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.223472 (+405096us) service_pool.cc:224] Handling call
0709 13:04:44.223812 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:44.224058 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:04:44.626863 (+402805us) log.cc:844] Serialized 14 byte log entry
0709 13:04:45.566043 (+939180us) log.cc:844] Serialized 14 byte log entry
0709 13:04:45.569839 (+  3796us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:45.569914 (+    75us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:45.569986 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:45.570077 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:45.570238 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13949,"prepare.queue_time_us":170464,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":43985,"spinlock_wait_cycles":5554944,"thread_start_us":1300844,"threads_started":4,"wal-append.queue_time_us":759218}
W20260709 13:04:45.578433  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:45.602380  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.565621 15329 raft_consensus.cc:740] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:45.583386  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:45.579699  9006 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:04:45.590529  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:45.588479  9002 raft_consensus.cc:1240] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:04:45.596788  9008 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 25. Current term is 26. Ops: []
I20260709 13:04:45.562897 15449 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:45.561152 15813 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:45.573676 15742 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:45.562667 15577 consensus_queue.cc:1059] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:04:45.600900 15756 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:45.617980  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:45.587987 15711 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:45.600155  9000 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 21.
I20260709 13:04:45.619508 15711 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.620019 15711 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:45.616031  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:45.614537  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:45.630231  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:45.631104  9008 raft_consensus.cc:2393] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 27: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:45.631049  8325 raft_consensus.cc:1275] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:04:45.613828 15795 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:04:45.635187 15455 consensus_meta.cc:287] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.531s	user 0.009s	sys 0.002s
I20260709 13:04:45.637741 15329 consensus_queue.cc:260] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:45.638873 15329 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:45.640728  8398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 3015) took 1965 ms. Trace:
I20260709 13:04:45.642340  8402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 1319) took 1944 ms. Trace:
I20260709 13:04:45.642794 15488 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:45.643705 15644 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:45.644124  9009 raft_consensus.cc:2393] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 21: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:45.633255  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:45.644860  8328 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
I20260709 13:04:45.645861  8323 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:04:45.647823  8323 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:04:45.641144  8398 rpcz_store.cc:276] 0709 13:04:43.675192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.748849 (+ 73657us) service_pool.cc:224] Handling call
0709 13:04:43.749250 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:43.749692 (+   442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:43.749733 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:43.750260 (+   527us) log.cc:844] Serialized 26 byte log entry
0709 13:04:44.225799 (+475539us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:44.732781 (+506982us) log.cc:844] Serialized 14 byte log entry
0709 13:04:44.733090 (+   309us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.733187 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:45.638753 (+905566us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b18004226e0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:04:45.640431 (+  1678us) raft_consensus.cc:1685] finished
0709 13:04:45.640524 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:45.640655 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24537,"prepare.queue_time_us":277584,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":3002,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":17184,"spinlock_wait_cycles":28270464,"thread_start_us":940039,"threads_started":2,"wal-append.queue_time_us":756584}
I20260709 13:04:45.649088  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:45.645110 15427 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:45.652041  8329 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:04:45.652526  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3625) took 1585 ms. Trace:
I20260709 13:04:45.642815  8402 rpcz_store.cc:276] 0709 13:04:43.698134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.104851 (+406717us) service_pool.cc:224] Handling call
0709 13:04:44.105915 (+  1064us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:45.188864 (+1082949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:45.642109 (+453245us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54001cbe54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:45.642281 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2445307264}
I20260709 13:04:45.653059  8397 rpcz_store.cc:276] 0709 13:04:44.067083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.350236 (+283153us) service_pool.cc:224] Handling call
0709 13:04:44.350680 (+   444us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:44.689686 (+339006us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:44.689718 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:44.696249 (+  6531us) log.cc:844] Serialized 52 byte log entry
0709 13:04:44.970844 (+274595us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:45.007695 (+ 36851us) log.cc:844] Serialized 14 byte log entry
0709 13:04:45.042273 (+ 34578us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:45.042368 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:45.649622 (+607254us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b1800451f60. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882569f2a6 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1
0709 13:04:45.650230 (+   608us) spinlock_profiling.cc:243] Waited 30.4 ms on lock 0x7b1800451f60. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:04:45.652218 (+  1988us) raft_consensus.cc:1685] finished
0709 13:04:45.652242 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:45.652462 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34286,"prepare.queue_time_us":36455,"prepare.run_wall_time_us":6992,"raft.queue_time_us":303543,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":28205,"spinlock_wait_cycles":132586240,"thread_start_us":569813,"threads_started":2,"wal-append.queue_time_us":388740}
I20260709 13:04:45.655716  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:45.656452 15706 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:45.657090  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:45.659415  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:45.662593  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:45.663856  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:45.666900  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:45.667579  8325 raft_consensus.cc:2393] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 24: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:45.660261 15683 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:45.585229 15775 consensus_queue.cc:1048] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:04:45.673674 15554 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 13:04:45.674129 15809 log.cc:927] Time spent T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:04:45.674757 15756 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:45.675026  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:45.675812  8323 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
I20260709 13:04:45.676333 15772 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:45.677101  8323 raft_consensus.cc:1240] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:45.677480  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:45.681605  8325 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 27: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:45.683135  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:45.684024 15795 raft_consensus.cc:697] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:45.687059  8324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 3442) took 1168 ms. Trace:
I20260709 13:04:45.687542  8324 rpcz_store.cc:276] 0709 13:04:44.518469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.522398 (+  3929us) service_pool.cc:224] Handling call
0709 13:04:44.572594 (+ 50196us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:44.966301 (+393707us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:44.966584 (+   283us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:44.967759 (+  1175us) log.cc:844] Serialized 26 byte log entry
0709 13:04:45.556437 (+588678us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:45.556481 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:45.556548 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:45.686860 (+130312us) raft_consensus.cc:1685] finished
0709 13:04:45.686888 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:45.687007 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22496,"raft.queue_time_us":123675,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":14804,"thread_start_us":886973,"threads_started":2,"wal-append.queue_time_us":680365}
I20260709 13:04:45.660254 15569 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:45.645164 15488 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:45.682181  8323 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:04:45.617515 15721 consensus_queue.cc:1059] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.068s
I20260709 13:04:45.696115  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:45.629086 15711 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:45.696566 15683 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:04:45.699115  8972 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:45.677179 15756 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:45.705101 15706 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:45.699512 15772 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:45.706471 15449 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:45.707175  8369 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:45.708009 15772 consensus_queue.cc:1059] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:45.708276 15449 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:45.708798 15772 raft_consensus.cc:3055] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:45.709219 15772 raft_consensus.cc:740] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:45.698661 15703 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:45.709693 15703 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.710124 15703 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:45.710000 15772 consensus_queue.cc:260] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:45.711226 15772 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:45.714522  8369 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:45.715605 15449 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:45.716004 15449 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.696718 15471 consensus_queue.cc:1048] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 13:04:45.716367 15449 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:45.720582 15812 consensus_queue.cc:1059] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:45.708425 15536 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:04:45.727490  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.685024 15795 consensus_queue.cc:237] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:45.744656  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:45.745553  8331 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:45.752892  9003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56954 (request call id 1285) took 1440 ms. Trace:
I20260709 13:04:45.753607 15797 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.733405 15683 raft_consensus.cc:697] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 26 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:45.754242 15797 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:45.755157 15683 consensus_queue.cc:237] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:45.759661 15797 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:45.735013 15812 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 13:04:45.762465 15812 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:45.763166 15812 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:45.764355  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:45.764972  8397 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:45.766463 15753 raft_consensus.cc:3055] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:45.767002 15753 raft_consensus.cc:740] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:45.767896 15753 consensus_queue.cc:260] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:45.768885 15753 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:45.772250 15545 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:45.775215  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.753362  9003 rpcz_store.cc:276] 0709 13:04:44.312258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.312849 (+   591us) service_pool.cc:224] Handling call
0709 13:04:45.752685 (+1439836us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54006ce3d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:45.752806 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":396665728}
I20260709 13:04:45.754407  8249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 1266) took 2438 ms. Trace:
I20260709 13:04:45.785851  8972 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:45.786304  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.779881  8249 rpcz_store.cc:276] 0709 13:04:43.315802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.180477 (+864675us) service_pool.cc:224] Handling call
0709 13:04:44.180804 (+   327us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:44.205469 (+ 24665us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:44.205504 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:44.212734 (+  7230us) log.cc:844] Serialized 52 byte log entry
0709 13:04:44.666621 (+453887us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:44.687441 (+ 20820us) log.cc:844] Serialized 14 byte log entry
0709 13:04:44.692239 (+  4798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.692449 (+   210us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:45.751298 (+1058849us) spinlock_profiling.cc:243] Waited 5.32 ms on lock 0x7b2400168998. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255579c4 00007f8825555acb 00007f8825710dc9 00007f8825710a55 00007f882570ffb0 00007f8828386488 00007f8828386be8 00007f882835e398 00007f882835976a 00007f8828358bb6
0709 13:04:45.754072 (+  2774us) raft_consensus.cc:1685] finished
0709 13:04:45.754188 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:45.754343 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23782,"prepare.queue_time_us":135460,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":5694,"raft.queue_time_us":20068,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":20079,"spinlock_wait_cycles":12225920,"thread_start_us":451213,"threads_started":1,"wal-append.queue_time_us":715387}
I20260709 13:04:45.765182 15812 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:45.791105 15488 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:45.792186 15711 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:45.733273 15536 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:45.796414  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:45.797015  8330 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:04:45.725970 15449 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:45.800884  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:45.802462  8328 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:04:45.803891  8331 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 25.
I20260709 13:04:45.793088 15706 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:45.809165  8369 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:45.816201 15703 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:45.825539  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:45.826695 15569 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.826778 15812 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:04:45.827395 15569 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:45.827643 15812 raft_consensus.cc:697] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 25 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:45.831784  8251 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 25 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 24. Current term is 25. Ops: []
I20260709 13:04:45.829025 15742 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:45.829919 15711 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:45.852552 15711 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:45.853034 15711 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:45.854082  8330 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:45.859440 15449 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:45.859447 15769 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:45.875744  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:45.876160  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:45.876513  8323 raft_consensus.cc:3060] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:45.876715  8330 raft_consensus.cc:3060] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:04:45.884805  8323 raft_consensus.cc:2468] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
I20260709 13:04:45.887152  8330 raft_consensus.cc:2468] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 23.
I20260709 13:04:45.893131 15675 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.893888 15675 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:45.860217 15536 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:45.828804 15812 consensus_queue.cc:237] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:45.867519  8328 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:04:45.869223 15703 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:45.870932  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:45.957216  9003 raft_consensus.cc:2393] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 21: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:45.871930  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:04:45.947407  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.958774  8972 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:45.963470  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:45.963923  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:45.964252  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:45.965824  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:45.966523  8329 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 28.
I20260709 13:04:45.963581  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:45.967628  8330 raft_consensus.cc:2393] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 21: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:45.963089  8372 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:45.970846 15675 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:45.971861 15536 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:45.968502 15683 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:45.982802 15569 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:45.974006 15703 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:04:45.991465  8295 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:45.968339 15780 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.995677  8295 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:46.001394  8323 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:46.005225  8295 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:46.004325 15722 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:04:46.007194 15797 raft_consensus.cc:2764] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:04:46.004395 15798 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.004601 15753 raft_consensus.cc:2764] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:04:45.981276 15711 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:46.012849  8295 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:46.015376 15722 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:46.015420  8295 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:46.003849 15780 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:46.023346  8295 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:46.028756  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:46.031239  8298 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:46.033915  8298 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:46.023027 15769 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:46.034839 15769 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:46.034992 15722 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:46.035427 15722 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:46.025151 15545 consensus_queue.cc:1048] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:04:46.036451 15722 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:46.013319 15798 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:46.023537 15795 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:46.035740 15753 raft_consensus.cc:2804] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:46.038103 15753 raft_consensus.cc:697] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:46.039526 15798 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:46.040555  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.039089 15753 consensus_queue.cc:237] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:46.042182 15690 raft_consensus.cc:3055] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:04:46.037910 15795 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:46.045287  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:46.035353 15769 raft_consensus.cc:697] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 21 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:46.053994  8369 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:46.066234  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:46.067608  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:46.068634  8329 raft_consensus.cc:2410] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:04:46.068475  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:46.071745  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:46.070262  8369 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:46.071362  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:46.072513  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:46.073275  8326 raft_consensus.cc:2410] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 27 because replica has last-logged OpId of term: 25 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 12.
I20260709 13:04:46.073557  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:46.049628 15690 raft_consensus.cc:740] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 26 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:46.074620  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:46.073611  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:46.077749  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:46.077973  8329 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 24.
I20260709 13:04:46.074610  8323 raft_consensus.cc:2393] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 23: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:46.076550  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:46.076987  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:46.086009  8325 raft_consensus.cc:1275] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:04:46.089066  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:46.060633 15711 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:46.078727 15742 raft_consensus.cc:2804] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:46.091295  8369 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:46.053682 15772 consensus_queue.cc:1059] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:04:46.095439  8974 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:46.096932  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:46.097695  8323 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:46.118139  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.121799 15777 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.124130 15777 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:46.127991  8325 raft_consensus.cc:1240] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:46.128604 15771 raft_consensus.cc:3055] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:46.129031 15771 raft_consensus.cc:740] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:46.129312 15577 raft_consensus.cc:3055] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:46.134316  8325 raft_consensus.cc:1275] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:04:46.094035 15703 raft_consensus.cc:2804] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:04:46.099650 15828 log.cc:927] Time spent T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:04:46.137466  9009 raft_consensus.cc:1275] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:04:46.137894  9005 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:46.139014  9002 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 28 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 13:04:46.098284 15288 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:46.095660 15795 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:46.141445  9001 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:46.102890 15569 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:46.145793  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:46.148082  8330 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 13:04:46.099677 15471 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:46.153892  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.146275 15488 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.134039 15577 raft_consensus.cc:740] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:46.156263  8251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 1279) took 1584 ms. Trace:
I20260709 13:04:46.186383  8251 rpcz_store.cc:276] 0709 13:04:44.571633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.571768 (+   135us) service_pool.cc:224] Handling call
0709 13:04:44.572091 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:45.832230 (+1260139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:46.156046 (+323816us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400177d54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:46.156206 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2896500096}
W20260709 13:04:46.158341  8151 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45448 (request call id 288) took 17377 ms (client timeout 15000 ms). Trace:
I20260709 13:04:46.166456  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 1264) took 2851 ms. Trace:
I20260709 13:04:46.187979  8255 rpcz_store.cc:276] 0709 13:04:43.315252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.177814 (+862562us) service_pool.cc:224] Handling call
0709 13:04:44.178152 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:44.178353 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:45.169805 (+991452us) spinlock_profiling.cc:243] Waited 4.03 ms on lock 0x7b180036baa0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88283664bc 00007f88256b448b 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256ce455 00007f882c1b63c7 00007f88287c89a7 00007f88287d80bf
0709 13:04:45.169939 (+   134us) log.cc:844] Serialized 14 byte log entry
0709 13:04:45.309061 (+139122us) log.cc:844] Serialized 14 byte log entry
0709 13:04:45.652890 (+343829us) spinlock_profiling.cc:243] Waited 7.3 ms on lock 0x7b180036baa0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882569f2a6 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1
0709 13:04:45.653384 (+   494us) spinlock_profiling.cc:243] Waited 8.88 ms on lock 0x7b180036baa0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:04:46.165951 (+512567us) spinlock_profiling.cc:243] Waited 9.1 ms on lock 0x7b180036baa0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882564bc0a 00007f882ed8b219 00007f88256d3a40 00007f88256ce4e4 00007f88256ce455 00007f88287b6060 00007f88287c8716 00007f88287d9287 00007f88287d9208
0709 13:04:46.166099 (+   148us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:46.166150 (+    51us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:46.166195 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:46.166272 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:46.166398 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":840510,"prepare.queue_time_us":468489,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":4385,"raft.queue_time_us":865488,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":7810,"spinlock_wait_cycles":67396608,"thread_start_us":605293,"threads_started":2,"wal-append.queue_time_us":164677}
I20260709 13:04:46.168455 14967 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:46.186115  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:46.172091 15538 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.174322 15800 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.174453 15508 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.174309  8222 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:46.138558 15675 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.099987 15767 consensus_queue.cc:1059] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:04:46.102757 15683 consensus_queue.cc:1059] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.049s
I20260709 13:04:46.179468  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:46.180226  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.182955  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:46.213017  8400 raft_consensus.cc:2410] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 27 because replica has last-logged OpId of term: 25 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 12.
I20260709 13:04:46.132565 15769 consensus_queue.cc:237] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:46.185669  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:46.214846  8404 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:04:46.190826  8406 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:46.191893  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:46.192628  8397 raft_consensus.cc:1275] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:04:46.205173  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:46.209367  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.143002 15742 raft_consensus.cc:697] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 21 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:46.198652 15711 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 13:04:46.219139  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:46.219482 15742 consensus_queue.cc:237] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:46.220175 15711 consensus_queue.cc:1059] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:46.222218  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:46.222857  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:46.223501  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:46.223784  8405 raft_consensus.cc:1275] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:04:46.224896  8256 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:04:46.193246 15488 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:46.194932 15288 raft_consensus.cc:697] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:46.226289  8401 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:04:46.226687 15288 consensus_queue.cc:237] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:46.224929  8257 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 23. Current term is 24. Ops: []
I20260709 13:04:46.205065 15800 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:46.193629 15732 consensus_queue.cc:1048] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:04:46.229998 15732 consensus_queue.cc:1243] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Peer ff7a6fe533e94c1bae555b4cae427728 log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:04:46.230115 15800 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:46.230556  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:46.216584  8399 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:46.217800  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:46.232438  8325 pending_rounds.cc:85] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 10
I20260709 13:04:46.232759  8325 pending_rounds.cc:107] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 13:04:46.232614  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:46.232999  8325 raft_consensus.cc:2889] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:46.233217  8397 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:46.233343 15777 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:46.233752  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:46.234134  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:46.182869 15771 consensus_queue.cc:260] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:46.234757  8400 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:04:46.235244 15771 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:46.235708  8327 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "038cd0877a674c2996410c9f327df911"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:04:46.236248  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:46.236713 15813 consensus_queue.cc:1175] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ff7a6fe533e94c1bae555b4cae427728" responder_term: 20 status { last_received { term: 19 index: 12 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 19. Current term is 20. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:04:46.236321  8327 raft_consensus.cc:606] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 21 LEADER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:04:46.223503  8248 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 13.
W20260709 13:04:46.239753 15835 log.cc:927] Time spent T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.248s	user 0.002s	sys 0.004s
I20260709 13:04:46.222878 15581 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:46.242591  8327 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 12. (term mismatch)
I20260709 13:04:46.223819  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:46.224835  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:46.243496  8250 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 28.
I20260709 13:04:46.244254  8223 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:46.243232  8327 pending_rounds.cc:85] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 11
I20260709 13:04:46.245461  8327 pending_rounds.cc:107] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 13:04:46.245795  8327 raft_consensus.cc:2889] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:46.246260  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:46.248713  9003 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:04:46.197712 15780 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:46.257887  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:46.273170  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:46.258718  8404 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
W20260709 13:04:46.248245 15831 log.cc:927] Time spent T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:04:46.242401  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:46.254105  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:46.160985  8146 catalog_manager.cc:5697] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:46.283695  8402 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:04:46.278836  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.242660 15767 consensus_queue.cc:1243] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer ff7a6fe533e94c1bae555b4cae427728 log is divergent from this leader: its last log entry 20.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:04:46.252228  8323 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:04:46.198894 15538 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:46.287093  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:46.289876  8400 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 21.
I20260709 13:04:46.212066 15675 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:46.294035  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:46.296180  8220 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:04:46.296756  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 122521us
I20260709 13:04:46.297235 15752 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.298523 15488 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:46.299269 15629 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.300228 15627 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:46.301191 15591 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:46.304792  8323 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:04:46.323501  8323 raft_consensus.cc:606] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:04:46.323885  8323 raft_consensus.cc:614] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:46.305724  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:46.197196 15471 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:46.212158 15508 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:46.319794  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:46.326920  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 105567us
I20260709 13:04:46.327335  8397 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 14.
I20260709 13:04:46.278677 15703 raft_consensus.cc:697] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 23 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:46.324419  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:46.303402 15674 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:46.323086 15591 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.337750 15591 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:46.338081  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2884) took 1105 ms. Trace:
I20260709 13:04:46.338444  8253 rpcz_store.cc:276] 0709 13:04:45.232255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.232376 (+   121us) service_pool.cc:224] Handling call
0709 13:04:45.232683 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:45.232885 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:45.249066 (+ 16181us) log.cc:844] Serialized 14 byte log entry
0709 13:04:46.256649 (+1007583us) log.cc:844] Serialized 14 byte log entry
0709 13:04:46.337742 (+ 81093us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:46.337796 (+    54us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:46.337839 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:46.337906 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:46.338026 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":980551,"prepare.queue_time_us":19724,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":836884,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":14374,"thread_start_us":189291,"threads_started":2,"wal-append.queue_time_us":182940}
W20260709 13:04:46.340732  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 44641.9us
W20260709 13:04:46.341560  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.341511  8400 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:04:46.342772 15508 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:46.323036 15577 consensus_queue.cc:260] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:46.331230 15703 consensus_queue.cc:237] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:46.348176  8402 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:04:46.345589  8222 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:46.351858  8220 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:46.354657  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:46.355850  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:46.358186  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:46.187880  8151 rpcz_store.cc:269] 0709 13:04:28.780725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.781195 (+   470us) service_pool.cc:224] Handling call
0709 13:04:46.158154 (+17376959us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:37.174571 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:37.175277 (+   706us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:37.175327 (+    50us) write_op.cc:435] Acquired schema lock
0709 13:04:37.175654 (+   327us) tablet.cc:662] Decoding operations
0709 13:04:37.267251 (+ 91597us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:37.285646 (+ 18395us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:37.285697 (+    51us) tablet.cc:685] Acquiring locks for 23 operations
0709 13:04:37.287638 (+  1941us) tablet.cc:701] Row locks acquired
0709 13:04:37.287695 (+    57us) write_op.cc:260] PREPARE: finished
0709 13:04:37.287877 (+   182us) write_op.cc:270] Start()
0709 13:04:37.288056 (+   179us) write_op.cc:276] Timestamp: P: 1783602277287806 usec, L: 0
0709 13:04:37.288106 (+    50us) op_driver.cc:348] REPLICATION: starting
0709 13:04:37.382906 (+ 94800us) log.cc:844] Serialized 8900 byte log entry
0709 13:04:38.677045 (+1294139us) op_driver.cc:464] REPLICATION: finished
0709 13:04:38.999686 (+322641us) write_op.cc:301] APPLY: starting
0709 13:04:38.999791 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:39.519180 (+519389us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:39.519290 (+   110us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:40.465357 (+946067us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:40.465920 (+   563us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=46,mrs_lookups=0
0709 13:04:40.540167 (+ 74247us) write_op.cc:312] APPLY: finished
0709 13:04:40.540812 (+   645us) log.cc:844] Serialized 204 byte log entry
0709 13:04:41.016718 (+475906us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:41.016920 (+   202us) write_op.cc:454] Released schema lock
0709 13:04:41.017020 (+   100us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:41.020561 (+  3541us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:41.020595 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":210583,"reported_tablets":39,"child_traces":[["op",{"apply.queue_time_us":274294,"apply.run_cpu_time_us":80113,"apply.run_wall_time_us":2020871,"cfile_cache_hit":94,"cfile_cache_hit_bytes":100313,"num_ops":23,"prepare.queue_time_us":322394,"prepare.run_cpu_time_us":46148,"prepare.run_wall_time_us":427938,"raft.queue_time_us":503896,"raft.run_cpu_time_us":45711,"raft.run_wall_time_us":370234,"replication_time_us":1338370,"thread_start_us":1154517,"threads_started":5,"wal-append.queue_time_us":1063354,"wal-append.run_cpu_time_us":67230,"wal-append.run_wall_time_us":2991081}]]}
I20260709 13:04:46.336052 15557 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:46.322889 15629 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:46.327826 15627 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:46.353920 15754 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:46.322448 15752 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:46.385186  8223 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:46.386405  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 201707us
I20260709 13:04:46.343890 15674 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:46.390650  8220 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:46.389003  8222 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:46.394707  8372 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:46.436420  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:46.454473  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:46.473377  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 263357us
I20260709 13:04:46.471899  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:46.388339 15673 raft_consensus.cc:3055] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:04:46.476720  9003 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 22.
W20260709 13:04:46.492205  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 107119us
I20260709 13:04:46.401647 15767 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:46.501410  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 110854us
W20260709 13:04:46.502072  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 113180us
I20260709 13:04:46.502203  8220 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:46.468780 15557 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:46.473348 15753 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.394290 15699 consensus_queue.cc:1059] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.102s
I20260709 13:04:46.505494  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:46.397310 15813 consensus_queue.cc:1048] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.098s
I20260709 13:04:46.399979 15795 consensus_queue.cc:1059] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.103s
I20260709 13:04:46.515945  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:46.516757  8253 raft_consensus.cc:2410] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 13:04:46.517503  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:46.518333  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:46.519469  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:46.520161  8250 raft_consensus.cc:2393] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 23: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:46.394032 15722 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:46.478250 15695 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:46.397578 15536 consensus_queue.cc:1059] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.095s
I20260709 13:04:46.482182 15673 raft_consensus.cc:740] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:46.515671  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:46.545413  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:46.499264 15471 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:46.527271  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:46.550352  8250 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 23. Current term is 24. Ops: []
I20260709 13:04:46.518911  8251 raft_consensus.cc:2376] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e64761f43e4240319c41117cf886d22f in term 22. Re-sending same reply.
I20260709 13:04:46.528208  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:46.553970  8255 raft_consensus.cc:1275] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:04:46.400578 15837 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.114s
I20260709 13:04:46.509979 15553 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:46.544665  8256 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:04:46.548017  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:46.544072  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:46.515110 15556 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:46.563601  8405 raft_consensus.cc:2393] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 22: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:46.526268 15608 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:46.567834  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:46.548166 15538 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:46.572919  8323 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 21 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:46.574380  8257 raft_consensus.cc:1240] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:04:46.577486  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:46.584793  8369 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:46.514282 15753 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:46.572559 15608 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.564352 15556 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:46.594779  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:46.602032  8257 raft_consensus.cc:2393] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 23: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:04:46.595711  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 70256.7us
I20260709 13:04:46.594727  8248 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 28 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 13:04:46.603335  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:46.604784  8248 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 13:04:46.598153  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 84123.8us
I20260709 13:04:46.606503  8222 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:46.598635  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:46.608088  8256 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:46.609138  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:46.599401  8252 raft_consensus.cc:2468] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
I20260709 13:04:46.610097  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:46.610438  8249 raft_consensus.cc:1275] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:04:46.610929  9003 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:46.583463 15553 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:46.618207  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:46.622124  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:46.627300  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:46.628226  8331 raft_consensus.cc:1275] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:04:46.629031  8323 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 13:04:46.629160  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:46.629479  8323 pending_rounds.cc:85] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 11
I20260709 13:04:46.629727  8323 pending_rounds.cc:107] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 13:04:46.629940  8323 raft_consensus.cc:2889] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:46.594971 15488 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.573297 15695 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.047s
I20260709 13:04:46.631631  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:46.600951 15608 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:46.630983  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:46.585268 15536 consensus_queue.cc:1059] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:04:46.606334 15471 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:46.589680 15538 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:04:46.607553 15556 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:46.675464  8324 raft_consensus.cc:2410] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:04:46.678323  8326 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:46.647524  8332 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 24. Current term is 25. Ops: []
I20260709 13:04:46.597559 15795 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:04:46.620321 15675 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:46.619513 15288 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:46.587348 15796 consensus_queue.cc:1048] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.089s
I20260709 13:04:46.703990  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:46.699594  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 61828.4us
I20260709 13:04:46.707935  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:46.619593 15767 consensus_queue.cc:1059] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.086s
I20260709 13:04:46.676903 15840 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:46.721489  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:46.721695  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:46.718191  8397 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 21.
I20260709 13:04:46.720376  8220 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:46.674295 15629 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:46.698114 15508 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.698946 15780 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:46.773298  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.597388 15673 consensus_queue.cc:260] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:46.703150 15471 raft_consensus.cc:697] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 18 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:46.714543 15627 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:46.674778 15488 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:46.761185 15674 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:46.855259  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 248815us
I20260709 13:04:46.847820  8223 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:04:46.848513  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 128218us
I20260709 13:04:46.864267  8397 raft_consensus.cc:3055] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:46.864845  8397 raft_consensus.cc:740] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:46.775286 15767 consensus_queue.cc:1243] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Peer ff7a6fe533e94c1bae555b4cae427728 log is divergent from this leader: its last log entry 21.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:04:46.783034 15772 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:46.865842  8397 consensus_queue.cc:260] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:46.880745  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 97414.8us
I20260709 13:04:46.883997  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:46.760591 15288 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:46.851001 15852 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:04:46.869254 15817 log.cc:927] Time spent T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:04:46.854491 15629 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:46.892555 15864 raft_consensus.cc:3055] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:46.893047 15864 raft_consensus.cc:740] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:46.894145 15847 log.cc:927] Time spent T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.191s	user 0.000s	sys 0.000s
I20260709 13:04:46.892095 15629 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:46.896775 15843 log.cc:927] Time spent T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
W20260709 13:04:46.892733 15851 log.cc:927] Time spent T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 13:04:46.783648 15508 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:46.908949  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:46.913772  8222 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:46.909837  8400 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 24.
I20260709 13:04:46.923945 15471 consensus_queue.cc:237] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:46.925201 15577 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:46.845077 15780 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:46.846339 15855 consensus_queue.cc:1048] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.129s
I20260709 13:04:46.883025 15627 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:46.887003 15674 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:46.925109 15591 consensus_meta.cc:287] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.587s	user 0.009s	sys 0.002s
I20260709 13:04:46.942204  8222 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:46.882226 15673 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:46.890084 15722 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:46.957876  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:04:46.977860  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.866356 15861 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.101s
I20260709 13:04:46.928447 15553 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:47.010947  8400 raft_consensus.cc:2393] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 22: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:04:47.011138  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 69041.3us
I20260709 13:04:47.039193  8222 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:46.888778 15772 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:47.003423  8972 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:04:46.964215 15850 log.cc:927] Time spent T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:04:46.952198 15663 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:46.941473 15752 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:47.050853  8222 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:46.945602 15536 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.095s
I20260709 13:04:47.038079 15553 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.059594  8222 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:47.071527  8222 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:47.043763 15753 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:47.064231  8256 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:04:47.059041 15663 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:47.098869 15856 log.cc:927] Time spent T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:04:47.048498 15864 consensus_queue.cc:260] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:47.110786 15488 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:47.067495 15795 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.010449 15591 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:47.113165 15288 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:47.002141 15841 consensus_queue.cc:1059] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.071s
I20260709 13:04:47.062013 15538 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:47.145725 15538 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:47.145862 15841 raft_consensus.cc:3055] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:04:47.146174 15538 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:47.146350 15841 raft_consensus.cc:740] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:47.147397 15841 consensus_queue.cc:260] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:47.148746 15841 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:47.121331 15868 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:04:47.150079 15868 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:47.150451 15868 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:47.150926 15868 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:47.122300 15855 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:47.125620  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 54181.5us
I20260709 13:04:47.052807 15863 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:04:46.978256 15627 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:47.155700 15627 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:46.951939 15852 consensus_queue.cc:1059] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.105s
I20260709 13:04:47.068213 15752 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:47.156472 15752 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:47.156703 15581 consensus_meta.cc:287] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.915s	user 0.010s	sys 0.000s
I20260709 13:04:47.156929 15752 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:47.090341 15865 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.092s
I20260709 13:04:47.157222 15538 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:47.158887  8328 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 27.
I20260709 13:04:47.159893 15538 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:46.998525 15674 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:47.141423 15795 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:47.162961 15795 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:47.075922 15553 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:47.061417 15871 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:47.163563 15871 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.163936 15871 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:47.166411 15674 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:47.175081 15771 consensus_meta.cc:287] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.939s	user 0.010s	sys 0.002s
I20260709 13:04:47.176035  8403 raft_consensus.cc:2410] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:04:47.179697 14967 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:47.180197 15508 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:47.055799  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 52512.8us
I20260709 13:04:47.139111 15775 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.134279 15867 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:47.197559 15855 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:47.198017 15775 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:47.199635 15846 log.cc:927] Time spent T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.477s	user 0.002s	sys 0.000s
W20260709 13:04:47.182792  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.201576  8372 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:47.202021 15853 log.cc:927] Time spent T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.436s	user 0.002s	sys 0.000s
I20260709 13:04:47.203639 15771 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.204156 15771 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:47.205298 15777 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:47.205718 15777 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.206075 15777 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:47.206271 15771 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:47.143057  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.180258 14967 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:47.194828  8402 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 21.
I20260709 13:04:47.210088  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:47.210549  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:47.195767 15488 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:04:47.211618  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:47.212599  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:47.211170  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:47.213266  8398 raft_consensus.cc:2393] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 20: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:47.198107 15855 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:47.213694  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:47.200418 15775 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:47.144809 15591 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:47.143745 15864 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:47.135419 15663 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:47.150090 15780 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:47.197265 15867 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:47.217456  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:47.218952 15795 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.215413 14967 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:47.219599 15771 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.269419 15795 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:47.269673 15771 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:47.270888 14967 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:47.271263 15553 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:47.271811 15795 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:47.271891 15771 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:47.220922  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.273449 15772 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:47.273989  8369 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:47.274046 15752 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:47.249228  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:47.251675  9002 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:04:47.256968  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:47.277117  9000 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
I20260709 13:04:47.257133  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:47.278266  8398 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
W20260709 13:04:47.286864 15556 consensus_meta.cc:287] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.571s	user 0.007s	sys 0.002s
I20260709 13:04:47.287451 15556 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:47.300683  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:47.306613  8222 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:47.307518  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:47.302753  8372 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:47.309602  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:47.262809 15690 consensus_queue.cc:260] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:47.264783  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:47.267356  8401 pending_rounds.cc:85] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 12
I20260709 13:04:47.232175  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:47.311038  8401 pending_rounds.cc:107] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 13:04:47.311285  9004 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:47.311445  8401 raft_consensus.cc:2889] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:47.276562 15752 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:47.258029  8402 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 25 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 24. Current term is 25. Ops: []
I20260709 13:04:47.323653  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:47.325289  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:47.326002  8402 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:47.319610 15711 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:47.330243 15690 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:47.350772  8402 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18.
I20260709 13:04:47.332221 15775 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:04:47.351804  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.331648 15795 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:47.345811 15608 consensus_meta.cc:287] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.645s	user 0.008s	sys 0.003s
I20260709 13:04:47.339083 15752 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:47.329108 15873 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:47.329912 15767 consensus_queue.cc:1059] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.024s
W20260709 13:04:47.377393 15647 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.004s	sys 0.008s
I20260709 13:04:47.346726 15855 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:47.345817 15556 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:47.352841 15775 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.343981 15570 consensus_queue.cc:1059] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.077s
W20260709 13:04:47.425150  8972 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.255.2:38599 timed out after 1.721s (SENT) [suppressed 16 similar messages]
I20260709 13:04:47.413201 15752 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:47.427306 15775 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:47.416968 15608 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:47.430994 15867 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:47.433537 15752 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:47.435171 15570 consensus_queue.cc:1243] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 25.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 13:04:47.425719 15647 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.005s	sys 0.008s
W20260709 13:04:47.439812 15625 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.077s	user 0.004s	sys 0.023s
W20260709 13:04:47.434002 15754 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.031s	user 0.005s	sys 0.019s
W20260709 13:04:47.458680 15625 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.100s	user 0.006s	sys 0.025s
W20260709 13:04:47.461982 15754 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.062s	user 0.005s	sys 0.019s
I20260709 13:04:47.465821 15754 raft_consensus.cc:697] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:47.467379 15754 consensus_queue.cc:237] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:47.435947 15868 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:47.444221 15871 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:47.555651  8252 raft_consensus.cc:3055] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:04:47.556905  8252 raft_consensus.cc:740] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:47.557834  8220 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.000s	sys 0.012s
W20260709 13:04:47.558534  8220 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.000s	sys 0.012s
W20260709 13:04:47.559091  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 700276us
I20260709 13:04:47.559334  8252 consensus_queue.cc:260] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:47.568307  8253 raft_consensus.cc:2368] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 24. Current term is 25.
I20260709 13:04:47.560886  8220 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:47.570914  8220 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:47.578763  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:47.583521  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:47.584610  8400 raft_consensus.cc:2393] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 14: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:04:47.579392  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 269861us
I20260709 13:04:47.578784  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:47.590680  8369 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:47.593503  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:47.594844  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 169577us
W20260709 13:04:47.597379  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.596104  8146 catalog_manager.cc:5697] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f reported cstate change: config changed from index 11 to 6, term changed from 22 to 18, leader changed from <none> to f865c4d761f5446eb12b3a029234772e (127.6.255.4), VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) added. New cstate: current_term: 18 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } attrs { replace: true } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:47.614038 15553 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:47.615417 15553 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:04:47.616391 15553 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:47.620576 15608 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:47.616614 15471 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:47.623787 15471 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:47.624779 15644 raft_consensus.cc:3055] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:47.632066 15644 raft_consensus.cc:740] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:47.646379  8250 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 20. Current term is 21. Ops: [20.11-20.11]
W20260709 13:04:47.673712  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.664997 15644 consensus_queue.cc:260] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:47.676668 15644 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:47.676766 15871 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:47.676910 15868 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:47.695420  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55264 (request call id 2789) took 1179 ms. Trace:
I20260709 13:04:47.696044  8253 rpcz_store.cc:276] 0709 13:04:46.515601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.517245 (+  1644us) service_pool.cc:224] Handling call
0709 13:04:47.695074 (+1177829us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54002ee354. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:47.695276 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2413920128}
I20260709 13:04:47.699441  8254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 1284) took 1480 ms. Trace:
I20260709 13:04:47.681267 15663 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:47.708452 15663 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:47.711068  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.714190  8972 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:47.718552 15841 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:04:47.719285 15841 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 28 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:47.720340 15841 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 28 FOLLOWER]: Advancing to term 29
W20260709 13:04:47.723856  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 113070us
I20260709 13:04:47.720265  8403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 3640) took 1524 ms. Trace:
I20260709 13:04:47.699895  8254 rpcz_store.cc:276] 0709 13:04:46.219233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.224019 (+  4786us) service_pool.cc:224] Handling call
0709 13:04:46.224310 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:46.349845 (+125535us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:46.349880 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:46.362266 (+ 12386us) log.cc:844] Serialized 26 byte log entry
0709 13:04:46.510352 (+148086us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:46.510397 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.698643 (+1188246us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b540009ecd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:47.698718 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:47.698795 (+    77us) raft_consensus.cc:1685] finished
0709 13:04:47.698809 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:47.699201 (+   392us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":32507,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":3390,"spinlock_wait_cycles":273815936,"thread_start_us":99393,"threads_started":1,"wal-append.queue_time_us":215326}
I20260709 13:04:47.725353  8403 rpcz_store.cc:276] 0709 13:04:46.195734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.195849 (+   115us) service_pool.cc:224] Handling call
0709 13:04:47.719794 (+1523945us) spinlock_profiling.cc:243] Waited 959 ms on lock 0x7b54000fc8d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:47.720052 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2205342976}
W20260709 13:04:47.740278  8406 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 3644) took 1518 ms (client timeout 1586 ms). Trace:
W20260709 13:04:47.740916  8406 rpcz_store.cc:269] 0709 13:04:46.221578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.245134 (+ 23556us) service_pool.cc:224] Handling call
0709 13:04:47.739860 (+1494726us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54000221d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:47.740054 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3113462016}
I20260709 13:04:47.755722 15795 raft_consensus.cc:3055] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:04:47.756270 15795 raft_consensus.cc:740] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 23 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:04:47.756632  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.757629 15561 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:47.758661 15561 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:47.750267  8972 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:47.771024  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:04:47.776757  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:47.781270 15800 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.264s	user 0.006s	sys 0.002s
W20260709 13:04:47.781778 15800 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.270s	user 0.007s	sys 0.002s
I20260709 13:04:47.784704  8402 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:47.780516  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:47.790762 15776 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:47.793206  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 69355us
I20260709 13:04:47.759928 15795 consensus_queue.cc:260] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:47.804615 15776 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.805332 15776 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:47.807695 15776 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:47.810308  8400 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 26. Current term is 27. Ops: []
I20260709 13:04:47.808398 15840 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:47.798406 15561 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:47.813482  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 81654us
I20260709 13:04:47.808743 15795 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:47.822748  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 3466) took 1194 ms. Trace:
W20260709 13:04:47.823717 15862 log.cc:927] Time spent T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.941s	user 0.002s	sys 0.002s
W20260709 13:04:47.825073  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.829161 15581 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:47.787298 15752 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:47.839654 15629 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:47.851943  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:47.852677  9004 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:04:47.852818  9008 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
W20260709 13:04:47.853807 15892 log.cc:927] Time spent T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:04:47.855365  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:47.856658  8974 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:47.857414  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39306 (request call id 3038) took 1153 ms. Trace:
I20260709 13:04:47.857856  8399 rpcz_store.cc:276] 0709 13:04:46.703473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.703737 (+   264us) service_pool.cc:224] Handling call
0709 13:04:47.857013 (+1153276us) spinlock_profiling.cc:243] Waited 701 ms on lock 0x7b54002ef9d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:47.857212 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1612027776}
I20260709 13:04:47.858379 15871 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:47.858749 15706 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:47.859252 15706 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:47.859764 15706 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:47.861879  8222 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:47.864774  8397 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:47.867457  9003 raft_consensus.cc:1240] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:04:47.877139  8402 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
I20260709 13:04:47.884014 15839 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:47.886693  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:47.886763  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:47.889142 15449 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.015s
I20260709 13:04:47.892794  8222 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:47.913569 15848 log.cc:927] Time spent T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 1.320s	user 0.000s	sys 0.002s
I20260709 13:04:47.914552 15629 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:47.869901 15553 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:47.916519 15553 raft_consensus.cc:697] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 18 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:47.918174  8328 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 3452) took 1850 ms (client timeout 2038 ms). Trace:
W20260709 13:04:47.918872  8328 rpcz_store.cc:269] 0709 13:04:46.067085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.070837 (+  3752us) service_pool.cc:224] Handling call
0709 13:04:47.917565 (+1846728us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540008ff54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:47.917869 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2458545024}
I20260709 13:04:47.922158  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:47.922925 15845 log.cc:927] Time spent T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 1.349s	user 0.000s	sys 0.002s
I20260709 13:04:47.923178  8330 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 25.
I20260709 13:04:47.925019  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 3114) took 1693 ms. Trace:
I20260709 13:04:47.925395  8325 rpcz_store.cc:276] 0709 13:04:46.231692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.231970 (+   278us) service_pool.cc:224] Handling call
0709 13:04:46.232252 (+   282us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:46.233711 (+  1459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:46.233739 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:46.234246 (+   507us) log.cc:844] Serialized 52 byte log entry
0709 13:04:46.397986 (+163740us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:46.398028 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:46.398091 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:47.921902 (+1523811us) spinlock_profiling.cc:243] Waited 46.6 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:04:47.924509 (+  2607us) raft_consensus.cc:1685] finished
0709 13:04:47.924533 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:47.924817 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1107,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":81,"spinlock_wait_cycles":107092864,"thread_start_us":153563,"threads_started":1,"wal-append.queue_time_us":276723}
I20260709 13:04:47.929585  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:47.932179  8250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 2792) took 1404 ms. Trace:
W20260709 13:04:47.933666 15777 consensus_meta.cc:287] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.727s	user 0.007s	sys 0.015s
I20260709 13:04:47.934329 15777 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:47.893451 15840 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:47.933679  8402 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Advancing to term 25
W20260709 13:04:47.937975 15842 log.cc:927] Time spent T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 1.434s	user 0.004s	sys 0.000s
W20260709 13:04:47.938265  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.942855 15536 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:47.943810 15536 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:47.894557 15581 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:47.946408  8252 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:47.947666 15581 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:47.897759 15813 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:47.948653 15813 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:47.951370 15813 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:47.898483 15812 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:47.932816  8250 rpcz_store.cc:276] 0709 13:04:46.527546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.551361 (+ 23815us) service_pool.cc:224] Handling call
0709 13:04:46.551760 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:47.647058 (+1095298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.931677 (+284619us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540034bcd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:47.932088 (+   411us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2515594624}
I20260709 13:04:47.893221 15795 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.903200  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:47.954447  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:47.955004  8399 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:47.904179  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:47.957504  8401 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:47.959050  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:47.961575 15795 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:47.912184  9004 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 22.
I20260709 13:04:47.952431 15812 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:47.964119 15753 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:47.965757 15722 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.901815  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:47.962507 15538 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.966775 15897 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:47.966944  8403 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 16.
I20260709 13:04:47.967135 15538 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:47.967581 15897 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:47.911882 15844 log.cc:927] Time spent T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 1.326s	user 0.003s	sys 0.000s
I20260709 13:04:47.823444  8329 rpcz_store.cc:276] 0709 13:04:46.628087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.632108 (+  4021us) service_pool.cc:224] Handling call
0709 13:04:46.632385 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:46.632582 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:46.694455 (+ 61873us) log.cc:844] Serialized 14 byte log entry
0709 13:04:47.811046 (+1116591us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b180016ae00. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882564bc0a 00007f882ed8b219 00007f88256d3a40 00007f88256ce4e4 00007f88256ce455 00007f88287b6060 00007f88287c8716 00007f88287d9287 00007f88287d9208
0709 13:04:47.821854 (+ 10808us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:47.821934 (+    80us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:47.822070 (+   136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.822231 (+   161us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:47.822475 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3909,"prepare.queue_time_us":64930,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":1132682,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":285,"spinlock_wait_cycles":6114560,"wal-append.queue_time_us":15261}
I20260709 13:04:47.972007 15545 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.972721 15545 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:47.973131 15795 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:47.974852  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:47.976029 15812 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:47.975286  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:47.978478 15538 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:47.979187  8402 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 25.
W20260709 13:04:47.982103 15893 log.cc:927] Time spent T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 13:04:47.969631 15722 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:47.985030  8399 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:47.951412 15536 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:47.987085 15722 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:47.994561 15840 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:47.995743  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:47.997256  8252 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:04:47.998373 15840 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:47.999109 15840 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:48.001420 15840 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:48.002847 15752 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:47.964766 15753 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:47.873977  8146 catalog_manager.cc:5697] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 7 to 14, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 14 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:04:47.988725 15891 log.cc:927] Time spent T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.263s	user 0.000s	sys 0.001s
I20260709 13:04:48.009656  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 2804) took 1402 ms. Trace:
I20260709 13:04:48.013924  8252 rpcz_store.cc:276] 0709 13:04:46.607615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.611165 (+  3550us) service_pool.cc:224] Handling call
0709 13:04:46.611372 (+   207us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.935464 (+1324092us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b5800a355d0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f882871a0aa 00007f882879c447 00007f88287b4fe7 00007f88287b67de 00007f88287c0b49 00007f88287c1416 00007f88287bdd8f 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4
0709 13:04:47.997624 (+ 62160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.009234 (+ 11610us) spinlock_profiling.cc:243] Waited 943 ms on lock 0x7b540026be54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:48.009578 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3012910208}
W20260709 13:04:48.014740 15775 consensus_meta.cc:287] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.571s	user 0.008s	sys 0.008s
I20260709 13:04:48.015436 15775 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:48.017025  8406 raft_consensus.cc:2393] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 21: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:04:48.021944  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:47.988090  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 141075us
I20260709 13:04:48.027010  8369 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:47.977301  8325 raft_consensus.cc:1240] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 26. Current term is 27. Ops: []
I20260709 13:04:48.029000 15840 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:48.036755 15675 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.037572 15675 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:48.040518 15675 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:48.045258 15871 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.046015 15871 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:48.049149 15871 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:47.875286 15776 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:48.050361 15776 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:47.990301 15875 log.cc:927] Time spent T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.539s	user 0.005s	sys 0.000s
I20260709 13:04:48.002753  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:47.991593 15553 consensus_queue.cc:237] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:48.073861  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.053170 15776 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:48.068436  8222 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:48.076802  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:48.078883  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:48.072844 15581 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.081530  8405 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 13:04:48.081955 15536 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:48.078838  8328 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:37920
I20260709 13:04:48.085268  8399 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:48.085847  8328 raft_consensus.cc:606] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:04:48.084410 15536 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:48.087092  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:48.087436 15536 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:48.088150  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:48.089289  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:48.090395  8325 raft_consensus.cc:2410] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:04:48.083529  8405 pending_rounds.cc:85] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 10
I20260709 13:04:48.078770  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:48.079824  8222 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:48.091395  8400 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:48.095590  8399 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:48.076823 15752 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:48.096524  8329 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 24.
I20260709 13:04:48.096776  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 3034) took 1512 ms. Trace:
I20260709 13:04:48.097591  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:48.090817  8405 pending_rounds.cc:107] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 13:04:48.100158  8405 raft_consensus.cc:2889] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:48.102531  8406 raft_consensus.cc:1217] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 25 LEADER]: Deduplicated request from leader. Original: 22.12->[24.13-24.13]   Dedup: 24.13->[]
I20260709 13:04:48.110522  8406 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 25 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 24. Current term is 25. Ops: [24.13-24.13]
I20260709 13:04:48.104333 15753 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:48.113878  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:48.096144 15545 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:48.114719  9003 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:04:48.098673  8404 rpcz_store.cc:276] 0709 13:04:46.583980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.584145 (+   165us) service_pool.cc:224] Handling call
0709 13:04:46.601026 (+ 16881us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:46.638678 (+ 37652us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:46.638728 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:46.639321 (+   593us) log.cc:844] Serialized 26 byte log entry
0709 13:04:46.703023 (+ 63702us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:46.708827 (+  5804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:46.709006 (+   179us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:47.822887 (+1113881us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b4400022b4c. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255f7c85 00007f88255f86a2 00007f882836a1db 00007f8828365aaf 00007f882835986e 00007f8828358bb6 00007f88283587d8 00007f8828362221 00007f88283621d9 00007f8828362169
0709 13:04:47.928999 (+106112us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b240020c378. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f88255579c4 00007f8825555acb 00007f8825710dc9 00007f8825710a55 00007f882570ffb0 00007f8828386488 00007f8828386be8 00007f882835e398 00007f882835976a 00007f8828358bb6
0709 13:04:48.095405 (+166406us) raft_consensus.cc:1685] finished
0709 13:04:48.095425 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:48.096567 (+  1142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37134,"raft.queue_time_us":37817,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":47054,"spinlock_wait_cycles":32856192,"thread_start_us":27025,"threads_started":1,"wal-append.queue_time_us":143358}
I20260709 13:04:48.126076 15629 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:04:48.123440 15690 consensus_meta.cc:287] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.687s	user 0.007s	sys 0.008s
W20260709 13:04:48.129631  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 273116us
I20260709 13:04:48.129695  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:48.116757 15897 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:48.135394 15870 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.112206 15813 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.138993  8330 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:04:48.108902 15752 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:48.139556  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:48.133992  8397 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 12. (term mismatch)
I20260709 13:04:48.141494  8397 pending_rounds.cc:85] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 11
I20260709 13:04:48.142226  8329 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 27.
I20260709 13:04:48.143668  8397 pending_rounds.cc:107] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 13:04:48.143985  8397 raft_consensus.cc:2889] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:48.141213  8324 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 25.
W20260709 13:04:48.151489  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.152376  8252 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9"
dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:51220
I20260709 13:04:48.154206  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:48.158353  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.146924  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:48.165272  9003 raft_consensus.cc:2368] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 14. Current term is 15.
I20260709 13:04:48.130177 15581 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:48.169395  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 3040) took 1323 ms. Trace:
I20260709 13:04:48.141729 15449 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:48.170205  8397 rpcz_store.cc:276] 0709 13:04:46.845673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.846271 (+   598us) service_pool.cc:224] Handling call
0709 13:04:46.864104 (+ 17833us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.863878 (+999774us) spinlock_profiling.cc:243] Waited 857 ms on lock 0x7b58007a4bd0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f882871a0aa 00007f882879c447 00007f88287b4fe7 00007f88287b67de 00007f88287c0b49 00007f88287c1416 00007f88287bdd8f 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4
0709 13:04:48.164462 (+300584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.169339 (+  4877us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1970824064}
I20260709 13:04:48.173273  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:48.162683  8297 connection.cc:256] RPC call timeout handler was delayed by -1.082471747999989s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1928s for a response
I20260709 13:04:48.165550 15840 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:48.177649  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 97901.4us
I20260709 13:04:48.181514  8222 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:48.168644 15841 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:48.172685  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:48.189358  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:48.192750  9001 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 29 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 28. Current term is 29. Ops: []
I20260709 13:04:48.194967  9000 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:48.199414 15841 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 29 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:48.194859  8222 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:48.173087 15777 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:48.184474  8252 raft_consensus.cc:606] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:04:48.206422  8252 raft_consensus.cc:614] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:48.166636 15813 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:48.207394  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:48.206732  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:48.191694  8329 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:04:48.216938  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:48.217576  9001 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:48.190179 15840 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:48.222754 15752 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:48.223672  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 142658us
I20260709 13:04:48.142606 15870 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:48.227140 15870 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:48.228746  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.229744 15775 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:48.230657  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:48.231186  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:48.231933  8397 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 26.
I20260709 13:04:48.232117  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:48.232609  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:48.232883  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:48.233075 15897 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.233516  8399 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 21.
I20260709 13:04:48.233754 15897 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:48.236159 15897 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:48.236421 15880 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:48.236828 15880 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:48.237223 15880 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:48.237960  9001 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 23.
I20260709 13:04:48.239974  9008 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 27.
W20260709 13:04:48.241921 15732 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 13:04:48.245543  9003 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:48.246435 15813 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:48.255585 15840 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:48.259194 15812 consensus_queue.cc:1059] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.17, Next index: 18, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:48.260211 15812 raft_consensus.cc:3055] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 28 LEADER]: Stepping down as leader of term 28
I20260709 13:04:48.260480  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:48.260608 15812 raft_consensus.cc:740] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 28 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:48.270955  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:48.272585  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 84380.2us
I20260709 13:04:48.278074  8406 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:48.281287 15753 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.281790  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:48.281864  8372 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:48.282819  9002 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:48.283495 15840 raft_consensus.cc:2804] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:48.284024  8329 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:04:48.284765 15840 raft_consensus.cc:697] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 23 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:48.286357  8252 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:04:48.287189 15538 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.288002  8250 raft_consensus.cc:1240] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:04:48.287889 15538 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:48.289180  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:48.288529 15840 consensus_queue.cc:237] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:48.290089  8254 raft_consensus.cc:2410] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 23 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:04:48.290107  8329 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b23a66e34de947348d5940084701c646"
dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
mode: GRACEFUL
new_leader_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:37920
W20260709 13:04:48.290325  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.290840  8329 raft_consensus.cc:606] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Received request to transfer leadership to TS e64761f43e4240319c41117cf886d22f
I20260709 13:04:48.291347  8329 raft_consensus.cc:614] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:48.291971  8372 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:48.298362  9002 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
I20260709 13:04:48.274824 15752 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:04:48.330273  8372 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:48.292339 15870 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:04:48.329164  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:48.329908  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:48.297066 15897 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:04:48.341933  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.343909  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:48.338804  8222 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:48.207707 15449 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:48.264942 15812 consensus_queue.cc:260] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:48.341586  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:48.281962 15753 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:48.343897  8254 raft_consensus.cc:1275] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:04:48.353178  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:48.353933  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:48.354687  8326 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:04:48.355739  8323 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:48.350610  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:48.328548 15813 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:04:48.343230  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:48.367225  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:48.336555 15870 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.359642  9000 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:48.329573 15545 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:48.365006  9005 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
I20260709 13:04:48.365443 15538 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:48.365981  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:48.366088  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:48.373348  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:48.377683  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:48.370019  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:48.378140  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:48.378268 15898 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:48.378827  8323 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:48.379130 15898 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:48.379468  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:48.378935  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:48.381031 15812 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:04:48.381134  8327 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:04:48.382246  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:48.383294  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:48.385797  8327 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:04:48.387377  8323 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 25.
W20260709 13:04:48.389611  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 244893us
I20260709 13:04:48.391626 15897 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:48.394294 15897 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:48.396129  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.396025 15545 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:48.398005  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 66364.4us
I20260709 13:04:48.398175  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:48.398763  8972 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:48.401170  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:48.401996 15753 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:48.370160 15813 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.402937 15813 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:04:48.362036 15775 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:48.406137 15775 raft_consensus.cc:697] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 21 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:48.406906 15449 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:48.407450 15775 consensus_queue.cc:237] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:48.407895  8372 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:48.409089 15732 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:48.372556 15870 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:04:48.409475 15732 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.409914 15732 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:48.369189 15795 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:48.415402 15795 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:48.378481  8326 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:48.347218 15752 raft_consensus.cc:697] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 25 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:48.371404 15706 consensus_meta.cc:287] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.511s	user 0.013s	sys 0.000s
I20260709 13:04:48.422755  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:48.348482 15675 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.076s
I20260709 13:04:48.428499  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:48.399022 15545 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:48.431718  9003 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:48.434417  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:48.381274 15898 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:48.425686 15841 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:48.436213  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:48.425720 15871 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:48.397224 15536 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:48.444083  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.432602 15897 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:48.450989  8222 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:48.455276  8254 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 26.
I20260709 13:04:48.461061  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:48.464840  8328 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:48.461807  8327 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 26.
I20260709 13:04:48.429415 15706 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:48.447199 15841 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.477322  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:48.479025  8369 connection.cc:256] RPC call timeout handler was delayed by -1.059526945000016s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17270000000000002s for a response
I20260709 13:04:48.469009  8252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:48.465445  8222 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:48.449479 15871 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.478358  8254 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:48.466758 15629 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:48.492089  9004 raft_consensus.cc:2393] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 22: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:48.487098 15629 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:48.494683  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:48.464454 15538 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.495699 15538 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:48.496367 15799 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:48.497895 15538 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:48.500471 15798 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:48.501529 15769 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:48.502339 15769 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:48.504839 15752 consensus_queue.cc:237] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:48.509509 15581 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:48.509747  8252 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:04:48.485852 15897 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:48.512841 15897 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:48.512519 15732 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:48.516422  8295 connection.cc:256] RPC call timeout handler was delayed by -1.3948000289999527s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1878s for a response
I20260709 13:04:48.518216 15795 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:48.496027  8255 raft_consensus.cc:1240] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:48.494848 15629 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:48.517122  8297 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:48.523155  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.482115 15841 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:48.476132 15901 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:48.486799 15871 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:48.527998  8326 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:48.504891 15769 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:48.538491  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:48.540427  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:48.536068 15553 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:48.523612 15690 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.545820  8295 leader_election.cc:400] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 22. Current term is 23.
I20260709 13:04:48.546229  8326 raft_consensus.cc:2410] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 27 because replica has last-logged OpId of term: 25 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 12.
I20260709 13:04:48.546350  8295 leader_election.cc:403] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:48.501075 15798 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:48.530699 15799 raft_consensus.cc:697] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:48.549403  8328 raft_consensus.cc:2410] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:04:48.545495  8254 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:48.552943  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:48.554106  9003 raft_consensus.cc:1275] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:04:48.566931  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:48.573318  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:48.574324  8323 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 26.
I20260709 13:04:48.550710 15553 raft_consensus.cc:697] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:48.548650 15777 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:48.551491 15581 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.600744  8326 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 22.
I20260709 13:04:48.602516  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:48.555997 15880 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:48.600503 15795 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.005s
I20260709 13:04:48.605930  8223 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:48.606154 15732 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:48.607432 15880 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:48.608162 15880 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.608668 15880 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:48.576568 15897 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 22. Current term is 23.
W20260709 13:04:48.620695  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 103274us
I20260709 13:04:48.622517  8295 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:48.623798  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:48.550899 15877 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:04:48.626554  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:48.627436  8323 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 27.
I20260709 13:04:48.610148  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:48.604883 15581 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:48.628566  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:48.629181  9003 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:48.629359  8326 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 27.
I20260709 13:04:48.614037 15629 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:48.609092  8297 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:48.549364 15538 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.048s
I20260709 13:04:48.609565 15545 raft_consensus.cc:993] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728: : Instructing follower e64761f43e4240319c41117cf886d22f to start an election
I20260709 13:04:48.557827 15690 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:48.639683  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:48.642911  8323 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:04:48.646919  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:48.632138 15897 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:48.647816  8326 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
I20260709 13:04:48.657022  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:48.661064  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:48.660998  8369 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:48.664479 15732 consensus_queue.cc:1048] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:48.633345 15777 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:48.667070  8297 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:48.667827  8323 raft_consensus.cc:3060] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:48.676980  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 54592.8us
I20260709 13:04:48.610810 15553 consensus_queue.cc:237] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:48.670243 15753 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:48.670104 15897 raft_consensus.cc:1081] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Signalling peer e64761f43e4240319c41117cf886d22f to start an election
I20260709 13:04:48.669571 15536 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:48.696322  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 260079us
I20260709 13:04:48.680958 15706 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:48.710788  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.723062  9003 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
W20260709 13:04:48.745504  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:48.733541  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 181982us
I20260709 13:04:48.761406  8372 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:48.713727 15795 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:48.767261 15840 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:48.771142  8326 raft_consensus.cc:1275] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:04:48.771613  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:48.712484 15812 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:04:48.770148 15908 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:04:48.782233  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:48.782951  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.679517 15799 consensus_queue.cc:237] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:48.783552  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.795589 15629 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:48.797848  8328 raft_consensus.cc:1217] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Deduplicated request from leader. Original: 15.10->[18.11-19.12]   Dedup: 18.11->[19.12-19.12]
I20260709 13:04:48.798612  8328 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 19. Current term is 20. Ops: [18.11-19.12]
I20260709 13:04:48.802016  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:48.802939  8401 raft_consensus.cc:2410] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 27 because replica has last-logged OpId of term: 25 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 12.
I20260709 13:04:48.802793  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:48.803391  8400 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:48.804121  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:48.804900  8401 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:48.810794  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 99543.3us
I20260709 13:04:48.821393  8323 raft_consensus.cc:2468] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 23.
I20260709 13:04:48.773706 15538 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.828934  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:48.778846 15840 raft_consensus.cc:697] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 21 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:48.782642 15861 raft_consensus.cc:2804] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:48.842654  8295 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:48.788523 15752 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.824158 15798 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:48.828469 15899 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:48.880126  8323 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
W20260709 13:04:48.885388  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 188325us
I20260709 13:04:48.848089 15861 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:48.852309 15690 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:48.884681  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 320066us
I20260709 13:04:48.829286 15812 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:48.889451  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 233241us
W20260709 13:04:48.897051 15904 log.cc:927] Time spent T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.244s	user 0.001s	sys 0.001s
I20260709 13:04:48.882630 15899 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:04:48.915491  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 189693us
W20260709 13:04:48.899920  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 126592us
I20260709 13:04:48.907713  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 3468) took 2046 ms. Trace:
I20260709 13:04:48.888900 15538 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:48.888914 15861 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:48.885093 15877 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.853456 15752 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:48.919998 15899 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:48.895413 15900 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:48.928442 15900 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:48.930160 15910 log.cc:927] Time spent T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:04:48.935364 15861 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:48.943801  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.947109  8400 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:04:48.938532  8369 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:48.951296  8974 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:48.920297 15777 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:48.954893  8401 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:48.918665  8332 rpcz_store.cc:276] 0709 13:04:46.719477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.719659 (+   182us) service_pool.cc:224] Handling call
0709 13:04:46.725388 (+  5729us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:46.725756 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:46.952173 (+226417us) log.cc:844] Serialized 14 byte log entry
0709 13:04:48.433077 (+1480904us) log.cc:844] Serialized 14 byte log entry
0709 13:04:48.709417 (+276340us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:48.713275 (+  3858us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:48.727467 (+ 14192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.730797 (+  3330us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:48.765736 (+ 34939us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":860061,"prepare.queue_time_us":450305,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":6696,"raft.queue_time_us":845981,"raft.run_cpu_time_us":321,"raft.run_wall_time_us":36596,"thread_start_us":1035513,"threads_started":5,"wal-append.queue_time_us":497067,"wal-append.run_cpu_time_us":1501,"wal-append.run_wall_time_us":1005997}
W20260709 13:04:48.962002  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 73350.3us
W20260709 13:04:48.962136  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 44763.3us
W20260709 13:04:48.961172 15870 consensus_meta.cc:287] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.551s	user 0.012s	sys 0.000s
I20260709 13:04:48.963721 15877 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:48.937997 15927 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:04:48.971037 15581 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:48.975411 15841 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:49.002351 15901 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:49.002836 15901 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:49.003737 15841 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:49.005983 15753 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:49.006510 15753 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:48.978233  8369 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:48.979235 15769 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:48.982592  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 140061us
W20260709 13:04:49.012300 15911 log.cc:927] Time spent T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.138s	user 0.001s	sys 0.001s
I20260709 13:04:48.983383 15798 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:04:49.016078  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.017551  8369 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:48.986150  8974 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:48.986603  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 70218.3us
I20260709 13:04:48.986845 15812 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:48.965546 15538 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:48.958235 15916 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:48.967655 15871 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:48.988278 15752 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:48.965171 15840 consensus_queue.cc:237] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:48.991401 15877 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:48.993119 15813 consensus_meta.cc:287] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.590s	user 0.008s	sys 0.002s
I20260709 13:04:48.972488 15777 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 13:04:48.994719  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 99510.8us
I20260709 13:04:48.997602 15870 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:49.017783  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:49.020545  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:49.021858 15775 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:49.022954  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:49.028546  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:48.984773  8222 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:04:49.078967  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 94246.1us
W20260709 13:04:49.040505  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 54456.8us
I20260709 13:04:49.041935  8297 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:49.042990 15927 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.049283  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:49.072398  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:49.081517  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:49.052189 15841 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:49.078172  8327 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 27.
I20260709 13:04:49.080854  8222 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:49.084501  8325 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:49.085050  8404 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 24.
I20260709 13:04:49.058669 15812 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:49.089618  8369 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:49.087826  8223 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:49.089658  8330 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:04:49.090337  8222 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:49.076601 15777 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.098146  8972 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:49.073407 15899 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:49.087664 15877 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:49.114046 15877 raft_consensus.cc:697] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:49.115088 15690 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:04:49.115679 15690 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.115212 15877 consensus_queue.cc:237] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:49.116086 15690 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:04:49.116381  8223 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:49.091001 15812 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:49.118381  8400 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:49.092374 15752 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:49.082201 15813 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 28 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:49.121996  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 80164.3us
I20260709 13:04:49.124372  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:49.125989 15899 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:49.129554 15905 log.cc:927] Time spent T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.482s	user 0.003s	sys 0.000s
I20260709 13:04:49.130790 15449 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:49.107290 15769 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:49.133569  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:49.135499 15870 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:49.137547  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:49.133579 15852 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:04:49.133590 15920 log.cc:927] Time spent T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.210s	user 0.002s	sys 0.000s
I20260709 13:04:49.143326 15871 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 26 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:49.143875 15924 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:49.117267 15538 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:49.103609 15777 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:49.133289 15861 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:49.157944  8398 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
W20260709 13:04:49.161867  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 45622.1us
I20260709 13:04:49.174109  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:49.174793  8326 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:49.162909  8327 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:04:49.166030  8325 raft_consensus.cc:2468] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:04:49.120594 15812 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:49.179720  8972 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:49.180408  8974 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:49.179848  8223 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:49.154430 15795 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.172757 15852 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.158895 15752 raft_consensus.cc:697] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:49.202258 15922 log.cc:927] Time spent T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:04:49.204058 15795 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:49.206193  9003 raft_consensus.cc:1275] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:04:49.208436 15732 consensus_queue.cc:1048] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:04:49.204042 15690 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 28 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:49.210511  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:49.211998  9003 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
I20260709 13:04:49.213697  8372 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:49.215859 15732 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:49.216284 15732 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.216671 15732 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:49.208760 15795 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:49.215101 15690 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:49.229516  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 158343us
I20260709 13:04:49.230407  8369 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:49.231317 15795 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:49.231767 15795 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:49.232180 15795 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:49.237149  9000 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:04:49.238940 15938 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:04:49.240418 15449 consensus_queue.cc:1048] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:49.243151 15813 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:49.244482  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:49.244838  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:49.245301  9003 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 28: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:49.245507  8325 raft_consensus.cc:3060] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:04:49.246547 15753 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:49.248680 15753 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:49.249521 15880 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:49.110356 15927 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:49.250155  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 75876.4us
I20260709 13:04:49.250873  8223 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:49.251628 15927 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:49.252209  8223 leader_election.cc:400] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 14. Current term is 15.
I20260709 13:04:49.252514  8223 leader_election.cc:403] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:04:49.252945 15900 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:49.244988  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 71968.2us
I20260709 13:04:49.253657 15553 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:49.254082 15553 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.254428 15553 raft_consensus.cc:3060] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:49.254501  8974 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:49.256067 15928 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.256675 15928 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:49.257256 15871 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:49.257627 15924 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:49.258421 15924 raft_consensus.cc:3060] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:49.258951 15928 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:49.259198 15871 raft_consensus.cc:2804] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:49.259989 15871 raft_consensus.cc:697] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 15 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:49.259948 15752 consensus_queue.cc:237] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:49.261093 15871 consensus_queue.cc:237] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:49.268575  8297 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:49.269109  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:49.269755  8400 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 17.
I20260709 13:04:49.269994  8297 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:49.270130 15900 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:49.271016  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:49.275391  8325 raft_consensus.cc:2468] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 28.
I20260709 13:04:49.268616  8222 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:49.186733 15776 raft_consensus.cc:2804] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:04:49.298197  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 44746us
I20260709 13:04:49.290870  8222 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:49.311841  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:49.311918  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:49.312521  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:49.312680  8400 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 28: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:49.189452 15852 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:49.313135  9003 raft_consensus.cc:2393] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 26: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:49.313481  8405 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:49.313453  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:49.316890  8972 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:49.324173  8326 raft_consensus.cc:2468] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 26.
I20260709 13:04:49.325455  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:49.326464  8398 raft_consensus.cc:2410] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:04:49.194609 15870 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:49.334059  8406 raft_consensus.cc:1275] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:04:49.292598 15880 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:49.292389 15769 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:49.337822  9008 raft_consensus.cc:1275] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:04:49.338022  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:49.346843  9001 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:04:49.336531  8223 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:49.337366  8330 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 29.
I20260709 13:04:49.319692 15690 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:49.352953  8403 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:49.334883 15861 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:49.355932  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:49.359274  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:49.359601  9003 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 20. Current term is 21. Ops: []
W20260709 13:04:49.357031  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:49.362210  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.364108  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:49.344287 15880 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.364331  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:49.363788 15925 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.220s	user 0.008s	sys 0.013s
I20260709 13:04:49.343037 15769 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.338737 15776 raft_consensus.cc:697] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:49.365504  8223 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:49.369370  8402 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 23.
W20260709 13:04:49.377108  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.356434 15841 raft_consensus.cc:2804] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:04:49.358479 15861 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.373481 15940 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:04:49.372359 15880 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:49.383450  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:49.360852 15938 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:49.397809  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:49.397809  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:49.384778  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:49.399001  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:49.403750  8222 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:49.408756  8405 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 24.
I20260709 13:04:49.422016  8402 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 28: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:49.422363  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:49.413589  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 72037.1us
I20260709 13:04:49.416810  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:49.425700  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:49.377750 15769 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:49.383832 15928 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:49.422993  8405 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
I20260709 13:04:49.365653 15690 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:49.423017  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:49.383391 15644 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:04:49.422353  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 69073.9us
I20260709 13:04:49.426982  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.398176 15940 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.423771  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:49.398128 15861 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:49.437404  8327 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 28.
I20260709 13:04:49.426386  8972 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:49.444083 15939 consensus_queue.cc:1059] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.130s
I20260709 13:04:49.439572  8255 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 20.
I20260709 13:04:49.446147  8326 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18.
I20260709 13:04:49.446295  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:49.451926  8330 raft_consensus.cc:3055] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:49.447744  9005 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 21.
I20260709 13:04:49.452410  8330 raft_consensus.cc:740] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:49.446982  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:49.453284  8402 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 28: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:49.448436  9003 raft_consensus.cc:1240] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:49.453876  8254 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:04:49.420964 15841 raft_consensus.cc:697] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 28 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:49.461609  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:49.466753 15861 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:49.429313 15538 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:49.450579 15940 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:49.479717  8972 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:49.389851 15813 consensus_queue.cc:1048] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:04:49.475942  8398 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:49.483845  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:49.481985  8369 leader_election.cc:400] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 24. Current term is 25.
I20260709 13:04:49.484656  8369 leader_election.cc:403] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:49.486943  8974 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:49.488113 15706 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:49.488655 15706 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:49.489131 15706 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:49.491307 15871 consensus_queue.cc:1059] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:04:49.466949 15947 log.cc:927] Time spent T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:04:49.513827  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:49.517580  8330 consensus_queue.cc:260] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:49.518177  8250 raft_consensus.cc:1240] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:49.518608  8330 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:49.519381  8253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:49.520254  8257 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:49.520290  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.521821  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.522364  8249 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:49.523476  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:49.524811  8248 raft_consensus.cc:3060] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:04:49.525496  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.521417  8257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.470829 15732 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:49.530062 15732 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:49.530826 15732 consensus_queue.cc:1048] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:49.539170 15690 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:49.540053 15690 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:04:49.540637 15690 raft_consensus.cc:697] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 28 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:49.541651  8248 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 29.
I20260709 13:04:49.513337  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:49.546775  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.547474  8248 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 20.
I20260709 13:04:49.454965 15644 raft_consensus.cc:697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:49.548408  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 273040us
I20260709 13:04:49.548812  8251 raft_consensus.cc:2393] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 22: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:49.479194 15870 raft_consensus.cc:2804] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:04:49.477778 15538 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.565871  8330 raft_consensus.cc:1275] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 13:04:49.568631  8330 pending_rounds.cc:85] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 10
I20260709 13:04:49.436794 15938 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:49.522265  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.568951  8330 pending_rounds.cc:107] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:04:49.572057 15690 consensus_queue.cc:237] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:49.573086  8330 raft_consensus.cc:2889] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:49.581537  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:49.589587  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:49.469893 15673 consensus_queue.cc:1048] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.185s
I20260709 13:04:49.625916  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.482583 15901 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:49.578227 15538 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:04:49.637498  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 155764us
I20260709 13:04:49.620093  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:04:49.643766  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 60778.3us
I20260709 13:04:49.645836  8256 raft_consensus.cc:2468] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 26.
I20260709 13:04:49.652170  8398 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e"
dest_uuid: "e64761f43e4240319c41117cf886d22f"
 from {username='slave'} at 127.0.0.1:39306
I20260709 13:04:49.655797  8402 raft_consensus.cc:1240] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:49.656178  8398 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:49.656553  8398 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:49.658396 15775 consensus_meta.cc:287] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.575s	user 0.007s	sys 0.001s
I20260709 13:04:49.658860 15775 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:49.578958 15926 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.063s
I20260709 13:04:49.584452 15553 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:49.662245 15861 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:49.662624 15924 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 14. Current term is 15.
W20260709 13:04:49.663138 15941 log.cc:927] Time spent T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc: Append to log took a long time: real 0.295s	user 0.001s	sys 0.000s
I20260709 13:04:49.669663 15899 raft_consensus.cc:3055] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:49.670148 15899 raft_consensus.cc:740] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:49.671152 15899 consensus_queue.cc:260] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:49.672397 15899 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:49.675271 15852 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:49.676069  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:49.582006 15870 raft_consensus.cc:697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 26 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:49.670157  8400 raft_consensus.cc:2468] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 22.
W20260709 13:04:49.649214  8223 connection.cc:256] RPC call timeout handler was delayed by -1.3658712769999966s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:04:49.687237  8250 raft_consensus.cc:2393] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 18: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
W20260709 13:04:49.690661  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 59052.8us
I20260709 13:04:49.691788  8295 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:49.692813 15581 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:49.696540 15581 consensus_queue.cc:1048] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:49.708042  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:49.709213  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.709640  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:49.710472  8325 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:04:49.711490  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:49.682077  8398 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:49.713840  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.714805  8254 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
I20260709 13:04:49.704397  8295 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:49.715955  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.720103  8295 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
W20260709 13:04:49.720775  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 245866us
I20260709 13:04:49.722137  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:49.722808  8326 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
I20260709 13:04:49.722795  8328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 3489) took 1209 ms. Trace:
I20260709 13:04:49.724750  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:49.724462  8328 rpcz_store.cc:276] 0709 13:04:48.513574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.799589 (+286015us) service_pool.cc:224] Handling call
0709 13:04:48.799986 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:49.078412 (+278426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:49.078441 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:49.099700 (+ 21259us) log.cc:844] Serialized 26 byte log entry
0709 13:04:49.194644 (+ 94944us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:49.194685 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.194765 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:49.722574 (+527809us) raft_consensus.cc:1685] finished
0709 13:04:49.722595 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:49.722718 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":175159,"raft.queue_time_us":175897,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":115,"spinlock_wait_cycles":3456,"thread_start_us":173030,"threads_started":2,"wal-append.queue_time_us":229990}
W20260709 13:04:49.726687  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 156558us
I20260709 13:04:49.647617 15928 raft_consensus.cc:697] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:49.726145  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 246528us
I20260709 13:04:49.729635  8398 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:49.729730  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:49.730733  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.731052  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.733132  8323 raft_consensus.cc:1275] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:04:49.735137  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:49.731739  8249 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 26.
I20260709 13:04:49.647236 15964 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 24. Current term is 25.
I20260709 13:04:49.730696  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:49.641412 15753 consensus_queue.cc:1059] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 13:04:49.740737  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:49.740792  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.741667  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:49.749092  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.749675  8326 raft_consensus.cc:2393] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 28: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:49.742525  8332 raft_consensus.cc:1275] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:04:49.739595  8295 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:49.677367 15852 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:49.744984  8248 raft_consensus.cc:1275] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:04:49.646404 15795 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:49.752681  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.650974 15898 consensus_queue.cc:1059] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 13:04:49.729079 15581 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:49.581586 15776 consensus_queue.cc:237] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:49.768090  8220 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:49.775187  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.775902  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:04:49.776517  8404 consensus_meta.cc:287] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.505s	user 0.010s	sys 0.000s
I20260709 13:04:49.712770 15813 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:04:49.778525  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.780229  8404 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 29.
I20260709 13:04:49.781986  8398 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:49.752296 15926 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:49.793211  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:49.781605  8298 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:49.793833  8406 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:49.794250  8399 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:49.794399  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.795444  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.796819  8248 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
I20260709 13:04:49.654074 15841 consensus_queue.cc:237] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:49.791620 15777 consensus_meta.cc:287] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.602s	user 0.009s	sys 0.001s
I20260709 13:04:49.804775  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:49.747615 15861 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.045s
I20260709 13:04:49.804057  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.776896 15581 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:49.778797 15939 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:49.814245  8397 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:04:49.815747  8249 raft_consensus.cc:2393] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 20: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:49.752496 15900 consensus_queue.cc:1048] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:04:49.796411 15926 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:49.829361  8249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.835713  8249 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 24.
I20260709 13:04:49.837052  8249 raft_consensus.cc:1275] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 13:04:49.837337  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.838102  8250 raft_consensus.cc:1275] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:04:49.829622 15812 consensus_meta.cc:287] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.641s	user 0.006s	sys 0.005s
I20260709 13:04:49.831193  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:49.821604 15777 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:49.825127 15939 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.833725 15971 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:04:49.834452  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 57822.6us
W20260709 13:04:49.841594  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 73615.5us
I20260709 13:04:49.795190 15753 consensus_queue.cc:1059] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:04:49.842103  8254 raft_consensus.cc:3060] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:49.825264 15581 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:49.812508 15752 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:49.748333 15644 consensus_queue.cc:237] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:49.847893  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.860057 15953 consensus_queue.cc:1059] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:49.861302  8220 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:49.870740  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.871383  8255 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
I20260709 13:04:49.869666  8404 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:04:49.839735 15926 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:49.870857  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 48569.9us
I20260709 13:04:49.876706  8298 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:49.878779  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:49.845077 15939 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:49.847994 15900 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:49.884840  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:49.882007  8250 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 23. Current term is 25. Ops: []
W20260709 13:04:49.887830  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.835124 15916 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.040s
I20260709 13:04:49.892066  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
W20260709 13:04:49.892990  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.893685  8256 raft_consensus.cc:2393] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 20: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:49.815992 15870 consensus_queue.cc:237] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:49.845860 15798 consensus_queue.cc:1059] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:04:49.874184 15840 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:49.898198  8369 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:49.848505 15812 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:49.878759 15553 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:49.888690 15901 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 25 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:49.889091 15900 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.890084 15799 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:49.865031 15971 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:49.867060 15928 consensus_queue.cc:237] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:49.890997 15953 consensus_queue.cc:1243] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer 8c42259d7c4b463a9f485f42c320c81b log is divergent from this leader: its last log entry 20.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:04:49.899318 15753 consensus_queue.cc:1059] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.045s
W20260709 13:04:49.963043  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 101835us
I20260709 13:04:49.999986  8406 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 25.
I20260709 13:04:49.924674  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.923056  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:50.001518  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 78539.4us
I20260709 13:04:49.983417  8397 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
I20260709 13:04:49.936501  8401 raft_consensus.cc:1240] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:49.948706  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.935241  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:50.004415  8248 raft_consensus.cc:2368] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 24. Current term is 25.
I20260709 13:04:50.002700  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:50.014717  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 143088us
I20260709 13:04:50.019778  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:49.923731 15898 raft_consensus.cc:3055] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:50.028124  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:50.028182  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.924279 15449 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:50.028543  8220 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:49.950462 15982 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:50.034380  8248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:49.922308 15900 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:50.018251 15974 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:50.053898 15974 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.032071  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:50.033051  8220 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:50.054394 15974 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:50.055389  8220 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:50.056423 15861 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:04:50.056806 15861 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.057156 15861 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:50.007731 15975 raft_consensus.cc:3055] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:50.057622 15975 raft_consensus.cc:740] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:50.058566 15975 consensus_queue.cc:260] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:50.059928 15975 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:50.038311  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:50.018911 15962 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:50.061828 15962 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.062252 15962 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:50.062403  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:50.062930 15899 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.063491 15899 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } attrs { replace: true } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:50.063967  8250 raft_consensus.cc:1240] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 29 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 28. Current term is 29. Ops: [28.17-28.17]
I20260709 13:04:50.040619 15777 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:50.029606  8972 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:50.028584 15898 raft_consensus.cc:740] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:49.921379 15840 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:50.042451 15976 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:04:50.075309  8253 raft_consensus.cc:2468] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15.
I20260709 13:04:50.034507 15799 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:50.020136 15956 consensus_queue.cc:1048] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.026s
W20260709 13:04:50.046280 15880 consensus_meta.cc:287] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.610s	user 0.010s	sys 0.000s
I20260709 13:04:49.963507 15901 consensus_queue.cc:1059] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.063s
I20260709 13:04:50.049777 15644 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.050453 15798 raft_consensus.cc:3055] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:04:50.051131 15769 consensus_meta.cc:287] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.602s	user 0.005s	sys 0.006s
I20260709 13:04:50.109760 15769 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:50.072145 15953 raft_consensus.cc:2804] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 29 FOLLOWER]: Leader election won for term 29
I20260709 13:04:49.970772 15966 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.143s
I20260709 13:04:50.068408  8220 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:50.039968 15971 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:50.040504 15538 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:50.018868 15795 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:50.067507  8972 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:04:50.074126  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 60541.4us
W20260709 13:04:50.044703 15706 consensus_meta.cc:287] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.555s	user 0.008s	sys 0.000s
I20260709 13:04:50.090579  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:50.073377 15924 raft_consensus.cc:2804] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:04:50.147573 15924 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:50.147997 15924 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:50.105394 15880 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:50.150270  8252 raft_consensus.cc:1240] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:04:50.151253 15880 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:50.109104 15798 raft_consensus.cc:740] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:49.970587 15968 consensus_queue.cc:1059] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.191s
I20260709 13:04:50.154747 15968 consensus_queue.cc:1243] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer 8c42259d7c4b463a9f485f42c320c81b log is divergent from this leader: its last log entry 17.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:04:50.155232 15968 raft_consensus.cc:3055] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:50.155544 15968 raft_consensus.cc:740] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:50.156325 15968 consensus_queue.cc:260] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:50.157684  8403 raft_consensus.cc:1240] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:50.158125  8295 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:50.158987 15968 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:50.159116  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:50.160753  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:50.115823  8254 raft_consensus.cc:2468] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
I20260709 13:04:50.117354 15901 consensus_queue.cc:1243] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer 8c42259d7c4b463a9f485f42c320c81b log is divergent from this leader: its last log entry 21.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:04:50.121743  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:04:50.126750  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 58447.2us
I20260709 13:04:50.166324  8220 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:50.129732  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:50.143322 15954 raft_consensus.cc:3055] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:50.143085 15706 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:50.144795 15538 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:50.145216 15769 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:50.145660  8257 raft_consensus.cc:2393] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 25: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:50.147029  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:50.149253  8397 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:50.174229  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:50.206679  8402 raft_consensus.cc:1813] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Handling vote request from an unknown peer ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:50.207000  8405 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 21 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:04:50.136646 15953 raft_consensus.cc:697] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 29 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:50.207758  8402 raft_consensus.cc:2410] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:04:50.117486 15644 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:50.160758 15812 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:50.209661  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 51605.9us
I20260709 13:04:50.117398 15900 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:50.163632 15939 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:50.216272 15690 raft_consensus.cc:3055] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:04:50.217258 15940 consensus_meta.cc:287] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.667s	user 0.007s	sys 0.002s
I20260709 13:04:50.216128 15924 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:50.217675 15899 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:50.222116  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:50.223757 15852 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:50.221843 15986 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:50.212653  8295 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:50.230942  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 163542us
I20260709 13:04:50.213271  8297 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:50.173475 15954 raft_consensus.cc:740] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:50.246124  8251 raft_consensus.cc:2410] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 23 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:04:50.233814  8256 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 18 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:50.208067 15971 raft_consensus.cc:2764] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:04:50.249893 15989 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:50.249100 15954 consensus_queue.cc:260] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:50.243934  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:50.236055 15690 raft_consensus.cc:740] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:50.245343 15986 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:50.251804 15986 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:50.252089 15690 consensus_queue.cc:260] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:50.253196 15690 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:50.258258  8406 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:04:50.239586 15928 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:50.259334  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:50.259997  8402 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:04:50.255771  8972 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:50.260506 15928 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:50.260994 15928 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:50.262171  8401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3696) took 1276 ms. Trace:
I20260709 13:04:50.262446  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:50.262527  8401 rpcz_store.cc:276] 0709 13:04:49.005683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.052334 (+ 46651us) service_pool.cc:224] Handling call
0709 13:04:49.053222 (+   888us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:50.005587 (+952365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.261982 (+256395us) spinlock_profiling.cc:243] Waited 883 ms on lock 0x7b54002675d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:50.262121 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2030738176}
I20260709 13:04:50.256402  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:50.240725  9001 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:50.247836  8295 leader_election.cc:400] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Vote denied by peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679) with higher term. Message: Invalid argument: T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 24. Current term is 25.
I20260709 13:04:50.248708  8297 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:50.250319 15989 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:50.224884 15581 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:50.254513  8222 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:04:50.278429  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 65556.2us
I20260709 13:04:50.253290  8974 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:50.282011  8331 raft_consensus.cc:1240] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:04:50.287485  9004 raft_consensus.cc:1240] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:50.304909  8250 raft_consensus.cc:1240] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:50.286098  8222 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:50.173535 15898 consensus_queue.cc:260] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:50.306043  8250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:50.314750  8297 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:50.316195  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:50.318287  8256 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:50.318742  9001 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:50.319453  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:50.279841 15899 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:50.334435  8295 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:50.340471  8297 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:50.285166 15989 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:50.263042 15986 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:50.307379 15752 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:50.353818  9009 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 26 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 13:04:50.304318 15971 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:50.314774 15536 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:04:50.243067 15940 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:50.367934  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 48798.5us
I20260709 13:04:50.363583  8402 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 25.
I20260709 13:04:50.344565 15899 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.378192  8295 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:50.349680 15769 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:50.355160 15998 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:50.362737 15753 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:50.384598 15899 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:50.414867  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 58834.4us
I20260709 13:04:50.415758  8297 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:50.417618  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 99782.5us
I20260709 13:04:50.382892 15990 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:50.420540  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:50.383148 15840 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:50.376048 15953 consensus_queue.cc:237] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.17, Last appended by leader: 17, Current term: 29, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:50.381881 15861 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:50.496582  8256 raft_consensus.cc:1275] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:04:50.498301  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.451109 15900 leader_election.cc:290] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:50.385365 15921 consensus_queue.cc:1059] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.096s
W20260709 13:04:50.474267 16002 log.cc:927] Time spent T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:04:50.467835 16000 log.cc:927] Time spent T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:04:50.477989 15995 log.cc:927] Time spent T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
I20260709 13:04:50.381561 16007 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.107s
I20260709 13:04:50.411202 15982 consensus_queue.cc:1059] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 13:04:50.455996 15753 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:50.466653 15644 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:50.490690 15769 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:50.461512 15938 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:50.494088 15536 raft_consensus.cc:697] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 25 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:50.496801 15988 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:50.505312 15706 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:50.577491  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 299661us
W20260709 13:04:50.604851  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.605124 15940 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:50.605409 16005 log.cc:927] Time spent T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
I20260709 13:04:50.607471 15776 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:50.608174 15776 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:50.609814 15969 consensus_queue.cc:1059] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:04:50.610453 15986 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:50.572000 15939 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:50.585075 15900 raft_consensus.cc:3055] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:04:50.614152 16011 log.cc:927] Time spent T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:04:50.614498 15900 raft_consensus.cc:740] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:50.615545 15900 consensus_queue.cc:260] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:50.578500 15966 raft_consensus.cc:3055] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:50.637856 15966 raft_consensus.cc:740] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:50.638502 15536 consensus_queue.cc:237] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:50.638890 15966 consensus_queue.cc:260] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:50.640702 15966 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:50.617134 15900 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:50.645874 15776 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:50.650369  9005 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
W20260709 13:04:50.652113 15962 consensus_meta.cc:287] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.589s	user 0.012s	sys 0.000s
I20260709 13:04:50.566582 15924 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:50.652940 15962 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:50.659694  8405 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:50.661157  9006 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 24. Current term is 25. Ops: []
I20260709 13:04:50.662956  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:50.664438  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:50.665459 15581 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:50.667110 15861 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:50.667868 15962 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:50.668416 15985 log.cc:927] Time spent T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.676s	user 0.000s	sys 0.002s
I20260709 13:04:50.671084 15970 raft_consensus.cc:3055] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:50.671604 15970 raft_consensus.cc:740] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:50.587109 15988 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.674059 15988 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:50.676152 15798 consensus_queue.cc:260] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:50.581949  8253 raft_consensus.cc:1240] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 26. Current term is 27. Ops: [26.19-26.19]
W20260709 13:04:50.581914  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 272605us
I20260709 13:04:50.504197 15990 consensus_queue.cc:1059] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.066s
I20260709 13:04:50.598254  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:50.603112  8223 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:50.604285 16006 log.cc:927] Time spent T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
W20260709 13:04:50.682129  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.672628 15970 consensus_queue.cc:260] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:50.706552  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 103525us
I20260709 13:04:50.681519 15798 raft_consensus.cc:3060] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:50.731806  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:50.733296  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:50.708860 15986 consensus_queue.cc:1059] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.025s
I20260709 13:04:50.734356 16019 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:50.735958  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:50.737939 15928 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:50.685379 15752 raft_consensus.cc:697] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 22 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:50.738673  8399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3700) took 1465 ms. Trace:
I20260709 13:04:50.741504  9008 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:04:50.742712 15971 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:50.747071  8974 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:50.768287  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 397973us
I20260709 13:04:50.747653 15956 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.748545  8405 raft_consensus.cc:1240] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:50.750070  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
W20260709 13:04:50.772697  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 355594us
I20260709 13:04:50.750771  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:50.751281  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:50.751573  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:04:50.755275 15926 consensus_meta.cc:287] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.864s	user 0.004s	sys 0.007s
I20260709 13:04:50.756642  8398 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:04:50.749147  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:50.756340  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.759615  9007 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:50.764307  8399 rpcz_store.cc:276] 0709 13:04:49.272804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.273917 (+  1113us) service_pool.cc:224] Handling call
0709 13:04:49.312090 (+ 38173us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:49.794735 (+482645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.738451 (+943716us) spinlock_profiling.cc:243] Waited 482 ms on lock 0x7b5400277754. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:50.738617 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1108404480}
I20260709 13:04:50.772969  8251 raft_consensus.cc:2393] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 20: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:50.802347  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:50.803165  8297 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:50.766268 15974 consensus_meta.cc:287] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.711s	user 0.012s	sys 0.001s
I20260709 13:04:50.843288  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:50.808807  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.821780  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:04:50.869985  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 57225.6us
I20260709 13:04:50.829814  9009 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:04:50.828083  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:50.872710  8405 raft_consensus.cc:2393] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 22: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:50.769112  8295 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:50.857218  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:50.765523 15553 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.868371  8254 raft_consensus.cc:3060] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:04:50.881276  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 112278us
I20260709 13:04:50.766469 15927 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:50.795903 15690 raft_consensus.cc:2804] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:04:50.885021  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 81956.4us
I20260709 13:04:50.889401  8297 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:50.890875  8297 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:50.876315  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 433367us
I20260709 13:04:50.898299  8220 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:50.899163 15921 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:50.900223 15921 consensus_queue.cc:1048] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:50.910787  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:50.900348  8220 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:50.901415  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b" is_pre_election: true
I20260709 13:04:50.803099 15970 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:50.820859 15769 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:50.879786 15989 consensus_meta.cc:287] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.509s	user 0.009s	sys 0.003s
I20260709 13:04:50.874660  8222 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e
I20260709 13:04:50.920403  8298 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:50.923185  8251 raft_consensus.cc:2393] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 18: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:50.923756  8406 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:50.884135 16018 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:50.888250 15986 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:04:50.924299  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 49746.1us
I20260709 13:04:50.912781 15982 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:50.942091  9001 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
W20260709 13:04:50.942624  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 174156us
I20260709 13:04:50.942854  8254 raft_consensus.cc:2468] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 27.
I20260709 13:04:50.796900 15956 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:50.951738  8251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:50.911033 16028 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:50.954427  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:50.919029 15690 raft_consensus.cc:697] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 22 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:50.806128 15926 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:50.956146  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:50.844341 16021 consensus_queue.cc:1059] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.180s
I20260709 13:04:50.873726 15974 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:50.957448  8254 raft_consensus.cc:2376] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 15. Re-sending same reply.
I20260709 13:04:50.844661 16020 consensus_queue.cc:1059] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.177s
I20260709 13:04:50.938279 16007 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:04:50.941861 15966 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:50.884768 15899 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:50.910007 15553 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:50.965965  8254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8c42259d7c4b463a9f485f42c320c81b"
I20260709 13:04:50.923755 15769 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:50.843602 15752 consensus_queue.cc:237] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:50.932165 15989 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:50.916966 15927 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:50.969301  8401 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 28 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 13:04:50.970501  8397 raft_consensus.cc:1275] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:04:50.959651 16018 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:04:50.985618  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 65292.5us
I20260709 13:04:50.991839  8254 raft_consensus.cc:1217] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Deduplicated request from leader. Original: 17.14->[19.15-19.15]   Dedup: 19.15->[]
I20260709 13:04:50.992916  8254 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:04:50.952137 15644 consensus_queue.cc:1059] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:04:50.968186 15966 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.984424 15928 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:50.997151 16007 raft_consensus.cc:697] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 25 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:50.969956 15939 consensus_queue.cc:1059] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:04:51.005596  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:50.962445 15986 consensus_queue.cc:1059] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:04:51.030638  8251 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 23.
I20260709 13:04:50.996460 15989 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:51.040130  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 104164us
I20260709 13:04:51.003114  8254 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:04:51.040571  8251 raft_consensus.cc:3060] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:04:51.042306  8297 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:50.999715  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 78137.7us
W20260709 13:04:51.007328  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:51.007701 16031 log.cc:927] Time spent T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:04:51.007464 15960 consensus_queue.cc:1048] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:51.008181 15944 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:51.009635 15690 consensus_queue.cc:237] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:51.049520  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:51.050742  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:51.051436  8400 raft_consensus.cc:2368] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 24. Current term is 25.
I20260709 13:04:51.052181  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:51.018000 16007 consensus_queue.cc:237] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:51.021567 15769 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:51.052847 15989 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:51.053443 15989 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:51.054697  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:51.055287  8331 raft_consensus.cc:3055] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:51.055482 15989 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:51.055680  8331 raft_consensus.cc:740] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:51.056244  8326 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 24. Current term is 25. Ops: []
I20260709 13:04:51.056545  8331 consensus_queue.cc:260] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:51.057085  8329 raft_consensus.cc:1240] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:51.027087 15988 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:51.057364  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:51.039304 15974 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:51.029090 15916 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:04:51.001488 15966 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:51.031772 15553 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:51.032820 15927 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:51.000797  9007 raft_consensus.cc:2468] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 22.
I20260709 13:04:51.033627  8402 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
I20260709 13:04:51.035678  9009 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:04:50.992596 16021 consensus_queue.cc:1243] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer f865c4d761f5446eb12b3a029234772e log is divergent from this leader: its last log entry 26.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 13:04:51.036834  8369 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:51.046763 15798 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:51.044445  8295 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:51.048128  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:51.048574  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:51.096932  8406 raft_consensus.cc:2393] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 27: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:51.040313 15939 consensus_queue.cc:1243] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Peer e64761f43e4240319c41117cf886d22f log is divergent from this leader: its last log entry 21.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 13:04:51.030673  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.074298  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:51.076704  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
W20260709 13:04:51.099661  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 62941.5us
I20260709 13:04:51.100051  8329 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:51.076006  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:51.099795  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.101004  8326 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
W20260709 13:04:51.079756  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.055505  8400 raft_consensus.cc:1240] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:51.056499  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:51.082039  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 26 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:51.103461  8332 raft_consensus.cc:2393] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 27: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:51.082805  8331 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:51.089936  8251 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 28 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 13:04:51.091993  9005 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:51.093550  8298 leader_election.cc:304] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:51.094853  9004 raft_consensus.cc:2393] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 25: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:51.037781 15901 raft_consensus.cc:3055] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:51.103243 15927 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:51.145498  8248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 2828) took 1403 ms. Trace:
W20260709 13:04:51.149593  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 105195us
I20260709 13:04:51.151593  8369 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:51.153622  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 60239.8us
I20260709 13:04:51.158011  8297 leader_election.cc:400] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 24. Current term is 25.
I20260709 13:04:51.158485  8297 leader_election.cc:403] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 13:04:51.160218  8369 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:51.108192 15901 raft_consensus.cc:740] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:51.110551 15880 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:51.188349  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:51.190660  8369 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:51.194010  8369 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:51.199793  8369 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:51.155855 15769 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:51.156157 15964 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:51.203900  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:51.208297  8400 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 29 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
W20260709 13:04:51.210461  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.212419  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:51.213203  8326 raft_consensus.cc:2393] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 21: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:51.142665 15795 consensus_queue.cc:1048] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
W20260709 13:04:51.215569  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 60768.9us
I20260709 13:04:51.182101 15944 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:51.212255 15840 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:51.156775 15974 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:04:51.156466 15960 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:04:51.193214 15986 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.192890 15775 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:51.198406 15989 raft_consensus.cc:2804] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:51.188903 16045 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:51.198069 15449 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 13:04:51.201841 15962 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:51.225857 15964 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:51.221869 16018 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:51.237968 16046 log.cc:927] Time spent T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:04:51.237808 16018 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:51.238725 15964 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:51.239730 15989 raft_consensus.cc:697] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 22 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:51.239905 16018 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:51.229784 16036 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
I20260709 13:04:51.240753 15989 consensus_queue.cc:237] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:51.241313  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:51.229902 15926 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:51.242138  8403 raft_consensus.cc:2393] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 21: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:51.245850  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:51.246434 15901 consensus_queue.cc:260] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:51.247895 15901 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:51.250742 15986 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:51.251190  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:51.251875  8401 raft_consensus.cc:2393] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 25: Already voted for candidate 8c42259d7c4b463a9f485f42c320c81b in this term.
I20260709 13:04:51.252909 15986 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:51.253417 15877 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:51.254004 15877 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:51.256294 15877 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:51.256470 15898 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:51.226231 15899 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:51.257892 15986 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:51.223225  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:51.228266 15812 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:04:51.259675  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:51.258839 15986 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:51.260998  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:51.215437 15769 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:51.220410 16043 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:51.269254  8332 raft_consensus.cc:1275] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 13:04:51.269119  8326 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:51.273921  8329 raft_consensus.cc:2468] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
W20260709 13:04:51.273831  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 60715.4us
I20260709 13:04:51.262228  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:51.263243  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:51.263712  8972 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:51.276147  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:51.272599  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:51.277304  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:51.278303  9005 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 21.
I20260709 13:04:51.278403 15987 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:51.280087  8332 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:51.261406 15956 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:51.276813  8972 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:51.282275  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:51.282985 15916 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:51.283176  8298 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:51.283121 15987 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 13:04:51.283690  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:51.284799  9005 raft_consensus.cc:2393] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 25: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:51.285063 15899 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:51.285501 15899 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.285845 15899 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:51.286634  9005 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Rejecting Update request from peer ff7a6fe533e94c1bae555b4cae427728 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:51.287235  8332 raft_consensus.cc:1275] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:04:51.289186 15939 consensus_queue.cc:1059] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:51.277510  8325 raft_consensus.cc:2393] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 22: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:51.293394  8332 raft_consensus.cc:1275] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:04:51.146052  8248 rpcz_store.cc:276] 0709 13:04:49.741960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.073717 (+331757us) service_pool.cc:224] Handling call
0709 13:04:50.074979 (+  1262us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:50.112003 (+ 37024us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:50.142616 (+ 30613us) log.cc:844] Serialized 14 byte log entry
0709 13:04:50.414647 (+272031us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:50.442350 (+ 27703us) log.cc:844] Serialized 26 byte log entry
0709 13:04:50.442574 (+   224us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:51.082276 (+639702us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b18003dd020. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882564bc0a 00007f882ed8b219 00007f882c69258f 00007f88256d1571 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd
0709 13:04:51.107394 (+ 25118us) log.cc:844] Serialized 14 byte log entry
0709 13:04:51.107682 (+   288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.107791 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:51.145178 (+ 37387us) raft_consensus.cc:1685] finished
0709 13:04:51.145210 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:51.145396 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":741378,"prepare.queue_time_us":30421,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":139778,"raft.run_cpu_time_us":495,"raft.run_wall_time_us":74174,"spinlock_wait_cycles":6151040,"thread_start_us":118976,"threads_started":1,"wal-append.queue_time_us":407250}
I20260709 13:04:51.296557  8332 raft_consensus.cc:1275] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 29 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 13:04:51.297224 15900 raft_consensus.cc:3055] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:51.297326 15927 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:51.297952 15900 raft_consensus.cc:740] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:51.299081 15900 consensus_queue.cc:260] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:51.300721 15900 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:51.268613 15769 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:51.261708 15967 raft_consensus.cc:3055] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:04:51.321592  8331 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
I20260709 13:04:51.334683  8369 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:51.335116  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:51.337538 15960 consensus_queue.cc:1059] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:51.338356 15962 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:51.338586  8369 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:51.338783 15962 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:51.339357 15962 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:51.339494  8369 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:51.339805  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:51.340436 15852 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:51.340852  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:51.262615 15986 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:51.342269  8403 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:04:51.342324 16054 raft_consensus.cc:2804] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:51.342760 15965 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:51.342983 16054 raft_consensus.cc:697] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 19 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:51.343257 15965 raft_consensus.cc:697] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:51.343945 16054 consensus_queue.cc:237] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:51.344269 15965 consensus_queue.cc:237] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:51.344674 15916 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.009s
I20260709 13:04:51.338877 15960 raft_consensus.cc:3055] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:51.346210 15960 raft_consensus.cc:740] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:51.340981 15852 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:51.348529 15852 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:51.348603 15962 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:51.347083 15960 consensus_queue.cc:260] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:51.351006  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:51.350973  8331 raft_consensus.cc:1275] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 13:04:51.322825 15990 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:51.320990 15967 raft_consensus.cc:740] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:51.380621  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.415009  8324 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 13:04:51.415592  8324 pending_rounds.cc:85] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728: Aborting all ops after (but not including) 11
I20260709 13:04:51.415866  8324 pending_rounds.cc:107] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 13:04:51.416183  8324 raft_consensus.cc:2889] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:51.418533  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:51.410576  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:51.411417  8372 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:51.436010  9008 raft_consensus.cc:2393] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 18: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:51.330669 15953 consensus_queue.cc:1059] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.105s
I20260709 13:04:51.334494 15956 consensus_queue.cc:1059] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.104s
I20260709 13:04:51.456758  8326 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 25.
I20260709 13:04:51.356703 15775 consensus_queue.cc:1059] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.092s
I20260709 13:04:51.361663 15690 consensus_queue.cc:1048] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.090s
I20260709 13:04:51.363636 15916 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 14, Time since last communication: 0.091s
I20260709 13:04:51.451975 15449 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:51.438192 15795 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:51.439589 15968 consensus_queue.cc:1048] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 5
I20260709 13:04:51.454903 15960 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:51.435400 15990 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:51.491994 15990 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:51.493490 15966 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:51.495829 15966 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:51.497166 15449 raft_consensus.cc:3055] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:51.497661 15449 raft_consensus.cc:740] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:51.498641 15449 consensus_queue.cc:260] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:51.499946 15449 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:51.501596 15795 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:51.504036 15795 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:51.483613 15799 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:51.505574 15799 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:51.506459 15536 raft_consensus.cc:3055] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:51.506945 15536 raft_consensus.cc:740] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:51.508021 15799 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:51.510200 15581 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 13:04:51.511439  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39306 (request call id 3069) took 1596 ms. Trace:
I20260709 13:04:51.513053 16043 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for earlier term 24. Current term is 25.
I20260709 13:04:51.514443  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:51.515890  8252 raft_consensus.cc:1240] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:51.508107 15536 consensus_queue.cc:260] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:51.511894  8404 rpcz_store.cc:276] 0709 13:04:49.915238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.925538 (+ 10300us) service_pool.cc:224] Handling call
0709 13:04:49.925894 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:50.105513 (+179619us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:50.137109 (+ 31596us) log.cc:844] Serialized 14 byte log entry
0709 13:04:50.981765 (+844656us) log.cc:844] Serialized 14 byte log entry
0709 13:04:50.982053 (+   288us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:50.982105 (+    52us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:50.982152 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.511163 (+529011us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b54003e6a54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:51.511221 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:51.511372 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":418860,"prepare.queue_time_us":265215,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":89,"raft.queue_time_us":396236,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":8503,"spinlock_wait_cycles":417687552,"thread_start_us":384001,"threads_started":2,"wal-append.queue_time_us":331745}
I20260709 13:04:51.518483 15962 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:51.482957 15967 consensus_queue.cc:260] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:51.528270  8252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 1318) took 1243 ms. Trace:
I20260709 13:04:51.530216  8327 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:04:51.534565  8252 rpcz_store.cc:276] 0709 13:04:50.284520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.478483 (+193963us) service_pool.cc:224] Handling call
0709 13:04:50.478763 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:51.509316 (+1030553us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b54003c58a0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f8828735dc0 00007f88287c01c0 00007f88287c1231 00007f88287bdd8f 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c
0709 13:04:51.527991 (+ 18675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.528210 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":251683712}
W20260709 13:04:51.535552  8297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.537187  9004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56940 (request call id 1450) took 1204 ms. Trace:
I20260709 13:04:51.535094 15967 raft_consensus.cc:3060] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:51.541884 15690 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:04:51.537739  9004 rpcz_store.cc:276] 0709 13:04:50.315602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.315959 (+   357us) service_pool.cc:224] Handling call
0709 13:04:51.520056 (+1204097us) spinlock_profiling.cc:243] Waited 777 ms on lock 0x7b54004ec854. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:51.520235 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1786086784}
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
W20260709 13:04:51.552285  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 109724us
I20260709 13:04:51.557029  8255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 2817) took 1844 ms. Trace:
I20260709 13:04:51.557857  8255 rpcz_store.cc:276] 0709 13:04:49.712880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.872035 (+159155us) service_pool.cc:224] Handling call
0709 13:04:49.883013 (+ 10978us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:49.979576 (+ 96563us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:49.979620 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:50.904977 (+925357us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b2c000014c8. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:04:50.919575 (+ 14598us) log.cc:844] Serialized 26 byte log entry
0709 13:04:50.999599 (+ 80024us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:50.999634 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.556642 (+557008us) spinlock_profiling.cc:243] Waited 72.1 ms on lock 0x7b540034b7d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:51.556709 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:51.556770 (+    61us) raft_consensus.cc:1685] finished
0709 13:04:51.556787 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:51.556955 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":96,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":21726,"spinlock_wait_cycles":214184960,"thread_start_us":34445,"threads_started":1,"wal-append.queue_time_us":139088}
I20260709 13:04:51.530926 15899 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:51.563302 15899 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:51.565852  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:51.566447  9009 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:51.570719  8323 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 18.
I20260709 13:04:51.557348 15581 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:51.583496 15581 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:51.590361  7164 tablet_server.cc:179] TabletServer@127.6.255.1:0 shutting down...
I20260709 13:04:51.592797 15974 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:51.607071 15972 log.cc:927] Time spent T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 1.683s	user 0.008s	sys 0.011s
I20260709 13:04:51.610014 15644 raft_consensus.cc:3055] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:04:51.610689 15644 raft_consensus.cc:740] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:51.611888 15644 consensus_queue.cc:260] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:51.593385 15974 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:51.630141  8249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 2815) took 1917 ms. Trace:
I20260709 13:04:51.617750 15980 raft_consensus.cc:3055] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:51.613332 15644 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:04:51.637315  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.634796 15980 raft_consensus.cc:740] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:51.641218 15980 consensus_queue.cc:260] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:51.641954  8403 raft_consensus.cc:1275] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:04:51.642400 15980 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:51.646121 15581 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:51.647768  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:51.648649 15974 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:51.647711  8973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.631489  8249 rpcz_store.cc:276] 0709 13:04:49.712322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.837781 (+125459us) service_pool.cc:224] Handling call
0709 13:04:49.838046 (+   265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:49.838400 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:04:49.870814 (+ 32414us) log.cc:844] Serialized 14 byte log entry
0709 13:04:50.582891 (+712077us) log.cc:844] Serialized 14 byte log entry
0709 13:04:50.586936 (+  4045us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:50.588080 (+  1144us) log.cc:844] Serialized 26 byte log entry
0709 13:04:50.690776 (+102696us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:51.192373 (+501597us) log.cc:844] Serialized 14 byte log entry
0709 13:04:51.194497 (+  2124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.194595 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:51.626657 (+432062us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b5400139aa0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f8828733dbe 00007f882873e930 00007f882873e8d8 00007f882873e858 00007f882873e813 00007f882873da9c 00007f882e629626 00007f882e623dd8 00007f882877a7cb 00007f882877d2c0
0709 13:04:51.629832 (+  3175us) raft_consensus.cc:1685] finished
0709 13:04:51.629864 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:51.630060 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":635451,"prepare.queue_time_us":1427979,"prepare.run_cpu_time_us":138,"prepare.run_wall_time_us":126,"raft.queue_time_us":48239,"raft.run_cpu_time_us":510,"raft.run_wall_time_us":49106,"spinlock_wait_cycles":237401984,"thread_start_us":640501,"threads_started":2,"wal-append.queue_time_us":428908}
I20260709 13:04:51.667575  9001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56940 (request call id 1452) took 1279 ms. Trace:
I20260709 13:04:51.668198  9001 rpcz_store.cc:276] 0709 13:04:50.387889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.388075 (+   186us) service_pool.cc:224] Handling call
0709 13:04:51.667309 (+1279234us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b5400476b54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:51.667490 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":840233856}
W20260709 13:04:51.669764  8298 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.620260  9009 raft_consensus.cc:2468] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 22.
I20260709 13:04:51.675936  8298 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:51.683560 15924 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:51.685837 15975 raft_consensus.cc:2804] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:04:51.724159  8295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.713251 15975 raft_consensus.cc:697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 22 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:51.736793  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:51.737811  8331 raft_consensus.cc:2368] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 18. Current term is 19.
I20260709 13:04:51.738785  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:51.739565  8331 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
I20260709 13:04:51.740536  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:51.718340 15861 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:51.741391  8331 raft_consensus.cc:2393] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 18: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:51.729621 15841 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.714838 15924 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:51.750265  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:51.751813  8331 raft_consensus.cc:2410] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 20 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
W20260709 13:04:51.753147  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.755152  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:51.756399  8331 raft_consensus.cc:2410] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:04:51.758520  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:51.760007  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:51.761577  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:51.765771  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:51.750756 15861 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:51.749323 15841 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:51.783178  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:51.783708  8220 leader_election.cc:400] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 18. Current term is 19.
I20260709 13:04:51.783828  8329 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:51.786528  8253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 1316) took 1502 ms. Trace:
I20260709 13:04:51.786578  8331 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 25.
I20260709 13:04:51.787202  8220 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:51.787071  8253 rpcz_store.cc:276] 0709 13:04:50.283936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.473176 (+189240us) service_pool.cc:224] Handling call
0709 13:04:50.473589 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:50.680780 (+207191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.786274 (+1105494us) spinlock_profiling.cc:243] Waited 65.8 ms on lock 0x7b540034c1d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:51.786467 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":151276416}
I20260709 13:04:51.788025  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:51.788149 15967 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:51.788805  8331 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 25: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:51.789345  8220 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:51.790180 15967 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:51.790300  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:51.792405  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:51.793743  8331 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:51.794647  8220 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:51.795519  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:51.796828  8331 raft_consensus.cc:2393] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 18: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:04:51.776122 16062 log.cc:927] Time spent T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:04:51.798868  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:51.801065  8331 raft_consensus.cc:2410] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:04:51.798528 15967 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:51.801905  8220 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:51.802281 15813 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.826289  8329 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:51.759876 15975 consensus_queue.cc:237] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:51.827965  8220 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:51.824144 15967 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:51.836104  8329 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:04:51.835009 15629 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:51.829170 15813 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:51.852085 15813 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:51.837402 15967 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:04:51.847195 16065 log.cc:927] Time spent T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.068s	user 0.001s	sys 0.001s
I20260709 13:04:51.860350  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:51.856397 15967 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:51.854214 15553 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 13:04:51.865383 16068 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:51.866139 16068 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:51.868434  8325 raft_consensus.cc:1275] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:04:51.862185 15924 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:51.870448  8328 raft_consensus.cc:1240] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 13:04:51.856746 15629 raft_consensus.cc:697] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 19 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:51.872056  8325 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 24. Current term is 25. Ops: []
I20260709 13:04:51.863219 15813 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:51.874765  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 81215.9us
I20260709 13:04:51.876828 15954 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:51.878981  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.868501 16068 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:51.872828 15553 consensus_queue.cc:1048] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 13:04:51.882936 15924 consensus_queue.cc:1059] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 13:04:51.900871  8325 raft_consensus.cc:1240] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 22 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:51.879249 15813 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:51.894680 15629 consensus_queue.cc:237] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:51.918361  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:51.918637  9002 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56954 (request call id 1313) took 2574 ms (client timeout 1740 ms). Trace:
I20260709 13:04:51.919095  8406 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 20.
W20260709 13:04:51.919092  9002 rpcz_store.cc:269] 0709 13:04:49.344455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.344640 (+   185us) service_pool.cc:224] Handling call
0709 13:04:51.918366 (+2573726us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54007198d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:51.918536 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3308468608}
I20260709 13:04:51.914486 15841 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:51.923357 15553 consensus_queue.cc:1059] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:51.924232 15553 consensus_queue.cc:1243] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer ff7a6fe533e94c1bae555b4cae427728 log is divergent from this leader: its last log entry 22.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:04:51.924697 15553 raft_consensus.cc:3055] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:51.925014 15553 raft_consensus.cc:740] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:51.925896 15553 consensus_queue.cc:260] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:51.926370 15813 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:51.927295 15553 raft_consensus.cc:3060] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:51.914485 15965 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:51.928746 15965 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:51.930858  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:51.931569  9002 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 22.
I20260709 13:04:51.932721  8325 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:51.937806  9003 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:04:51.938951  8324 raft_consensus.cc:1240] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:51.913875 15581 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:51.949780 16023 consensus_queue.cc:1059] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:04:51.953755 16023 raft_consensus.cc:3055] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:51.954198 16023 raft_consensus.cc:740] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:51.955029 16023 consensus_queue.cc:260] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:51.955821  8297 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:51.945712 15965 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:51.956228 16023 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:51.956940 16080 log.cc:927] Time spent T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:04:51.957201 15997 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:51.958050 16072 log.cc:927] Time spent T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:04:51.963043  8369 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:51.965994  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.970263  8257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55264 (request call id 2832) took 1223 ms. Trace:
W20260709 13:04:51.970863  8223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.969718  8369 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:51.977123  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.980775  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:51.981277  9002 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:51.987588  9007 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:04:51.989279  9006 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 24. Current term is 25. Ops: []
I20260709 13:04:51.970691  8257 rpcz_store.cc:276] 0709 13:04:50.746992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.752009 (+  5017us) service_pool.cc:224] Handling call
0709 13:04:50.752346 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:50.799514 (+ 47168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.799561 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:51.490229 (+690668us) spinlock_profiling.cc:243] Waited 4.18 ms on lock 0x7b2c000014c8. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:04:51.490707 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 13:04:51.718127 (+227420us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:51.718180 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.718518 (+   338us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:51.961504 (+242986us) raft_consensus.cc:1685] finished
0709 13:04:51.961523 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:51.970198 (+  8675us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46740,"raft.queue_time_us":46858,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":25068,"spinlock_wait_cycles":9622400,"thread_start_us":227048,"threads_started":1,"wal-append.queue_time_us":346232}
I20260709 13:04:51.986796 15901 consensus_queue.cc:1048] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.011s
I20260709 13:04:51.974037 15965 raft_consensus.cc:2804] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:04:51.997586 15992 consensus_queue.cc:1048] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.003s
I20260709 13:04:51.991128 16068 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:52.001722 16071 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.007166 15690 consensus_queue.cc:1059] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:04:51.949832 15581 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:52.012362  8332 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 22 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:51.970214 16076 consensus_queue.cc:1059] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.029s
I20260709 13:04:52.007229 15795 raft_consensus.cc:3055] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:04:52.018684 16068 raft_consensus.cc:697] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 23 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:52.024271  9003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56940 (request call id 1461) took 1259 ms. Trace:
I20260709 13:04:52.019261 15965 raft_consensus.cc:697] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 27 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:52.014554 16016 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:52.017751  8331 raft_consensus.cc:1240] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 24. Current term is 25. Ops: []
I20260709 13:04:52.028312  9003 rpcz_store.cc:276] 0709 13:04:50.760663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.934776 (+174113us) service_pool.cc:224] Handling call
0709 13:04:50.935191 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:51.938235 (+1003044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:52.020014 (+ 81779us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400533254. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:52.020193 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2304971648}
W20260709 13:04:52.029670  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.019681 16071 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:52.027498 15795 raft_consensus.cc:740] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 24 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:52.043814 15988 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 25 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:52.044314 15795 consensus_queue.cc:260] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:52.045660 15795 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:04:52.050729 16081 log.cc:927] Time spent T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:04:52.040287 15924 consensus_queue.cc:1059] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:04:52.058025 16078 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.200s	user 0.002s	sys 0.000s
I20260709 13:04:52.059407 16071 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:52.063642  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:52.064158 15861 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:52.064838 15706 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:52.064958 15916 consensus_queue.cc:1048] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:52.065485 15706 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:52.065896  8329 raft_consensus.cc:1240] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 13:04:52.067109 15916 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:52.067695 15776 raft_consensus.cc:3055] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:52.068113 15776 raft_consensus.cc:740] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:52.068560 15916 consensus_queue.cc:1048] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:04:52.032685 16016 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:52.073813  8369 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:52.074205 16016 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:52.075248 15944 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:52.075989 15938 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:52.076607 15967 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:52.075047 16068 consensus_queue.cc:237] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:52.079522  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.079401  8331 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:04:52.082537  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:52.083135  8329 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 22.
W20260709 13:04:52.088165  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 44559.9us
I20260709 13:04:52.090646  8323 raft_consensus.cc:1240] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:04:52.086912  8974 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:52.092924  8974 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:52.097299 16071 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:52.097796 16071 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.098165 16071 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:52.109452  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:52.109141  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:52.110553  8329 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:52.120177  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:52.120210  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:52.122653  8323 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 25.
I20260709 13:04:52.109609 15690 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:52.124249 15706 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:52.065593 15965 consensus_queue.cc:237] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:52.113055 15967 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:52.162341  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:52.121055 15924 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.164418  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:52.157763 15962 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.169028  8329 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
W20260709 13:04:52.169215  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.186542  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:04:52.168838  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.192529  8401 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:04:52.120688 15861 consensus_queue.cc:1059] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:04:52.191536  8222 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:52.194872  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:52.157364 15581 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:52.199496  8222 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:52.200650  8401 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:04:52.161018 15980 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.205675  8222 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:52.206946  8399 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:04:52.215301  8403 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 25.
I20260709 13:04:52.216543  8256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49220 (request call id 1315) took 1932 ms. Trace:
I20260709 13:04:52.217069  8256 rpcz_store.cc:276] 0709 13:04:50.283651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.370512 (+ 86861us) service_pool.cc:224] Handling call
0709 13:04:50.480998 (+110486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:50.498510 (+ 17512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:52.213963 (+1715453us) spinlock_profiling.cc:243] Waited 95.9 ms on lock 0x7b5400308750. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:52.216447 (+  2484us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":220460928}
I20260709 13:04:52.200919 15921 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:52.225893  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:52.226600  8327 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 25.
I20260709 13:04:52.244387  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:52.208638 15644 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:52.206198 15581 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:52.215320  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:52.193832 15962 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:52.215728 16015 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:52.293823  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:52.300683  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:52.288108 16085 log.cc:927] Time spent T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 13:04:52.213380 15980 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:52.315893  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:52.325670  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:52.312085  8328 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:52.287822 15795 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:52.318315  8329 raft_consensus.cc:1240] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 24. Current term is 25. Ops: []
I20260709 13:04:52.333766  8329 raft_consensus.cc:1275] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:04:52.299154 16077 log.cc:927] Time spent T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:04:52.207846 15924 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:52.358915  8328 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:04:52.222991 16016 consensus_queue.cc:1059] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.087s
I20260709 13:04:52.224977 15861 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.087s
I20260709 13:04:52.323668  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:52.331036  8327 raft_consensus.cc:2393] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 22: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:04:52.326352 16087 log.cc:927] Time spent T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.244s	user 0.000s	sys 0.001s
I20260709 13:04:52.293097 15581 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:52.388839  8328 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: []
W20260709 13:04:52.390563  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.355820 15795 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:52.399160  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 185343us
I20260709 13:04:52.401576  9000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 1601) took 1717 ms. Trace:
I20260709 13:04:52.402179  9000 rpcz_store.cc:276] 0709 13:04:50.683962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.889524 (+205562us) service_pool.cc:224] Handling call
0709 13:04:50.922181 (+ 32657us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:50.922428 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 13:04:50.948827 (+ 26399us) log.cc:844] Serialized 14 byte log entry
0709 13:04:52.309670 (+1360843us) spinlock_profiling.cc:243] Waited 2.75 ms on lock 0x7b180038b3c0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e23a 00007f88287c8a2a
0709 13:04:52.401074 (+ 91404us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:52.401152 (+    78us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:04:52.401220 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:52.401308 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:52.401504 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":26181,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":2620,"raft.queue_time_us":97,"spinlock_wait_cycles":6320000,"thread_start_us":91070,"threads_started":1}
I20260709 13:04:52.409221 15553 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:52.412081  8222 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:52.412830 15553 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:52.420512 15980 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:52.437176 15944 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.414441 15644 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 13:04:52.449193  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:52.451309  8403 raft_consensus.cc:2393] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 22: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:52.464340 15536 raft_consensus.cc:3060] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:52.452380 15971 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.474846 15971 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:52.479876 15924 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:52.479926  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:52.480909  8330 raft_consensus.cc:2376] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 25. Re-sending same reply.
W20260709 13:04:52.470100  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.419602 15553 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.065s
I20260709 13:04:52.443190 15841 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:52.452524 15944 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:52.494243  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.491807  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:52.485905 15581 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:52.498942 16023 consensus_meta.cc:287] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.542s	user 0.009s	sys 0.002s
I20260709 13:04:52.500228 15966 raft_consensus.cc:3055] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:04:52.500654 15966 raft_consensus.cc:740] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:52.502064 16086 raft_consensus.cc:3055] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:52.502583 16086 raft_consensus.cc:740] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:52.503628 16086 consensus_queue.cc:260] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:52.494356  8972 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
W20260709 13:04:52.515306  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.520474  8223 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:52.522446  8223 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:52.522087  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:52.524866  8223 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:52.536087  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.508637 16086 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:52.551499  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:52.552304  9000 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 25.
I20260709 13:04:52.546828  8972 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:52.524139 16067 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:52.526284 16023 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:52.527148 15967 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 28 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:04:52.565188  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:52.508980 15841 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:52.547075 15962 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:52.550190 15971 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:52.580516  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:52.576663  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:52.578058  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:52.535494 16071 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:52.586861  8372 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:52.548918 15553 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:04:52.560429 15924 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.019s
W20260709 13:04:52.601861  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 55128.4us
I20260709 13:04:52.572918 16067 raft_consensus.cc:697] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:04:52.602485  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 58132.4us
I20260709 13:04:52.524747 15966 consensus_queue.cc:260] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:52.607456  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:52.609135  8295 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 24 similar messages]
I20260709 13:04:52.661032  9008 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
W20260709 13:04:52.666051  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 100841us
I20260709 13:04:52.603065 15795 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:52.672178  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 85444.7us
I20260709 13:04:52.613560 15690 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:52.682834 15997 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.688246 15997 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:52.690980 15997 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:52.692243  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:52.693177  8398 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
I20260709 13:04:52.694417  8297 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:52.710243 15944 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:52.712900  8403 raft_consensus.cc:3055] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:52.713367  8403 raft_consensus.cc:740] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:52.679265 15449 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:52.683985 15690 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:52.716639  8403 consensus_queue.cc:260] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:52.685365 16097 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:04:52.759169 16067 consensus_queue.cc:237] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:52.770406 15988 consensus_queue.cc:1059] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:04:52.675079 15962 consensus_queue.cc:1059] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 13:04:52.782564 15962 raft_consensus.cc:3055] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:52.783051 15962 raft_consensus.cc:740] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:52.784030 15962 consensus_queue.cc:260] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:52.718407 15938 consensus_queue.cc:1048] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 13:04:52.758476 15690 raft_consensus.cc:3060] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:52.745072 15966 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:52.803774  8403 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:52.798902 15962 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:52.809777 16097 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:04:52.830454  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 213964us
W20260709 13:04:52.829726 16015 consensus_meta.cc:287] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.527s	user 0.008s	sys 0.002s
W20260709 13:04:52.848419  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 189442us
I20260709 13:04:52.850633 16015 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:52.838303  8223 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:52.843271  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 234147us
I20260709 13:04:52.842999  9000 raft_consensus.cc:1240] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:04:52.849383  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:52.848869 15795 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:52.874118 15795 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:52.877070 15795 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:52.883785  8402 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:04:52.862840 15841 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:52.870126 16015 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:52.899024 16016 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:52.902261 15690 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:52.904924 15690 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:52.914670  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 232597us
W20260709 13:04:52.915944  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 242564us
I20260709 13:04:52.848592 16097 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:52.886070 15673 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 13:04:52.925313 15690 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:52.926981 15690 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:52.929149 15690 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:52.923874  8372 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:52.933944  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:52.933802  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 239477us
I20260709 13:04:52.934439 15753 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:52.941581 15975 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.941835 15753 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.942359 15753 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:52.942090 15975 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:52.944398 15975 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:52.945446  8330 raft_consensus.cc:1275] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:04:52.937153 15899 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.946583  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:52.946981  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:52.946805 16102 log.cc:927] Time spent T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:04:52.947599  8326 raft_consensus.cc:2468] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 20.
I20260709 13:04:52.946610 15899 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:52.950379  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:52.885828 16020 consensus_queue.cc:1059] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:04:52.954090  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:52.968923  8372 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:04:52.973057  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 56468.7us
I20260709 13:04:52.953562 15899 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:52.914357 15553 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.055s
I20260709 13:04:52.914776 16023 consensus_queue.cc:1059] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.054s
W20260709 13:04:52.986210 16105 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 13:04:52.955555 15841 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:04:52.991721  8372 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:52.974671 15795 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 13:04:53.002328 16020 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.021509  8403 raft_consensus.cc:1275] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 13:04:53.021962  8403 pending_rounds.cc:85] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 10
I20260709 13:04:53.022213  8403 pending_rounds.cc:107] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 18.11
W20260709 13:04:53.001530 15536 consensus_meta.cc:287] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.536s	user 0.012s	sys 0.000s
I20260709 13:04:52.995134 15799 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.999079 15967 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.994936 16015 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:52.996160 16016 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
W20260709 13:04:53.075480  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.013842 15899 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:53.043850 15841 raft_consensus.cc:697] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 25 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:53.027932 15644 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.019340 16020 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:53.091995  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:53.084326 16105 log.cc:927] Time spent T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:04:53.077473 16071 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:53.141597  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 45071.7us
I20260709 13:04:53.135800  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57186 (request call id 3514) took 1418 ms. Trace:
I20260709 13:04:53.145992  8332 rpcz_store.cc:276] 0709 13:04:51.717422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.868047 (+150625us) service_pool.cc:224] Handling call
0709 13:04:51.868300 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:52.020420 (+152120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.135478 (+1115058us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b540037ad54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:53.135718 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":330817792}
I20260709 13:04:53.022449  8403 raft_consensus.cc:2889] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:53.081547 15536 raft_consensus.cc:3055] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:53.088116 16076 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:53.118166 16111 log.cc:927] Time spent T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:04:53.015162 15776 consensus_queue.cc:260] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.096534 16107 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:53.103201 15732 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:53.084853 15967 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:53.079416 15799 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.086400 16016 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.153707 15536 raft_consensus.cc:740] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:53.085323 16015 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.154979 15553 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 13:04:53.143890 15644 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.141989 16112 consensus_queue.cc:1048] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:04:53.175242 16115 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:53.201787 15536 consensus_queue.cc:260] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.214854 15799 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:53.211081 16075 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:53.217113 16015 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:53.171447 16076 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.192843 16119 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.204538 15776 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:53.097252 15899 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.178719 15732 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.232401  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 3172) took 1011 ms. Trace:
I20260709 13:04:53.183210 16107 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.209939 16118 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:53.239454  8325 rpcz_store.cc:276] 0709 13:04:52.221205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.225240 (+  4035us) service_pool.cc:224] Handling call
0709 13:04:52.294826 (+ 69586us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:52.295077 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:04:52.325595 (+ 30518us) log.cc:844] Serialized 14 byte log entry
0709 13:04:52.946003 (+620408us) spinlock_profiling.cc:243] Waited 5.6 ms on lock 0x7b180049edc0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e23a 00007f88287c8a2a
0709 13:04:53.231754 (+285751us) log.cc:844] Serialized 14 byte log entry
0709 13:04:53.231932 (+   178us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:53.232009 (+    77us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:53.232076 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.232168 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.232327 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":29726,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":3425,"raft.queue_time_us":1545,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":2527,"spinlock_wait_cycles":12883328,"thread_start_us":285296,"threads_started":1}
I20260709 13:04:53.222132 16097 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:53.230705 16119 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.263594 16119 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:53.264895 15753 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:53.204564 15841 consensus_queue.cc:237] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:53.261026 16086 consensus_meta.cc:287] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.699s	user 0.012s	sys 0.000s
W20260709 13:04:53.274489  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 281334us
I20260709 13:04:53.268146 15753 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:53.283829  8330 raft_consensus.cc:1275] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:04:53.284581  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:53.242851 15732 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:53.286597  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:53.283785  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:53.275163 16112 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.382637  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:53.408802  8324 raft_consensus.cc:2410] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 13:04:53.245092 16118 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:53.425554  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 405727us
I20260709 13:04:53.427147  8220 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:53.430274  8220 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:53.452929  8220 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:53.379501 15984 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.286765 15966 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:53.369303 16086 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.348984 15975 raft_consensus.cc:493] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:53.350287 15581 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:53.381409 15753 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.464705 15984 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:53.470983 16109 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.192s	user 0.000s	sys 0.000s
W20260709 13:04:53.471578  8369 leader_election.cc:336] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:53.471701 16121 log.cc:927] Time spent T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.267s	user 0.000s	sys 0.001s
I20260709 13:04:53.472769 15984 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:53.473312  8369 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:53.474570  8369 leader_election.cc:336] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.475015  8369 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:53.381815 16097 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.477113  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:53.480523  8329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 3186) took 1160 ms. Trace:
W20260709 13:04:53.482966  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 491389us
I20260709 13:04:53.484716  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:53.490839  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:53.491988  9000 raft_consensus.cc:2410] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 13:04:53.481335  8329 rpcz_store.cc:276] 0709 13:04:52.320524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.358116 (+ 37592us) service_pool.cc:224] Handling call
0709 13:04:52.366911 (+  8795us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:52.482315 (+115404us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:04:52.482649 (+   334us) log.cc:844] Serialized 14 byte log entry
0709 13:04:52.681111 (+198462us) log.cc:844] Serialized 14 byte log entry
0709 13:04:52.908386 (+227275us) spinlock_profiling.cc:243] Waited 3 ms on lock 0x7b1800217a20. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256d13f3 00007f88256d37c1 00007f88256d3779 00007f88256d3709 00007f88256d36d1 00007f88256d29cd 000000000038f354 000000000038f199 00007f88256b483a
0709 13:04:52.932696 (+ 24310us) spinlock_profiling.cc:243] Waited 48.3 ms on lock 0x7b1800217a20. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f882564bc0a 00007f882ed8b219 00007f88256d3a40 00007f88256ce4e4 00007f88256ce455 00007f88287b6060 00007f88287c8716 00007f88287d9287 00007f88287d9208
0709 13:04:52.932874 (+   178us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:52.932939 (+    65us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:52.933002 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.480192 (+547190us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b540014ffd4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:53.480261 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.480445 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":259548,"prepare.queue_time_us":691,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":78,"raft.queue_time_us":252409,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":6206,"spinlock_wait_cycles":382806912,"thread_start_us":196327,"threads_started":1,"wal-append.queue_time_us":350046}
I20260709 13:04:53.513727 16120 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:53.514470 16120 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.517095 16120 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:53.462630 15921 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:53.458590 16015 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:53.458926 15799 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:53.559020  8222 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:53.287822 15880 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.469779 16016 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:53.473337 15984 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.556327 15967 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:53.563975 16015 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:04:53.602404  8974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.932s	user 0.032s	sys 0.109s
W20260709 13:04:53.620498  8974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.950s	user 0.035s	sys 0.111s
W20260709 13:04:53.567868  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 422540us
I20260709 13:04:53.469002 15966 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:53.476411 16086 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.453486 16112 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.556960 15753 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:53.556629 15975 raft_consensus.cc:515] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.567179 16076 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:53.567342 15644 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:53.642361 15965 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.007s	sys 0.003s
W20260709 13:04:53.646456  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 183489us
I20260709 13:04:53.573923 15984 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:53.556730 15581 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.665830  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:53.622539 16015 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:53.678265  9003 raft_consensus.cc:2410] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:04:53.571774 15880 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:53.672283 15965 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.007s	sys 0.004s
I20260709 13:04:53.678073  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:53.640648 16119 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:53.646189 16116 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:53.682240  9006 raft_consensus.cc:1275] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:04:53.562883 16097 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.684244  9007 raft_consensus.cc:1275] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 13:04:53.669204  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 47157.1us
I20260709 13:04:53.669627 15968 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:53.685091 16131 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:53.698467 16128 log.cc:927] Time spent T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:04:53.688285 16107 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:53.707094  8323 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:04:53.679435 15629 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:53.701673  8220 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:53.700559 16134 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:53.700410 16132 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.715224  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3747) took 1144 ms. Trace:
I20260709 13:04:53.699988 16130 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:53.709865 15967 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:53.698776 15968 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.695307 16116 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.722581 15799 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.750257 15975 leader_election.cc:290] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:53.750798 15753 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:53.751542 15984 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:53.751922 15581 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:53.752590 15449 consensus_meta.cc:287] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 1.007s	user 0.012s	sys 0.000s
I20260709 13:04:53.752983 15449 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:53.725494 16132 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:53.754972 15776 consensus_meta.cc:287] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.521s	user 0.007s	sys 0.002s
I20260709 13:04:53.722950 16016 raft_consensus.cc:2804] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:53.755623 16016 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.756004 16016 raft_consensus.cc:3060] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:53.757617 16141 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.757632 15799 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:53.758124 16141 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.723148 16127 consensus_queue.cc:1059] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:04:53.760782 16127 consensus_queue.cc:1243] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer ff7a6fe533e94c1bae555b4cae427728 log is divergent from this leader: its last log entry 22.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
W20260709 13:04:53.761078 16125 log.cc:927] Time spent T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:04:53.761270 16127 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.761888 16127 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:53.760293 16129 log.cc:927] Time spent T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:04:53.765534 16124 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.766152 16124 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.767148 16118 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:53.766839 16015 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:53.768261 16112 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:53.757300 15732 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.772872 15732 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:53.719815 15629 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:53.773717 15629 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:04:53.774315 15962 consensus_meta.cc:287] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.943s	user 0.012s	sys 0.000s
I20260709 13:04:53.775274 16086 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:53.775955 15899 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:53.776614 16143 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.777109 16143 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:53.724191 16134 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.778657 16016 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:53.779225 16143 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:53.724768  8406 rpcz_store.cc:276] 0709 13:04:52.570711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.579274 (+  8563us) service_pool.cc:224] Handling call
0709 13:04:52.603408 (+ 24134us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:52.833954 (+230546us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:52.833989 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:52.860979 (+ 26990us) log.cc:844] Serialized 26 byte log entry
0709 13:04:53.170400 (+309421us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:53.170450 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.170540 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:53.715009 (+544469us) raft_consensus.cc:1685] finished
0709 13:04:53.715029 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.715167 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69054,"raft.queue_time_us":69431,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":6713,"thread_start_us":470104,"threads_started":2,"wal-append.queue_time_us":779764}
W20260709 13:04:53.783324  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.783939  8397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39308 (request call id 3746) took 1293 ms. Trace:
I20260709 13:04:53.788384  8397 rpcz_store.cc:276] 0709 13:04:52.490062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.490182 (+   120us) service_pool.cc:224] Handling call
0709 13:04:52.490520 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:52.490856 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:52.490886 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:52.495155 (+  4269us) log.cc:844] Serialized 26 byte log entry
0709 13:04:52.942883 (+447728us) spinlock_profiling.cc:243] Waited 59.7 ms on lock 0x7b18003b8ea0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88256b3e07 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256d0c02 00007f882835820b 00007f882835dd23 00007f882835e0d1 00007f8828779a0f
0709 13:04:53.192501 (+249618us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:53.192544 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.192613 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:53.739678 (+547065us) raft_consensus.cc:1685] finished
0709 13:04:53.739699 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.783871 (+ 44172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4771,"raft.queue_time_us":3247,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":13668,"spinlock_wait_cycles":137332480,"thread_start_us":249138,"threads_started":1,"wal-append.queue_time_us":1037917}
W20260709 13:04:53.786464  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 227537us
I20260709 13:04:53.779822 16134 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:53.781984 15795 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.014s	sys 0.007s
I20260709 13:04:53.801929  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:53.804081  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:53.811873  8402 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:53.780092 15992 raft_consensus.cc:3055] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:04:53.817746  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:53.817898  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:53.797601 15962 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.797015 15581 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:53.799449 16118 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:53.739140 16130 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:53.815766 15795 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.014s	sys 0.007s
W20260709 13:04:53.835332  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 360779us
W20260709 13:04:53.836439  8369 leader_election.cc:336] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.824561  8399 raft_consensus.cc:2393] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 22: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:53.799732 16143 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.793972 15921 raft_consensus.cc:697] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:53.830735  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:53.842005  9006 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:53.834221  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:53.820075 15944 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.815730 16068 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.809692 16131 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:53.798693 15899 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:53.816468 15992 raft_consensus.cc:740] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 26 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
W20260709 13:04:53.843860  8369 leader_election.cc:336] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:53.845396  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.845577  8369 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:53.856746  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:53.857411  8399 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:53.874747  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:53.859336  8402 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 21.
I20260709 13:04:53.884807  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:53.835474 15962 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:53.893841  8406 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:04:53.844350 16140 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:53.905910  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:53.836203 16118 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.915259  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:53.915480  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:53.899205 16142 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:04:53.836303 15581 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.857941 15975 raft_consensus.cc:493] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:53.906840  8327 raft_consensus.cc:2468] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
I20260709 13:04:53.916903  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:53.917220  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 120707us
I20260709 13:04:53.927326  8398 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:53.921299  8297 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:53.918361  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:53.918665  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:53.916365  8401 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 18.
I20260709 13:04:53.892621 15732 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:53.931542  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:53.931543  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:53.932687  8399 raft_consensus.cc:2468] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 21.
I20260709 13:04:53.924237  8329 raft_consensus.cc:1240] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:53.935604  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:53.936378  8329 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 21.
I20260709 13:04:53.898583 16112 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:53.895176 16148 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:53.847774 15944 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:53.937386  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:53.939332  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:53.858603 15899 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:53.856030 16068 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:53.858827 16143 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:53.917392 16139 raft_consensus.cc:493] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:53.858711 16131 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:53.961941 16153 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:53.910348 16150 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:53.912585 15753 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:53.952943  8327 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 25.
I20260709 13:04:53.900055 15921 consensus_queue.cc:237] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:53.964165  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 120367us
I20260709 13:04:53.973734  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:53.981338  8369 leader_election.cc:336] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.960887 15776 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:53.981900  8369 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:53.978616  8972 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:53.996994  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 95007.2us
W20260709 13:04:54.003608  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 71268.8us
I20260709 13:04:53.974123 16016 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:53.917052 16140 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.937995 15975 raft_consensus.cc:515] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:54.015093 16130 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:54.016474 16068 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:54.016595 16131 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:54.017565 16147 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.076s
I20260709 13:04:53.954996 16112 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.932884 15992 consensus_queue.cc:260] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:54.065922  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 144699us
I20260709 13:04:54.010814 16133 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:54.074054  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 82691.2us
I20260709 13:04:53.982992 15753 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:53.962100 16139 raft_consensus.cc:515] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:54.069796  8220 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:53.997150 15967 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 13:04:53.959183 16148 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:54.081574  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.990922 15776 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:54.089350  8220 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:53.977931 16153 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:54.055495 15673 raft_consensus.cc:3055] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:54.019373 15629 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:54.109879 16160 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:54.112279  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 114464us
W20260709 13:04:54.117741  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.977959 16150 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:54.092038 15992 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:54.090801 15799 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:54.104772 16147 raft_consensus.cc:2804] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:54.132740 16147 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.133296 16147 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:54.138926 15975 leader_election.cc:290] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:54.139683 15975 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:54.140134 15975 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.140547 15975 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:54.127523 15753 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:54.128352 16153 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:54.130393 16150 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:54.130774 16139 leader_election.cc:290] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:54.132149 15629 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:54.106925 15673 raft_consensus.cc:740] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:54.178472  8297 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:54.180823  8369 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:54.132627 15776 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:54.185583  8369 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:54.152245 15962 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:54.191552  8297 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:54.169450 15581 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:54.169407 15753 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:54.173170 16150 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:54.110934 16160 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:54.213871  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:54.217082  8328 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 22.
I20260709 13:04:54.195191 16131 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:54.196079 16068 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:54.221416  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:54.186993 15938 consensus_queue.cc:1048] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:04:54.225162  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:54.235558  9006 raft_consensus.cc:2468] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 26.
I20260709 13:04:54.220551  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:54.251391  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:54.251777  8328 raft_consensus.cc:1275] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 13:04:54.252137  8398 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 22.
I20260709 13:04:54.230703  8324 raft_consensus.cc:2468] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 22.
W20260709 13:04:54.253486  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.255486  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:54.257925  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:54.216964 16154 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.215303 16118 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:54.256378  8972 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:54.264233  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:54.261389  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:54.218503 15899 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:54.270596  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:54.272189  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:54.279636  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:54.264914  8328 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:04:54.248814 16143 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:54.280526  8405 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 21.
I20260709 13:04:54.215058 15753 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:54.275565  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:54.217258 16150 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:54.290939  8331 raft_consensus.cc:3060] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:54.292739  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:54.307755  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:54.308121  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:54.308869  8324 raft_consensus.cc:2393] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 25: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:04:54.310650 16174 log.cc:927] Time spent T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:04:54.219894 15673 consensus_queue.cc:260] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:54.315824 15673 raft_consensus.cc:3060] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:54.293681  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:54.267697 16156 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.317543 16156 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:54.319131  8323 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57202 (request call id 3175) took 2096 ms (client timeout 2145 ms). Trace:
W20260709 13:04:54.319593  8323 rpcz_store.cc:269] 0709 13:04:52.223025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.223175 (+   150us) service_pool.cc:224] Handling call
0709 13:04:53.476983 (+1253808us) spinlock_profiling.cc:243] Waited 3.61 ms on lock 0x7b58000e3498. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f8825699eee 0000000000375939 00007f882d70d1e0 00007f882d7d60c4 00007f882d7d6131 00007f882d7840a0 00007f882d783984 00007f8826cfe6d4 00007f8826cfe662
0709 13:04:54.318956 (+841973us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b5400364554. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:54.319070 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":469311360}
I20260709 13:04:54.320168 16156 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:54.293769 16168 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:54.321352 16168 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.321798 16168 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:54.322710  8147 catalog_manager.cc:5697] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 15 to 16, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 16 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:54.294088 15581 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:54.325067 15581 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:54.325937 16075 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:04:54.326191 16159 log.cc:927] Time spent T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:04:54.326442 16075 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:54.325706  8147 catalog_manager.cc:5697] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 11 to 17, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 17 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:54.295199 16165 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 13:04:54.295410 16158 log.cc:927] Time spent T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:04:54.295356 16112 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:54.300133 15753 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:54.300944 16152 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:54.260176 16147 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:54.301688 16140 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:54.303037 16160 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:54.295502 16148 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:54.297192  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:54.296169 16179 log.cc:927] Time spent T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:04:54.277204 16154 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:54.342238 16154 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:54.271152 16176 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:54.343357 16176 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.343811 16176 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:54.346709 16015 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 26 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:54.348629  8298 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:54.351539  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.337925  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:54.337673  8405 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 17.
I20260709 13:04:54.282626 16067 raft_consensus.cc:2804] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:54.333410 16165 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.354779  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:54.358939  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.334254  8328 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:04:54.335278 16152 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:54.374558  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:54.375401  8406 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 21.
I20260709 13:04:54.376906  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:54.379200  8369 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:54.363452  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.373456  8331 raft_consensus.cc:1275] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:04:54.374466 16153 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:54.404721  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:54.378198 16152 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:54.372843 16067 raft_consensus.cc:697] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 22 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:54.370225 16165 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:54.417932  8369 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:54.426324  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81804f59e3124d27998ea80fc1c36663" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:54.419596  8369 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:54.429725  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:54.431285  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.404654 16015 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.405915 15938 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:54.382716 16160 consensus_queue.cc:1048] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 13:04:54.407208 15581 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:54.438409  8369 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:54.384644 16118 consensus_queue.cc:1048] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:04:54.385309 15997 consensus_queue.cc:1048] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:04:54.444638  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:54.435609 16130 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:54.438050 16068 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:54.455399 15938 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:54.464757 16130 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.507468  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:54.443506 16015 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:54.568451  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:54.568766 16186 log.cc:927] Time spent T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 13:04:54.570246  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0493b0dbbc746408a5db9e698678d83" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:54.570595  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766ac43473084aa6a9fdce530520947c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:54.571448  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038cd0877a674c2996410c9f327df911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:54.572018  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:54.573669 16191 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.574482  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954d2ad09dcb4697adccd661625a841b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:54.574304 16191 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:54.576828 16191 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:54.585326  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:54.484129 15938 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.591982  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:54.582722  8974 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:54.467818 15813 consensus_queue.cc:1048] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.100s
I20260709 13:04:54.488090 16130 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:54.589824  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:54.484270 16068 raft_consensus.cc:697] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 18 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:04:54.609942 16173 log.cc:927] Time spent T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
W20260709 13:04:54.617328  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.619616 15962 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:54.620127 15962 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:54.623550  8147 catalog_manager.cc:5697] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 12 to 13, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 13 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:54.625195  8147 catalog_manager.cc:5697] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 6 to 11, term changed from 18 to 25, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to <none>, VOTER 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) evicted. New cstate: current_term: 25 committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:54.626717  8147 catalog_manager.cc:5697] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e reported cstate change: config changed from index 5 to 12, term changed from 14 to 15, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to <none>, VOTER ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) evicted. New cstate: current_term: 15 committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:54.628263  8147 catalog_manager.cc:5697] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:54.630096  8147 catalog_manager.cc:5697] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 13 to 15, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 15 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:54.632007  8147 catalog_manager.cc:5697] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 13 to 18, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 18 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:54.633616  8147 catalog_manager.cc:5697] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:54.635120  8147 catalog_manager.cc:5697] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:54.638424 16181 log.cc:927] Time spent T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.231s	user 0.000s	sys 0.003s
I20260709 13:04:54.642517 15916 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:54.637905 16068 consensus_queue.cc:237] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:54.482462 15975 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:54.591125 16150 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:54.594837 15938 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:54.598213 16147 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:54.654765  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 207025us
W20260709 13:04:54.655341  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.661656 16124 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:54.663779 16015 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:54.561887 16067 consensus_queue.cc:237] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:54.673947 15975 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:54.645358 16118 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:54.674935 16118 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:54.675622 16184 log.cc:927] Time spent T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.211s	user 0.000s	sys 0.002s
I20260709 13:04:54.650197 16146 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.680341 16146 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:54.682301 15813 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.682590 16146 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:54.682837 15813 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:54.685036 15813 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:54.698308 16170 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:54.698975 16170 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:54.649698 16176 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:54.703086 16176 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:54.702987  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:54.703874  8399 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:54.703843  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:54.706079  8406 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:54.705099 15962 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:54.711277 16075 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:54.713943 16075 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:54.716786 15992 consensus_meta.cc:287] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: Time spent flushing consensus metadata: real 0.574s	user 0.010s	sys 0.000s
I20260709 13:04:54.677191 16118 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:54.722658  8399 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 19.
I20260709 13:04:54.723011  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:54.746596  8328 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
I20260709 13:04:54.711192 16165 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:54.737234  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:54.752127  8323 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 25.
I20260709 13:04:54.737555  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:54.688980 16130 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:54.739244  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:54.747833  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:54.753182  8405 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 22.
I20260709 13:04:54.754735  8328 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:54.732299  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:54.763932 16020 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:54.766934 16141 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:54.769726 16116 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:54.771211 16097 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:54.779744 15938 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:54.783444  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:54.786221  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:54.749095  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:54.790316  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.790928  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:54.792749  8323 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
I20260709 13:04:54.751067  9006 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 27.
I20260709 13:04:54.786325  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:54.808348  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:54.788480  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:54.809412  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:54.809739  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:54.804322 16127 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:54.800738 15983 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:54.801018 16146 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:54.807533  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:54.810238  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:54.811057  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:54.826243  8323 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:54.826820  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:54.813643  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:54.813882  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:54.748690  8406 raft_consensus.cc:2468] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 20.
W20260709 13:04:54.815049  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 134426us
I20260709 13:04:54.739950 16170 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:54.815776  8328 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 26.
I20260709 13:04:54.830765  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:54.815547  8405 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 25. Current term is 26. Ops: [25.19-25.19]
I20260709 13:04:54.808894  8329 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
I20260709 13:04:54.831805  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:54.831876  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:54.832584  8328 raft_consensus.cc:2393] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 25: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:54.832593  8329 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 25.
I20260709 13:04:54.750137  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:54.833465  8324 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
I20260709 13:04:54.833567  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:54.829164  8295 leader_election.cc:336] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.645s (SENT)
W20260709 13:04:54.835807  8295 leader_election.cc:336] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.588s (SENT)
I20260709 13:04:54.836309  8295 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:54.834457  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:54.838657  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:54.840391  8220 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:54.773118 16152 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:54.843616  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:54.843914  8220 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:54.844313  8406 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:54.845216  8220 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:54.846717  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:54.847383  8328 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
I20260709 13:04:54.848263  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:54.850102  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:54.851270  8330 raft_consensus.cc:2393] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 21: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:54.851939  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:54.854168  8328 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:04:54.854789  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:54.856190  8401 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 25. Current term is 26. Ops: []
I20260709 13:04:54.824888 16146 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.824447 15983 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.860280  8330 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 25. Current term is 26. Ops: [25.15-25.15]
I20260709 13:04:54.861284  8328 raft_consensus.cc:1217] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 22 LEADER]: Deduplicated request from leader. Original: 18.13->[19.14-21.15]   Dedup: 19.14->[21.15-21.15]
I20260709 13:04:54.862461  8220 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:54.872478 15938 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:54.873153 16165 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 26 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:54.860150  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:54.889452  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:54.901126  8330 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:54.839893 15975 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:54.853853 15924 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:54.854575 16097 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:54.902539  8397 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 19. Current term is 20. Ops: [19.11-19.11]
W20260709 13:04:54.875118 16168 consensus_meta.cc:287] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.553s	user 0.007s	sys 0.004s
W20260709 13:04:54.911870  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.875168  8328 raft_consensus.cc:1240] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 22 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 21. Current term is 22. Ops: [19.14-21.15]
I20260709 13:04:54.856668 16141 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:54.914094 16141 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.914469 16141 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:54.911528 16097 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.916378  8323 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 26.
I20260709 13:04:54.894120 16146 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:54.891390 15983 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:54.898666 16020 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:54.923800  8330 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 26.
I20260709 13:04:54.925917  8220 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:54.927030 16165 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:04:54.905040 15924 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.927791 16165 raft_consensus.cc:697] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 26 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:54.930670 16130 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:54.931638  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:54.921900 16097 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:54.938647  8406 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 24.
I20260709 13:04:54.942313 16165 consensus_queue.cc:237] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:54.943496 16133 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:54.943899 16133 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:54.944272 16133 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:54.932677  8372 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:54.939265  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:54.946521  8399 raft_consensus.cc:2393] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 21: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:04:54.947957  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.924861 16020 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.911888 16147 consensus_queue.cc:1059] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
W20260709 13:04:54.950768  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 368105us
I20260709 13:04:54.953722  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:54.950937 16130 raft_consensus.cc:2804] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:04:54.961901 16130 raft_consensus.cc:697] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 26 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:54.962783 16130 consensus_queue.cc:237] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:54.916622 16015 consensus_queue.cc:1059] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 13:04:54.936342 15924 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:54.954545  8974 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:54.917172 16168 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:54.973908  8298 leader_election.cc:304] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:54.976861  8298 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:54.941335 16118 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:04:54.983024  8399 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:54.952275 16020 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:54.991546  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07268be9b33649e8b0e8bddf7a3bcabe" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:54.987488 16170 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 13:04:55.001407  8220 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:55.002691 16191 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:04:55.001601 16015 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:55.003762 16015 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:55.004202  8325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 1255) took 1103 ms. Trace:
I20260709 13:04:55.004551  8325 rpcz_store.cc:276] 0709 13:04:53.900613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.900750 (+   137us) service_pool.cc:224] Handling call
0709 13:04:53.901036 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.066665 (+165629us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:54.066701 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:54.066744 (+    43us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:04:54.066775 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.003987 (+937212us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b540037b254. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:55.004031 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:55.004149 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":380458240}
W20260709 13:04:55.003263  8295 leader_election.cc:336] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.679s (SENT)
I20260709 13:04:55.006492  8295 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:55.012192  8295 leader_election.cc:336] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.746s (SENT)
I20260709 13:04:55.012467  8329 raft_consensus.cc:2393] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 19: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:55.012660  8295 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:55.014858  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:54.985461 16196 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:54.981455 15975 raft_consensus.cc:2749] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:04:55.019788  8295 leader_election.cc:336] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.786s (SENT)
I20260709 13:04:55.022451  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:54.994398 15921 raft_consensus.cc:3055] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:04:55.024519  8295 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:55.025058 15921 raft_consensus.cc:740] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:55.025710 16170 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:55.026062 15921 consensus_queue.cc:260] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:55.027765 15921 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:55.018035 16160 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:55.038077  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:55.038905  8399 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
I20260709 13:04:55.022092 16178 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:04:55.054621  8295 leader_election.cc:336] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.914s (SENT)
I20260709 13:04:55.055320  8295 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:55.030709 16196 raft_consensus.cc:697] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 21 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:55.063596 16196 consensus_queue.cc:237] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:55.067830 16152 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:55.032672 16015 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:55.074352  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:55.076804  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:55.037757 16185 log.cc:927] Time spent T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.564s	user 0.000s	sys 0.003s
I20260709 13:04:55.089673  8222 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:55.102576  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 48043.4us
I20260709 13:04:55.107932 15673 consensus_queue.cc:1059] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:04:55.086961  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.114260  8222 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:55.117110 15673 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:55.118666 15944 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:55.101709 16147 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:55.122875 15975 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:04:55.137282  8404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34462 (request call id 1393) took 1335 ms. Trace:
I20260709 13:04:55.137871  8404 rpcz_store.cc:276] 0709 13:04:53.801392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.806485 (+  5093us) service_pool.cc:224] Handling call
0709 13:04:53.806731 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.941683 (+134952us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:53.941730 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:53.941789 (+    59us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:53.941834 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.137034 (+1195200us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b54003e6f54. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bee01 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6
0709 13:04:55.137084 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:55.137226 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":309782784}
I20260709 13:04:55.123039 16015 consensus_queue.cc:1059] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.013s
I20260709 13:04:55.185987 16015 raft_consensus.cc:3055] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:04:55.186574 16015 raft_consensus.cc:740] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:55.193369  8297 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:55.195549  8297 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:55.196808  8297 leader_election.cc:304] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:55.197855  8297 leader_election.cc:304] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:55.202036  8297 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:55.203349  8297 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:55.204595 16170 raft_consensus.cc:2749] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:55.202442 15975 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:55.207965 16170 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:55.206724 15975 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:55.211537 15975 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:55.211676 15880 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:55.201297 15984 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:55.202342 16178 raft_consensus.cc:2749] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:55.207527 16115 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:55.213497 16176 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:55.187379 16097 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:55.231215  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:55.231550  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:55.230801  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.231015 16115 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:55.229490 15966 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:55.240300  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:55.240492 16132 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:55.250293  8223 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.345s	user 0.042s	sys 0.065s
W20260709 13:04:55.250761  8223 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.345s	user 0.042s	sys 0.065s
I20260709 13:04:55.219442 16191 raft_consensus.cc:697] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 26 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:55.251097  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 2.35905e+06us
I20260709 13:04:55.255774  8222 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:55.195779 16015 consensus_queue.cc:260] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:55.274421  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:55.232406 16176 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:55.272651  8222 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:55.275930  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 415855us
I20260709 13:04:55.271082  8372 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:55.281101  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:55.281955  8406 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
I20260709 13:04:55.282814  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:55.283507  8325 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:55.284422  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:55.285163  8326 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:55.287132  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:55.290405  9002 raft_consensus.cc:1240] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 25. Current term is 26. Ops: [25.19-25.19]
I20260709 13:04:55.292520  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:55.227799 16133 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:55.300642  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:55.291625  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:55.245302 16118 raft_consensus.cc:697] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 26 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:55.302908  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
W20260709 13:04:55.307700  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.310891 16203 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:55.311493 16203 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:55.313828  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:55.314085  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:55.314954 16205 log.cc:927] Time spent T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 13:04:55.314253 16200 log.cc:927] Time spent T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:04:55.275161 16147 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 13:04:55.315231  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:55.323976  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:55.282470 16176 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:55.315652  9005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:55.313164  8325 raft_consensus.cc:2468] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:04:55.384903  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:55.594285  8332 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:55.437346  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:55.594347  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:55.500793  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 297495us
W20260709 13:04:55.565007  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 292410us
I20260709 13:04:55.554257  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:55.436928  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:55.609165  8297 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:55.410531  8972 leader_election.cc:336] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.637s (SENT)
I20260709 13:04:55.363922  9009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:55.610503  8972 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:55.505765  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 301961us
I20260709 13:04:55.610821  9009 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
W20260709 13:04:55.414611  8295 leader_election.cc:336] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.950s (SENT)
I20260709 13:04:55.612485  8295 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:55.372931  9007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:55.613598  8326 raft_consensus.cc:2468] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:04:55.328464 16015 raft_consensus.cc:3060] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:04:55.353847 16201 log.cc:927] Time spent T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
W20260709 13:04:55.452716 16200 log.cc:927] Time spent T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
W20260709 13:04:55.581226 15983 consensus_meta.cc:287] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.649s	user 0.001s	sys 0.009s
I20260709 13:04:55.566210 16133 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:55.339632 15968 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:55.428778 16208 log.cc:927] Time spent T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:04:55.615159  8327 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 22.
I20260709 13:04:55.663962  9005 raft_consensus.cc:1275] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:04:55.624429  8222 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:55.633396  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:55.665757  8402 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 21.
I20260709 13:04:55.441201 16215 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:04:55.669334  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 45015.1us
I20260709 13:04:55.670199  8222 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:55.637667 16195 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:55.635143  8297 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:55.480721 16211 log.cc:927] Time spent T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
W20260709 13:04:55.673738  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:55.673880  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 160309us
I20260709 13:04:55.318418 16075 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:55.421684 16147 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.400549 16145 raft_consensus.cc:3055] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:04:55.561070 16214 log.cc:927] Time spent T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:04:55.421772 16023 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.424499 16086 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.433782 15967 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.498739 15921 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:55.640038  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 225454us
W20260709 13:04:55.651615  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 241123us
W20260709 13:04:55.963575  8295 leader_election.cc:336] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 2.012s (SENT)
I20260709 13:04:55.964030  8295 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:55.373783 16130 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:55.659631 15983 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:55.677896 15984 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:04:55.859305  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 184572us
W20260709 13:04:55.967800  8295 leader_election.cc:336] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 2.134s (SENT)
I20260709 13:04:55.944854 15536 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:55.912659  8221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:55.850572  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 180453us
I20260709 13:04:55.854983 16225 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:04:55.651262 16219 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:55.849411  8297 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:55.975446  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 297916us
W20260709 13:04:55.974840  8972 leader_election.cc:336] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.760s (SENT)
I20260709 13:04:55.861215  9008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56948 (request call id 1618) took 2043 ms. Trace:
I20260709 13:04:55.976135  8972 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:55.857967  8974 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:55.916359 16097 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:55.872654 15880 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:55.917135 16132 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:55.986361 16223 log.cc:927] Time spent T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:04:55.986667 16132 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:55.988795 16132 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:55.918987 16176 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:55.990742 16176 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:55.914216 16193 log.cc:927] Time spent T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.242s	user 0.000s	sys 0.008s
I20260709 13:04:55.483103 16118 consensus_queue.cc:237] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:55.960837 16023 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:55.850901 16203 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:55.653401 16149 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:55.685253 16147 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:55.994549 16023 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:55.851527  8331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57202 (request call id 3213) took 1165 ms. Trace:
I20260709 13:04:55.995431  8331 rpcz_store.cc:276] 0709 13:04:54.686504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.686650 (+   146us) service_pool.cc:224] Handling call
0709 13:04:54.686998 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:54.687344 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:04:54.687657 (+   313us) log.cc:844] Serialized 14 byte log entry
0709 13:04:54.984738 (+297081us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:54.993126 (+  8388us) log.cc:844] Serialized 26 byte log entry
0709 13:04:54.993282 (+   156us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:54.993318 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.993534 (+   216us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:55.851311 (+857777us) raft_consensus.cc:1685] finished
0709 13:04:55.851333 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:55.851467 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1203,"prepare.queue_time_us":629,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":32,"raft.queue_time_us":12175,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":7932,"thread_start_us":294853,"threads_started":1,"wal-append.queue_time_us":515080}
I20260709 13:04:55.996737 16215 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
W20260709 13:04:55.892719  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.691256 15975 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:55.385329 16191 consensus_queue.cc:237] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:55.911181 16146 consensus_meta.cc:287] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.982s	user 0.010s	sys 0.004s
I20260709 13:04:56.002152  8331 raft_consensus.cc:1240] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 25. Current term is 26. Ops: []
W20260709 13:04:55.858284  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.679121 16145 raft_consensus.cc:740] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:55.888482 16126 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:04:56.013306  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 141041us
I20260709 13:04:55.522306 15966 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:56.014130  8298 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:56.014680 15966 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:56.015223 16149 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:04:56.015690 16149 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:56.016108 16149 raft_consensus.cc:3060] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:56.016816 15966 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:55.985265 16204 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:56.019019 16147 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:55.975564 16160 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:55.850046 15673 consensus_meta.cc:287] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.732s	user 0.011s	sys 0.010s
W20260709 13:04:55.960536 16020 consensus_meta.cc:287] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.965s	user 0.010s	sys 0.002s
I20260709 13:04:55.976121  8297 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:55.974174 15984 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:04:56.031911  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 55872.9us
I20260709 13:04:56.032896  8297 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:55.978078  8972 leader_election.cc:336] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.767s (SENT)
I20260709 13:04:55.979192  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:55.979285  8222 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:56.034210  8330 raft_consensus.cc:2468] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 29.
W20260709 13:04:56.035367  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 57322.1us
I20260709 13:04:55.959085 16168 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:55.960749 16086 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:55.981132  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 123294us
I20260709 13:04:55.976577  9008 rpcz_store.cc:276] 0709 13:04:53.818113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.819142 (+  1029us) service_pool.cc:224] Handling call
0709 13:04:53.819412 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.819598 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:55.379692 (+1560094us) spinlock_profiling.cc:243] Waited 4.47 ms on lock 0x7b18004807c0. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882e599de1 00007f882e5a65b1 00007f88283664bc 00007f88256b448b 00007f882e50eab9 00007f88256d069b 00007f88256ce93f 00007f88256ce455 00007f882c1b63c7 00007f88287c89a7 00007f88287d80bf
0709 13:04:55.379891 (+   199us) log.cc:844] Serialized 14 byte log entry
0709 13:04:55.860722 (+480831us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:55.860806 (+    84us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:55.860872 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.860981 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:55.861143 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":118563,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":2482,"raft.queue_time_us":110213,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":24344,"spinlock_wait_cycles":10291968,"thread_start_us":486922,"threads_started":2,"wal-append.queue_time_us":561892}
I20260709 13:04:55.981738 16166 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:55.995132  8323 raft_consensus.cc:2368] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 22. Current term is 23.
I20260709 13:04:56.001237  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:55.970048 16165 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:56.009296  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.010927 15983 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:56.011153 15924 consensus_meta.cc:287] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.027s	user 0.006s	sys 0.007s
I20260709 13:04:56.013114 16133 consensus_queue.cc:1048] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 13:04:55.968737  8295 leader_election.cc:336] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 2.109s (SENT)
I20260709 13:04:56.064764  8295 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:56.037770  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:56.065557  8397 raft_consensus.cc:3060] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:56.038290  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:56.039894  8325 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:04:56.066334  8402 raft_consensus.cc:2376] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 21. Re-sending same reply.
I20260709 13:04:56.067395  8325 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:04:55.966439 15921 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:55.982420 15968 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:56.041191  9003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:56.041761  8974 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:56.036365  8972 leader_election.cc:336] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.779s (SENT)
I20260709 13:04:56.077347  8297 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:56.077780  8972 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:56.078073  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:56.048172  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:56.078969  8332 raft_consensus.cc:2410] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 27 because replica has last-logged OpId of term: 26 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 18.
I20260709 13:04:56.076817  8974 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:56.079881  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:55.967378 15967 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:55.972236 16130 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:56.059288  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 80089.5us
I20260709 13:04:56.061545  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:56.093088  8331 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:56.096006  8404 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:56.096312  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:56.096973  8402 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:04:56.029318 15673 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:56.110909 16067 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:56.083844 16086 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:56.112694  9004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 1358) took 1126 ms. Trace:
W20260709 13:04:56.113173 16212 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.633s	user 0.001s	sys 0.002s
I20260709 13:04:56.114547  8974 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:56.116780 16206 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:56.117211 16206 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:56.117586 16206 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:56.118762 15921 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:56.119953  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.120410 15975 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:56.121235 15975 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:04:56.122654  8222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.122732  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:56.081069 16195 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:56.123696  8330 raft_consensus.cc:2393] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 19: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:56.058485  8220 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:56.126078  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 67720.4us
I20260709 13:04:56.113138  9004 rpcz_store.cc:276] 0709 13:04:54.986418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.987104 (+   686us) service_pool.cc:224] Handling call
0709 13:04:54.987504 (+   400us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:54.995243 (+  7739us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:54.995280 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:55.115236 (+119956us) spinlock_profiling.cc:243] Waited 9.06 ms on lock 0x7b480002cdd8. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f882d6e7aa1 00007f882d6deb37 00007f882d6de9ca 00007f882d7d626f 00007f882d7d5f37 00007f882d7ea55c 00007f882d7ea508 00007f882d7ea488 00007f882d7ea443 00007f882d7e96ec 00007f882c713a96
0709 13:04:55.590481 (+475245us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b2c00047a38. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f88270b7a11 00007f88270c7372 00007f88287f9a01 00007f88287be43a 00007f88287bd83e 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2
0709 13:04:55.596292 (+  5811us) log.cc:844] Serialized 52 byte log entry
0709 13:04:55.652978 (+ 56686us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:55.653022 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.653110 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:56.112452 (+459342us) raft_consensus.cc:1685] finished
0709 13:04:56.112474 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:56.112637 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3032,"raft.run_cpu_time_us":2587,"raft.run_wall_time_us":117131,"spinlock_wait_cycles":46746752,"thread_start_us":55047,"threads_started":1,"wal-append.queue_time_us":473852}
I20260709 13:04:56.069597  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:56.046540 16020 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:56.071353 15924 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:56.131121 16020 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:56.131698 15924 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:56.133285 15967 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:56.082459 16145 consensus_queue.cc:260] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:56.070688  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 101982us
I20260709 13:04:56.043242 15984 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:56.135594 16130 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:56.045855 16176 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:56.046375 16219 raft_consensus.cc:2804] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:55.997377 16215 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:56.083964  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 47621.6us
I20260709 13:04:56.096897  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:56.097190  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:56.148672  8404 raft_consensus.cc:2393] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 21: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:56.062532  9000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56954 (request call id 1357) took 1076 ms. Trace:
I20260709 13:04:56.100070  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:56.101501  8324 raft_consensus.cc:1240] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:56.019766 16146 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:56.148993  9000 rpcz_store.cc:276] 0709 13:04:54.986117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.986251 (+   134us) service_pool.cc:224] Handling call
0709 13:04:54.986571 (+   320us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:54.986932 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:54.986960 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:54.995958 (+  8998us) log.cc:844] Serialized 52 byte log entry
0709 13:04:55.271088 (+275130us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:55.271129 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.271199 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:56.062208 (+791009us) raft_consensus.cc:1685] finished
0709 13:04:56.062240 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:56.062452 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2684,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":16903,"thread_start_us":274726,"threads_started":1,"wal-append.queue_time_us":501564}
I20260709 13:04:56.154815  8399 raft_consensus.cc:2393] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 19: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:56.155138  8402 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:56.107581  8397 raft_consensus.cc:2468] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 23.
I20260709 13:04:56.108539 16116 consensus_queue.cc:1059] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:56.110124 15841 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:04:56.128672 16132 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:56.064352 16023 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:56.068583  8369 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:56.126230 15673 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:56.135624  8295 leader_election.cc:336] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 2.175s (SENT)
I20260709 13:04:56.214807  8295 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:56.137902  8332 raft_consensus.cc:2393] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 21: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:56.146577  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:56.136377 16141 consensus_meta.cc:287] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 1.220s	user 0.011s	sys 0.002s
I20260709 13:04:56.160392  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:56.177652 16086 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:04:56.057561 16165 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:04:56.179239 16215 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:56.186473 16126 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting leader election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:56.189102 16127 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:56.190039 16115 consensus_meta.cc:287] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.914s	user 0.009s	sys 0.011s
I20260709 13:04:56.160122 16219 raft_consensus.cc:697] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:56.191123  8222 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:04:56.234719  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 43679.4us
I20260709 13:04:56.191788 16146 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:56.147606 16176 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:56.192468 15968 raft_consensus.cc:3055] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:56.192504 16147 consensus_queue.cc:1059] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:04:56.197840  8405 raft_consensus.cc:2368] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 25. Current term is 26.
W20260709 13:04:56.201792 16015 consensus_meta.cc:287] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.570s	user 0.006s	sys 0.005s
W20260709 13:04:56.209327  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 140929us
I20260709 13:04:56.213335  8331 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
W20260709 13:04:56.219169  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 83572.2us
I20260709 13:04:56.255882  8369 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:56.209714 16206 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:56.225863 16168 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:56.259063  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:56.230708 16127 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:56.217615 16075 raft_consensus.cc:2804] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:56.240653  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:56.228058 16126 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:56.263980  8397 raft_consensus.cc:2393] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 23: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:56.220748 16141 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:56.228209 16145 raft_consensus.cc:3060] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:56.256809  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 142331us
I20260709 13:04:56.225633 15975 raft_consensus.cc:2804] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:56.261009  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:56.257474  8369 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:56.270710  8401 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:56.248349  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 171061us
I20260709 13:04:56.263655 16146 raft_consensus.cc:2804] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:56.274406  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:56.241987 16215 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:56.251283 16176 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:56.250939 15968 raft_consensus.cc:740] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:56.233897 16115 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:56.277382 16152 consensus_queue.cc:1059] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:56.296509  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:56.297329  8327 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:56.281673 16165 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:56.283070 16206 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:56.283560 16141 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 26 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:56.259238 15984 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:56.287784  8399 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:04:56.288197  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:56.308372  8397 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Advancing to term 26
I20260709 13:04:56.288859  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:56.293322  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:56.268584 16075 raft_consensus.cc:697] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 19 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:56.301138  8369 leader_election.cc:400] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 22. Current term is 23.
I20260709 13:04:56.273150 16166 raft_consensus.cc:697] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:56.303938  8297 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:56.304520  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 47383.3us
I20260709 13:04:56.293776 15975 raft_consensus.cc:697] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 21 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:56.309633  8331 raft_consensus.cc:2393] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 19: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:56.309633  8332 raft_consensus.cc:2393] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 23: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:56.314318  8398 raft_consensus.cc:1240] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:56.316071  8369 leader_election.cc:403] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 13:04:56.303369  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:56.316812  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:56.323693  8400 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:04:56.318518  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:56.319324  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39308 (request call id 3762) took 1465 ms. Trace:
W20260709 13:04:56.320933  8295 leader_election.cc:336] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 2.213s (SENT)
I20260709 13:04:56.328006  8295 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:56.329933  8220 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:56.324970  8405 rpcz_store.cc:276] 0709 13:04:54.854300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.854541 (+   241us) service_pool.cc:224] Handling call
0709 13:04:56.316427 (+1461886us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54003160d4. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287c22fa 00007f882d783b17 00007f8826cfe6d4 00007f8826cfe662 00007f8826cfe591 00007f8826cfe50c 00007f8826cfd7c2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:56.319268 (+  2841us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3087008128}
I20260709 13:04:56.338503  8222 leader_election.cc:400] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 election: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 25. Current term is 26.
I20260709 13:04:56.338960  8222 leader_election.cc:403] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 13:04:56.339341  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:56.341249  8220 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:56.346539  8401 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:56.306623 16215 raft_consensus.cc:697] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 23 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:56.354813  8397 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 26.
I20260709 13:04:56.300087 16146 raft_consensus.cc:697] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 19 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:56.370524 16146 consensus_queue.cc:237] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:56.341406 16165 consensus_queue.cc:1059] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:04:56.374569 16165 raft_consensus.cc:3055] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:56.374962 16165 raft_consensus.cc:740] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:56.351372  8222 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:04:56.376890  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 71613.8us
I20260709 13:04:56.353976  8327 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:56.355355  8328 raft_consensus.cc:1240] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 17. Current term is 18. Ops: []
W20260709 13:04:56.356854  8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.294752 16133 consensus_queue.cc:1059] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
W20260709 13:04:56.379703  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 58697.6us
W20260709 13:04:56.361709 16241 log.cc:927] Time spent T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
W20260709 13:04:56.362459 16238 log.cc:927] Time spent T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
W20260709 13:04:56.363031 16239 log.cc:927] Time spent T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:04:56.315490 16219 consensus_queue.cc:237] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:56.364320  8397 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:04:56.364859 16215 consensus_queue.cc:237] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:56.339897 16152 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599) with higher term. Message: Invalid argument: T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for earlier term 22. Current term is 23.
I20260709 13:04:56.368463 16166 consensus_queue.cc:237] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:56.412900  8400 raft_consensus.cc:2468] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 23.
I20260709 13:04:56.377112  8222 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:56.398432  8220 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:56.367238 16015 raft_consensus.cc:3055] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:56.345721 16141 raft_consensus.cc:3060] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:56.355816 16147 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:56.369410  8223 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:56.383013 15921 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:56.383090 15924 raft_consensus.cc:3060] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:56.355027 16118 raft_consensus.cc:2764] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 13:04:56.422315  8974 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:04:56.423725  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 54399.9us
I20260709 13:04:56.428573  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:56.401978 16242 raft_consensus.cc:2804] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:56.429133  8323 raft_consensus.cc:3060] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:56.375787 16165 consensus_queue.cc:260] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:56.428393  8223 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:56.376461 16023 consensus_queue.cc:1059] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.046s
I20260709 13:04:56.436717  8401 raft_consensus.cc:1240] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:56.429411 15975 consensus_queue.cc:237] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:56.440083 15938 raft_consensus.cc:3055] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:04:56.440537 15938 raft_consensus.cc:740] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 25 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:56.416824 16015 raft_consensus.cc:740] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:56.421442 15644 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:04:56.443477  8223 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:56.426934 16020 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:56.424458 16191 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:56.434590 16204 raft_consensus.cc:2804] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:56.461241  8223 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:56.431387 15921 raft_consensus.cc:697] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 23 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:56.481487  8223 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:56.446318 15966 raft_consensus.cc:3055] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:56.482419  8401 raft_consensus.cc:1275] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:04:56.376370 15968 consensus_queue.cc:260] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:56.447000 16222 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:56.446120 16249 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:56.445468 16242 raft_consensus.cc:697] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 23 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:04:56.464285 15967 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:56.492278  8223 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:56.489923  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.441666 16116 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 13:04:56.427232 16075 consensus_queue.cc:237] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:56.466290 16023 consensus_queue.cc:1059] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 13:04:56.466459 15644 raft_consensus.cc:697] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 26 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:56.536440 16242 consensus_queue.cc:237] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:56.495934 16221 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:56.535694  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 214780us
W20260709 13:04:56.538367  8295 leader_election.cc:336] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 2.230s (SENT)
I20260709 13:04:56.538751  8295 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:56.508865  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.512972  8323 raft_consensus.cc:2468] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 26.
W20260709 13:04:56.544411  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 52768.6us
I20260709 13:04:56.545644  8223 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:56.547252  8295 leader_election.cc:336] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 2.274s (SENT)
I20260709 13:04:56.547986  8295 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:56.472136 16204 raft_consensus.cc:697] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 23 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:56.486712 16191 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:56.484983 15966 raft_consensus.cc:740] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:56.553887  8223 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:56.496477 16165 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:56.501803 16195 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:56.459048 15938 consensus_queue.cc:260] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:56.558219  8295 leader_election.cc:336] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 2.291s (SENT)
I20260709 13:04:56.562152  8295 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:56.551415 16221 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:04:56.552163 16086 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:56.490456 16206 consensus_queue.cc:1059] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.065s
W20260709 13:04:56.566917  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.558722 16240 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:56.564248 16253 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:56.600664  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:56.559787 15921 consensus_queue.cc:237] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:56.607149 16127 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:56.610368  8296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.563921 16195 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:56.581856 16067 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.584041 15968 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:56.611091 16149 consensus_meta.cc:287] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.595s	user 0.007s	sys 0.004s
I20260709 13:04:56.577183 16168 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:56.590581 16115 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:56.675716  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:56.593266  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:56.677218  8328 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 19.
I20260709 13:04:56.595189  8326 raft_consensus.cc:1275] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:04:56.568373 15984 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:56.614256  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:56.618023  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:56.682662  8397 raft_consensus.cc:2468] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:56.565050 16023 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:56.623898  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:56.689675  8325 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:56.628985  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:56.568267 15938 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:56.631217  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:56.630605 16152 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:56.692894  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 56582.3us
I20260709 13:04:56.631947  9004 raft_consensus.cc:3055] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:04:56.566293 16086 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:56.635812  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:56.638993  8972 consensus_peers.cc:597] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:56.699600  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 60661.5us
I20260709 13:04:56.641170  9006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:56.584453  8146 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45456 (request call id 286) took 63793 ms (client timeout 15000 ms). Trace:
I20260709 13:04:56.624419 15841 consensus_queue.cc:1048] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 13:04:56.654322  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:56.670795  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 59882.4us
I20260709 13:04:56.630872 15966 consensus_queue.cc:260] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:56.709154  8222 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:04:56.700673  8972 consensus_peers.cc:597] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:56.717178  8972 consensus_peers.cc:597] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:56.717645  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:56.718036  8972 consensus_peers.cc:597] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:56.718498  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:56.679458  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 214149us
I20260709 13:04:56.686000  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:56.635553 16130 raft_consensus.cc:3055] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:56.688086  8329 raft_consensus.cc:1240] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:04:56.688752  8220 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:56.690639  9008 raft_consensus.cc:2410] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 13:04:56.694006  9004 raft_consensus.cc:740] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:56.710225 16191 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.724378 16191 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:56.725593 15880 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:56.726966 16191 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:56.728029 16152 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:56.724045  9004 consensus_queue.cc:260] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:56.729315  9004 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:56.730341 16152 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:56.637537 15536 consensus_queue.cc:1059] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.079s
I20260709 13:04:56.731688 15536 raft_consensus.cc:3055] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:56.732116 15536 raft_consensus.cc:740] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:56.732884 16015 consensus_queue.cc:260] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:56.733601 16225 raft_consensus.cc:2804] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:56.734087 16225 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:56.734493 16225 raft_consensus.cc:3060] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:56.734645 16015 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:56.735718  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:56.737911  8398 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:56.738723  8327 raft_consensus.cc:1240] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 18. Current term is 19. Ops: [18.13-18.13]
W20260709 13:04:56.741024  8369 leader_election.cc:336] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.504s (SENT)
W20260709 13:04:56.742941  8369 leader_election.cc:336] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.568s (SENT)
I20260709 13:04:56.743477  8369 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:56.741766  8328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:56.744282  8328 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 26.
I20260709 13:04:56.748508  8223 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:56.654995 16195 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:56.718931  8972 consensus_peers.cc:597] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:56.757063  8220 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:56.758134  8223 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:56.762567  8220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:56.763418  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 44554.2us
I20260709 13:04:56.612380 15644 consensus_queue.cc:237] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:56.714890 16086 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:56.779031  8295 leader_election.cc:336] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 3.063s (SENT)
I20260709 13:04:56.703027 15673 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:56.774473  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:56.742417 16130 raft_consensus.cc:740] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
W20260709 13:04:56.776026  8972 leader_election.cc:336] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:56.788297  8972 leader_election.cc:336] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:56.788796  8972 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:56.761809 16020 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.637676 16067 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:56.785382  9004 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 18. (term mismatch)
W20260709 13:04:56.773698  8369 leader_election.cc:336] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.739s (SENT)
I20260709 13:04:56.812121  9004 pending_rounds.cc:85] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e: Aborting all ops after (but not including) 17
I20260709 13:04:56.812230  8369 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:56.812654  9004 pending_rounds.cc:107] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e: Aborting uncommitted NO_OP operation due to leader change: 25.18
I20260709 13:04:56.812927  9004 raft_consensus.cc:2889] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:56.779934  8295 leader_election.cc:336] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 3.085s (SENT)
I20260709 13:04:56.675292 16149 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:56.732969 15536 consensus_queue.cc:260] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:56.806668 16116 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:56.790627  8972 leader_election.cc:336] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:56.822180  8972 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:56.823347  8972 leader_election.cc:336] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:56.824348  8972 leader_election.cc:336] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:56.774581 15880 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:04:56.675305 16126 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:56.825774 15880 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:56.834699 15880 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:56.835698 16126 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:56.771826 16152 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:56.769824 16127 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:56.814160  8295 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:56.838678  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:56.839758 16195 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:04:56.815385  8369 leader_election.cc:336] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 1.751s (SENT)
I20260709 13:04:56.816457  9006 raft_consensus.cc:2410] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 28 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 18.
I20260709 13:04:56.816880  8325 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 26.
W20260709 13:04:56.826421  8972 consensus_peers.cc:597] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:56.786114 16168 raft_consensus.cc:493] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:56.845292  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 65386.6us
W20260709 13:04:56.846278  8295 leader_election.cc:336] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Timed out: RequestConsensusVote RPC to 127.6.255.1:44679 timed out after 3.126s (SENT)
I20260709 13:04:56.787391 16221 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:56.854251  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.639691 16204 consensus_queue.cc:237] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:56.861896  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.865469 15966 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:56.866844  8972 consensus_peers.cc:597] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:56.817566 16020 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:56.867506  8972 leader_election.cc:336] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:56.867959  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:56.868901 16130 consensus_queue.cc:260] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:56.870268 16130 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:56.827149 15983 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:56.874240 16168 raft_consensus.cc:515] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:56.876854 16168 leader_election.cc:290] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:56.878180 16263 log.cc:927] Time spent T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:04:56.879070 15536 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:56.879549 15880 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:56.785269 16191 consensus_queue.cc:1059] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
W20260709 13:04:56.882517 16258 log.cc:927] Time spent T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:04:56.882846  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:56.868969  8222 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:56.884609 16215 raft_consensus.cc:3060] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:56.840277 16195 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:56.888593 15880 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:56.881093 16020 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:56.866578  8369 consensus_peers.cc:597] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:56.889115  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:56.841781 16116 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:56.880008  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:56.898048  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:56.899861  8222 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:56.899991  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 43734.3us
I20260709 13:04:56.901019 15880 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:56.908233 16149 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:56.910069  8295 consensus_peers.cc:597] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:56.910919  8295 consensus_peers.cc:597] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:56.912482 16260 log.cc:927] Time spent T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:04:56.913884  8324 raft_consensus.cc:2410] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 28 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 18.
I20260709 13:04:56.914804 16197 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:56.917683  8372 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:56.913924  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 46477.8us
W20260709 13:04:56.930660  8298 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.006s	sys 0.002s
W20260709 13:04:56.931226  8298 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.006s	sys 0.002s
W20260709 13:04:56.931531  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 537864us
W20260709 13:04:56.925043  8369 leader_election.cc:336] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:56.937165  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:56.937331  9006 raft_consensus.cc:1240] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 21. Current term is 22. Ops: []
W20260709 13:04:56.914825  8295 leader_election.cc:336] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:56.944336  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 53036.6us
W20260709 13:04:56.945924  8295 leader_election.cc:336] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:56.948105  8295 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:56.945151  8369 consensus_peers.cc:597] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:56.950068  8369 consensus_peers.cc:597] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:56.949994  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:56.950960  8326 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:04:56.951728  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 181231us
I20260709 13:04:56.953336  8223 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:56.856892 16250 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:04:56.954232  8295 leader_election.cc:336] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:56.951220  8369 leader_election.cc:336] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:56.945362 16145 consensus_meta.cc:287] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.677s	user 0.011s	sys 0.001s
I20260709 13:04:56.955376  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:56.964717  8972 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:04:56.951530 16176 consensus_meta.cc:287] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728: Time spent flushing consensus metadata: real 0.648s	user 0.009s	sys 0.002s
I20260709 13:04:56.949517 15983 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:56.956650  8223 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:56.967907  8295 leader_election.cc:336] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:56.968741  8323 raft_consensus.cc:1240] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:56.971794  8295 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:56.970418  8369 leader_election.cc:336] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:56.975960  8223 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:56.980077 16225 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:56.971563 16176 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:56.985989 16176 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:56.986858  8330 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:56.991936 16145 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:04:56.995033 16147 consensus_meta.cc:287] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.569s	user 0.007s	sys 0.002s
I20260709 13:04:56.963059 16221 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:56.962860 16249 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:56.947166 16130 consensus_queue.cc:1059] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
W20260709 13:04:57.009181  8295 leader_election.cc:336] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:56.975262 16118 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:57.015126 16152 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.016928  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:57.019356  8326 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 27.
W20260709 13:04:56.712371  8146 rpcz_store.cc:269] 0709 13:03:52.790803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.817757 (+ 26954us) service_pool.cc:224] Handling call
0709 13:03:53.210780 (+393023us) spinlock_profiling.cc:243] Waited 57.5 ms on lock 0x7b5400200490. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f882569a0b2 00007f882e613a89 00007f882e5abda6 00007f882e57d088 00007f882e67b37e 00007f882cb3c2a4 00007f882cb3c232 00007f882cb3c161 00007f882cb3c0dc
0709 13:04:56.584347 (+63373567us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:48.675436 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:48.874064 (+198628us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:48.885545 (+ 11481us) write_op.cc:435] Acquired schema lock
0709 13:04:48.891101 (+  5556us) tablet.cc:662] Decoding operations
0709 13:04:48.970532 (+ 79431us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:48.970626 (+    94us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:48.970658 (+    32us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:04:48.971289 (+   631us) tablet.cc:701] Row locks acquired
0709 13:04:48.971344 (+    55us) write_op.cc:260] PREPARE: finished
0709 13:04:48.971531 (+   187us) write_op.cc:270] Start()
0709 13:04:48.971704 (+   173us) write_op.cc:276] Timestamp: P: 1783602288971457 usec, L: 0
0709 13:04:48.971745 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 13:04:48.975551 (+  3806us) log.cc:844] Serialized 1345 byte log entry
0709 13:04:49.917868 (+942317us) op_driver.cc:464] REPLICATION: finished
0709 13:04:50.361839 (+443971us) write_op.cc:301] APPLY: starting
0709 13:04:50.375527 (+ 13688us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:50.621735 (+246208us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:50.621874 (+   139us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:50.624458 (+  2584us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:50.624856 (+   398us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=6,mrs_lookups=0
0709 13:04:50.624950 (+    94us) write_op.cc:312] APPLY: finished
0709 13:04:50.625337 (+   387us) log.cc:844] Serialized 41 byte log entry
0709 13:04:50.706461 (+ 81124us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:50.730846 (+ 24385us) write_op.cc:454] Released schema lock
0709 13:04:50.740125 (+  9279us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:50.740599 (+   474us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:50.740633 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":80496,"reported_tablets":50,"spinlock_wait_cycles":132230784,"child_traces":[["op",{"apply.queue_time_us":327856,"apply.run_cpu_time_us":9689,"apply.run_wall_time_us":422262,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16673,"num_ops":3,"prepare.queue_time_us":401693,"prepare.run_cpu_time_us":13038,"prepare.run_wall_time_us":448899,"raft.queue_time_us":201356,"raft.run_cpu_time_us":28256,"raft.run_wall_time_us":268335,"replication_time_us":916263,"thread_start_us":495557,"threads_started":4,"wal-append.queue_time_us":329141,"wal-append.run_cpu_time_us":31169,"wal-append.run_wall_time_us":2306501}]]}
I20260709 13:04:56.977290 15967 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:57.010335  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.975476 15968 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:57.022575  8295 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:57.003628 16147 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367) with higher term. Message: Invalid argument: T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 25. Current term is 26.
I20260709 13:04:57.026042 15880 raft_consensus.cc:3055] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:57.040382 15880 raft_consensus.cc:740] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:57.026523 16130 raft_consensus.cc:3055] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:04:57.028977  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:57.023819 15924 consensus_meta.cc:287] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.594s	user 0.009s	sys 0.000s
W20260709 13:04:57.035630  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:57.097028  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 121149us
I20260709 13:04:57.047569  8405 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:04:57.048601 16130 raft_consensus.cc:740] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:57.035331 16152 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:57.059015 16195 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:57.062714 16067 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:57.041371 15880 consensus_queue.cc:260] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:57.049759 16221 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.021593 16176 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:57.045181 16249 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:57.047170  8220 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:57.098706  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:57.099684  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:57.100318  9004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39384c6d69ad4f658d2e777dfdcbb49c" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:57.102479  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:57.106110  8405 raft_consensus.cc:1240] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 21. Current term is 22. Ops: [21.13-21.13]
W20260709 13:04:57.120890  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.038506 15968 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.143208 16152 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:57.103247 16147 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:04:57.145121  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 98043.2us
I20260709 13:04:57.145547 15921 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:04:57.148873  8401 raft_consensus.cc:1275] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:04:57.144721 16130 consensus_queue.cc:260] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:57.150298  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:57.186367  8406 raft_consensus.cc:2393] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f865c4d761f5446eb12b3a029234772e in current term 23: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:57.113282 16221 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:57.115181 15924 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:57.156072  8397 raft_consensus.cc:1275] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:04:57.156368 15880 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:57.157080  8399 raft_consensus.cc:1275] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:04:57.161375  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:57.196461  9008 raft_consensus.cc:2410] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:04:57.144634 16020 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.024s
W20260709 13:04:57.107841 16275 log.cc:927] Time spent T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.062s	user 0.000s	sys 0.002s
I20260709 13:04:57.044482 16215 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:57.176185  9000 raft_consensus.cc:2468] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 27.
I20260709 13:04:57.188781  8323 raft_consensus.cc:1275] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:04:57.114794 16272 consensus_queue.cc:1059] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.085s
I20260709 13:04:57.194800  8324 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:04:57.201403 16149 consensus_queue.cc:1048] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:57.189312 15921 raft_consensus.cc:697] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 27 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:57.202214 16245 consensus_queue.cc:1048] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:57.176626 15968 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:57.224464  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:57.234352  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
W20260709 13:04:57.227569  8295 leader_election.cc:336] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:57.236034  8295 leader_election.cc:336] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:57.236691  8295 leader_election.cc:336] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.236968  8295 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:57.205092 16130 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:57.239446  8295 leader_election.cc:336] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:57.242071  8295 leader_election.cc:336] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.242810  8295 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:57.243925  8295 leader_election.cc:336] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.244289  8295 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:57.245586  8295 leader_election.cc:336] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:57.247146 16273 log.cc:927] Time spent T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:04:57.225064 16225 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:57.248615  8295 leader_election.cc:336] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:57.249583  8295 leader_election.cc:336] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.245206 16166 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:04:57.242568 16191 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 13:04:57.250610  8295 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:57.251531 16166 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:57.252540 16176 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:57.238039 16220 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.254587 16220 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:57.243640 15644 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:04:57.238215 16160 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:57.231079 15536 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.260408 15536 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:57.262006 15921 consensus_queue.cc:237] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:57.266459 16265 log.cc:927] Time spent T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
W20260709 13:04:57.280462 16278 log.cc:927] Time spent T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.088s	user 0.001s	sys 0.001s
W20260709 13:04:57.285759  8295 leader_election.cc:336] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.288095  8323 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 13:04:57.289973 15967 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:04:57.291697  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:57.297151  8295 leader_election.cc:336] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.298485  8295 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:57.306725  9001 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 13:04:57.307236  9001 pending_rounds.cc:85] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e: Aborting all ops after (but not including) 14
I20260709 13:04:57.307510  9001 pending_rounds.cc:107] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 13:04:57.307845  9001 raft_consensus.cc:2889] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:57.219529 16249 consensus_queue.cc:1048] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.026s
W20260709 13:04:57.312799  8295 leader_election.cc:336] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.314226  8295 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:57.317701  8330 raft_consensus.cc:1275] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 13:04:57.312666  8298 leader_election.cc:304] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:57.317638 16176 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:57.320281 16176 raft_consensus.cc:2749] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:57.320560  8298 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:57.321599 16272 consensus_queue.cc:1048] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:57.323319 15968 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:57.326076 15968 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:04:57.303056 16086 consensus_meta.cc:287] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.512s	user 0.010s	sys 0.000s
W20260709 13:04:57.338052  8295 leader_election.cc:336] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.288125 16221 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:57.343475  8369 consensus_peers.cc:597] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:57.346452  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 369448us
W20260709 13:04:57.347452  8369 leader_election.cc:336] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.349118  8369 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:57.350747  8295 consensus_peers.cc:597] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:57.333833 15938 consensus_meta.cc:287] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: Time spent flushing consensus metadata: real 0.618s	user 0.007s	sys 0.004s
W20260709 13:04:57.354754  8295 leader_election.cc:336] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:57.354684  8369 leader_election.cc:336] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:57.354673  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.336306 16149 raft_consensus.cc:2804] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:04:57.365015  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:57.378269  8369 consensus_peers.cc:597] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:57.379972  8369 consensus_peers.cc:597] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:57.380851  8295 leader_election.cc:336] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:57.382364  8295 leader_election.cc:336] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.382807  8405 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
W20260709 13:04:57.342988 16141 consensus_meta.cc:287] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.922s	user 0.013s	sys 0.000s
I20260709 13:04:57.365316  8398 raft_consensus.cc:1275] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:04:57.383653  8295 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:57.400198  8295 leader_election.cc:336] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.353856 16176 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:57.401583  8295 leader_election.cc:336] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.402057  8295 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:57.384721  8369 consensus_peers.cc:597] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:57.358402 16075 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:57.357029 16242 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 13:04:57.406177 16276 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:04:57.406364  8369 leader_election.cc:336] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.338191 16086 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:57.422703  8369 leader_election.cc:336] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.425771  8223 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:57.423782  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:57.432832  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:57.399727 16220 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:57.401793 16141 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:57.440866  8369 leader_election.cc:336] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.441421  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:57.460232  8223 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:57.463534  8369 leader_election.cc:336] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.464013  8369 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:57.465368 16277 log.cc:927] Time spent T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
I20260709 13:04:57.467386 16015 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:57.458523 16259 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:57.471069  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:57.471107  8406 raft_consensus.cc:1275] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 13:04:57.471832  8404 raft_consensus.cc:2393] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 23: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
I20260709 13:04:57.402667 16020 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 13:04:57.435133 16160 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:57.477515  8332 raft_consensus.cc:1240] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Rejecting Update request from peer f865c4d761f5446eb12b3a029234772e for earlier term 25. Current term is 26. Ops: []
I20260709 13:04:57.465521 15536 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:57.490299  7164 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 8282)
I20260709 13:04:57.495222 16086 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:57.498111  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.458709 16225 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:57.498360  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:57.499125  8326 raft_consensus.cc:2393] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 19: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:57.500466  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:57.501171  9001 raft_consensus.cc:2368] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 19. Current term is 20.
I20260709 13:04:57.479245  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:57.477516 16282 log.cc:927] Time spent T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:04:57.473906 16220 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:57.512653  8223 leader_election.cc:400] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 19. Current term is 20.
I20260709 13:04:57.461512 16145 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:57.534117  8223 leader_election.cc:403] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
W20260709 13:04:57.522558  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 59039.5us
W20260709 13:04:57.540620  8369 leader_election.cc:336] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:57.542313  8369 leader_election.cc:336] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.542768  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:57.543871  8975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:57.545449  8369 leader_election.cc:336] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:57.546602  8369 leader_election.cc:336] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.531283 16075 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:57.478835 16272 consensus_queue.cc:1048] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.067s
I20260709 13:04:57.546061 15966 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:57.558039  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 156470us
W20260709 13:04:57.568200  8295 leader_election.cc:336] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.568701  8295 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:57.547574 16130 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:57.544076 16145 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:57.555420  8972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.014s	sys 0.023s
I20260709 13:04:57.550038  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:57.582038 16160 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:04:57.588748  8972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.015s	sys 0.023s
W20260709 13:04:57.603618  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 638951us
W20260709 13:04:57.601251  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 54674.3us
I20260709 13:04:57.587597 16149 raft_consensus.cc:697] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 27 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
W20260709 13:04:57.607524  8369 leader_election.cc:336] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.605229 16259 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:57.619645 16149 consensus_queue.cc:237] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:57.620471 15924 raft_consensus.cc:3055] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:57.619125  8972 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:57.621042 15924 raft_consensus.cc:740] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
W20260709 13:04:57.612637  8369 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 123 similar messages]
I20260709 13:04:57.622080 15924 consensus_queue.cc:260] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:57.627779  8972 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:57.629266 16288 log.cc:927] Time spent T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:04:57.624554  8369 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:57.630722 16130 raft_consensus.cc:2804] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:04:57.631278 16130 raft_consensus.cc:697] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 26 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:57.635237  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:57.638628 16145 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:57.627547 15841 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:57.642205  8331 raft_consensus.cc:1240] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Rejecting Update request from peer e64761f43e4240319c41117cf886d22f for earlier term 26. Current term is 27. Ops: []
I20260709 13:04:57.643682 16116 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:57.646289 16130 consensus_queue.cc:237] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:57.654553  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:57.654644  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:57.655288  8325 raft_consensus.cc:3060] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:57.656870  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:57.663049  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.672552  8325 raft_consensus.cc:2468] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 20.
I20260709 13:04:57.614282 16290 consensus_queue.cc:1048] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 13:04:57.698127 16141 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 13:04:57.756197  8324 raft_consensus.cc:1275] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:04:57.772634  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:57.777388  8324 raft_consensus.cc:1275] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:04:57.778167  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:57.782238  8223 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:57.783694 16141 raft_consensus.cc:3060] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:57.788594  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 89954.6us
W20260709 13:04:57.790230  8972 leader_election.cc:336] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:57.791637  8972 leader_election.cc:336] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:57.792797  8972 consensus_peers.cc:597] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:57.797461  8295 leader_election.cc:336] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.786669 15966 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825) with higher term. Message: Invalid argument: T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for earlier term 19. Current term is 20.
I20260709 13:04:57.798857 15967 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:57.799576  8295 leader_election.cc:336] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.800058 16146 consensus_queue.cc:1059] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:57.800344  8295 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:57.801281 16276 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:57.801793 15938 raft_consensus.cc:3055] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:04:57.802219 15938 raft_consensus.cc:740] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 26 LEADER]: Becoming Follower/Learner. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:57.801457 16146 consensus_queue.cc:1048] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:04:57.803342  8295 consensus_peers.cc:597] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:57.803211  8369 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:57.803126 15938 consensus_queue.cc:260] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:57.804257 16220 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:57.804551 15938 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:57.800103 15967 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:57.805130 16220 consensus_queue.cc:1048] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:04:57.806535  8369 leader_election.cc:336] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.806810 15967 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:57.807440  8369 leader_election.cc:336] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:57.808310  8369 leader_election.cc:336] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:57.809120  8369 consensus_peers.cc:597] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:57.810621  8369 consensus_peers.cc:597] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:57.811406  8369 leader_election.cc:336] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:57.812592  8369 leader_election.cc:336] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:57.813256  8369 leader_election.cc:336] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:57.814072  8369 leader_election.cc:336] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:57.814368 16298 log.cc:927] Time spent T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:04:57.814103  8295 consensus_peers.cc:597] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:57.815012  8369 leader_election.cc:336] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:57.815814  8369 leader_election.cc:336] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:57.816603  8369 leader_election.cc:336] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.816977  8369 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:57.818627  8369 leader_election.cc:336] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.819139  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:57.819892 16068 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 13:04:57.820652  8369 consensus_peers.cc:597] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:57.821704  8369 leader_election.cc:336] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.822193  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:57.823525  8369 leader_election.cc:336] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.824278 15983 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:57.823992  8369 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
W20260709 13:04:57.826202  8369 leader_election.cc:336] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:57.827275  8369 leader_election.cc:336] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.827699  8369 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:57.830674 16075 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:57.831008  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 298061us
I20260709 13:04:57.831483 16075 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:57.820725 16068 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:57.836284  8326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:57.836891  8329 raft_consensus.cc:1240] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Rejecting Update request from peer 8c42259d7c4b463a9f485f42c320c81b for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:57.841374  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:57.843436  8327 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:57.844305 16145 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.847869 16145 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:57.850186 16145 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:57.850898 16279 log.cc:927] Time spent T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.618s	user 0.002s	sys 0.005s
I20260709 13:04:57.856030 16141 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:57.875370  8972 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:57.878974 16197 raft_consensus.cc:2804] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:57.881983 16197 raft_consensus.cc:697] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 20 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:04:57.884228 16197 consensus_queue.cc:237] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:57.896036 15924 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:57.955101  8972 leader_election.cc:336] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:57.956593  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:57.962105  8974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.972297 15880 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.979548 15880 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:57.984328  8404 raft_consensus.cc:1275] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:04:57.995995 16146 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.997313 16146 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:58.044289 16146 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:58.040728 16020 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.048952 16020 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:58.049748 15880 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:58.051765 16020 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:58.011603 15841 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:58.021974 16147 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:58.063988 16312 log.cc:927] Time spent T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:04:58.066668 16191 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.068428  8220 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:58.092242  8332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41664 (request call id 1276) took 1416 ms. Trace:
I20260709 13:04:58.095155  8332 rpcz_store.cc:276] 0709 13:04:56.675937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.676229 (+   292us) service_pool.cc:224] Handling call
0709 13:04:57.475165 (+798936us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:57.485102 (+  9937us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.091780 (+606678us) spinlock_profiling.cc:243] Waited 799 ms on lock 0x7b54003c8f50. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287bd8ad 00007f882d782419 00007f8826cf9bd4 00007f8826cf9b62 00007f8826cf9a91 00007f8826cf9a0c 00007f8826cf8cc2 00007f8826af948c 00007f8826af88c6 00007f8826af826f
0709 13:04:58.092034 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1836595072}
I20260709 13:04:58.102222  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:58.103516  8330 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:04:58.103760 16126 consensus_queue.cc:1048] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:04:58.140640  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 72698us
I20260709 13:04:58.143055  8220 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:58.145203  8327 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 13:04:58.145673  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:58.152462  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:58.138080 16147 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:58.176394 16147 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:58.177603  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:58.178622  8327 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:58.193193  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:58.194202  8400 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
I20260709 13:04:58.133195 16191 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:58.202526 15921 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:58.208312  8400 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:04:58.209645  8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 66838.5us
I20260709 13:04:58.209913 15880 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:58.210541 15880 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.210893 15880 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:58.212913 15644 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:58.214867 16191 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:58.229557  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:58.230844  8400 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 16. (term mismatch)
I20260709 13:04:58.231287  8400 pending_rounds.cc:85] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f: Aborting all ops after (but not including) 15
I20260709 13:04:58.231645  8400 pending_rounds.cc:107] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f: Aborting uncommitted NO_OP operation due to leader change: 26.16
I20260709 13:04:58.231994  8400 raft_consensus.cc:2889] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:58.240254  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:58.241871  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 77785.6us
I20260709 13:04:58.241954 15880 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:58.253638 15880 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:58.263271  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:58.263198  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:58.268508 16315 log.cc:927] Time spent T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:04:58.268991  8405 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 22.
I20260709 13:04:58.271299  8369 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:58.276701 15938 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:04:58.277135  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 92638.8us
I20260709 13:04:58.277298 15938 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.283085  8220 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:58.290230  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:58.291175  8397 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:58.285348 15938 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:58.292272 15880 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:58.301723 16149 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:58.302431 16149 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:58.292330 16176 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.014s
I20260709 13:04:58.317209  9002 raft_consensus.cc:1275] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 13:04:58.300794 15880 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.325548 16160 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:58.330258  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:58.331897 16176 consensus_queue.cc:1048] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 13:04:58.305383 16149 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:58.340133  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:58.341777  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:58.349078  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:58.351485 16023 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.362607  8405 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:04:58.368438 16145 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.369112 16145 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:58.356937 15880 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:58.380645 16023 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:58.383158 16023 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:58.384083 16160 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:58.387089 16160 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:58.388500  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:58.389670  8400 raft_consensus.cc:2393] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 22: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:58.393687  8222 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:58.394397 16075 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.394980 16075 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:58.396102 16020 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:58.397464  8324 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:58.401000  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:58.401219 16145 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:58.402052  8330 raft_consensus.cc:2410] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 22 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
W20260709 13:04:58.414341 15924 consensus_meta.cc:287] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b: Time spent flushing consensus metadata: real 0.517s	user 0.019s	sys 0.011s
I20260709 13:04:58.414777  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:58.415498  9002 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 22.
I20260709 13:04:58.401873 16145 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:58.417243  8372 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:58.418888 16130 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:58.419305 16130 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.419638 16130 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:58.422860  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:58.396035  8222 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:58.430598  8222 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:58.432539 16075 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:58.423460  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 46413.4us
W20260709 13:04:58.434151  8295 consensus_peers.cc:597] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:58.436975  8972 leader_election.cc:336] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:58.437801  8972 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:58.439721 16204 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:58.448112  8405 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 24.
I20260709 13:04:58.449615 16023 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:58.462762  8972 leader_election.cc:336] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:58.463737  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:58.464885 16145 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:58.463791 16023 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:58.470709  8222 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:58.482023 16130 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:58.485677  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:58.487305  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:58.487007 16023 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:58.487905  9008 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:58.490761  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:04:58.492379  8972 leader_election.cc:336] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:58.492803 16130 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:58.494339  7164 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 8282)
I20260709 13:04:58.495160  8323 raft_consensus.cc:1275] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:04:58.500429  9008 raft_consensus.cc:2468] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 23.
I20260709 13:04:58.510985  8372 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:58.511231 16023 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:58.511955 16130 raft_consensus.cc:2804] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:58.512497 16130 raft_consensus.cc:697] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 23 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:58.513480 16130 consensus_queue.cc:237] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:58.515936  8972 consensus_peers.cc:597] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:58.517746  8369 leader_election.cc:336] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:58.518102 16126 consensus_queue.cc:1048] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:04:58.518317  8369 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:58.522114 16068 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:58.523389  8369 consensus_peers.cc:597] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:58.524814  8330 raft_consensus.cc:1275] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 13:04:58.532948  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:58.534615  8324 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 24.
I20260709 13:04:58.526079 16145 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:58.544718  8222 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:58.548415 15644 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:58.551061  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:58.551388 15644 raft_consensus.cc:697] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 24 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:58.552085  8400 raft_consensus.cc:2410] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b for term 22 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:04:58.552578 15644 consensus_queue.cc:237] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:58.550992  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:58.556471 16197 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:04:58.550839 16147 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.560575 16147 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:58.564508 16147 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:58.566129  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:58.562130  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:58.559487  8222 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:58.571403 16147 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:58.572639  8222 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:58.573386 16147 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:58.573745 16020 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.574425 16020 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:58.549419  8369 consensus_peers.cc:597] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:58.576700  8369 leader_election.cc:336] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:58.576797 16020 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:58.577121  8369 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:58.577477  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:58.581007  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:58.581712  8405 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 17.
I20260709 13:04:58.583498  8222 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:58.590067  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:58.590835  8330 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 18.
I20260709 13:04:58.592301 16068 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:58.542734 16145 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.600556 16145 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:58.607620 15924 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:58.608170 15924 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.608610 15924 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:58.631430  8150 catalog_manager.cc:5697] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:58.633780  8220 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:58.635308  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 47909.3us
I20260709 13:04:58.641956 16132 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:58.633266  8150 catalog_manager.cc:5697] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:04:58.628125 16020 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.662926  8150 catalog_manager.cc:5697] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 12 to 22, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 22 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:58.667198 15938 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:58.665992  8147 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55754 (request call id 204) took 31156 ms (client timeout 15000 ms). Trace:
I20260709 13:04:58.668642 16020 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:58.669708  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:58.671056 16020 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:58.671811  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:58.671594  8150 catalog_manager.cc:5697] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 24 to 25, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 25 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:58.673691  8150 catalog_manager.cc:5697] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:58.676573 15938 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:58.625357 15880 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:58.684202 16075 consensus_queue.cc:1048] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:58.685551 15880 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:58.686193  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:58.686800  8325 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:04:58.656607 16204 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:58.699344  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:58.700030  8324 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:58.700596  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 90366.7us
I20260709 13:04:58.703449  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:58.704053  8405 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:58.711663  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:58.709137  9000 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:58.714851  8150 catalog_manager.cc:5697] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:04:58.669253  8147 rpcz_store.cc:269] 0709 13:04:27.510140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.510508 (+   368us) service_pool.cc:224] Handling call
0709 13:04:58.665741 (+31155233us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:55.941190 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:56.102379 (+161189us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:56.136806 (+ 34427us) write_op.cc:435] Acquired schema lock
0709 13:04:56.136829 (+    23us) tablet.cc:662] Decoding operations
0709 13:04:56.179070 (+ 42241us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:56.179127 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:56.179163 (+    36us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:04:56.179978 (+   815us) tablet.cc:701] Row locks acquired
0709 13:04:56.179994 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:04:56.180147 (+   153us) write_op.cc:270] Start()
0709 13:04:56.180259 (+   112us) write_op.cc:276] Timestamp: P: 1783602296180097 usec, L: 0
0709 13:04:56.180276 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:04:56.180935 (+   659us) log.cc:844] Serialized 3856 byte log entry
0709 13:04:56.505759 (+324824us) op_driver.cc:464] REPLICATION: finished
0709 13:04:56.654966 (+149207us) write_op.cc:301] APPLY: starting
0709 13:04:56.680700 (+ 25734us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:56.824740 (+144040us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:56.824856 (+   116us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:57.273008 (+448152us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:57.304763 (+ 31755us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=20,mrs_lookups=0
0709 13:04:57.310546 (+  5783us) write_op.cc:312] APPLY: finished
0709 13:04:57.384235 (+ 73689us) log.cc:844] Serialized 97 byte log entry
0709 13:04:57.399333 (+ 15098us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:57.410368 (+ 11035us) write_op.cc:454] Released schema lock
0709 13:04:57.410431 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:57.410823 (+   392us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:57.410854 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2364578,"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":118383,"apply.run_cpu_time_us":20054,"apply.run_wall_time_us":755837,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45947,"num_ops":10,"prepare.queue_time_us":348361,"prepare.run_cpu_time_us":30379,"prepare.run_wall_time_us":349090,"raft.queue_time_us":95269,"raft.run_cpu_time_us":7103,"raft.run_wall_time_us":82652,"replication_time_us":311638,"thread_start_us":317626,"threads_started":4,"wal-append.queue_time_us":192518,"wal-append.run_cpu_time_us":39954,"wal-append.run_wall_time_us":2092565}]]}
I20260709 13:04:58.723400  8150 catalog_manager.cc:5697] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 15 to 16, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 16 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:58.725013  8150 catalog_manager.cc:5697] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 16 to 19, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 19 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:58.708278 16147 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:58.733707  8324 raft_consensus.cc:2468] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
I20260709 13:04:58.734025 16147 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:58.733800  8150 catalog_manager.cc:5697] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 15 to 17, leader changed from <none> to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 17 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:58.736931 16147 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:58.737363  8150 catalog_manager.cc:5697] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 16 to 22, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 22 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:58.739529  8223 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:58.713567 16145 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:58.740975  8150 catalog_manager.cc:5697] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 17 to 19, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 19 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:58.746930 16145 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 election: Requested vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:58.745671  8150 catalog_manager.cc:5697] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 16 to 17, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 17 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:58.753115  8150 catalog_manager.cc:5697] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 25 to 26, leader changed from <none> to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 26 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:58.759645 15880 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:58.759773  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 50490.7us
I20260709 13:04:58.760715 15880 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.763893  8223 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:58.761654 15921 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:04:58.766237  8369 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:58.766543 15921 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:58.767287 16075 raft_consensus.cc:2804] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:58.768872 15921 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:58.761351 15880 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:58.780530  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:58.780648  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:04:58.781273  8331 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 21.
I20260709 13:04:58.781744  9002 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:58.782294 15924 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:58.783036  8220 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:58.785856  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:58.792268  8325 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:58.789177  8150 catalog_manager.cc:5697] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 21 to 23, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 23 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:58.797534  8405 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:58.798810 15921 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:58.799821 16075 raft_consensus.cc:697] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:04:58.800608 16147 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:58.801483 16147 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.802120 16147 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:58.803735 15921 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.804330 15921 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:58.816423 15644 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.810513  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:58.819423 15644 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:04:58.820497  8972 consensus_peers.cc:597] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:58.827659  8223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 63848.9us
I20260709 13:04:58.838792  8220 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:58.859241  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:58.859860  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:58.860716  8331 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:58.852766  8150 catalog_manager.cc:5697] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 18 to 19, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 19 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:04:58.807955 15921 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:58.874311  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:58.878861  9002 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 24.
I20260709 13:04:58.810758 15924 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:58.816179 16145 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.898053  8223 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:04:58.802325 16075 consensus_queue.cc:237] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:58.906059  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:58.906291  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:58.909238  8404 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:58.910054  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:04:58.908308  8369 leader_election.cc:336] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:58.914400 16225 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:58.915030 16225 raft_consensus.cc:697] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 23 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:58.915577  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:58.919878 15644 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:58.892421 16191 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.923614  8222 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:58.903151 15921 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.933467  8404 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 24.
W20260709 13:04:58.922087  8369 consensus_peers.cc:597] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:58.938225  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:58.891799 16145 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:58.928802  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:58.948360  8402 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:04:58.921247 16132 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:58.936087 16023 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:58.945747 15921 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:58.960652 16132 raft_consensus.cc:697] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 24 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:58.964242 15921 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:58.964486 16132 consensus_queue.cc:237] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:58.965591 15938 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.966185 15938 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:58.967279 16145 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:58.916056 16225 consensus_queue.cc:237] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:58.968580 15938 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:58.973791  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:58.975968  8223 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:58.985036  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:58.981797  8295 consensus_peers.cc:597] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:58.936075 16191 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:58.988338  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:58.987991  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:58.986517 15921 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:59.012158  8369 leader_election.cc:336] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:59.026641 15880 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:59.027405 15880 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.027925 15880 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:59.035058 16191 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:59.042388 15644 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:04:59.043159 15644 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:59.048522  9001 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:04:59.050218 15644 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:59.051558 15880 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:04:59.051783  8295 leader_election.cc:336] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:59.052335  8295 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
W20260709 13:04:59.054456  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 45049.4us
W20260709 13:04:59.041158 16328 log.cc:927] Time spent T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.138s	user 0.003s	sys 0.000s
I20260709 13:04:59.060541  8402 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
W20260709 13:04:59.059074  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 121011us
W20260709 13:04:59.067611  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 52488.5us
W20260709 13:04:59.068567  8295 leader_election.cc:336] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:59.069123  8295 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:04:59.048887 16147 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:59.070770  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:59.071281 16160 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:59.070511 16075 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 13:04:59.073304  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:04:59.075160 16327 log.cc:927] Time spent T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:04:59.075533  8139 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 161018us
I20260709 13:04:59.066465  8222 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:59.086705 16147 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:59.091434 16147 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:59.093313  8222 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:59.101929  8331 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 19.
I20260709 13:04:59.105958 16147 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:59.082423 16176 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:59.137468  8314 tablet_service.cc:1558] Processing DeleteTablet for tablet 430fa172910841a49c0c80f2cbf76ae5 with delete_type TABLET_DATA_TOMBSTONED (TS ff7a6fe533e94c1bae555b4cae427728 not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:49298
I20260709 13:04:59.116560  8222 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
W20260709 13:04:59.150441  8369 leader_election.cc:336] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:59.151268  8398 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:04:59.151297  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:04:59.152623  8399 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:59.153903  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:59.153210  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:59.157160 16147 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:59.195132 16191 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:59.202070  8399 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:59.171572 16132 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.202721 16225 raft_consensus.cc:493] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.244681 16225 raft_consensus.cc:515] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:59.247342 16225 leader_election.cc:290] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:59.248718 16020 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.250685 16132 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:59.193480 15644 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:59.310110  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 171708us
I20260709 13:04:59.256011 16160 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:04:59.331489  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 159199us
W20260709 13:04:59.353654  8222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 191300us
I20260709 13:04:59.355098  8220 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:59.357599  8220 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:59.307137 15644 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:59.364466 15644 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:59.364984 15924 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:59.361778  8222 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:04:59.370577 16145 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:59.371122 16145 raft_consensus.cc:697] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 23 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:59.372144 16145 consensus_queue.cc:237] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:59.373040  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:59.300843 16020 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:04:59.326653  8139 catalog_manager.cc:4755] TS 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 81804f59e3124d27998ea80fc1c36663: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
W20260709 13:04:59.381611  8139 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 55012.7us
I20260709 13:04:59.390890  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:59.393858  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:59.382297 16023 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:04:59.358500 16160 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:59.419096 16147 raft_consensus.cc:697] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 19 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:04:59.454479 16347 tablet_replica.cc:333] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:04:59.431870 16176 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:59.495390  7164 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 8282)
I20260709 13:04:59.498946  8330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:59.504060 16176 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.504308 16132 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:59.505028 16342 log.cc:927] Time spent T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 13:04:59.504745 16176 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:59.505694 16020 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:59.504379 16344 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.520040  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:59.525031  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:59.526041  8324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:59.527891  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:59.531471  8327 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:59.535778  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:59.542200  8401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 28 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:59.544595 16160 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:59.547065 16147 consensus_queue.cc:237] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:59.549036  8405 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:04:59.534423 16344 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:59.552665  8330 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
I20260709 13:04:59.553766  8332 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:04:59.554505  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:59.554690  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 28 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:59.555059  8329 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:59.556070  8332 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:04:59.556418  8323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82abcf8a2a574841972243a029be749d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:59.557128  8323 raft_consensus.cc:2393] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 20: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:04:59.557964 16176 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:59.552733 16344 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:04:59.562179 16347 raft_consensus.cc:2243] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.563154 16347 raft_consensus.cc:2272] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.568367  8404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:59.571091  8220 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:59.577909  8220 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:59.576455  8328 raft_consensus.cc:3060] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:04:59.575440  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:59.582350  8398 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 21.
I20260709 13:04:59.578485 16225 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:59.584836  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:04:59.586566 15983 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:59.586863 16225 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:59.586943 15983 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:59.588457 15983 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:59.588728  8220 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:59.594113 16225 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:59.596716 16225 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:04:59.600412 16347 ts_tablet_manager.cc:1916] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:59.607862 16225 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:04:59.616063 15644 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:04:59.616714 15644 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:59.613901  8297 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:04:59.619532 16344 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:59.619755 15644 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:59.619916 16344 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:59.620466 16344 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:59.622045  8329 raft_consensus.cc:1275] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:04:59.622936 15644 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.623574 15644 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:59.624027  8328 raft_consensus.cc:1275] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:04:59.625728 15644 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:59.630172  8398 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:04:59.631301  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:59.632710  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:59.633420  8402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:04:59.630179  8220 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:59.628806 15983 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:59.642091  8220 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f
I20260709 13:04:59.656375  8223 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:59.644234  8327 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 23.
I20260709 13:04:59.675701  8220 leader_election.cc:304] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:04:59.656160 15880 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:04:59.657145 15644 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:59.685256  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:59.676584 16020 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:59.693603  9000 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:04:59.669047 16023 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 13:04:59.682242 15644 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:59.684396 16191 raft_consensus.cc:3060] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:59.698580  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:59.676923 16146 consensus_queue.cc:1048] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 13:04:59.683454 16358 consensus_queue.cc:1048] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
W20260709 13:04:59.707911  8369 consensus_peers.cc:597] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:59.700021 15644 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:59.696342 16356 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.741626 16141 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:04:59.742625 16358 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:04:59.743180 16358 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:59.745280 16358 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:59.753587  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:59.754395  8327 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 23: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
W20260709 13:04:59.785529  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 277834us
I20260709 13:04:59.802194 16344 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:04:59.807564  8223 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:59.788039 16147 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.810173  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b5f14332d64504837ae4e2df77e409" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:59.817740 15983 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:04:59.818898  8327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:04:59.819823  8327 raft_consensus.cc:2393] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 22: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:04:59.831260  8223 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:59.730028 16356 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:59.839574 15880 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:59.883545 16191 raft_consensus.cc:2749] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:59.837575 16147 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:59.864934  8298 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:04:59.859167 16132 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:59.910975  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 46172.4us
I20260709 13:04:59.839184 16225 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.052s
W20260709 13:04:59.920181  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 107047us
W20260709 13:04:59.916625  8369 leader_election.cc:336] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:59.923130  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 111852us
I20260709 13:04:59.923630  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:04:59.925096  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 52076.4us
W20260709 13:04:59.937608  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 98197.5us
I20260709 13:04:59.913818 16132 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:04:59.947093 16344 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:04:59.948976 16372 log.cc:927] Time spent T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:04:59.949715 16385 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:59.953495 16355 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:59.958508 16132 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:59.963927  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:59.974067  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:59.973618  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:04:59.976825 16356 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:59.983255  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:04:59.986627  8332 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:04:59.988395  8220 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:00.017215 16360 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:00.021778  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:05:00.022719  8400 raft_consensus.cc:2393] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 22: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
W20260709 13:05:00.014102  8295 leader_election.cc:336] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:00.024221  8295 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:05:00.026564 16249 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:00.031605 16147 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:04:59.999037 15924 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:05:00.043217  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:05:00.047384  8325 raft_consensus.cc:1275] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:05:00.049778  8295 leader_election.cc:336] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:00.050187  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 133607us
W20260709 13:05:00.051268  8369 leader_election.cc:336] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:00.051690  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:00.052441  8223 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:00.065912 16020 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:00.066097 16334 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:05:00.072695 16385 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:00.077148  8295 leader_election.cc:336] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:00.077719  8295 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:05:00.082377 16360 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:05:00.122491 15644 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:00.125734 15644 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:05:00.126093  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 46279.5us
I20260709 13:05:00.129850  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:00.132573  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:05:00.133162  8405 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:00.146459  8406 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:05:00.147570  8222 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:05:00.151993  8405 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 24.
I20260709 13:05:00.149204 16352 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:05:00.179721  8222 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f; no voters: 
I20260709 13:05:00.185316  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:05:00.188548  8332 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:00.187569 16352 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 13:05:00.198922 16023 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:05:00.206030 16347 ts_tablet_manager.cc:1929] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 14.11
I20260709 13:05:00.207362 16347 log.cc:1199] T 430fa172910841a49c0c80f2cbf76ae5 P ff7a6fe533e94c1bae555b4cae427728: Deleting WAL directory at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0/minicluster-data/ts-1-root/wals/430fa172910841a49c0c80f2cbf76ae5
I20260709 13:05:00.209236 16347 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49298 (request call id 0) took 1094 ms. Trace:
I20260709 13:05:00.212841 16347 rpcz_store.cc:276] 0709 13:04:59.131280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:59.137074 (+  5794us) service_pool.cc:224] Handling call
0709 13:04:59.429873 (+292799us) ts_tablet_manager.cc:1151] Deleting tablet 430fa172910841a49c0c80f2cbf76ae5
0709 13:04:59.443562 (+ 13689us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:05:00.072122 (+628560us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:00.205892 (+133770us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:00.209146 (+  3254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25226,"tablet-delete.queue_time_us":263721,"thread_start_us":115036,"threads_started":1}
I20260709 13:05:00.214668  8139 catalog_manager.cc:5028] TS ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): tablet 430fa172910841a49c0c80f2cbf76ae5 (table test-workload [id=9be9e9c7ed2c46b2b2ae65f2089d588a]) successfully deleted
I20260709 13:05:00.218155 15644 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:05:00.220333 16352 consensus_queue.cc:1048] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:00.226471  8332 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c42259d7c4b463a9f485f42c320c81b in term 24.
I20260709 13:05:00.241725  8398 raft_consensus.cc:1275] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:05:00.236545 16352 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.252360 16352 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:00.254742 16352 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:00.264421  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:00.270193  8329 raft_consensus.cc:2393] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 22: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:05:00.230309 15644 raft_consensus.cc:697] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 24 LEADER]: Becoming Leader. State: Replica: 8c42259d7c4b463a9f485f42c320c81b, State: Running, Role: LEADER
I20260709 13:05:00.272634 15921 consensus_queue.cc:1048] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 13:05:00.349959 15644 consensus_queue.cc:237] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:00.377508 16225 raft_consensus.cc:493] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.378104 16225 raft_consensus.cc:515] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:00.380872 16225 leader_election.cc:290] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:05:00.374951 16397 log.cc:927] Time spent T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:05:00.385480  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:05:00.386524  8972 consensus_peers.cc:597] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:00.388396  8972 consensus_peers.cc:597] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:00.397981  8972 consensus_peers.cc:597] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:00.406291  8150 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45456 (request call id 288) took 16709 ms (client timeout 15000 ms). Trace:
W20260709 13:05:00.406913  8150 rpcz_store.cc:269] 0709 13:04:43.696611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.696879 (+   268us) service_pool.cc:224] Handling call
0709 13:05:00.406172 (+16709293us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:59.364027 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:59.364689 (+   662us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:59.364747 (+    58us) write_op.cc:435] Acquired schema lock
0709 13:04:59.364764 (+    17us) tablet.cc:662] Decoding operations
0709 13:04:59.367298 (+  2534us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:59.367389 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:59.367420 (+    31us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:04:59.369013 (+  1593us) tablet.cc:701] Row locks acquired
0709 13:04:59.369072 (+    59us) write_op.cc:260] PREPARE: finished
0709 13:04:59.369275 (+   203us) write_op.cc:270] Start()
0709 13:04:59.369498 (+   223us) write_op.cc:276] Timestamp: P: 1783602299369198 usec, L: 0
0709 13:04:59.369560 (+    62us) op_driver.cc:348] REPLICATION: starting
0709 13:04:59.493739 (+124179us) log.cc:844] Serialized 5834 byte log entry
0709 13:04:59.804926 (+311187us) op_driver.cc:464] REPLICATION: finished
0709 13:04:59.815128 (+ 10202us) spinlock_profiling.cc:243] Waited 5.68 ms on lock 0x7b54003bf454. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8828727571 00007f88287b9e28 00007f882c1f4581 00007f882c1f3d58 00007f882c1f89f1 00007f882c1f89a9 00007f882c1f8939 00007f882c1f8901 00007f882c1f7b2d 000000000038f354 000000000038f199
0709 13:05:00.025956 (+210828us) write_op.cc:301] APPLY: starting
0709 13:05:00.026086 (+   130us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:00.259115 (+233029us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:00.259157 (+    42us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:00.287722 (+ 28565us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:00.288083 (+   361us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=30,mrs_lookups=0
0709 13:05:00.288135 (+    52us) write_op.cc:312] APPLY: finished
0709 13:05:00.288595 (+   460us) log.cc:844] Serialized 139 byte log entry
0709 13:05:00.288799 (+   204us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:00.288960 (+   161us) write_op.cc:454] Released schema lock
0709 13:05:00.289042 (+    82us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:00.300238 (+ 11196us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:00.302762 (+  2524us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":630895,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":185613,"apply.run_cpu_time_us":23164,"apply.run_wall_time_us":286987,"cfile_cache_hit":62,"cfile_cache_hit_bytes":66857,"num_ops":15,"prepare.queue_time_us":275709,"prepare.run_cpu_time_us":18162,"prepare.run_wall_time_us":451349,"raft.queue_time_us":63938,"raft.run_cpu_time_us":28476,"raft.run_wall_time_us":113393,"replication_time_us":435171,"spinlock_wait_cycles":13064960,"thread_start_us":533438,"threads_started":4,"wal-append.queue_time_us":196944}]]}
I20260709 13:05:00.430696  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:00.437711  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:00.439934  8406 raft_consensus.cc:2393] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c42259d7c4b463a9f485f42c320c81b in current term 22: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:05:00.445170  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac7f1e5192d48a98a850cb0ece17ad5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:00.450219  8146 catalog_manager.cc:5697] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 13 to 23, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 23 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:05:00.460850  8220 leader_election.cc:304] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:00.452479  8146 catalog_manager.cc:5697] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:00.443616 16023 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:00.472568  8972 consensus_peers.cc:597] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:00.482133 16023 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:05:00.495998  7164 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 8282)
I20260709 13:05:00.494385  8222 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:00.500133 15921 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:00.466971  8146 catalog_manager.cc:5697] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 23 to 26. New cstate: current_term: 26 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:05:00.521968  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:00.524691  8146 catalog_manager.cc:5697] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:00.505609 16147 raft_consensus.cc:2749] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 13:05:00.544049  8146 catalog_manager.cc:5697] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 16 to 17, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 17 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:00.546211  8146 catalog_manager.cc:5697] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 14 to 17, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 17 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:00.553681  8146 catalog_manager.cc:5697] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 17 to 24, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 24 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:05:00.543128  8369 consensus_peers.cc:597] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:00.540764 16400 log.cc:927] Time spent T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:05:00.563261  8146 catalog_manager.cc:5697] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:00.566926  8139 catalog_manager.cc:4755] TS 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 81804f59e3124d27998ea80fc1c36663: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 13:05:00.576591  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeca7eb827a04189af4de94af55ffb91" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:05:00.591816  8972 consensus_peers.cc:597] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:00.563823 16023 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:05:00.595355  8972 consensus_peers.cc:597] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:00.602200 16141 raft_consensus.cc:493] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:05:00.602803 16141 raft_consensus.cc:515] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:00.606434 15644 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.607048 15644 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:00.609166 15644 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:00.610807  8222 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:00.612591 15921 raft_consensus.cc:2749] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:00.619661 16141 leader_election.cc:290] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:00.621620  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:00.622239  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:05:00.628618  8295 consensus_peers.cc:597] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:00.629525 15924 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:00.648553  8369 consensus_peers.cc:597] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:00.653636  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94bcf9755aa4f0abddcd5b9fbc2939f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:05:00.657451  8972 consensus_peers.cc:597] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:00.662190 15644 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.662900  8222 leader_election.cc:304] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:05:00.662829 15644 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:05:00.663823  8369 consensus_peers.cc:597] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:00.664026  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:05:00.664463  8369 consensus_peers.cc:597] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:00.657507  8146 catalog_manager.cc:5697] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 19 to 21, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 21 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:05:00.665223 15644 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:00.665493  8146 catalog_manager.cc:5697] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 13 to 28. New cstate: current_term: 28 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:05:00.669456  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:00.667332  8222 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:00.670984 16387 raft_consensus.cc:2749] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:00.677847 16387 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:00.678797  8146 catalog_manager.cc:5697] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 21 to 22, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 22 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:05:00.679191 16141 raft_consensus.cc:493] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f865c4d761f5446eb12b3a029234772e)
I20260709 13:05:00.680100 16141 raft_consensus.cc:515] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:00.680302  8146 catalog_manager.cc:5697] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 16 to 20, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 20 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:00.682600 16141 leader_election.cc:290] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:00.686540  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:05:00.689769  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:05:00.687665  8369 consensus_peers.cc:597] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:00.691759  8369 consensus_peers.cc:597] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:00.697927  8146 catalog_manager.cc:5697] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:05:00.699558  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430fa172910841a49c0c80f2cbf76ae5" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:00.706266  8222 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:00.713951  8146 catalog_manager.cc:5697] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:05:00.715546 16141 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:00.716728  8222 leader_election.cc:304] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:05:00.728950 16141 raft_consensus.cc:2749] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:00.729588  8146 catalog_manager.cc:5697] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 11 to 13, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 13 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:00.725512  8369 consensus_peers.cc:597] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:00.748668  8146 catalog_manager.cc:5697] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 17 to 25, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 25 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:05:00.751083 16387 raft_consensus.cc:493] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.751278  8146 catalog_manager.cc:5697] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 18 to 22, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 22 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:05:00.751641 16387 raft_consensus.cc:515] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:00.752693  8146 catalog_manager.cc:5697] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 14 to 21. New cstate: current_term: 21 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:00.753980 16387 leader_election.cc:290] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:00.755671  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:00.756443  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9159db9f370400d98f8503d06a33a33" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:00.759738  8222 leader_election.cc:304] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:00.760599 16387 raft_consensus.cc:2749] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:00.791105  8146 catalog_manager.cc:5697] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 16 to 17, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 17 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:00.792933  8146 catalog_manager.cc:5697] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 15 to 22, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 22 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } }
I20260709 13:05:00.803424  8329 raft_consensus.cc:1275] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:05:00.805115 16141 consensus_queue.cc:1048] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:05:00.805838  8972 consensus_peers.cc:597] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:00.818137  8146 catalog_manager.cc:5697] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
W20260709 13:05:00.863752  8369 consensus_peers.cc:597] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:00.864018  8399 raft_consensus.cc:1275] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Refusing update from remote peer 8c42259d7c4b463a9f485f42c320c81b: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:05:00.865599 15644 consensus_queue.cc:1048] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:00.866411  8146 catalog_manager.cc:5697] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 20 to 25, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 25 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:05:00.868779  8146 catalog_manager.cc:5697] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 19 to 23, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 23 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:05:00.875795  8146 catalog_manager.cc:5697] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 19 to 22, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 22 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:05:00.895159  8146 catalog_manager.cc:5697] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 18 to 21, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 21 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:05:00.909198 15644 raft_consensus.cc:493] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.909794 15644 raft_consensus.cc:515] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } }
I20260709 13:05:00.912460  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:00.915661  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1fdbe7b73a540ddb2e5cb473a8404de" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:00.917419  8220 leader_election.cc:304] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:00.918344 16387 raft_consensus.cc:2749] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:00.919546 15644 leader_election.cc:290] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:05:00.932386 15644 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.933133 15644 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:05:00.938035  8295 consensus_peers.cc:597] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:00.944115  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:05:00.945982  8295 consensus_peers.cc:597] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:00.951133  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "767eba28679649519167576023555b60" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:05:00.953609  8223 leader_election.cc:304] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:00.954512 16387 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:00.950014 15644 leader_election.cc:290] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:00.985222 15644 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.985770 15644 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:00.988520  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:00.991375 16387 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:05:00.991920 16387 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:00.994014 16387 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:00.994730 15644 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:00.996272  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:00.997885  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:01.000219  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:01.000595  8222 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:01.001912  8220 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:01.003229 16141 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:01.004037 16141 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:01.028321  8369 consensus_peers.cc:597] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:01.029299  8295 consensus_peers.cc:597] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:01.044060  8369 consensus_peers.cc:597] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:01.070187  8369 consensus_peers.cc:597] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:01.115267 15644 raft_consensus.cc:493] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.115851 15644 raft_consensus.cc:515] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:01.125460  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:01.129273 15644 leader_election.cc:290] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:01.131244  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2e021fd91643b49aa8e553436c4b69" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:01.135447  8222 leader_election.cc:304] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:01.136729 16145 raft_consensus.cc:2749] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:05:01.185962  8369 consensus_peers.cc:597] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:01.227336  8972 consensus_peers.cc:597] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:01.232283  8972 consensus_peers.cc:597] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:01.260852 16141 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.261523 16141 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:01.265080  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:01.272826  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7c15354a2a84f868942919f011af448" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:05:01.274603  8223 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:01.270560 16141 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:01.276441 15924 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:05:01.304243 15924 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.304927 15924 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:01.308063  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:01.311278 15924 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:01.311615  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d966cf0fb9e24bd9993fb60505fdc62c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:01.317092  8222 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:01.319036 16145 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:05:01.405200  8369 consensus_peers.cc:597] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:01.471798  8295 consensus_peers.cc:597] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:01.477089 16141 raft_consensus.cc:493] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.477773 16141 raft_consensus.cc:515] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:01.480101 16141 leader_election.cc:290] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:01.482322  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:01.484501  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed318aa932504f70b21cfa8e646283ac" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:01.488135  8220 leader_election.cc:304] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:01.490279 15644 raft_consensus.cc:2749] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:05:01.496800  7164 thread.cc:527] Waited for 5000ms trying to join with heartbeat (tid 8282)
I20260709 13:05:01.501530  8146 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45448 (request call id 289) took 7823 ms. Trace:
I20260709 13:05:01.502030  8146 rpcz_store.cc:276] 0709 13:04:53.682913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.228639 (+3545726us) service_pool.cc:224] Handling call
0709 13:05:01.501415 (+4272776us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:00.981371 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:00.981945 (+   574us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:00.981979 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:05:00.981996 (+    17us) tablet.cc:662] Decoding operations
0709 13:05:00.983819 (+  1823us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:00.983902 (+    83us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:00.983932 (+    30us) tablet.cc:685] Acquiring locks for 25 operations
0709 13:05:01.026170 (+ 42238us) tablet.cc:701] Row locks acquired
0709 13:05:01.026192 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:05:01.026348 (+   156us) write_op.cc:270] Start()
0709 13:05:01.026593 (+   245us) write_op.cc:276] Timestamp: P: 1783602301026298 usec, L: 0
0709 13:05:01.026628 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 13:05:01.027409 (+   781us) log.cc:844] Serialized 9635 byte log entry
0709 13:05:01.119896 (+ 92487us) op_driver.cc:464] REPLICATION: finished
0709 13:05:01.213931 (+ 94035us) write_op.cc:301] APPLY: starting
0709 13:05:01.241632 (+ 27701us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:01.339426 (+ 97794us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:01.339531 (+   105us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:01.407391 (+ 67860us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:01.407782 (+   391us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=50,mrs_lookups=0
0709 13:05:01.407851 (+    69us) write_op.cc:312] APPLY: finished
0709 13:05:01.408436 (+   585us) log.cc:844] Serialized 220 byte log entry
0709 13:05:01.409318 (+   882us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:01.409580 (+   262us) write_op.cc:454] Released schema lock
0709 13:05:01.409644 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:01.410099 (+   455us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:01.410131 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":490528,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":85821,"apply.run_cpu_time_us":34281,"apply.run_wall_time_us":198368,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108677,"num_ops":25,"prepare.queue_time_us":43505,"prepare.run_cpu_time_us":6354,"prepare.run_wall_time_us":46355,"replication_time_us":93062,"thread_start_us":93457,"threads_started":2}]]}
W20260709 13:05:01.585522  8295 consensus_peers.cc:597] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:01.639415 16141 raft_consensus.cc:493] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.640031 16141 raft_consensus.cc:515] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:01.642447 16141 leader_election.cc:290] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:01.645505  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:01.650492  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32240ced15f4c36a9ce65ca5a13634a" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:05:01.665326  8152 catalog_manager.cc:5697] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 22 to 26, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 26 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:01.669538  8152 catalog_manager.cc:5697] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:01.671996  8152 catalog_manager.cc:5697] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b reported cstate change: term changed from 14 to 15, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 15 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:01.677515  8149 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45456 (request call id 285) took 92359 ms (client timeout 15000 ms). Trace:
I20260709 13:05:01.680846 15924 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:01.682932 15924 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:05:01.682157  8149 rpcz_store.cc:269] 0709 13:03:29.317977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.325632 (+  7655us) service_pool.cc:224] Handling call
0709 13:05:01.677285 (+92351653us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1094,"reported_tablets":50}
I20260709 13:05:01.685057 15924 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:01.686169  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:01.680616  8223 leader_election.cc:304] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:05:01.704619  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2323b22f124b407c95caf5525949dd6c" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:05:01.704249 15924 raft_consensus.cc:2749] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:01.709568  8223 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:01.711190 15924 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:01.806047 16141 raft_consensus.cc:493] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.806620 16141 raft_consensus.cc:515] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:01.808905 16141 leader_election.cc:290] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:01.812011  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:05:01.818132  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb79894cecb541e3b7dd164271cadc16" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:01.819854  8220 leader_election.cc:304] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:01.821163 16141 raft_consensus.cc:2749] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:01.842654  8149 catalog_manager.cc:5697] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 22 to 23, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 23 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:05:01.844436  8149 catalog_manager.cc:5697] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 17 to 26, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 26 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:05:01.846464  8149 catalog_manager.cc:5697] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 18 to 23. New cstate: current_term: 23 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:05:01.847958  8149 catalog_manager.cc:5697] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 17 to 20, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 20 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:05:01.850605  8149 catalog_manager.cc:5697] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 25 to 26, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 26 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:05:01.852311  8149 catalog_manager.cc:5697] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:05:01.853734  8149 catalog_manager.cc:5697] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 23 to 27, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 27 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
W20260709 13:05:01.849767  8152 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45434 (request call id 286) took 26841 ms (client timeout 15000 ms). Trace:
W20260709 13:05:01.854880  8152 rpcz_store.cc:269] 0709 13:04:35.008487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.008675 (+   188us) service_pool.cc:224] Handling call
0709 13:05:01.849698 (+26841023us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:01.749853 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:01.750465 (+   612us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:01.750508 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:05:01.750529 (+    21us) tablet.cc:662] Decoding operations
0709 13:05:01.751353 (+   824us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:01.751435 (+    82us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:01.751468 (+    33us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:05:01.751864 (+   396us) tablet.cc:701] Row locks acquired
0709 13:05:01.751885 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:05:01.752035 (+   150us) write_op.cc:270] Start()
0709 13:05:01.752152 (+   117us) write_op.cc:276] Timestamp: P: 1783602301751985 usec, L: 0
0709 13:05:01.752175 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:05:01.752841 (+   666us) log.cc:844] Serialized 1355 byte log entry
0709 13:05:01.798070 (+ 45229us) op_driver.cc:464] REPLICATION: finished
0709 13:05:01.798780 (+   710us) write_op.cc:301] APPLY: starting
0709 13:05:01.798876 (+    96us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:01.800778 (+  1902us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:01.800886 (+   108us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:01.802860 (+  1974us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:01.803065 (+   205us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=6,mrs_lookups=0
0709 13:05:01.803115 (+    50us) write_op.cc:312] APPLY: finished
0709 13:05:01.803325 (+   210us) log.cc:844] Serialized 41 byte log entry
0709 13:05:01.803494 (+   169us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:01.803595 (+   101us) write_op.cc:454] Released schema lock
0709 13:05:01.803641 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:01.804067 (+   426us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:01.804108 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":70195,"reported_tablets":50,"spinlock_wait_cycles":4864,"child_traces":[["op",{"apply.queue_time_us":483,"apply.run_cpu_time_us":5224,"apply.run_wall_time_us":5225,"cfile_cache_hit":12,"cfile_cache_hit_bytes":12546,"num_ops":3,"prepare.queue_time_us":39406,"prepare.run_cpu_time_us":3094,"prepare.run_wall_time_us":3397,"replication_time_us":45737,"thread_start_us":49137,"threads_started":1}]]}
I20260709 13:05:01.876271  8149 catalog_manager.cc:5697] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 16 to 23, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 23 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:05:01.877971  8149 catalog_manager.cc:5697] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 20 to 26, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 26 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:05:01.879525  8149 catalog_manager.cc:5697] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 15 to 19, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 19 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:05:01.881102  8149 catalog_manager.cc:5697] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:05:01.882521  8149 catalog_manager.cc:5697] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 17 to 20, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 20 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:05:01.883939  8149 catalog_manager.cc:5697] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 13 to 18, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 18 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:05:01.885480  8149 catalog_manager.cc:5697] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:01.887487  8149 catalog_manager.cc:5697] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:05:01.888945  8149 catalog_manager.cc:5697] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:01.888942  8295 consensus_peers.cc:597] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:01.890857  8149 catalog_manager.cc:5697] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 26 to 28. New cstate: current_term: 28 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:05:01.892266  8149 catalog_manager.cc:5697] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 17 to 19, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 19 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:05:01.893250 16141 raft_consensus.cc:493] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:05:01.893846 16141 raft_consensus.cc:515] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:01.893567  8149 catalog_manager.cc:5697] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 21 to 22, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 22 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:05:01.895107  8149 catalog_manager.cc:5697] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 16 to 22, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 22 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:01.896006 16141 leader_election.cc:290] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:01.896749  8149 catalog_manager.cc:5697] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:01.898361  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:01.898202  8149 catalog_manager.cc:5697] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 26 to 27, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 27 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:01.899557  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e1a14c7c9f49e2b92fcea2a1bb191d" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:01.899775  8149 catalog_manager.cc:5697] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 15 to 20, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 20 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:01.900823 16416 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.901326 16416 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:01.901650  8149 catalog_manager.cc:5697] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 15 to 19, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 19 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:05:01.903364 16416 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:05:01.903072  8149 catalog_manager.cc:5697] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 18 to 23. New cstate: current_term: 23 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:05:01.904413  8149 catalog_manager.cc:5697] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:05:01.905324  8220 leader_election.cc:304] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:01.906774 16141 raft_consensus.cc:2749] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:01.905676  8149 catalog_manager.cc:5697] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 18 to 21, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 21 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:05:01.908351  8149 catalog_manager.cc:5697] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 26 to 29, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 29 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } attrs { replace: true } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:05:01.910015  8149 catalog_manager.cc:5697] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:05:01.909320  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:05:01.910957  8369 leader_election.cc:336] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:01.911674  8325 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 22.
I20260709 13:05:01.911675  8149 catalog_manager.cc:5697] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:05:01.913504  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:05:01.915081  8149 catalog_manager.cc:5697] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 20 to 23, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 23 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:05:01.916200 16416 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:05:01.916623 16416 raft_consensus.cc:493] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.917132 16416 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:01.916656  8149 catalog_manager.cc:5697] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:05:01.918339  8149 catalog_manager.cc:5697] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 18 to 26, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 26 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
I20260709 13:05:01.919718  8149 catalog_manager.cc:5697] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 19 to 21, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 21 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:05:01.931453 16416 raft_consensus.cc:515] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:01.934803 16416 leader_election.cc:290] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:05:01.938575  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc5b6425934c0aa85546e0b33f4abd" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:05:01.941798  8325 raft_consensus.cc:3060] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:05:01.945678  8369 leader_election.cc:336] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:01.948828  8325 raft_consensus.cc:2468] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 23.
I20260709 13:05:01.961687  8369 leader_election.cc:304] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:05:01.971354 16416 raft_consensus.cc:2804] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:05:01.975394 16416 raft_consensus.cc:697] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 23 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:05:01.976917 16416 consensus_queue.cc:237] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:05:02.109807  8369 consensus_peers.cc:597] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:02.131793 15924 raft_consensus.cc:493] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.132416 15924 raft_consensus.cc:515] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:02.135574  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:02.137506 15924 leader_election.cc:290] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:02.139947  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83839ff3ce80447bb72b3c8d15b630ff" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:05:02.152962  8223 leader_election.cc:304] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:02.154491 15644 raft_consensus.cc:2749] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:02.224076 16145 raft_consensus.cc:493] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.224673 16145 raft_consensus.cc:515] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:02.227928  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:02.229732 16145 leader_election.cc:290] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:02.230865  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fade5d45b9804cc98377ac6ab69be94b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:02.232544  8220 leader_election.cc:304] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:02.233731 16141 raft_consensus.cc:2749] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:02.281472  8153 catalog_manager.cc:5697] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:05:02.281495  8139 catalog_manager.cc:4755] TS 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 81804f59e3124d27998ea80fc1c36663: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 13:05:02.297853 16141 raft_consensus.cc:493] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.298484 16141 raft_consensus.cc:515] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:02.301887  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:02.311347 16141 leader_election.cc:290] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:02.314292  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007e904f46ef4a1a978f97d557d2a65b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:05:02.319343  8223 leader_election.cc:304] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:05:02.321450 16141 raft_consensus.cc:2749] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:02.394776  8369 consensus_peers.cc:597] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:02.413712 16145 raft_consensus.cc:493] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.414321 16145 raft_consensus.cc:515] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:02.419147  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:02.416628 16145 leader_election.cc:290] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:02.460135  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdf12882ac9e4f7d9121a5fcb9d4efaf" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:05:02.497293  7164 thread.cc:527] Waited for 6000ms trying to join with heartbeat (tid 8282)
I20260709 13:05:02.558185 15924 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.558988 15924 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:02.567569  8331 raft_consensus.cc:1275] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:05:02.567952 15924 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:02.568750  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:02.569525 16416 consensus_queue.cc:1048] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:02.589984  8223 leader_election.cc:304] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:02.590893 16145 raft_consensus.cc:2749] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:05:02.599908  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10cffa7c6411440ca77cee6731faca67" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:05:02.627146  8369 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 279 similar messages]
I20260709 13:05:02.648423 16141 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:05:02.649034 16141 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:02.649875  8223 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:05:02.652640 16141 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:02.655490 16145 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:05:02.656535  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:02.669152  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c80a6b33dc431eac0fe35b37d4ea23" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
W20260709 13:05:02.690732  8153 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55754 (request call id 203) took 56414 ms (client timeout 15000 ms). Trace:
W20260709 13:05:02.691355  8153 rpcz_store.cc:269] 0709 13:04:06.276904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.277205 (+   301us) service_pool.cc:224] Handling call
0709 13:05:02.322484 (+56045279us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:02.322515 (+    31us) write_op.cc:454] Released schema lock
0709 13:05:02.690653 (+368138us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:02.286328 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:02.286935 (+   607us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:02.286978 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:05:02.286995 (+    17us) tablet.cc:662] Decoding operations
0709 13:05:02.287369 (+   374us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:02.287431 (+    62us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:02.287461 (+    30us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:05:02.287634 (+   173us) tablet.cc:701] Row locks acquired
0709 13:05:02.287647 (+    13us) write_op.cc:260] PREPARE: finished
0709 13:05:02.287781 (+   134us) write_op.cc:270] Start()
0709 13:05:02.287886 (+   105us) write_op.cc:276] Timestamp: P: 1783602302287736 usec, L: 0
0709 13:05:02.287913 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:05:02.288535 (+   622us) log.cc:844] Serialized 530 byte log entry
0709 13:05:02.290539 (+  2004us) op_driver.cc:464] REPLICATION: finished
0709 13:05:02.290965 (+   426us) write_op.cc:301] APPLY: starting
0709 13:05:02.291038 (+    73us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:02.291805 (+   767us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:02.291899 (+    94us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:02.292597 (+   698us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:02.292768 (+   171us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:05:02.292813 (+    45us) write_op.cc:312] APPLY: finished
0709 13:05:02.292998 (+   185us) log.cc:844] Serialized 25 byte log entry
0709 13:05:02.293180 (+   182us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:02.293264 (+    84us) write_op.cc:454] Released schema lock
0709 13:05:02.293301 (+    37us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":155652,"reported_tablets":32,"spinlock_wait_cycles":15616,"child_traces":[["op",{"apply.queue_time_us":216,"apply.run_cpu_time_us":2657,"apply.run_wall_time_us":2657,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4182,"num_ops":1,"prepare.queue_time_us":132,"prepare.run_cpu_time_us":2463,"prepare.run_wall_time_us":2462,"replication_time_us":2512}]]}
I20260709 13:05:02.698742 16145 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.699321 16145 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:02.701637 16145 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:02.704105  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:02.716950  8223 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:05:02.718108 15924 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:02.729336  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf42cb9d00c6491eb6b0cf8b896f7c5b" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:05:02.731572 15924 raft_consensus.cc:493] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.732142 15924 raft_consensus.cc:515] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:02.734766  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:02.737057 15924 leader_election.cc:290] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:02.738544  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3e4cc2f40346c0bc3bc739e3675685" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:02.741478  8222 leader_election.cc:304] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:02.742370 16141 raft_consensus.cc:2749] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:02.754647  8223 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:05:02.759965  8154 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55754 (request call id 202) took 72378 ms (client timeout 15000 ms). Trace:
W20260709 13:05:02.760453  8154 rpcz_store.cc:269] 0709 13:03:50.382013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.382241 (+   228us) service_pool.cc:224] Handling call
0709 13:05:02.759757 (+72377516us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":270437,"reported_tablets":32}
I20260709 13:05:02.762049 15924 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:02.764603  8148 catalog_manager.cc:5697] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 13 to 15, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 15 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:02.774930  8972 consensus_peers.cc:597] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:02.818267  8369 consensus_peers.cc:597] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:02.823196  8972 consensus_peers.cc:597] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:02.879292  8972 consensus_peers.cc:597] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:02.909181  8149 catalog_manager.cc:5697] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } }
W20260709 13:05:02.929260  8972 consensus_peers.cc:597] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:02.967002  8151 catalog_manager.cc:5697] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:02.969723  8151 catalog_manager.cc:5697] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 17 to 18, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 18 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:02.971501  8151 catalog_manager.cc:5697] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 22 to 23, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 23 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:02.978060  8148 catalog_manager.cc:5244] ChangeConfig:REMOVE_PEER RPC for tablet 430fa172910841a49c0c80f2cbf76ae5 with cas_config_opid_index 5: aborting the task: latest config opid_index 12; task opid_index 5
W20260709 13:05:02.980391  8148 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45456 (request call id 287) took 45811 ms (client timeout 15000 ms). Trace:
I20260709 13:05:02.973189  8151 catalog_manager.cc:5697] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 19 to 20, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 20 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:05:02.983253  8151 catalog_manager.cc:5697] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:02.987561  8151 catalog_manager.cc:5697] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:02.980778  8148 rpcz_store.cc:269] 0709 13:04:17.228854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.229124 (+   270us) service_pool.cc:224] Handling call
0709 13:04:18.764174 (+1535050us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b5400200490. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f882569a0b2 00007f882e613a89 00007f882e5abda6 00007f882e57d088 00007f882e67b37e 00007f882cb3c2a4 00007f882cb3c232 00007f882cb3c161 00007f882cb3c0dc
0709 13:05:02.980311 (+44216137us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:02.836492 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:02.837084 (+   592us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:02.837124 (+    40us) write_op.cc:435] Acquired schema lock
0709 13:05:02.837144 (+    20us) tablet.cc:662] Decoding operations
0709 13:05:02.838058 (+   914us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:02.838133 (+    75us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:02.838165 (+    32us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:05:02.838348 (+   183us) tablet.cc:701] Row locks acquired
0709 13:05:02.838364 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:05:02.838496 (+   132us) write_op.cc:270] Start()
0709 13:05:02.838606 (+   110us) write_op.cc:276] Timestamp: P: 1783602302838446 usec, L: 0
0709 13:05:02.838625 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:05:02.839268 (+   643us) log.cc:844] Serialized 530 byte log entry
0709 13:05:02.885466 (+ 46198us) op_driver.cc:464] REPLICATION: finished
0709 13:05:02.894352 (+  8886us) write_op.cc:301] APPLY: starting
0709 13:05:02.894443 (+    91us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:02.895733 (+  1290us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:02.895827 (+    94us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:02.896569 (+   742us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:02.896748 (+   179us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:05:02.896798 (+    50us) write_op.cc:312] APPLY: finished
0709 13:05:02.897004 (+   206us) log.cc:844] Serialized 25 byte log entry
0709 13:05:02.897182 (+   178us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:02.897264 (+    82us) write_op.cc:454] Released schema lock
0709 13:05:02.897303 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:02.928719 (+ 31416us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:02.928754 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":48457,"reported_tablets":50,"spinlock_wait_cycles":4771200,"child_traces":[["op",{"apply.queue_time_us":8616,"apply.run_cpu_time_us":3250,"apply.run_wall_time_us":34322,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8309,"num_ops":1,"prepare.queue_time_us":44378,"prepare.run_cpu_time_us":2623,"prepare.run_wall_time_us":14017,"replication_time_us":46664,"thread_start_us":53860,"threads_started":2}]]}
W20260709 13:05:03.025934  8972 consensus_peers.cc:597] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:03.052443  8369 consensus_peers.cc:597] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:03.064116  8150 catalog_manager.cc:5697] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 20 to 21, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 21 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:05:03.067708  8295 consensus_peers.cc:597] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:03.069070  8151 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55754 (request call id 205) took 17018 ms (client timeout 15000 ms). Trace:
I20260709 13:05:03.070514  8150 catalog_manager.cc:5697] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:05:03.072273  8150 catalog_manager.cc:5697] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 23 to 26, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 26 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:03.074321  8150 catalog_manager.cc:5697] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:05:03.069473  8151 rpcz_store.cc:269] 0709 13:04:46.053495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.504682 (+451187us) service_pool.cc:224] Handling call
0709 13:05:03.069000 (+16564318us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:02.996792 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:02.997460 (+   668us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:02.997501 (+    41us) write_op.cc:435] Acquired schema lock
0709 13:05:02.997514 (+    13us) tablet.cc:662] Decoding operations
0709 13:05:02.998288 (+   774us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:02.998343 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:02.998370 (+    27us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:05:02.998933 (+   563us) tablet.cc:701] Row locks acquired
0709 13:05:02.998949 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:05:02.999089 (+   140us) write_op.cc:270] Start()
0709 13:05:02.999206 (+   117us) write_op.cc:276] Timestamp: P: 1783602302999045 usec, L: 0
0709 13:05:02.999226 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:05:02.999874 (+   648us) log.cc:844] Serialized 2416 byte log entry
0709 13:05:03.002317 (+  2443us) op_driver.cc:464] REPLICATION: finished
0709 13:05:03.002958 (+   641us) write_op.cc:301] APPLY: starting
0709 13:05:03.003038 (+    80us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:03.006514 (+  3476us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:03.006636 (+   122us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:03.012331 (+  5695us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:03.012557 (+   226us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=12,mrs_lookups=0
0709 13:05:03.012620 (+    63us) write_op.cc:312] APPLY: finished
0709 13:05:03.012955 (+   335us) log.cc:844] Serialized 65 byte log entry
0709 13:05:03.013804 (+   849us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:03.013933 (+   129us) write_op.cc:454] Released schema lock
0709 13:05:03.013983 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:03.014351 (+   368us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:03.014383 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":19482,"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":431,"apply.run_cpu_time_us":8922,"apply.run_wall_time_us":11353,"cfile_cache_hit":24,"cfile_cache_hit_bytes":25092,"num_ops":6,"prepare.queue_time_us":3525,"prepare.run_cpu_time_us":3406,"prepare.run_wall_time_us":5361,"replication_time_us":2968}]]}
I20260709 13:05:03.078755  8150 catalog_manager.cc:5697] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
I20260709 13:05:03.080895  8150 catalog_manager.cc:5697] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 19 to 21, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 21 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:03.083460  8150 catalog_manager.cc:5697] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 16 to 17, leader changed from f865c4d761f5446eb12b3a029234772e (127.6.255.4) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 17 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:05:03.091775  8369 consensus_peers.cc:597] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:03.116832  8295 consensus_peers.cc:597] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:03.168313 16145 raft_consensus.cc:493] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.168900 16145 raft_consensus.cc:515] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:03.171242 16145 leader_election.cc:290] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:03.174306  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:05:03.175681  8972 consensus_peers.cc:597] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:03.178143  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3d99c80602e45da9c43a49cf83b8211" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:05:03.182439  8220 leader_election.cc:304] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:03.183988 15924 raft_consensus.cc:2749] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:05:03.189826 16145 raft_consensus.cc:493] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.190351 16145 raft_consensus.cc:515] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:03.193456  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:03.196519  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c2abe28fd449b094f4ccff4bac396e" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:03.198124  8222 leader_election.cc:304] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:03.198925 16141 raft_consensus.cc:2749] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:03.199276 16145 leader_election.cc:290] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:05:03.211122  8369 consensus_peers.cc:597] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:03.211889  8150 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45456 (request call id 289) took 2933 ms. Trace:
I20260709 13:05:03.212271  8150 rpcz_store.cc:276] 0709 13:05:00.282254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.454487 (+172233us) service_pool.cc:224] Handling call
0709 13:05:00.516745 (+ 62258us) spinlock_profiling.cc:243] Waited 3.67 ms on lock 0x7b5400200490. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f882569a640 00007f882e571b76 00007f882e67bc3e 00007f882cb3c2a4 00007f882cb3c232 00007f882cb3c161 00007f882cb3c0dc 00007f882cb3b392 00007f8826af948c 00007f8826af88c6
0709 13:05:03.211783 (+2695038us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:03.088127 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:03.088653 (+   526us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:03.088685 (+    32us) write_op.cc:435] Acquired schema lock
0709 13:05:03.088701 (+    16us) tablet.cc:662] Decoding operations
0709 13:05:03.089462 (+   761us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:03.089518 (+    56us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:03.089544 (+    26us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:05:03.090091 (+   547us) tablet.cc:701] Row locks acquired
0709 13:05:03.090107 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:05:03.090242 (+   135us) write_op.cc:270] Start()
0709 13:05:03.090339 (+    97us) write_op.cc:276] Timestamp: P: 1783602303090196 usec, L: 0
0709 13:05:03.090361 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:05:03.091285 (+   924us) log.cc:844] Serialized 2793 byte log entry
0709 13:05:03.093668 (+  2383us) op_driver.cc:464] REPLICATION: finished
0709 13:05:03.094088 (+   420us) write_op.cc:301] APPLY: starting
0709 13:05:03.094166 (+    78us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:03.098537 (+  4371us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:03.098643 (+   106us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:03.103091 (+  4448us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:03.103315 (+   224us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=14,mrs_lookups=0
0709 13:05:03.103379 (+    64us) write_op.cc:312] APPLY: finished
0709 13:05:03.106837 (+  3458us) log.cc:844] Serialized 73 byte log entry
0709 13:05:03.107798 (+   961us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:03.107930 (+   132us) write_op.cc:454] Released schema lock
0709 13:05:03.107986 (+    56us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:03.108323 (+   337us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:03.108352 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":927,"reported_tablets":50,"spinlock_wait_cycles":8430848,"child_traces":[["op",{"apply.queue_time_us":232,"apply.run_cpu_time_us":9909,"apply.run_wall_time_us":14195,"cfile_cache_hit":28,"cfile_cache_hit_bytes":29274,"num_ops":7,"prepare.queue_time_us":105,"prepare.run_cpu_time_us":3390,"prepare.run_wall_time_us":5189,"replication_time_us":3184,"spinlock_wait_cycles":1213184}]]}
W20260709 13:05:03.234794  8369 consensus_peers.cc:597] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:03.236632 16141 raft_consensus.cc:493] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.237207 16141 raft_consensus.cc:515] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:03.239300 16141 leader_election.cc:290] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:03.241725  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 28 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:05:03.251518  8295 consensus_peers.cc:597] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:03.255529  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec275b714efb4b8b92919025fbd79b1f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 28 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:03.257239  8220 leader_election.cc:304] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:03.258049 15924 raft_consensus.cc:493] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:05:03.258543 15924 raft_consensus.cc:515] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:03.260754 16141 raft_consensus.cc:2749] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:05:03.268350  8972 consensus_peers.cc:597] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:03.272540  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
W20260709 13:05:03.276046  8369 consensus_peers.cc:597] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:03.276705  8972 consensus_peers.cc:597] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:03.260665 15924 leader_election.cc:290] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:05:03.292732  8369 consensus_peers.cc:597] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:03.294040  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3340359c8445d8950a3e68e0070928" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:05:03.298214  8369 consensus_peers.cc:597] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:03.303625  8220 leader_election.cc:304] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:03.305042 16145 raft_consensus.cc:2749] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:03.338675  8146 catalog_manager.cc:5697] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 15 to 24, leader changed from e64761f43e4240319c41117cf886d22f (127.6.255.3) to 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1). New cstate: current_term: 24 leader_uuid: "8c42259d7c4b463a9f485f42c320c81b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } } }
W20260709 13:05:03.340550  8295 consensus_peers.cc:597] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:03.368235 16141 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.368878 16141 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:03.373543  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:05:03.375754  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4516247b88e34a1d89ec9eb29e0ecb7f" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:03.377556  8222 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:05:03.378425 16141 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:03.379197 15924 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:03.425870  8369 consensus_peers.cc:597] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:03.484669  8146 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55754 (request call id 206) took 2312 ms. Trace:
I20260709 13:05:03.485123  8146 rpcz_store.cc:276] 0709 13:05:01.171924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.517609 (+345685us) service_pool.cc:224] Handling call
0709 13:05:03.390407 (+1872798us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:03.390436 (+    29us) write_op.cc:454] Released schema lock
0709 13:05:03.484577 (+ 94141us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:03.381476 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:03.382065 (+   589us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:03.382105 (+    40us) write_op.cc:435] Acquired schema lock
0709 13:05:03.382126 (+    21us) tablet.cc:662] Decoding operations
0709 13:05:03.382450 (+   324us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:03.382504 (+    54us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:03.382531 (+    27us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:05:03.382713 (+   182us) tablet.cc:701] Row locks acquired
0709 13:05:03.382731 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:05:03.382865 (+   134us) write_op.cc:270] Start()
0709 13:05:03.382991 (+   126us) write_op.cc:276] Timestamp: P: 1783602303382818 usec, L: 0
0709 13:05:03.383015 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:05:03.383628 (+   613us) log.cc:844] Serialized 530 byte log entry
0709 13:05:03.385939 (+  2311us) op_driver.cc:464] REPLICATION: finished
0709 13:05:03.386388 (+   449us) write_op.cc:301] APPLY: starting
0709 13:05:03.386477 (+    89us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:03.387384 (+   907us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:03.387501 (+   117us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:03.388224 (+   723us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:03.388380 (+   156us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:05:03.388426 (+    46us) write_op.cc:312] APPLY: finished
0709 13:05:03.388624 (+   198us) log.cc:844] Serialized 25 byte log entry
0709 13:05:03.388807 (+   183us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:03.388887 (+    80us) write_op.cc:454] Released schema lock
0709 13:05:03.388931 (+    44us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":57481,"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":245,"apply.run_cpu_time_us":2804,"apply.run_wall_time_us":2804,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4182,"num_ops":1,"prepare.queue_time_us":102,"prepare.run_cpu_time_us":2485,"prepare.run_wall_time_us":2485,"replication_time_us":2805}]]}
W20260709 13:05:03.497838  7164 thread.cc:527] Waited for 7000ms trying to join with heartbeat (tid 8282)
I20260709 13:05:03.507347  8147 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45434 (request call id 287) took 3426 ms. Trace:
I20260709 13:05:03.507818  8147 rpcz_store.cc:276] 0709 13:05:00.149629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.149848 (+   219us) service_pool.cc:224] Handling call
0709 13:05:03.507276 (+3357428us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58512,"reported_tablets":50}
I20260709 13:05:03.536633  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:03.538618  7164 tablet_replica.cc:333] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.540385  7164 raft_consensus.cc:2243] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.541170  7164 pending_rounds.cc:70] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b: Trying to abort 1 pending ops.
I20260709 13:05:03.546746  7164 pending_rounds.cc:77] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b: Aborting op as it isn't in flight: id { term: 21 index: 13 } timestamp: 7305634980036968448 op_type: NO_OP noop_request { }
I20260709 13:05:03.547382  7164 raft_consensus.cc:2889] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:03.548076  7164 raft_consensus.cc:2272] T 767eba28679649519167576023555b60 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:03.554608  8369 consensus_peers.cc:597] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:03.551524  7164 tablet_replica.cc:333] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.562533  7164 raft_consensus.cc:2243] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.563374  7164 pending_rounds.cc:70] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b: Trying to abort 2 pending ops.
I20260709 13:05:03.563788  7164 pending_rounds.cc:77] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b: Aborting op as it isn't in flight: id { term: 21 index: 14 } timestamp: 7305634927836971008 op_type: NO_OP noop_request { }
I20260709 13:05:03.564249  7164 raft_consensus.cc:2889] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:03.564646  7164 pending_rounds.cc:77] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b: Aborting op as it isn't in flight: id { term: 19 index: 13 } timestamp: 7305634880703246336 op_type: NO_OP noop_request { }
I20260709 13:05:03.565068  7164 raft_consensus.cc:2889] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:03.569330  7164 raft_consensus.cc:2272] T d966cf0fb9e24bd9993fb60505fdc62c P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:03.571336  8369 consensus_peers.cc:597] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:03.584244  7164 tablet_replica.cc:333] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.597786  7164 raft_consensus.cc:2243] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.598690  7164 raft_consensus.cc:2272] T 0c3340359c8445d8950a3e68e0070928 P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.602203  7164 tablet_replica.cc:333] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.611881  7164 raft_consensus.cc:2243] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.614682  7164 raft_consensus.cc:2272] T 27e1a14c7c9f49e2b92fcea2a1bb191d P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:03.612793  8972 consensus_peers.cc:597] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:03.618026  7164 tablet_replica.cc:333] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.642867  7164 raft_consensus.cc:2243] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.643615  7164 pending_rounds.cc:70] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b: Trying to abort 1 pending ops.
I20260709 13:05:03.645618  7164 pending_rounds.cc:77] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b: Aborting op as it isn't in flight: id { term: 18 index: 11 } timestamp: 7305634972233392128 op_type: NO_OP noop_request { }
I20260709 13:05:03.646075  7164 raft_consensus.cc:2889] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:03.646443  7164 raft_consensus.cc:2272] T 1d3e4cc2f40346c0bc3bc739e3675685 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.649551  7164 tablet_replica.cc:333] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.650846  7164 raft_consensus.cc:2243] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.651672  7164 raft_consensus.cc:2272] T bac7f1e5192d48a98a850cb0ece17ad5 P 8c42259d7c4b463a9f485f42c320c81b [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.686842  7164 tablet_replica.cc:333] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.687960  7164 raft_consensus.cc:2243] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.698385  7164 raft_consensus.cc:2272] T f19e818631f543b28a8bec9ba4547527 P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.701514  7164 tablet_replica.cc:333] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.702677  7164 raft_consensus.cc:2243] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.703436  7164 raft_consensus.cc:2272] T 98c2abe28fd449b094f4ccff4bac396e P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.718827  7164 tablet_replica.cc:333] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.719950  7164 raft_consensus.cc:2243] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.720662  7164 raft_consensus.cc:2272] T b3d99c80602e45da9c43a49cf83b8211 P 8c42259d7c4b463a9f485f42c320c81b [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.723869  7164 tablet_replica.cc:333] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.725068  7164 raft_consensus.cc:2243] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.725857  7164 raft_consensus.cc:2272] T 6a2e021fd91643b49aa8e553436c4b69 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.728827  7164 tablet_replica.cc:333] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.729802  7164 raft_consensus.cc:2243] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.741897  7164 raft_consensus.cc:2272] T e94bcf9755aa4f0abddcd5b9fbc2939f P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.751456  7164 tablet_replica.cc:333] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.752447  7164 raft_consensus.cc:2243] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.757030  7164 pending_rounds.cc:70] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b: Trying to abort 2 pending ops.
I20260709 13:05:03.757305  7164 pending_rounds.cc:77] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b: Aborting op as it isn't in flight: id { term: 17 index: 12 } timestamp: 7305634964024221696 op_type: NO_OP noop_request { }
I20260709 13:05:03.757726  7164 raft_consensus.cc:2889] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:03.758056  7164 pending_rounds.cc:77] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b: Aborting op as it isn't in flight: id { term: 15 index: 11 } timestamp: 7305634928572137472 op_type: NO_OP noop_request { }
I20260709 13:05:03.758395  7164 raft_consensus.cc:2889] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:03.758741  7164 raft_consensus.cc:2272] T 2323b22f124b407c95caf5525949dd6c P 8c42259d7c4b463a9f485f42c320c81b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.765599  7164 tablet_replica.cc:333] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.779922  7164 raft_consensus.cc:2243] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.780717  7164 raft_consensus.cc:2272] T cb79894cecb541e3b7dd164271cadc16 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.784351  7164 tablet_replica.cc:333] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.788720  7164 raft_consensus.cc:2243] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.789839  7164 raft_consensus.cc:2272] T b23a66e34de947348d5940084701c646 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:03.791164  8155 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45434 (request call id 285) took 53576 ms (client timeout 15000 ms). Trace:
I20260709 13:05:03.792740  7164 tablet_replica.cc:333] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
W20260709 13:05:03.800181  8155 rpcz_store.cc:269] 0709 13:04:10.225983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.237313 (+ 11330us) service_pool.cc:224] Handling call
0709 13:04:11.306714 (+1069401us) spinlock_profiling.cc:243] Waited 62.4 ms on lock 0x7b5400200490. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f882569a640 00007f882e571b76 00007f882e67bc3e 00007f882cb3c2a4 00007f882cb3c232 00007f882cb3c161 00007f882cb3c0dc 00007f882cb3b392 00007f8826af948c 00007f8826af88c6
0709 13:05:03.790983 (+52484269us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41359,"reported_tablets":50,"spinlock_wait_cycles":143602048}
I20260709 13:05:03.810892  7164 raft_consensus.cc:2243] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.812157  7164 raft_consensus.cc:2272] T 09943d7e76a54d4e80eb181cac80bf22 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:03.813133  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 72392us
I20260709 13:05:03.821615  7164 tablet_replica.cc:333] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.830235  7164 raft_consensus.cc:2243] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.831058  7164 raft_consensus.cc:2272] T eeca7eb827a04189af4de94af55ffb91 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.840286  7164 tablet_replica.cc:333] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.841411  7164 raft_consensus.cc:2243] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.842350  7164 raft_consensus.cc:2272] T 4b9b125a55cf4a5d9db580313045edde P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.855060  7164 tablet_replica.cc:333] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.862231  7164 raft_consensus.cc:2243] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.863119  7164 raft_consensus.cc:2272] T 83839ff3ce80447bb72b3c8d15b630ff P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.866577  7164 tablet_replica.cc:333] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.869223  7164 raft_consensus.cc:2243] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.882272  7164 raft_consensus.cc:2272] T cbcc5b6425934c0aa85546e0b33f4abd P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.886310  7164 tablet_replica.cc:333] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
W20260709 13:05:03.887149  8972 consensus_peers.cc:597] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:03.895478  7164 raft_consensus.cc:2243] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.900907  7164 raft_consensus.cc:2272] T 1112bcf4c4ea44e1933ee6287bc869f8 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.904031  7164 tablet_replica.cc:333] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
W20260709 13:05:03.899009  8369 consensus_peers.cc:597] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:03.919822  7164 raft_consensus.cc:2243] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.920703  7164 raft_consensus.cc:2272] T ec275b714efb4b8b92919025fbd79b1f P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.923584  7164 tablet_replica.cc:333] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.924427  7164 raft_consensus.cc:2243] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.924954  7164 raft_consensus.cc:2272] T b7c15354a2a84f868942919f011af448 P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.928016  7164 tablet_replica.cc:333] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.928901  7164 raft_consensus.cc:2243] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.929591  7164 raft_consensus.cc:2272] T e9159db9f370400d98f8503d06a33a33 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.932391  7164 tablet_replica.cc:333] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.933234  7164 raft_consensus.cc:2243] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.933993  7164 raft_consensus.cc:2272] T 4516247b88e34a1d89ec9eb29e0ecb7f P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.936722  7164 tablet_replica.cc:333] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.937566  7164 raft_consensus.cc:2243] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.938587  7164 raft_consensus.cc:2272] T 6efcdb47a4a24d3680780fb6deb13911 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.942730  7164 tablet_replica.cc:333] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.943665  7164 raft_consensus.cc:2243] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.944316  7164 raft_consensus.cc:2272] T 60c80a6b33dc431eac0fe35b37d4ea23 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.949993  7164 tablet_replica.cc:333] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.950954  7164 raft_consensus.cc:2243] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.951603  7164 raft_consensus.cc:2272] T 82abcf8a2a574841972243a029be749d P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.954591  7164 tablet_replica.cc:333] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.955463  7164 raft_consensus.cc:2243] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.956529  7164 raft_consensus.cc:2272] T 3c1d459f4b98433682d7d19a825eb4ef P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.959620  7164 tablet_replica.cc:333] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.960513  7164 raft_consensus.cc:2243] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.963380  7164 raft_consensus.cc:2272] T 20b7fa75b43141c0ba1e5e08972a3a8f P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.968273  7164 tablet_replica.cc:333] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.969261  7164 raft_consensus.cc:2243] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.969959  7164 pending_rounds.cc:70] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b: Trying to abort 2 pending ops.
I20260709 13:05:03.970252  7164 pending_rounds.cc:77] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b: Aborting op as it isn't in flight: id { term: 20 index: 16 } timestamp: 7305634975391182848 op_type: NO_OP noop_request { }
I20260709 13:05:03.970664  7164 raft_consensus.cc:2889] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:03.971107  7164 pending_rounds.cc:77] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b: Aborting op as it isn't in flight: id { term: 19 index: 15 } timestamp: 7305634956005240832 op_type: NO_OP noop_request { }
I20260709 13:05:03.971493  7164 raft_consensus.cc:2889] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:03.971896  7164 raft_consensus.cc:2272] T 007e904f46ef4a1a978f97d557d2a65b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.975112  7164 tablet_replica.cc:333] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.976006  7164 raft_consensus.cc:2243] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.976634  7164 pending_rounds.cc:70] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b: Trying to abort 2 pending ops.
I20260709 13:05:03.981626  7164 pending_rounds.cc:77] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b: Aborting op as it isn't in flight: id { term: 21 index: 15 } timestamp: 7305634979723186176 op_type: NO_OP noop_request { }
I20260709 13:05:03.982095  7164 raft_consensus.cc:2889] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:03.983481  7164 pending_rounds.cc:77] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b: Aborting op as it isn't in flight: id { term: 19 index: 14 } timestamp: 7305634942173396992 op_type: NO_OP noop_request { }
I20260709 13:05:03.983961  7164 raft_consensus.cc:2889] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:03.984615  7164 raft_consensus.cc:2272] T bf42cb9d00c6491eb6b0cf8b896f7c5b P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.991835  7164 tablet_replica.cc:333] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.992776  7164 raft_consensus.cc:2243] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.994215  7164 raft_consensus.cc:2272] T 625b0bdeab5d48eaa06a46928cbad7e4 P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.997301  7164 tablet_replica.cc:333] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:03.998286  7164 raft_consensus.cc:2243] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.998953  7164 pending_rounds.cc:70] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b: Trying to abort 1 pending ops.
I20260709 13:05:03.999234  7164 pending_rounds.cc:77] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b: Aborting op as it isn't in flight: id { term: 25 index: 18 } timestamp: 7305634962286088192 op_type: NO_OP noop_request { }
I20260709 13:05:03.999672  7164 raft_consensus.cc:2889] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:04.000077  7164 raft_consensus.cc:2272] T fdf12882ac9e4f7d9121a5fcb9d4efaf P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:04.012251  7164 tablet_replica.cc:333] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:04.013248  7164 raft_consensus.cc:2243] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:05:04.014474  7164 raft_consensus.cc:2272] T 5c7b40c72696450b884fa3a4659ccd84 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:04.027017  7164 tablet_replica.cc:333] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:04.027994  7164 raft_consensus.cc:2243] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:04.028648  7164 raft_consensus.cc:2272] T ed318aa932504f70b21cfa8e646283ac P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:04.035682  7164 tablet_replica.cc:333] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:04.036634  7164 raft_consensus.cc:2243] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:05:04.039685 16141 raft_consensus.cc:493] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.040196 16141 raft_consensus.cc:515] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:04.042238 16141 leader_election.cc:290] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e64761f43e4240319c41117cf886d22f (127.6.255.3:37367), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:04.044684  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:04.046161  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263b3742fd541f8bda5a6b6c10a8596" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 24 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:05:04.048805  8222 leader_election.cc:304] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e
I20260709 13:05:04.052131 16141 raft_consensus.cc:2749] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:04.039659  7164 raft_consensus.cc:2272] T 104e8624a4bd4222aa8e1be9cba98372 P 8c42259d7c4b463a9f485f42c320c81b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:04.059378  7164 tablet_replica.cc:333] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:04.060271  7164 raft_consensus.cc:2243] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:04.060935  7164 raft_consensus.cc:2272] T 1263b3742fd541f8bda5a6b6c10a8596 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:04.063659  7164 tablet_replica.cc:333] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
W20260709 13:05:04.064421  8369 consensus_peers.cc:597] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:04.064479  7164 raft_consensus.cc:2243] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:05:04.066175  7164 raft_consensus.cc:2272] T f66b0119c0d84d8cb95b2252fcb6d390 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:04.071131  8295 consensus_peers.cc:597] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:04.072090  7164 tablet_replica.cc:333] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:04.080679  7164 raft_consensus.cc:2243] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:05:04.081981  7164 raft_consensus.cc:2272] T bf584dd56b3149a0b92539bc7115815b P 8c42259d7c4b463a9f485f42c320c81b [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:04.096256 16141 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.096637  7164 tablet_replica.cc:333] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:04.096772 16141 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:04.098850 16141 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:04.100658  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:04.101082  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757eb9e36091440581fbfc9a6b000e74" candidate_uuid: "8c42259d7c4b463a9f485f42c320c81b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:04.103569  8220 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c42259d7c4b463a9f485f42c320c81b; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:04.104821 16141 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:04.105614  7164 raft_consensus.cc:2243] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:04.108206  7164 pending_rounds.cc:70] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b: Trying to abort 1 pending ops.
I20260709 13:05:04.108556  7164 pending_rounds.cc:77] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b: Aborting op as it isn't in flight: id { term: 19 index: 11 } timestamp: 7305634988123250688 op_type: NO_OP noop_request { }
I20260709 13:05:04.108984  7164 raft_consensus.cc:2889] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:04.109393  7164 raft_consensus.cc:2272] T 757eb9e36091440581fbfc9a6b000e74 P 8c42259d7c4b463a9f485f42c320c81b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:04.112228  7164 tablet_replica.cc:333] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:04.113019  7164 raft_consensus.cc:2243] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:04.113760  7164 raft_consensus.cc:2272] T d32240ced15f4c36a9ce65ca5a13634a P 8c42259d7c4b463a9f485f42c320c81b [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:04.116586  7164 tablet_replica.cc:333] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:04.117405  7164 raft_consensus.cc:2243] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:04.118191  7164 raft_consensus.cc:2272] T 430fa172910841a49c0c80f2cbf76ae5 P 8c42259d7c4b463a9f485f42c320c81b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:04.121564  7164 tablet_replica.cc:333] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:04.122469  7164 raft_consensus.cc:2243] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:04.123492  7164 pending_rounds.cc:70] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b: Trying to abort 2 pending ops.
I20260709 13:05:04.123898  7164 pending_rounds.cc:77] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b: Aborting op as it isn't in flight: id { term: 25 index: 19 } timestamp: 7305634984158986240 op_type: NO_OP noop_request { }
I20260709 13:05:04.124325  7164 raft_consensus.cc:2889] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:04.124680  7164 pending_rounds.cc:77] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b: Aborting op as it isn't in flight: id { term: 23 index: 18 } timestamp: 7305634941074182144 op_type: NO_OP noop_request { }
I20260709 13:05:04.125079  7164 raft_consensus.cc:2889] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:04.125506  7164 raft_consensus.cc:2272] T 10cffa7c6411440ca77cee6731faca67 P 8c42259d7c4b463a9f485f42c320c81b [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:04.128638  7164 tablet_replica.cc:333] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:04.129549  7164 raft_consensus.cc:2243] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:05:04.130682  7164 raft_consensus.cc:2272] T 338bba401dcd4d8c9642cd7b255dea63 P 8c42259d7c4b463a9f485f42c320c81b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:04.133904  7164 tablet_replica.cc:333] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:04.134723  7164 raft_consensus.cc:2243] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:04.135344  7164 raft_consensus.cc:2272] T fade5d45b9804cc98377ac6ab69be94b P 8c42259d7c4b463a9f485f42c320c81b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:04.140853  7164 tablet_replica.cc:333] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:04.141755  7164 raft_consensus.cc:2243] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:05:04.142891  7164 raft_consensus.cc:2272] T 6c6a6afa04ab437cbbd6011a11681763 P 8c42259d7c4b463a9f485f42c320c81b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:04.150617  7164 tablet_replica.cc:333] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:04.151443  7164 raft_consensus.cc:2243] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:05:04.152513  7164 raft_consensus.cc:2272] T 9d36c3e19b8c47bc9d26b071486fdec9 P 8c42259d7c4b463a9f485f42c320c81b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:04.155092  7164 tablet_replica.cc:333] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b: stopping tablet replica
I20260709 13:05:04.157523  7164 raft_consensus.cc:2243] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:04.159170  7164 pending_rounds.cc:70] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b: Trying to abort 1 pending ops.
I20260709 13:05:04.159462  7164 pending_rounds.cc:77] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b: Aborting op as it isn't in flight: id { term: 20 index: 11 } timestamp: 7305634903188996096 op_type: NO_OP noop_request { }
I20260709 13:05:04.159950  7164 raft_consensus.cc:2889] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:04.161849  7164 raft_consensus.cc:2272] T a1fdbe7b73a540ddb2e5cb473a8404de P 8c42259d7c4b463a9f485f42c320c81b [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:04.336797  8295 consensus_peers.cc:597] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:04.394663  8369 consensus_peers.cc:597] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:04.396829  8139 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 50264.6us
W20260709 13:05:04.447114  8139 catalog_manager.cc:4755] TS 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 81804f59e3124d27998ea80fc1c36663: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:04.458654  8295 consensus_peers.cc:597] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:04.734762  8369 consensus_peers.cc:597] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:04.779706 16204 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:04.780340 16204 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:04.782649 16204 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:04.797953  8972 leader_election.cc:336] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:04.798767  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:04.815927  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:04.819911 16204 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:05:04.915634 16416 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:04.916218 16416 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:04.918252 16416 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:05:04.920506  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:05:04.925704  8369 leader_election.cc:336] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:04.926430  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:04.931871 16416 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:05.021138 16425 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.021648 16409 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.021763 16425 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.022231 16409 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.023984 16425 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:05:05.024230 16409 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:05.025780  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:05.026432  8398 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 23.
I20260709 13:05:05.027848  8297 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:05:05.028069  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:05:05.029148  8369 leader_election.cc:336] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.029765 16430 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:05.029979  8369 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:05.030254 16430 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.030711 16430 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:05.030776 16425 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:05:05.032677  8295 leader_election.cc:336] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.054491 16438 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.055117 16438 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:05.059023 15968 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.059547 15968 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:05.060961 16438 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:05.063789  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:05.066958 16430 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:05:05.064153  8972 leader_election.cc:336] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.064818 15968 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:05.066236  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:05.074434  8974 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:05:05.076068 15968 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:05.076323 16425 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.076906 16425 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:05.079047 16425 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:05:05.082074  8972 leader_election.cc:336] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.082623  8972 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:05:05.082844  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:05:05.083374 15968 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:05.084254  9008 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
I20260709 13:05:05.085480  8372 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:05:05.086377 16425 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:05.086843 16425 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.087242 16425 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:05.090859  8369 leader_election.cc:336] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:05.104120  8295 leader_election.cc:336] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.107509 16430 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:05:05.108119  8369 consensus_peers.cc:597] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:05.109071 16439 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.109647 16439 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.110493 16416 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.133488 16416 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.135622 16416 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:05:05.112185  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23a66e34de947348d5940084701c646" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:05:05.136346  8399 raft_consensus.cc:3060] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:05:05.138195  8972 consensus_peers.cc:597] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:05.120249 16425 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:05.108139 16430 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.123418 16409 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.139379 16409 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.140524 16425 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:05.141503 16409 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:05.142864 16430 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.144939 16430 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:05.145543  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1112bcf4c4ea44e1933ee6287bc869f8" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:05:05.146297  9008 raft_consensus.cc:3060] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:05.147790  8369 leader_election.cc:336] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:05.148419  8295 leader_election.cc:336] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:05.150208  8369 leader_election.cc:336] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:05.154246  8295 leader_election.cc:336] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.156322  8403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:05.156395  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:05.154511  9008 raft_consensus.cc:2468] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 22.
I20260709 13:05:05.160578  8372 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:05:05.162307 16416 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.162850 16416 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:05.164950 16416 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:05.165774  8297 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:05:05.167064  8297 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:05:05.169617 16409 raft_consensus.cc:2749] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:05:05.171082 16409 raft_consensus.cc:2749] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:05.174981  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:05:05.175657  9001 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 17.
I20260709 13:05:05.177034  8372 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:05:05.178154  8369 leader_election.cc:336] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.178447 16441 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:05.178907 16441 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.179271 16441 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:05.181454  8399 raft_consensus.cc:2468] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 24.
I20260709 13:05:05.182595 16416 raft_consensus.cc:2804] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:05.183101 16416 raft_consensus.cc:697] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 22 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:05:05.183974 16416 consensus_queue.cc:237] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:05.188141  8297 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:05:05.189145 16430 raft_consensus.cc:2804] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:05:05.189658 16430 raft_consensus.cc:697] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 24 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:05:05.190565 16430 consensus_queue.cc:237] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.132742 16440 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.196271 16439 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:05.206408 16425 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.207214 16425 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.193046 16440 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:05.217620  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:05.218583  8399 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 23.
W20260709 13:05:05.223286  8295 leader_election.cc:336] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:05.226084  8295 leader_election.cc:336] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.227516  8297 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:05:05.229993 16425 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:05:05.232519 16440 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:05.235760  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:05:05.236421  9001 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 26.
I20260709 13:05:05.236608  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:05.237229  8331 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 23.
I20260709 13:05:05.237867  8298 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:05:05.238580 16409 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:05.239120 16409 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.239650 16409 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:05.239161 16451 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.240374 16451 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.241641 16440 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:05:05.243181 16440 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.244804 16451 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:05.245725  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:05.253561  8399 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 23.
I20260709 13:05:05.255455 16440 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:05:05.257743  8297 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:05:05.259315 16451 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:05.260232 16451 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.261137 16451 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:05.232295 16446 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.262178 16446 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.264384 16446 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:05:05.274937  8155 catalog_manager.cc:5697] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 23 to 24, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 24 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:05.279842 16451 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.281175 16444 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.286252 16444 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.286827 16449 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.287360 16449 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.288411  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d36c3e19b8c47bc9d26b071486fdec9" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:05:05.288435 16444 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:05:05.289108  8398 raft_consensus.cc:3060] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:05.292514 16444 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.293174  8146 catalog_manager.cc:5697] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 21 to 22, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 22 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:05.299918  8398 raft_consensus.cc:2468] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 24.
I20260709 13:05:05.293097 16444 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:05.298295 16451 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:05.302299  8297 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:05:05.303821  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:05.299053 16449 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:05:05.320170  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:05:05.327176  8295 leader_election.cc:336] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.340101 16448 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.321153  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:05:05.359156  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:05.312888  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:05.361281  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:05:05.359527 16448 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.305065 16430 raft_consensus.cc:2804] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:05:05.365967 16448 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:05:05.309767 16444 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:05.362002 16416 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.330139 16409 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.367136 16416 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.368076 16430 raft_consensus.cc:697] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 24 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:05:05.368511 16409 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:05.335443 16438 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.369334 16416 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:05:05.369113 16430 consensus_queue.cc:237] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.369624 16438 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:05.371814 16438 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:05:05.372303  8972 consensus_peers.cc:597] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:05.373872  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:05.374110  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7b40c72696450b884fa3a4659ccd84" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:05:05.374648  8398 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 24.
I20260709 13:05:05.374739  8399 raft_consensus.cc:3060] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:05.376117  8974 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:05:05.377065 16451 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.377754 16451 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.378476 15968 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:05:05.378862 15968 raft_consensus.cc:493] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.379190 15968 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:05:05.379558 16439 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.380149 16439 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.355724  8329 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 24.
W20260709 13:05:05.382220  8295 leader_election.cc:336] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.382422 16416 raft_consensus.cc:2764] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 13:05:05.382797 16409 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:05:05.390838  8369 leader_election.cc:336] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.391499  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:05.392321 16444 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:05:05.383524 16439 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:05.384865  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:05.310438 16441 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:05.390844 16409 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.397934 16409 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:05.398380  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:05.398751 16441 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:05.399008  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19e818631f543b28a8bec9ba4547527" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:05:05.399564  9008 raft_consensus.cc:3060] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:05.401724  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:05.402354  8398 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 26.
I20260709 13:05:05.403084 16451 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:05.403020 16444 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.404824 16444 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.406023 16443 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.404191  8297 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:05:05.406791  8369 leader_election.cc:336] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.405309  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:05.409731  8399 raft_consensus.cc:2468] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 24.
I20260709 13:05:05.410332  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:05.408461  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:05:05.411607  8972 leader_election.cc:336] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.407732  8331 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 25.
I20260709 13:05:05.406531 16443 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.411131 16439 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.422302 16439 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:05:05.423758  7164 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 8278)
I20260709 13:05:05.423542 16443 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:05:05.426416  8295 leader_election.cc:336] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:05.427317  8295 leader_election.cc:336] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:05.428841  8369 leader_election.cc:336] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.428074  9008 raft_consensus.cc:2468] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 18.
I20260709 13:05:05.428510  8295 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:05:05.428907  8332 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 27.
W20260709 13:05:05.429891  8369 leader_election.cc:336] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.430610  8372 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:05:05.438328  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:05.439306  8399 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 26.
I20260709 13:05:05.421455 16444 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:05:05.436710  8369 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:05.441707  8297 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:05:05.430449 16425 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:05:05.426517 16451 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:05:05.424259 16439 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:05.443406  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:05.443043 16448 raft_consensus.cc:2804] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:05.431244 16441 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.437640 16443 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:05:05.447089  8369 leader_election.cc:336] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.447224  8399 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 27.
I20260709 13:05:05.445091 16451 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.447774 16439 raft_consensus.cc:2804] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:05:05.453552 16448 raft_consensus.cc:697] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 18 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:05:05.486649 16451 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:05:05.456954  8295 leader_election.cc:336] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.446914 16444 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:05.444844 16454 raft_consensus.cc:2749] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:05.463686 16443 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.466567 16440 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:05.491377 16448 consensus_queue.cc:237] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:05:05.487843  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 59745.1us
I20260709 13:05:05.483814 16441 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.495580  9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09943d7e76a54d4e80eb181cac80bf22" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:05:05.496174  9008 raft_consensus.cc:3060] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:05:05.496206  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:05:05.484812 15968 raft_consensus.cc:515] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:05:05.490283  8295 leader_election.cc:336] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.489319 16439 raft_consensus.cc:697] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 24 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:05:05.498684  9001 raft_consensus.cc:2393] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 25: Already voted for candidate f865c4d761f5446eb12b3a029234772e in this term.
W20260709 13:05:05.496925  8972 consensus_peers.cc:597] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:05.498123  8295 leader_election.cc:336] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.500372 16444 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:05:05.500321 16439 consensus_queue.cc:237] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.501014 16444 raft_consensus.cc:493] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.499487  8295 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:05:05.507211  9008 raft_consensus.cc:2468] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 27.
W20260709 13:05:05.509876  8295 leader_election.cc:336] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.510875  8298 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f865c4d761f5446eb12b3a029234772e, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:05:05.511819 15968 leader_election.cc:290] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 25 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:05.513273 16430 raft_consensus.cc:2749] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:05.514181  8372 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, f865c4d761f5446eb12b3a029234772e
I20260709 13:05:05.509095 16409 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.522902 16409 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.514707 16454 raft_consensus.cc:2804] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:05:05.508415 16444 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:05:05.516316  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9b125a55cf4a5d9db580313045edde" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:05:05.525705 16409 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:05.519316 16451 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.525792 16416 raft_consensus.cc:3060] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:05:05.526656 16454 raft_consensus.cc:697] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 27 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:05:05.531440 16454 consensus_queue.cc:237] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:05.532981 16441 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:05:05.497210 16440 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:05.535662 16451 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:05.536906 16444 raft_consensus.cc:515] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.509931  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:05:05.539609  8972 consensus_peers.cc:597] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:05.539744  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:05:05.495575 16443 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:05.543255 16425 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:05:05.538938 16444 leader_election.cc:290] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:05:05.544428 16416 raft_consensus.cc:2749] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:05.545092 16425 raft_consensus.cc:493] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.545917 16425 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:05:05.544296  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:05:05.549515  8295 consensus_peers.cc:597] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:05.549821  8332 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:05:05.545614 16441 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 13:05:05.551885  8295 leader_election.cc:336] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.552134 16441 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
W20260709 13:05:05.552639  8295 leader_election.cc:336] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.552601 16441 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:05:05.562723  8295 leader_election.cc:336] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:05.565198  8295 leader_election.cc:336] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.565696  8399 raft_consensus.cc:2468] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f865c4d761f5446eb12b3a029234772e in term 25.
W20260709 13:05:05.566293  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 124642us
I20260709 13:05:05.567041  8297 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:05:05.568305  8297 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:05:05.568817 16459 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:05:05.569141 16443 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:05:05.569471  8974 leader_election.cc:304] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:05:05.569609 16443 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.570403 16456 raft_consensus.cc:2804] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:05:05.570468  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:05.571012 16456 raft_consensus.cc:697] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 25 LEADER]: Becoming Leader. State: Replica: f865c4d761f5446eb12b3a029234772e, State: Running, Role: LEADER
I20260709 13:05:05.571288  8398 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 23.
I20260709 13:05:05.570492 16443 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:05:05.571967 16456 consensus_queue.cc:237] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:05.573372  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:05.574066  8398 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 24.
I20260709 13:05:05.577145  8297 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:05:05.577452  8972 leader_election.cc:336] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.578079  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1d459f4b98433682d7d19a825eb4ef" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:05:05.578737  8399 raft_consensus.cc:3060] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:05:05.581976 16439 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:05.582450 16439 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.582867 16439 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:05.579175  8297 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:05:05.589618  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:05.590337  8331 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 24.
I20260709 13:05:05.591079 16451 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:05:05.591300  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b7fa75b43141c0ba1e5e08972a3a8f" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:05:05.591704 16451 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.592051  8332 raft_consensus.cc:3060] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:05:05.592293 16451 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:05:05.595963  8972 consensus_peers.cc:597] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:05.609767  8332 raft_consensus.cc:2468] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 25.
I20260709 13:05:05.562980 16425 raft_consensus.cc:515] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.612910 16425 leader_election.cc:290] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:05.616464  8295 consensus_peers.cc:597] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:05.625662 16443 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.628065 16443 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 28 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:05.569366 16459 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.630306 16459 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:05:05.631124 16416 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.633167 16416 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.635782 16439 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.638499 16416 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:05.638566  8295 leader_election.cc:336] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.640506 16441 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.643643  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:05:05.644683  8399 raft_consensus.cc:2468] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 27.
W20260709 13:05:05.632522  8369 leader_election.cc:336] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:05.613155  8972 consensus_peers.cc:597] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:05.647861  8369 leader_election.cc:336] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.647800  8297 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
W20260709 13:05:05.649080  8369 leader_election.cc:336] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.663383  8398 raft_consensus.cc:2393] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 28: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:05:05.662464 16459 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.666226 16441 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:05:05.669737  8297 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:05:05.673671 16430 raft_consensus.cc:2804] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:05:05.674170 16430 raft_consensus.cc:697] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 27 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:05:05.675047 16430 consensus_queue.cc:237] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.678066 16454 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 13:05:05.680912  8398 raft_consensus.cc:1275] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:05:05.683368 16443 consensus_queue.cc:1048] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:05:05.685170  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66b0119c0d84d8cb95b2252fcb6d390" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:05:05.685526  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a6afa04ab437cbbd6011a11681763" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:05:05.686298  8331 raft_consensus.cc:3060] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:05:05.687165  8398 raft_consensus.cc:3060] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:05.692945  8369 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:05:05.700805 16416 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:05.701232 16416 raft_consensus.cc:493] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.701650 16416 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:05.703060  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:05:05.711243  8369 leader_election.cc:304] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:05:05.693291 16439 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:05.713507 16441 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:05:05.716595 16448 raft_consensus.cc:2804] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:05:05.716593 16441 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.717157  8398 raft_consensus.cc:2468] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 24.
I20260709 13:05:05.717294 16448 raft_consensus.cc:697] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 25 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:05:05.718256 16448 consensus_queue.cc:237] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.719375  8297 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:05:05.720666 16439 raft_consensus.cc:2804] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:05:05.721182 16439 raft_consensus.cc:697] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 24 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:05:05.721386  9008 raft_consensus.cc:1275] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:05:05.725198  8295 leader_election.cc:336] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.726687 16416 raft_consensus.cc:515] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.727540  8331 raft_consensus.cc:2468] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 26.
I20260709 13:05:05.728835 16416 leader_election.cc:290] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:05:05.731858 16451 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.726945 16446 consensus_queue.cc:1048] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.007s
I20260709 13:05:05.717337 16441 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:05:05.740303  8295 leader_election.cc:336] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.722110 16439 consensus_queue.cc:237] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.751076  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
W20260709 13:05:05.751585  8369 leader_election.cc:336] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.751964  8331 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 21.
W20260709 13:05:05.756634  8369 leader_election.cc:336] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:05.757714  8369 leader_election.cc:336] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.763441  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:05:05.764302  8331 raft_consensus.cc:2393] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 28: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:05:05.777678  8369 leader_election.cc:304] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:05:05.765136 16451 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:05.779663  8399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:05:05.779072  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:05:05.788304  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625b0bdeab5d48eaa06a46928cbad7e4" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:05:05.791365  8331 raft_consensus.cc:3060] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:05.793144 16416 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:05.782485 16446 raft_consensus.cc:2804] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:05:05.793740 16416 raft_consensus.cc:493] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8c42259d7c4b463a9f485f42c320c81b)
I20260709 13:05:05.794303 16416 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:05.814289 16459 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:05.817639  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efcdb47a4a24d3680780fb6deb13911" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:05:05.818288  8398 raft_consensus.cc:3060] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:05:05.794219 16446 raft_consensus.cc:697] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 26 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:05:05.821055 16446 consensus_queue.cc:237] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.825531  8398 raft_consensus.cc:2468] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 27.
I20260709 13:05:05.826779  8297 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:05:05.828130  8146 catalog_manager.cc:5697] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 17 to 18, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 18 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:05.830333  8369 leader_election.cc:336] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:05.838465  8369 consensus_peers.cc:597] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:05.831873 16409 raft_consensus.cc:2804] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:05:05.834964  8155 catalog_manager.cc:5697] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e reported cstate change: term changed from 24 to 25, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to f865c4d761f5446eb12b3a029234772e (127.6.255.4). New cstate: current_term: 25 leader_uuid: "f865c4d761f5446eb12b3a029234772e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:05.839618 16409 raft_consensus.cc:697] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 27 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:05:05.840656 16409 consensus_queue.cc:237] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.836318  8150 catalog_manager.cc:5697] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 23 to 24, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 24 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:05.842662  8150 catalog_manager.cc:5697] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 23 to 24, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 24 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:05.844559  8295 leader_election.cc:336] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:05.845458  8295 consensus_peers.cc:597] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:05.844308  8150 catalog_manager.cc:5697] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 26 to 27, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 27 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:05.831082 16416 raft_consensus.cc:515] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.851910  8331 raft_consensus.cc:2468] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
W20260709 13:05:05.850903  8369 consensus_peers.cc:597] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:05.855504  8369 leader_election.cc:336] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:05.856809  8369 leader_election.cc:336] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:05.857260  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:05.858052 16444 raft_consensus.cc:2749] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 13:05:05.865742 16416 leader_election.cc:290] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:05.878120  8295 consensus_peers.cc:597] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:05.883167  8325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e8624a4bd4222aa8e1be9cba98372" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:05:05.884042  8325 raft_consensus.cc:3060] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:05.897691  8972 consensus_peers.cc:597] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:05.881837  8369 leader_election.cc:304] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:05:05.903229  8369 consensus_peers.cc:597] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:05.904021  8369 consensus_peers.cc:597] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:05.904553  8369 consensus_peers.cc:597] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:05.905911  8369 leader_election.cc:336] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:05.907119  8369 consensus_peers.cc:597] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:05.907552  8369 consensus_peers.cc:597] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:05.909556 16449 raft_consensus.cc:2804] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:05:05.922482  8972 consensus_peers.cc:597] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:05.927805  8369 leader_election.cc:336] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:05.928630  8369 consensus_peers.cc:597] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:05.934651 16441 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:05.937917  8399 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7a6fe533e94c1bae555b4cae427728 in current term 25: Already voted for candidate e64761f43e4240319c41117cf886d22f in this term.
I20260709 13:05:05.927990  8325 raft_consensus.cc:2468] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 22.
I20260709 13:05:05.940658  8297 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:05:05.941789 16451 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:05:05.944082  8295 consensus_peers.cc:597] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:05.919622 16449 raft_consensus.cc:697] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:05:05.945909 16449 consensus_queue.cc:237] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:05:05.949544  8295 consensus_peers.cc:597] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:05.937191 16441 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:05.980410  8295 consensus_peers.cc:597] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:05.983711  8399 raft_consensus.cc:1275] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Refusing update from remote peer f865c4d761f5446eb12b3a029234772e: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:05:05.994292 15968 consensus_queue.cc:1048] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:05:06.004484  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:05:06.005535  8331 raft_consensus.cc:2393] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate e64761f43e4240319c41117cf886d22f in current term 25: Already voted for candidate ff7a6fe533e94c1bae555b4cae427728 in this term.
I20260709 13:05:06.006230  8369 leader_election.cc:304] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:05:06.010630 16416 raft_consensus.cc:2804] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:06.016213  9001 raft_consensus.cc:1275] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:05:06.021152 16449 consensus_queue.cc:1048] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:06.052291  9008 raft_consensus.cc:1275] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 13:05:06.054183 16440 consensus_queue.cc:1048] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 13:05:06.071651  8295 consensus_peers.cc:597] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:06.178648  8369 leader_election.cc:336] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:06.179176  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:05:06.180208  8369 consensus_peers.cc:597] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:06.180649  8369 consensus_peers.cc:597] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:06.181607 16444 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:05:06.185073  8369 consensus_peers.cc:597] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:06.242365  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 90498.2us
I20260709 13:05:06.243103 16416 raft_consensus.cc:697] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 22 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:05:06.161275  8150 catalog_manager.cc:5697] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 23 to 24, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 24 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:06.244973  8150 catalog_manager.cc:5697] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 26 to 27, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 27 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:06.247469  8150 catalog_manager.cc:5697] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 26 to 27, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 27 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:06.200517  8155 catalog_manager.cc:5697] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 24 to 25, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 25 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:06.249959  8155 catalog_manager.cc:5697] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 25 to 26, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 26 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:06.259812 16416 consensus_queue.cc:237] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:06.276899  8398 raft_consensus.cc:1275] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:05:06.283625  8398 raft_consensus.cc:1275] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
W20260709 13:05:06.289544  8295 consensus_peers.cc:597] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:06.291541  8398 raft_consensus.cc:1275] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
W20260709 13:05:06.294346  8972 consensus_peers.cc:597] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:06.295045  8972 consensus_peers.cc:597] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:06.307065  8398 raft_consensus.cc:1275] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 13:05:06.317258 16409 consensus_queue.cc:1048] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:05:06.319257 16459 consensus_queue.cc:1048] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:06.318698 16409 consensus_queue.cc:1048] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:05:06.325500 16440 consensus_queue.cc:1048] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:05:06.330214  8295 consensus_peers.cc:597] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:06.331705  8398 raft_consensus.cc:1275] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
W20260709 13:05:06.356462  8369 consensus_peers.cc:597] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:06.357087  8369 consensus_peers.cc:597] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:06.363965 16440 consensus_queue.cc:1048] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:05:06.390944  8295 consensus_peers.cc:597] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:06.413738  8331 raft_consensus.cc:1275] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:05:06.417661  8369 consensus_peers.cc:597] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:06.418246  8369 consensus_peers.cc:597] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:06.424917  7164 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 8278)
W20260709 13:05:06.437757  8369 consensus_peers.cc:597] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:06.438508 16416 consensus_queue.cc:1048] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:06.444341  8331 raft_consensus.cc:1275] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:05:06.445514  8332 raft_consensus.cc:1275] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 13:05:06.447141 16416 consensus_queue.cc:1048] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:05:06.479434 16416 consensus_queue.cc:1048] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:05:06.487216  8147 catalog_manager.cc:5697] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 19 to 20, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 20 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:06.491088  8147 catalog_manager.cc:5697] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 21 to 22, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 22 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:06.517369 16493 log.cc:927] Time spent T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
W20260709 13:05:06.542618  8972 consensus_peers.cc:597] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:06.575749  8297 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 121645us
I20260709 13:05:06.730122  8331 raft_consensus.cc:1275] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 13:05:06.746405  8369 consensus_peers.cc:597] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:06.753732 16449 consensus_queue.cc:1048] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:05:06.787384  8369 consensus_peers.cc:597] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:06.901115  8369 consensus_peers.cc:597] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:07.085014  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 172953us
W20260709 13:05:07.425406  7164 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 8278)
W20260709 13:05:07.432353  8369 consensus_peers.cc:597] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:07.555627 16500 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.556210 16500 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:07.565822 16480 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.566434 16480 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:07.568450 16480 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:05:07.569205  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:07.569826  8397 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 25.
I20260709 13:05:07.573383  8297 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:05:07.574149 16409 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:05:07.574514 16409 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:07.574831 16409 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:05:07.573374  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728" is_pre_election: true
I20260709 13:05:07.577031  8331 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 28.
W20260709 13:05:07.576438  8972 consensus_peers.cc:597] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:07.578155  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
W20260709 13:05:07.578994  8369 leader_election.cc:336] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:07.579221 16449 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:05:07.579624 16449 raft_consensus.cc:493] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:07.579962 16449 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:05:07.583132 16409 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:07.587148  8400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338bba401dcd4d8c9642cd7b255dea63" candidate_uuid: "ff7a6fe533e94c1bae555b4cae427728" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f"
I20260709 13:05:07.587690  8400 raft_consensus.cc:3060] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:05:07.589664 16409 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:07.589596 16500 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:07.609462 16449 raft_consensus.cc:515] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:07.631577  8400 raft_consensus.cc:2468] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7a6fe533e94c1bae555b4cae427728 in term 26.
I20260709 13:05:07.632841  8297 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 
I20260709 13:05:07.635980 16500 raft_consensus.cc:2804] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:05:07.636497 16500 raft_consensus.cc:697] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 26 LEADER]: Becoming Leader. State: Replica: ff7a6fe533e94c1bae555b4cae427728, State: Running, Role: LEADER
I20260709 13:05:07.641367 16500 consensus_queue.cc:237] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:05:07.644457  8295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 73415.1us
W20260709 13:05:07.645296  8295 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111) [suppressed 386 similar messages]
I20260709 13:05:07.661303 16449 leader_election.cc:290] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 29 election: Requested vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:05:07.665253  8329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf584dd56b3149a0b92539bc7115815b" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "ff7a6fe533e94c1bae555b4cae427728"
I20260709 13:05:07.665932  8329 raft_consensus.cc:3060] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Advancing to term 29
W20260709 13:05:07.669796  8369 leader_election.cc:336] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:07.675340  8329 raft_consensus.cc:2468] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 29.
I20260709 13:05:07.676549  8369 leader_election.cc:304] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728; no voters: 8c42259d7c4b463a9f485f42c320c81b
I20260709 13:05:07.682396 16488 raft_consensus.cc:2804] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 29 FOLLOWER]: Leader election won for term 29
I20260709 13:05:07.682981 16488 raft_consensus.cc:697] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 29 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:05:07.687122 16488 consensus_queue.cc:237] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:05:07.693235  8295 consensus_peers.cc:597] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:07.694193  8295 consensus_peers.cc:597] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:07.695081  8295 leader_election.cc:336] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:07.749853  8295 leader_election.cc:336] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:07.757965  8155 catalog_manager.cc:5697] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 reported cstate change: term changed from 24 to 26, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2). New cstate: current_term: 26 leader_uuid: "ff7a6fe533e94c1bae555b4cae427728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:07.803854  8147 catalog_manager.cc:5697] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 27 to 29, leader changed from 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 29 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:07.932005  8369 consensus_peers.cc:597] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:08.001488  8972 consensus_peers.cc:597] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:08.005595  8972 consensus_peers.cc:597] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:08.021807  8972 consensus_peers.cc:597] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:08.074625  8329 raft_consensus.cc:1275] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 29 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 13:05:08.077739 16480 consensus_queue.cc:1048] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:05:08.094403  8972 consensus_peers.cc:597] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:08.135318  9030 debug-util.cc:398] Leaking SignalData structure 0x7b0800294300 after lost signal to thread 7167
W20260709 13:05:08.159296  9030 debug-util.cc:398] Leaking SignalData structure 0x7b08001aace0 after lost signal to thread 8189
W20260709 13:05:08.191656  9030 debug-util.cc:398] Leaking SignalData structure 0x7b08002a4f80 after lost signal to thread 8356
W20260709 13:05:08.213935  9030 debug-util.cc:398] Leaking SignalData structure 0x7b0800133580 after lost signal to thread 8430
W20260709 13:05:08.216533  9030 debug-util.cc:398] Leaking SignalData structure 0x7b08003c7f80 after lost signal to thread 9034
W20260709 13:05:08.221248  8972 consensus_peers.cc:597] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:08.230579  8400 raft_consensus.cc:1275] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Refusing update from remote peer ff7a6fe533e94c1bae555b4cae427728: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 13:05:08.233528 16409 consensus_queue.cc:1048] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:05:08.256944  8972 consensus_peers.cc:597] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:08.258539  8295 consensus_peers.cc:597] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:08.262154  8369 consensus_peers.cc:597] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:08.262745  8369 consensus_peers.cc:597] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:08.268302  8295 consensus_peers.cc:597] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:08.352777  8972 consensus_peers.cc:597] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:08.426152  7164 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 8278)
W20260709 13:05:08.444345  8369 consensus_peers.cc:597] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:08.446005  8369 consensus_peers.cc:597] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:08.447145  8369 consensus_peers.cc:597] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:08.447674  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 76505.7us
W20260709 13:05:08.517551  8369 consensus_peers.cc:597] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:08.532663  8369 consensus_peers.cc:597] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:08.577297  8972 consensus_peers.cc:597] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:08.582530  8369 consensus_peers.cc:597] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:08.597275  8369 consensus_peers.cc:597] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:08.638934  8369 consensus_peers.cc:597] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:08.648447  8369 consensus_peers.cc:597] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:08.663630  8369 consensus_peers.cc:597] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:08.665885  8139 catalog_manager.cc:4755] TS 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 81804f59e3124d27998ea80fc1c36663: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:08.734093  8369 consensus_peers.cc:597] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:08.775084  8295 consensus_peers.cc:597] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:08.802094  8972 consensus_peers.cc:597] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:08.865227  8369 consensus_peers.cc:597] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:08.910054  8972 consensus_peers.cc:597] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:08.918970  8295 consensus_peers.cc:597] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:08.922042  8369 consensus_peers.cc:597] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:08.928056  8295 consensus_peers.cc:597] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:09.072093  8369 consensus_peers.cc:597] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:09.098026  8369 consensus_peers.cc:597] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:09.153144  8295 consensus_peers.cc:597] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:09.161876  8369 consensus_peers.cc:597] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:09.207588  8295 consensus_peers.cc:597] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:09.223273  8295 consensus_peers.cc:597] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:09.272090  8295 consensus_peers.cc:597] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:09.305351  8295 consensus_peers.cc:597] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:09.426774  7164 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 8278)
W20260709 13:05:09.479555  8295 consensus_peers.cc:597] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:09.567333  8295 consensus_peers.cc:597] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:09.575358  8369 consensus_peers.cc:597] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:09.583812  8369 consensus_peers.cc:597] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:09.631228  8295 consensus_peers.cc:597] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:09.973879  8295 consensus_peers.cc:597] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:09.994064  8972 consensus_peers.cc:597] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:10.100248  8295 consensus_peers.cc:597] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:10.125840  8369 consensus_peers.cc:597] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:10.358708  8972 consensus_peers.cc:597] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:10.427498  7164 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 8278)
W20260709 13:05:10.542405  8972 consensus_peers.cc:597] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:10.619073  8972 consensus_peers.cc:597] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:10.660597  8369 consensus_peers.cc:597] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:10.698947  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 75894.8us
W20260709 13:05:10.700904  8972 consensus_peers.cc:597] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:10.701865  8972 consensus_peers.cc:597] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:10.750720  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 44927.2us
W20260709 13:05:10.785642  8295 consensus_peers.cc:597] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:10.804522  8972 consensus_peers.cc:597] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:10.860255  8295 consensus_peers.cc:597] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:10.917084  8369 consensus_peers.cc:597] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:10.936676  8369 consensus_peers.cc:597] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:10.949542  8369 consensus_peers.cc:597] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:11.010715  8369 consensus_peers.cc:597] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:11.016149  8369 consensus_peers.cc:597] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:11.045342  8972 consensus_peers.cc:597] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:11.055316  8369 consensus_peers.cc:597] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:11.130458  8972 consensus_peers.cc:597] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:11.176899  8369 consensus_peers.cc:597] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:11.187587  8369 consensus_peers.cc:597] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:11.288350  8295 consensus_peers.cc:597] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:11.314414  8369 consensus_peers.cc:597] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:11.339613  8369 consensus_peers.cc:597] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:11.350819  8369 consensus_peers.cc:597] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:11.351380  8369 consensus_peers.cc:597] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:11.377709  8369 consensus_peers.cc:597] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:11.391045  8295 consensus_peers.cc:597] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:11.394793  8295 consensus_peers.cc:597] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:11.409734  8972 consensus_peers.cc:597] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:11.423929  8972 consensus_peers.cc:597] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:11.429471  7164 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 8278)
W20260709 13:05:11.476387  8295 consensus_peers.cc:597] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:11.600379  8369 consensus_peers.cc:597] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:11.684494  8295 consensus_peers.cc:597] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:11.688822  8369 consensus_peers.cc:597] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:11.708372  8369 consensus_peers.cc:597] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:11.743633  8369 consensus_peers.cc:597] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:11.762694  8295 consensus_peers.cc:597] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:11.790057  8295 consensus_peers.cc:597] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:11.889683  8369 consensus_peers.cc:597] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:11.964814  8295 consensus_peers.cc:597] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:12.003844  8295 consensus_peers.cc:597] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:12.087888  8295 consensus_peers.cc:597] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:12.132602  8369 consensus_peers.cc:597] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:12.159626  8295 consensus_peers.cc:597] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:12.168076  8369 consensus_peers.cc:597] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:12.248062  8972 consensus_peers.cc:597] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:12.430663  7164 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 8278)
W20260709 13:05:12.569072  8295 consensus_peers.cc:597] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:12.644042  8369 consensus_peers.cc:597] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:12.650810  8295 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111) [suppressed 486 similar messages]
W20260709 13:05:12.671665  8295 consensus_peers.cc:597] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:12.802423  8972 consensus_peers.cc:597] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:12.859902  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 53946.8us
W20260709 13:05:12.940212  8972 consensus_peers.cc:597] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:12.943421  8972 consensus_peers.cc:597] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:12.985399  8972 consensus_peers.cc:597] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:13.160485  8295 consensus_peers.cc:597] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:13.209241  8369 consensus_peers.cc:597] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:13.431483  7164 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 8278)
W20260709 13:05:13.442525  8369 consensus_peers.cc:597] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:13.449767  8369 consensus_peers.cc:597] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:13.443099  8295 consensus_peers.cc:597] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:13.451447  8369 consensus_peers.cc:597] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:13.526777  8369 consensus_peers.cc:597] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:13.527712  8369 consensus_peers.cc:597] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:13.597723  8369 consensus_peers.cc:597] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:13.762029  8369 consensus_peers.cc:597] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:13.829147  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 52714.8us
W20260709 13:05:13.832558  8369 consensus_peers.cc:597] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:13.859103  8295 consensus_peers.cc:597] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:13.914554  8295 consensus_peers.cc:597] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:13.945509  8369 consensus_peers.cc:597] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:13.946604  8369 consensus_peers.cc:597] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:13.976392  8369 consensus_peers.cc:597] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:13.981529  8369 consensus_peers.cc:597] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:14.013723  8295 consensus_peers.cc:597] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:14.062811  8295 consensus_peers.cc:597] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:14.114504  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 59939.3us
W20260709 13:05:14.139825  8369 consensus_peers.cc:597] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:14.140364  8972 consensus_peers.cc:597] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:14.145961  8972 consensus_peers.cc:597] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:14.146590  8972 consensus_peers.cc:597] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:14.157920  8369 consensus_peers.cc:597] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:14.201647  8974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:37367, user_credentials={real_user=slave}} blocked reactor thread for 63923.8us
W20260709 13:05:14.221787  8369 consensus_peers.cc:597] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:14.226056  8972 consensus_peers.cc:597] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:14.354898  8295 consensus_peers.cc:597] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:14.363124  8972 consensus_peers.cc:597] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:14.363724  8295 consensus_peers.cc:597] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:14.431995  7164 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 8278)
W20260709 13:05:14.548220  8295 consensus_peers.cc:597] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:14.592177  8295 consensus_peers.cc:597] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:14.659569  8295 consensus_peers.cc:597] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:14.669129  8295 consensus_peers.cc:597] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:14.669577  8972 consensus_peers.cc:597] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:14.688344  8295 consensus_peers.cc:597] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:14.974259  8295 consensus_peers.cc:597] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:15.057991  8295 consensus_peers.cc:597] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:15.221570  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:38599, user_credentials={real_user=slave}} blocked reactor thread for 64458.3us
W20260709 13:05:15.308701  8369 consensus_peers.cc:597] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:15.311072  8369 consensus_peers.cc:597] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:15.313724  8369 consensus_peers.cc:597] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:15.314459  8369 consensus_peers.cc:597] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:15.411689  8972 consensus_peers.cc:597] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:15.433106  7164 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 8278)
W20260709 13:05:15.524072  8369 consensus_peers.cc:597] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:15.650588  8369 consensus_peers.cc:597] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:15.673074  8295 consensus_peers.cc:597] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:15.749402  8295 consensus_peers.cc:597] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:15.921914  8972 consensus_peers.cc:597] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:15.931424  8972 consensus_peers.cc:597] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:16.035989  8369 consensus_peers.cc:597] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:16.090732  8369 consensus_peers.cc:597] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:16.211411  8972 consensus_peers.cc:597] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:16.232971  8972 consensus_peers.cc:597] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:16.294453  8369 consensus_peers.cc:597] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:16.307778  8972 consensus_peers.cc:597] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:16.323588  8369 consensus_peers.cc:597] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:16.363871  8369 consensus_peers.cc:597] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:16.430114  8295 consensus_peers.cc:597] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:16.433773  7164 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 8278)
W20260709 13:05:16.491640  8295 consensus_peers.cc:597] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:16.496351  8295 consensus_peers.cc:597] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:16.501539  8369 consensus_peers.cc:597] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:16.522666  8972 consensus_peers.cc:597] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:16.535125  8295 consensus_peers.cc:597] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:16.551800  8972 consensus_peers.cc:597] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:16.571856  8369 consensus_peers.cc:597] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:16.609741  8369 consensus_peers.cc:597] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:16.740573  8295 consensus_peers.cc:597] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:16.745098  8369 consensus_peers.cc:597] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:16.768111  8972 consensus_peers.cc:597] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:16.786907  8295 consensus_peers.cc:597] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:16.799278  8295 consensus_peers.cc:597] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:16.808799  8369 consensus_peers.cc:597] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:16.826785  8369 consensus_peers.cc:597] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:16.901628  8369 consensus_peers.cc:597] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:16.907505  8972 consensus_peers.cc:597] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:16.919318  8139 catalog_manager.cc:4755] TS 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 81804f59e3124d27998ea80fc1c36663: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:17.009631  8369 consensus_peers.cc:597] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:17.117041  8369 consensus_peers.cc:597] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:17.117550  8369 consensus_peers.cc:597] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:17.138711  8295 consensus_peers.cc:597] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:17.168429  8369 consensus_peers.cc:597] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:17.171139  8295 consensus_peers.cc:597] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:17.264925  8972 consensus_peers.cc:597] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:17.323941  8295 consensus_peers.cc:597] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:17.331543  8295 consensus_peers.cc:597] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:17.354014  8369 consensus_peers.cc:597] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:17.422202  8295 consensus_peers.cc:597] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:17.434674  7164 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 8278)
W20260709 13:05:17.544150  8295 consensus_peers.cc:597] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:17.548331  8369 consensus_peers.cc:597] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:17.658888  8295 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111) [suppressed 433 similar messages]
W20260709 13:05:17.799412  8369 consensus_peers.cc:597] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:17.817723  8369 consensus_peers.cc:597] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:17.947082  8369 consensus_peers.cc:597] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:18.045146  8369 consensus_peers.cc:597] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:18.289717  8295 consensus_peers.cc:597] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:18.380399  8295 consensus_peers.cc:597] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:18.402460  8298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 61140.6us
W20260709 13:05:18.435410  7164 thread.cc:527] Waited for 14000ms trying to join with diag-logger (tid 8278)
W20260709 13:05:18.470919  8369 consensus_peers.cc:597] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:18.580940  8972 consensus_peers.cc:597] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:18.683208  8369 consensus_peers.cc:597] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:18.693085  8369 consensus_peers.cc:597] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:18.813127  8369 consensus_peers.cc:597] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:18.842528  8295 consensus_peers.cc:597] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:18.926967  8972 consensus_peers.cc:597] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:18.938189  8295 consensus_peers.cc:597] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:18.951450  8972 consensus_peers.cc:597] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:18.970717  8369 consensus_peers.cc:597] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:18.994565  8369 consensus_peers.cc:597] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:19.136075  8295 consensus_peers.cc:597] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:19.153072  8369 consensus_peers.cc:597] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:19.155236  8369 consensus_peers.cc:597] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:19.165601  8295 consensus_peers.cc:597] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:19.169533  8295 consensus_peers.cc:597] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:19.183987  8369 consensus_peers.cc:597] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:19.211357  8369 consensus_peers.cc:597] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:19.270545  8972 consensus_peers.cc:597] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:19.277056  8369 consensus_peers.cc:597] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:19.354964  8295 consensus_peers.cc:597] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:19.380493  8295 consensus_peers.cc:597] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:19.384306  8972 consensus_peers.cc:597] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:19.403472  8972 consensus_peers.cc:597] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:19.435925  7164 thread.cc:527] Waited for 15000ms trying to join with diag-logger (tid 8278)
W20260709 13:05:19.481496  8369 consensus_peers.cc:597] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:19.488688  8369 consensus_peers.cc:597] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:19.515477  8369 consensus_peers.cc:597] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:19.578451  8295 consensus_peers.cc:597] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:19.614107  8295 consensus_peers.cc:597] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:19.654838  8295 consensus_peers.cc:597] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:19.700130  8369 consensus_peers.cc:597] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:19.719311  8369 consensus_peers.cc:597] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:19.736404  8369 consensus_peers.cc:597] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:19.795596  8295 consensus_peers.cc:597] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:19.819540  8972 consensus_peers.cc:597] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:19.829185  8972 consensus_peers.cc:597] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:19.912053  8295 consensus_peers.cc:597] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:19.995318  8972 consensus_peers.cc:597] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:20.034968  8295 consensus_peers.cc:597] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:20.040598  8369 consensus_peers.cc:597] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:20.137619  8972 consensus_peers.cc:597] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:20.242249  8369 consensus_peers.cc:597] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:20.372380  8972 consensus_peers.cc:597] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:20.377590  8369 consensus_peers.cc:597] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:20.404413  8369 consensus_peers.cc:597] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:20.437012  7164 thread.cc:527] Waited for 16000ms trying to join with diag-logger (tid 8278)
W20260709 13:05:20.457055  8369 consensus_peers.cc:597] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:20.730741  8295 consensus_peers.cc:597] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:20.789280  8295 consensus_peers.cc:597] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:20.874554  8972 consensus_peers.cc:597] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:20.918965  8369 consensus_peers.cc:597] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:21.162038  8369 consensus_peers.cc:597] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:21.192232  8369 consensus_peers.cc:597] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:21.402891  8369 consensus_peers.cc:597] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:21.415970  8972 consensus_peers.cc:597] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:21.437482  7164 thread.cc:527] Waited for 17000ms trying to join with diag-logger (tid 8278)
W20260709 13:05:21.446866  8972 consensus_peers.cc:597] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:21.510561  8369 consensus_peers.cc:597] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:21.511261  8369 consensus_peers.cc:597] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:21.540198  8369 consensus_peers.cc:597] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:21.602118  8972 consensus_peers.cc:597] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:21.626703  8295 consensus_peers.cc:597] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:21.640595  8295 consensus_peers.cc:597] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:21.647961  8295 consensus_peers.cc:597] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:21.657670  8369 consensus_peers.cc:597] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:21.694585  8295 consensus_peers.cc:597] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:21.697239  8972 consensus_peers.cc:597] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:21.699293  8369 consensus_peers.cc:597] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:21.699295  8295 consensus_peers.cc:597] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:21.741572  8369 consensus_peers.cc:597] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:21.855423  8295 consensus_peers.cc:597] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:21.890372  8369 consensus_peers.cc:597] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:21.899936  8369 consensus_peers.cc:597] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:21.905167  8972 consensus_peers.cc:597] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:21.947641  8369 consensus_peers.cc:597] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:21.967221  8369 consensus_peers.cc:597] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:22.045684  8295 consensus_peers.cc:597] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:22.085371  8369 consensus_peers.cc:597] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:22.132443  8295 consensus_peers.cc:597] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:22.143595  8972 consensus_peers.cc:597] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:22.152387  8295 consensus_peers.cc:597] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:22.202526  8972 consensus_peers.cc:597] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:22.238433  8369 consensus_peers.cc:597] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:22.256183  8295 consensus_peers.cc:597] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:22.296028  8295 consensus_peers.cc:597] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:22.340827  8369 consensus_peers.cc:597] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:22.453035  8295 consensus_peers.cc:597] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:22.579308  8295 consensus_peers.cc:597] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:05:22.624691  7164 tablet_server.cc:196] TabletServer@127.6.255.1:0 shutdown complete.
W20260709 13:05:22.671319  8369 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111) [suppressed 466 similar messages]
W20260709 13:05:22.674610  8369 consensus_peers.cc:597] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:22.703099  8972 consensus_peers.cc:597] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:22.737421  8972 consensus_peers.cc:597] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:22.749684  8369 consensus_peers.cc:597] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:05:22.880728  7164 tablet_server.cc:179] TabletServer@127.6.255.2:0 shutting down...
W20260709 13:05:22.898278  8369 consensus_peers.cc:597] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:22.906370  8972 consensus_peers.cc:597] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:22.911921  8972 consensus_peers.cc:597] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:22.916288  8369 consensus_peers.cc:597] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:22.930048  8369 consensus_peers.cc:597] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:22.934561  8369 consensus_peers.cc:597] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:22.936302  8369 consensus_peers.cc:597] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:22.956745  8369 consensus_peers.cc:597] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:22.987828  8972 consensus_peers.cc:597] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:23.003583  8972 consensus_peers.cc:597] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.006191  8369 consensus_peers.cc:597] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.007012  8369 consensus_peers.cc:597] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.021780  8972 consensus_peers.cc:597] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.034744  8369 consensus_peers.cc:597] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.054617  8369 consensus_peers.cc:597] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.061528  8369 consensus_peers.cc:597] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.062289  8369 consensus_peers.cc:597] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.064908  8369 consensus_peers.cc:597] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.065627  8369 consensus_peers.cc:597] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.088850  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:23.098006  7164 tablet_replica.cc:333] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.101567  7164 raft_consensus.cc:2243] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.103143  7164 raft_consensus.cc:2272] T 1d3e4cc2f40346c0bc3bc739e3675685 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.136535  8369 consensus_peers.cc:597] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:05:23.106515  7164 tablet_replica.cc:333] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
W20260709 13:05:23.144070  8972 consensus_peers.cc:597] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.144373  7164 raft_consensus.cc:2243] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 22 LEADER]: Raft consensus shutting down.
W20260709 13:05:23.153273  8369 consensus_peers.cc:597] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.154402  8369 consensus_peers.cc:597] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.153769  7164 raft_consensus.cc:2272] T 767eba28679649519167576023555b60 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.165663  7164 tablet_replica.cc:333] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.168038  7164 raft_consensus.cc:2243] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.170336  7164 raft_consensus.cc:2272] T 27e1a14c7c9f49e2b92fcea2a1bb191d P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.173312  7164 tablet_replica.cc:333] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
W20260709 13:05:23.169397  8369 consensus_peers.cc:597] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.184027  8369 consensus_peers.cc:597] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.186816  7164 raft_consensus.cc:2243] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.188572  7164 raft_consensus.cc:2272] T 0c3340359c8445d8950a3e68e0070928 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.191425  7164 tablet_replica.cc:333] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.192407  7164 raft_consensus.cc:2243] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.263263  7164 raft_consensus.cc:2272] T 9d36c3e19b8c47bc9d26b071486fdec9 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.266376  7164 tablet_replica.cc:333] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.294307  7164 raft_consensus.cc:2243] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.298945  7164 raft_consensus.cc:2272] T a0493b0dbbc746408a5db9e698678d83 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.311154  8369 consensus_peers.cc:597] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:23.287887  8295 consensus_peers.cc:597] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:23.288300  8972 consensus_peers.cc:597] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:05:23.311720  7164 tablet_replica.cc:333] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.320240  7164 raft_consensus.cc:2243] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.323873  7164 raft_consensus.cc:2272] T d966cf0fb9e24bd9993fb60505fdc62c P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.328580  8972 consensus_peers.cc:597] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.322477  8369 consensus_peers.cc:597] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.330539  8369 consensus_peers.cc:597] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.327222  7164 tablet_replica.cc:333] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.341010  7164 raft_consensus.cc:2243] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.344463  7164 raft_consensus.cc:2272] T 98c2abe28fd449b094f4ccff4bac396e P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.348804  7164 tablet_replica.cc:333] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.354110  7164 raft_consensus.cc:2243] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.354946  7164 raft_consensus.cc:2272] T b3d99c80602e45da9c43a49cf83b8211 P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.352950  8369 consensus_peers.cc:597] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.373375  7164 tablet_replica.cc:333] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.374502  7164 raft_consensus.cc:2243] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.380667  7164 raft_consensus.cc:2272] T 2323b22f124b407c95caf5525949dd6c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.386998  7164 tablet_replica.cc:333] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.387955  7164 raft_consensus.cc:2243] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.407051  7164 raft_consensus.cc:2272] T cb79894cecb541e3b7dd164271cadc16 P ff7a6fe533e94c1bae555b4cae427728 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.409332  8972 consensus_peers.cc:597] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.410959  7164 tablet_replica.cc:333] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.411933  7164 raft_consensus.cc:2243] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.412959  7164 raft_consensus.cc:2272] T 56b5f14332d64504837ae4e2df77e409 P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.418363  7164 tablet_replica.cc:333] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.419302  7164 raft_consensus.cc:2243] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.420634  7164 raft_consensus.cc:2272] T b23a66e34de947348d5940084701c646 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.423895  7164 tablet_replica.cc:333] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.424882  7164 raft_consensus.cc:2243] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.426172  7164 raft_consensus.cc:2272] T 09943d7e76a54d4e80eb181cac80bf22 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.429277  7164 tablet_replica.cc:333] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.430199  7164 raft_consensus.cc:2243] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.431726  7164 raft_consensus.cc:2272] T eeca7eb827a04189af4de94af55ffb91 P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.438323  8369 consensus_peers.cc:597] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:23.451611  8972 consensus_peers.cc:597] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.452513  7164 tablet_replica.cc:333] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.457322  7164 raft_consensus.cc:2243] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.461730  7164 raft_consensus.cc:2272] T 81804f59e3124d27998ea80fc1c36663 P ff7a6fe533e94c1bae555b4cae427728 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.464643  7164 tablet_replica.cc:333] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.465581  7164 raft_consensus.cc:2243] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Raft consensus shutting down.
W20260709 13:05:23.467444  8369 consensus_peers.cc:597] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.457582  8972 consensus_peers.cc:597] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.482426  8295 consensus_peers.cc:597] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:05:23.466416  7164 raft_consensus.cc:2272] T cbcc5b6425934c0aa85546e0b33f4abd P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.495586  7164 tablet_replica.cc:333] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.502209  7164 raft_consensus.cc:2243] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.503024  7164 raft_consensus.cc:2272] T 766ac43473084aa6a9fdce530520947c P ff7a6fe533e94c1bae555b4cae427728 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.517107  7164 tablet_replica.cc:333] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.520720  7164 raft_consensus.cc:2243] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.540302  7164 raft_consensus.cc:2272] T 82abcf8a2a574841972243a029be749d P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.551066  8369 consensus_peers.cc:597] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:05:23.602802  7164 tablet_replica.cc:333] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.603783  7164 raft_consensus.cc:2243] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.604964  7164 raft_consensus.cc:2272] T eeecf7303d634c0d8983b810bb97c33c P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.607909  7164 tablet_replica.cc:333] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.608709  7164 raft_consensus.cc:2243] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.616262  7164 raft_consensus.cc:2272] T 83839ff3ce80447bb72b3c8d15b630ff P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.619817  7164 tablet_replica.cc:333] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.620594  7164 raft_consensus.cc:2243] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.621169  7164 raft_consensus.cc:2272] T ec275b714efb4b8b92919025fbd79b1f P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.623806  7164 tablet_replica.cc:333] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.624612  7164 raft_consensus.cc:2243] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.634825  7164 raft_consensus.cc:2272] T b7c15354a2a84f868942919f011af448 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.642696  7164 tablet_replica.cc:333] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.644152  7164 raft_consensus.cc:2243] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.644901  7164 raft_consensus.cc:2272] T e9159db9f370400d98f8503d06a33a33 P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.647828  7164 tablet_replica.cc:333] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.648706  7164 raft_consensus.cc:2243] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.649365  7164 raft_consensus.cc:2272] T 038cd0877a674c2996410c9f327df911 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.652146  7164 tablet_replica.cc:333] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.653017  7164 raft_consensus.cc:2243] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.654088  7164 raft_consensus.cc:2272] T 6a2e021fd91643b49aa8e553436c4b69 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.657627  7164 tablet_replica.cc:333] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.658466  7164 raft_consensus.cc:2243] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.660594  7164 raft_consensus.cc:2272] T 07268be9b33649e8b0e8bddf7a3bcabe P ff7a6fe533e94c1bae555b4cae427728 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.663869  7164 tablet_replica.cc:333] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.664710  7164 raft_consensus.cc:2243] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.669121  7164 raft_consensus.cc:2272] T 6efcdb47a4a24d3680780fb6deb13911 P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.672181  7164 tablet_replica.cc:333] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.672994  7164 raft_consensus.cc:2243] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.674157  7164 raft_consensus.cc:2272] T bf42cb9d00c6491eb6b0cf8b896f7c5b P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.676980  7164 tablet_replica.cc:333] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.677801  7164 raft_consensus.cc:2243] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Raft consensus shutting down.
W20260709 13:05:23.678553  8369 consensus_peers.cc:597] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:05:23.681521  7164 raft_consensus.cc:2272] T 625b0bdeab5d48eaa06a46928cbad7e4 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.689252  7164 tablet_replica.cc:333] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.692564  7164 raft_consensus.cc:2243] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.699956  7164 raft_consensus.cc:2272] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P ff7a6fe533e94c1bae555b4cae427728 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.710731  7164 tablet_replica.cc:333] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
W20260709 13:05:23.716830  8972 consensus_peers.cc:597] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:05:23.724308  7164 raft_consensus.cc:2243] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.728892  7164 raft_consensus.cc:2272] T ed318aa932504f70b21cfa8e646283ac P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.735881  7164 tablet_replica.cc:333] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.736951  7164 raft_consensus.cc:2243] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.737895  7164 raft_consensus.cc:2272] T fdf12882ac9e4f7d9121a5fcb9d4efaf P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.741465  7164 tablet_replica.cc:333] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.742507  7164 raft_consensus.cc:2243] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.756583  7164 raft_consensus.cc:2272] T 5c7b40c72696450b884fa3a4659ccd84 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.763967  7164 tablet_replica.cc:333] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.765051  7164 raft_consensus.cc:2243] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.766131  7164 raft_consensus.cc:2272] T a1fdbe7b73a540ddb2e5cb473a8404de P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.769634  7164 tablet_replica.cc:333] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.770565  7164 raft_consensus.cc:2243] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.773113  7164 raft_consensus.cc:2272] T fade5d45b9804cc98377ac6ab69be94b P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.782828  7164 tablet_replica.cc:333] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.793570  7164 raft_consensus.cc:2243] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.794422  7164 raft_consensus.cc:2272] T 6c6a6afa04ab437cbbd6011a11681763 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.825611  8369 consensus_peers.cc:597] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:23.833640  8972 consensus_peers.cc:597] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:05:23.837069  7164 tablet_replica.cc:333] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.838311  7164 raft_consensus.cc:2243] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.839470  7164 raft_consensus.cc:2272] T ae82ced6ff014453bbcfd5f9f14b1f8d P ff7a6fe533e94c1bae555b4cae427728 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.853955  7164 tablet_replica.cc:333] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.854789  7164 raft_consensus.cc:2243] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.856025  7164 raft_consensus.cc:2272] T 338bba401dcd4d8c9642cd7b255dea63 P ff7a6fe533e94c1bae555b4cae427728 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.860193  7164 tablet_replica.cc:333] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.861558  7164 raft_consensus.cc:2243] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.862830  7164 raft_consensus.cc:2272] T f66b0119c0d84d8cb95b2252fcb6d390 P ff7a6fe533e94c1bae555b4cae427728 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.865873  7164 tablet_replica.cc:333] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.866686  7164 raft_consensus.cc:2243] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.867825  7164 raft_consensus.cc:2272] T bf584dd56b3149a0b92539bc7115815b P ff7a6fe533e94c1bae555b4cae427728 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.870824  7164 tablet_replica.cc:333] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.871592  7164 raft_consensus.cc:2243] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.876492  7164 raft_consensus.cc:2272] T bac7f1e5192d48a98a850cb0ece17ad5 P ff7a6fe533e94c1bae555b4cae427728 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.880836  7164 tablet_replica.cc:333] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.881587  7164 raft_consensus.cc:2243] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.882225  7164 raft_consensus.cc:2272] T 104e8624a4bd4222aa8e1be9cba98372 P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.885200  7164 tablet_replica.cc:333] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.886632  7164 raft_consensus.cc:2243] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.888134  7164 raft_consensus.cc:2272] T 39384c6d69ad4f658d2e777dfdcbb49c P ff7a6fe533e94c1bae555b4cae427728 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.890731  7164 tablet_replica.cc:333] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.891460  7164 raft_consensus.cc:2243] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.894212  7164 raft_consensus.cc:2272] T 3c1d459f4b98433682d7d19a825eb4ef P ff7a6fe533e94c1bae555b4cae427728 [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.898481  8369 consensus_peers.cc:597] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:05:23.900141  7164 tablet_replica.cc:333] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.901042  7164 raft_consensus.cc:2243] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.902101  7164 raft_consensus.cc:2272] T 954d2ad09dcb4697adccd661625a841b P ff7a6fe533e94c1bae555b4cae427728 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.904978  7164 tablet_replica.cc:333] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.907085  7164 raft_consensus.cc:2243] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.908571  7164 raft_consensus.cc:2272] T 20b7fa75b43141c0ba1e5e08972a3a8f P ff7a6fe533e94c1bae555b4cae427728 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.911384  7164 tablet_replica.cc:333] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728: stopping tablet replica
I20260709 13:05:23.912055  7164 raft_consensus.cc:2243] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.913244  7164 raft_consensus.cc:2272] T 757eb9e36091440581fbfc9a6b000e74 P ff7a6fe533e94c1bae555b4cae427728 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.989744  8972 consensus_peers.cc:597] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:24.053046  8369 consensus_peers.cc:597] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:24.098572  8369 consensus_peers.cc:597] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:24.120648  8972 consensus_peers.cc:597] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:24.155437  8369 consensus_peers.cc:597] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:24.238232  8369 consensus_peers.cc:597] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:24.256767  8972 consensus_peers.cc:597] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:05:24.343097 16570 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:24.343712 16570 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:24.346415 16570 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:24.368858  8369 leader_election.cc:336] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:24.376415  8369 leader_election.cc:336] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:24.377043  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:24.378005 16570 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:05:24.390895  8369 consensus_peers.cc:597] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:05:24.432665 16566 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:24.433348 16566 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:24.435988 16566 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:24.507652  8372 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:33825, user_credentials={real_user=slave}} blocked reactor thread for 64010.9us
I20260709 13:05:24.517248 16566 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:24.517922 16566 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:24.520509 16566 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:24.521689  8972 leader_election.cc:336] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:24.527637  8972 leader_election.cc:336] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:24.530382  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:24.534896 16566 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:24.537055  8369 consensus_peers.cc:597] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:24.562685  8369 consensus_peers.cc:597] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:05:24.563297 16566 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:24.564215 16566 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:24.566663 16566 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:24.583143  8369 consensus_peers.cc:597] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:24.584717  8972 leader_election.cc:336] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:24.665977  8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 81295.1us
W20260709 13:05:24.676347  8369 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:44679, user_credentials={real_user=slave}} blocked reactor thread for 90955.8us
W20260709 13:05:24.715298  8369 consensus_peers.cc:597] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:24.720039  8972 leader_election.cc:336] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:24.720605  8972 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:24.721907 16578 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:24.729044 16575 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:24.729838 16575 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:24.732710 16575 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:24.753984  8972 leader_election.cc:336] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:24.763837 16578 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:24.764539 16578 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:24.768958  8398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:24.771119 16578 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:05:24.773468  8972 leader_election.cc:336] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:24.773998  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:24.775236 16578 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:05:24.799784  8972 consensus_peers.cc:597] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:05:24.800683 16575 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:24.802205 16575 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:24.804441 16575 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:24.806085  8369 consensus_peers.cc:597] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:24.810212  8369 consensus_peers.cc:597] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:05:24.815465 16579 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:24.816015 16579 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:24.818245 16579 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:05:24.826825 16575 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:24.827466 16575 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:24.830504 16575 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:24.832599  8369 leader_election.cc:336] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
W20260709 13:05:24.838778  8972 consensus_peers.cc:597] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:24.840070  8972 leader_election.cc:336] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:24.842702  8972 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:24.843605 16578 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:24.853703 16579 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:24.854264 16579 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:24.856356 16579 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:24.864837  8369 leader_election.cc:336] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:24.866335  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:24.873502 16575 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:24.889796 16578 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:24.890492 16578 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:24.892794 16578 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:24.903885  8369 consensus_peers.cc:597] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:24.908177  8369 leader_election.cc:336] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:24.916539  8369 leader_election.cc:336] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:24.918346 16578 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:24.919667 16578 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:24.921754 16578 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:24.944331  8972 leader_election.cc:336] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:24.945209  8369 leader_election.cc:336] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:24.946622  8369 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:24.949288 16575 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:24.951762 16575 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:24.954232 16575 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:24.957289  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:05:24.957963  9002 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:05:24.959283  8372 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
W20260709 13:05:24.961143  8972 leader_election.cc:336] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:24.962201 16579 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:24.963974 16579 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:24.964432 16579 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:24.961709 16575 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:24.988305 16575 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:24.988970 16575 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:24.991549 16575 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:05:24.992137  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:05:24.963406  8369 leader_election.cc:336] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:24.992992  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:24.993604 16578 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:05:24.995740 16579 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:05:24.997047  8369 leader_election.cc:336] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:24.997494 16575 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:05:25.001281  8972 consensus_peers.cc:597] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:05:25.004045  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeecf7303d634c0d8983b810bb97c33c" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:05:25.018170  9002 raft_consensus.cc:3060] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:25.019204 16580 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:25.019747 16580 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:25.023229  9001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:05:25.023869  9001 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 13.
I20260709 13:05:25.025326  8372 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:05:25.025892  9002 raft_consensus.cc:2468] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
I20260709 13:05:25.026206 16570 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:25.026440 16580 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:25.027352 16570 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:25.028080 16570 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:25.027194  8372 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:05:25.035651 16572 raft_consensus.cc:2804] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:25.036217 16572 raft_consensus.cc:697] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:05:25.037276 16572 consensus_queue.cc:237] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:25.048355 16570 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:25.061831 16575 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:25.062446 16575 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:25.051422 16579 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
W20260709 13:05:25.060622  8972 leader_election.cc:336] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:25.085887 16570 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:25.078089 16575 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:05:25.086095  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e0ee9efb7d40309bc7e4d2450bb1b5" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:05:25.090047  9002 raft_consensus.cc:3060] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:25.097180  9002 raft_consensus.cc:2468] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 14.
I20260709 13:05:25.098495  8372 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:05:25.104643  8147 catalog_manager.cc:5697] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 19 to 20, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 20 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:25.111665 16572 raft_consensus.cc:2804] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:25.113346 16572 raft_consensus.cc:697] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:05:25.114684 16572 consensus_queue.cc:237] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:05:25.088995  8972 leader_election.cc:336] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:25.120461  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:25.123822 16578 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:25.132678 16570 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:25.133586 16570 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:25.137761 16570 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:25.135763  8369 leader_election.cc:336] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:25.143044  8369 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:05:25.144130  8369 leader_election.cc:336] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:25.146026 16570 raft_consensus.cc:2749] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:05:25.156630  8369 leader_election.cc:336] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:25.157160  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:25.158969 16570 raft_consensus.cc:2749] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:05:25.185984 16575 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:25.186805 16575 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:25.189204 16575 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:25.185949  8369 leader_election.cc:336] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:25.206669  8369 leader_election.cc:336] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
W20260709 13:05:25.208920  8369 leader_election.cc:336] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
W20260709 13:05:25.210354  7164 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 8353)
I20260709 13:05:25.210512  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:05:25.211699  8369 leader_election.cc:336] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:25.212294 16572 raft_consensus.cc:2749] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:05:25.212844  8147 catalog_manager.cc:5697] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 13 to 14, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 14 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:25.212380  8369 leader_election.cc:336] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:25.279548 16578 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:25.280150 16578 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:25.285280  8397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "f865c4d761f5446eb12b3a029234772e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e64761f43e4240319c41117cf886d22f" is_pre_election: true
I20260709 13:05:25.286334 16578 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:05:25.288676  8369 leader_election.cc:336] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:25.290448  8369 leader_election.cc:336] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:25.291289  8369 leader_election.cc:336] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
W20260709 13:05:25.294034  8972 leader_election.cc:336] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:25.294622  8972 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:25.295459 16578 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:25.296751  8369 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:25.297703 16570 raft_consensus.cc:2749] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:25.297931  8369 leader_election.cc:336] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
W20260709 13:05:25.299245  8369 leader_election.cc:336] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:25.299779  8369 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:25.300575 16572 raft_consensus.cc:2749] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:05:25.326553  8369 leader_election.cc:336] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
W20260709 13:05:25.328665  8369 consensus_peers.cc:597] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:25.329134  8369 leader_election.cc:336] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:25.329566  8369 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:25.335882 16572 raft_consensus.cc:2749] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:05:25.337059 16570 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:25.337664 16570 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:25.339919 16570 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:25.341625  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e" is_pre_election: true
I20260709 13:05:25.343564  9002 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 19.
I20260709 13:05:25.344802  8372 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:05:25.345643 16570 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:25.346032 16570 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Starting leader election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:25.346412 16570 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:25.355638 16570 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:25.358939  9002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae82ced6ff014453bbcfd5f9f14b1f8d" candidate_uuid: "e64761f43e4240319c41117cf886d22f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f865c4d761f5446eb12b3a029234772e"
I20260709 13:05:25.359539  9002 raft_consensus.cc:3060] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:25.367492  9002 raft_consensus.cc:2468] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64761f43e4240319c41117cf886d22f in term 20.
I20260709 13:05:25.368724  8372 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e64761f43e4240319c41117cf886d22f, f865c4d761f5446eb12b3a029234772e; no voters: 
I20260709 13:05:25.369644 16572 raft_consensus.cc:2804] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:25.370151 16572 raft_consensus.cc:697] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Becoming Leader. State: Replica: e64761f43e4240319c41117cf886d22f, State: Running, Role: LEADER
I20260709 13:05:25.371094 16572 consensus_queue.cc:237] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:05:25.386307  8972 consensus_peers.cc:597] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:05:25.391340 16570 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: Requested vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), f865c4d761f5446eb12b3a029234772e (127.6.255.4:33825)
I20260709 13:05:25.396386  8147 catalog_manager.cc:5697] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f reported cstate change: term changed from 19 to 20, leader changed from ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2) to e64761f43e4240319c41117cf886d22f (127.6.255.3). New cstate: current_term: 20 leader_uuid: "e64761f43e4240319c41117cf886d22f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:25.401832  8972 consensus_peers.cc:597] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:25.433120  8369 consensus_peers.cc:597] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:25.433808  8369 leader_election.cc:336] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
W20260709 13:05:25.434602  8369 leader_election.cc:336] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
W20260709 13:05:25.449507  8369 consensus_peers.cc:597] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:25.451303  8369 consensus_peers.cc:597] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:25.454352  8369 consensus_peers.cc:597] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:25.502655  8369 consensus_peers.cc:597] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:25.526796  8369 consensus_peers.cc:597] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:25.533286  8369 consensus_peers.cc:597] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:25.536764  8972 consensus_peers.cc:597] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:25.549955  8972 consensus_peers.cc:597] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:25.554265  8369 consensus_peers.cc:597] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:25.558802  9002 raft_consensus.cc:1275] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:05:25.560954 16570 consensus_queue.cc:1048] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:05:25.565646  8369 consensus_peers.cc:597] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:25.586663  8972 consensus_peers.cc:597] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:25.589270  8369 consensus_peers.cc:597] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:25.628587  8972 consensus_peers.cc:597] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:25.657757  9002 raft_consensus.cc:1275] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:05:25.658643  8369 consensus_peers.cc:597] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:25.661664 16570 consensus_queue.cc:1048] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:05:25.684309  8369 consensus_peers.cc:597] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:25.685024  8972 consensus_peers.cc:597] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:25.742625  8369 consensus_peers.cc:597] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:25.749207  8369 consensus_peers.cc:597] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:25.790409  8369 consensus_peers.cc:597] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:25.805290  8369 consensus_peers.cc:597] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:25.806842  8369 consensus_peers.cc:597] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:25.808349  8972 consensus_peers.cc:597] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:25.810613  8369 consensus_peers.cc:597] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:25.811131  8369 consensus_peers.cc:597] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:25.821357  8369 consensus_peers.cc:597] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:25.832479  8369 consensus_peers.cc:597] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:25.838825  8369 consensus_peers.cc:597] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:25.859745  8369 consensus_peers.cc:597] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:25.888244  8369 consensus_peers.cc:597] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:25.922961  8369 consensus_peers.cc:597] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:25.935627 16575 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:25.936228 16575 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:25.938723 16575 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:25.946978  8369 leader_election.cc:336] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:25.947743  8369 leader_election.cc:336] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:25.948167  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:25.950248 16570 raft_consensus.cc:2749] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:25.953078  9002 raft_consensus.cc:1275] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Refusing update from remote peer e64761f43e4240319c41117cf886d22f: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 13:05:25.953289  8369 consensus_peers.cc:597] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:25.956583 16570 consensus_queue.cc:1048] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:05:25.968581  8972 consensus_peers.cc:597] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:26.005059  8972 consensus_peers.cc:597] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:26.039482  8369 consensus_peers.cc:597] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:26.055083  8369 consensus_peers.cc:597] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:26.071993  8972 consensus_peers.cc:597] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:26.088164  8369 consensus_peers.cc:597] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:26.106400  8972 consensus_peers.cc:597] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:26.160409  8972 consensus_peers.cc:597] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:26.186609  8369 consensus_peers.cc:597] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:26.231236  8369 consensus_peers.cc:597] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:26.249862  8972 consensus_peers.cc:597] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:05:26.283027 16578 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:26.283650 16578 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:26.285966 16578 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:26.295363  8972 leader_election.cc:336] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:26.296146  8972 leader_election.cc:336] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:26.296623  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:26.297865 16578 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:26.345263  8369 consensus_peers.cc:597] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:26.449636  8369 consensus_peers.cc:597] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:26.457647  8369 consensus_peers.cc:597] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:26.567649  8972 consensus_peers.cc:597] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:05:26.582556 16597 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:26.583122 16597 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:26.585193 16597 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:26.598234  8369 consensus_peers.cc:597] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:26.608747  8972 leader_election.cc:336] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:26.611720  8972 leader_election.cc:336] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:26.614737  8972 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:26.615501 16597 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:26.614307  8369 consensus_peers.cc:597] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:05:26.643302 16572 raft_consensus.cc:493] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:26.643837 16572 raft_consensus.cc:515] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:26.645819 16572 leader_election.cc:290] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:26.681357  8369 consensus_peers.cc:597] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:26.682057  8369 leader_election.cc:336] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:26.683113  8369 leader_election.cc:336] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:26.687058  8369 leader_election.cc:304] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:26.687903 16572 raft_consensus.cc:2749] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:26.720638 16578 raft_consensus.cc:493] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:26.721253 16578 raft_consensus.cc:515] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:26.723351 16578 leader_election.cc:290] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:26.732527  8972 leader_election.cc:336] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:26.733279  8972 leader_election.cc:336] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:26.733728  8972 leader_election.cc:304] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:26.735203 16578 raft_consensus.cc:2749] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:05:26.770218  8972 consensus_peers.cc:597] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:05:26.793637 16570 raft_consensus.cc:493] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:26.794155 16570 raft_consensus.cc:515] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:26.795970 16570 leader_election.cc:290] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:26.800947  8369 leader_election.cc:336] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:26.801640  8369 leader_election.cc:336] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:26.802094  8369 leader_election.cc:304] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:26.802824 16570 raft_consensus.cc:2749] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:26.841573 16570 raft_consensus.cc:493] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:26.842089 16570 raft_consensus.cc:515] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:26.843868 16570 leader_election.cc:290] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:26.866321  8369 leader_election.cc:336] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:26.872448  8369 leader_election.cc:336] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:26.873016  8369 leader_election.cc:304] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:26.873840 16598 raft_consensus.cc:2749] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:26.904245 16578 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:26.905117 16578 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:26.916229 16578 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:26.917337  8972 leader_election.cc:336] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:26.922394  8972 leader_election.cc:336] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:26.923002  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:26.924070 16578 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:05:26.997704  8972 consensus_peers.cc:597] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:05:27.099350 16598 raft_consensus.cc:493] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:27.099910 16598 raft_consensus.cc:515] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:27.102000 16598 leader_election.cc:290] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:27.107336  8369 leader_election.cc:336] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:27.112488 16577 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:27.113086 16577 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:27.115197 16577 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:27.127321  8972 leader_election.cc:336] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
W20260709 13:05:27.131340  8972 leader_election.cc:336] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:27.132467  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:27.133292 16578 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:27.167374 16598 raft_consensus.cc:493] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:27.167929 16598 raft_consensus.cc:515] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
W20260709 13:05:27.168255  8972 consensus_peers.cc:597] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:05:27.170413 16598 leader_election.cc:290] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:05:27.171049  7164 tablet_server.cc:196] TabletServer@127.6.255.2:0 shutdown complete.
W20260709 13:05:27.175602  8369 leader_election.cc:336] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:27.206794 16598 raft_consensus.cc:493] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:27.207335 16598 raft_consensus.cc:515] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:27.211434 16598 leader_election.cc:290] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:27.213625  8369 leader_election.cc:336] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:27.237776 16598 raft_consensus.cc:493] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:27.238315 16598 raft_consensus.cc:515] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:27.240298 16598 leader_election.cc:290] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:27.245093  8369 leader_election.cc:336] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
W20260709 13:05:27.290735  8972 consensus_peers.cc:597] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:05:27.331775  7164 tablet_server.cc:179] TabletServer@127.6.255.3:0 shutting down...
W20260709 13:05:27.332085  8972 consensus_peers.cc:597] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:27.374676  8974 consensus_peers.cc:597] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e -> Peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Couldn't send request to peer e64761f43e4240319c41117cf886d22f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:27.388151 16572 raft_consensus.cc:493] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:27.388792 16572 raft_consensus.cc:515] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:27.390952 16572 leader_election.cc:290] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:27.401402  8369 leader_election.cc:336] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
W20260709 13:05:27.402776  8974 consensus_peers.cc:597] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e -> Peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Couldn't send request to peer e64761f43e4240319c41117cf886d22f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:27.444262  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:27.447434  7164 tablet_replica.cc:333] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.448387  7164 raft_consensus.cc:2243] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.449505  7164 raft_consensus.cc:2272] T 1d3e4cc2f40346c0bc3bc739e3675685 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.452831  7164 tablet_replica.cc:333] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.453800  7164 raft_consensus.cc:2243] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.455000  7164 raft_consensus.cc:2272] T a0493b0dbbc746408a5db9e698678d83 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.458030  7164 tablet_replica.cc:333] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.458976  7164 raft_consensus.cc:2243] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.459849  7164 raft_consensus.cc:2272] T 27e1a14c7c9f49e2b92fcea2a1bb191d P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:27.461319  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.490445  7164 tablet_replica.cc:333] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.491851  7164 raft_consensus.cc:2243] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.492839  7164 raft_consensus.cc:2272] T 0c3340359c8445d8950a3e68e0070928 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.495724  7164 tablet_replica.cc:333] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.496583  7164 raft_consensus.cc:2243] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.498917  7164 raft_consensus.cc:2272] T f19e818631f543b28a8bec9ba4547527 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.501873  7164 tablet_replica.cc:333] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.502787  7164 raft_consensus.cc:2243] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.503474  7164 raft_consensus.cc:2272] T d966cf0fb9e24bd9993fb60505fdc62c P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.509922 16570 raft_consensus.cc:493] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:27.510390 16570 raft_consensus.cc:515] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:27.512357 16570 leader_election.cc:290] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:05:27.512789  7164 tablet_replica.cc:333] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.513748  7164 raft_consensus.cc:2243] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.514737  7164 raft_consensus.cc:2272] T 98c2abe28fd449b094f4ccff4bac396e P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:27.516268  8369 leader_election.cc:336] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:27.517472  7164 tablet_replica.cc:333] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.518499  7164 raft_consensus.cc:2243] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.519351  7164 raft_consensus.cc:2272] T bac7f1e5192d48a98a850cb0ece17ad5 P e64761f43e4240319c41117cf886d22f [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.522424  7164 tablet_replica.cc:333] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.523378  7164 raft_consensus.cc:2243] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.524293  7164 raft_consensus.cc:2272] T eeca7eb827a04189af4de94af55ffb91 P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.527168  7164 tablet_replica.cc:333] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
W20260709 13:05:27.527570  8974 consensus_peers.cc:597] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e -> Peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Couldn't send request to peer e64761f43e4240319c41117cf886d22f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:27.527999  7164 raft_consensus.cc:2243] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.529086  7164 raft_consensus.cc:2272] T 6a2e021fd91643b49aa8e553436c4b69 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.531852  7164 tablet_replica.cc:333] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.532697  7164 raft_consensus.cc:2243] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.533303  7164 raft_consensus.cc:2272] T 038cd0877a674c2996410c9f327df911 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.536744  7164 tablet_replica.cc:333] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.537595  7164 raft_consensus.cc:2243] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.538245  7164 raft_consensus.cc:2272] T e94bcf9755aa4f0abddcd5b9fbc2939f P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:27.546306  8972 consensus_peers.cc:597] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:05:27.547006  7164 tablet_replica.cc:333] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.547935  7164 raft_consensus.cc:2243] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.549029  7164 raft_consensus.cc:2272] T 56b5f14332d64504837ae4e2df77e409 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.552053  7164 tablet_replica.cc:333] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.552913  7164 raft_consensus.cc:2243] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.553741  7164 raft_consensus.cc:2272] T b23a66e34de947348d5940084701c646 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.556599  7164 tablet_replica.cc:333] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f: stopping tablet replica
W20260709 13:05:27.557565  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.557576  7164 raft_consensus.cc:2243] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.559134  7164 raft_consensus.cc:2272] T 4b9b125a55cf4a5d9db580313045edde P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:27.567814  8974 consensus_peers.cc:597] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e -> Peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Couldn't send request to peer e64761f43e4240319c41117cf886d22f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:27.574015  7164 tablet_replica.cc:333] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
W20260709 13:05:27.575434  8974 consensus_peers.cc:597] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e -> Peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Couldn't send request to peer e64761f43e4240319c41117cf886d22f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:27.576632  7164 raft_consensus.cc:2243] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.577406  7164 raft_consensus.cc:2272] T 81804f59e3124d27998ea80fc1c36663 P e64761f43e4240319c41117cf886d22f [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:27.584646  8974 consensus_peers.cc:597] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e -> Peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Couldn't send request to peer e64761f43e4240319c41117cf886d22f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:27.588075  7164 tablet_replica.cc:333] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.589002  7164 raft_consensus.cc:2243] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.590246  7164 raft_consensus.cc:2272] T cbcc5b6425934c0aa85546e0b33f4abd P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.593039  7164 tablet_replica.cc:333] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.593871  7164 raft_consensus.cc:2243] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.594440  7164 raft_consensus.cc:2272] T 766ac43473084aa6a9fdce530520947c P e64761f43e4240319c41117cf886d22f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.597110  7164 tablet_replica.cc:333] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.597941  7164 raft_consensus.cc:2243] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.607300  7164 raft_consensus.cc:2272] T 1112bcf4c4ea44e1933ee6287bc869f8 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.609565 16570 raft_consensus.cc:493] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:27.609989 16570 raft_consensus.cc:515] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } }
I20260709 13:05:27.611714 16570 leader_election.cc:290] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
I20260709 13:05:27.612399  7164 tablet_replica.cc:333] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.613307  7164 raft_consensus.cc:2243] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.614354  7164 raft_consensus.cc:2272] T eeecf7303d634c0d8983b810bb97c33c P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:27.616540  8369 leader_election.cc:336] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:27.618152  7164 tablet_replica.cc:333] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.619022  7164 raft_consensus.cc:2243] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.620150  7164 raft_consensus.cc:2272] T ec275b714efb4b8b92919025fbd79b1f P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.624146  7164 tablet_replica.cc:333] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.624929  7164 raft_consensus.cc:2243] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.625908  7164 raft_consensus.cc:2272] T e9159db9f370400d98f8503d06a33a33 P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.628531  7164 tablet_replica.cc:333] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.629319  7164 raft_consensus.cc:2243] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.630369  7164 raft_consensus.cc:2272] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P e64761f43e4240319c41117cf886d22f [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:27.638754  8974 consensus_peers.cc:597] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e -> Peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Couldn't send request to peer e64761f43e4240319c41117cf886d22f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:27.640038  7164 tablet_replica.cc:333] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.640978  7164 raft_consensus.cc:2243] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.642154  7164 raft_consensus.cc:2272] T ed318aa932504f70b21cfa8e646283ac P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.645037  7164 tablet_replica.cc:333] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.653677  7164 raft_consensus.cc:2243] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:05:27.656013  8974 consensus_peers.cc:597] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e -> Peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Couldn't send request to peer e64761f43e4240319c41117cf886d22f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:27.656595  7164 raft_consensus.cc:2272] T 4516247b88e34a1d89ec9eb29e0ecb7f P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.660152  7164 tablet_replica.cc:333] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.661084  7164 raft_consensus.cc:2243] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.663102  7164 raft_consensus.cc:2272] T 60c80a6b33dc431eac0fe35b37d4ea23 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.666925  7164 tablet_replica.cc:333] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.667711  7164 raft_consensus.cc:2243] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.668505  7164 raft_consensus.cc:2272] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:27.669682  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.671327  7164 tablet_replica.cc:333] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.672154  7164 raft_consensus.cc:2243] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.673255  7164 raft_consensus.cc:2272] T 625b0bdeab5d48eaa06a46928cbad7e4 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.676187  7164 tablet_replica.cc:333] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.676990  7164 raft_consensus.cc:2243] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.677709  7164 raft_consensus.cc:2272] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.680320  7164 tablet_replica.cc:333] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.681742  7164 raft_consensus.cc:2243] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.682480  7164 raft_consensus.cc:2272] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.685472  7164 tablet_replica.cc:333] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.686408  7164 raft_consensus.cc:2243] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.687739  7164 raft_consensus.cc:2272] T a1fdbe7b73a540ddb2e5cb473a8404de P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:27.689163  8974 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 684 similar messages]
W20260709 13:05:27.690397  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.692381  8974 consensus_peers.cc:597] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e -> Peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Couldn't send request to peer e64761f43e4240319c41117cf886d22f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:27.693131  7164 tablet_replica.cc:333] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.694085  7164 raft_consensus.cc:2243] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.695227  7164 raft_consensus.cc:2272] T 07268be9b33649e8b0e8bddf7a3bcabe P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:27.695312  8370 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.698473  7164 tablet_replica.cc:333] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.699359  7164 raft_consensus.cc:2243] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.700407  7164 raft_consensus.cc:2272] T 104e8624a4bd4222aa8e1be9cba98372 P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.703315  7164 tablet_replica.cc:333] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.704198  7164 raft_consensus.cc:2243] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.705263  7164 raft_consensus.cc:2272] T 1263b3742fd541f8bda5a6b6c10a8596 P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.708484  7164 tablet_replica.cc:333] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.709236  7164 raft_consensus.cc:2243] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.710366  7164 raft_consensus.cc:2272] T ae82ced6ff014453bbcfd5f9f14b1f8d P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.713259  7164 tablet_replica.cc:333] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.714128  7164 raft_consensus.cc:2243] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.714797  7164 raft_consensus.cc:2272] T 39384c6d69ad4f658d2e777dfdcbb49c P e64761f43e4240319c41117cf886d22f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.717322  7164 tablet_replica.cc:333] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.718156  7164 raft_consensus.cc:2243] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.718888  7164 raft_consensus.cc:2272] T 430fa172910841a49c0c80f2cbf76ae5 P e64761f43e4240319c41117cf886d22f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.721519  7164 tablet_replica.cc:333] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.722318  7164 raft_consensus.cc:2243] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.723374  7164 raft_consensus.cc:2272] T 10cffa7c6411440ca77cee6731faca67 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.726054  7164 tablet_replica.cc:333] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.726801  7164 raft_consensus.cc:2243] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.727519  7164 raft_consensus.cc:2272] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.730012  7164 tablet_replica.cc:333] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.730818  7164 raft_consensus.cc:2243] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.731576  7164 raft_consensus.cc:2272] T d32240ced15f4c36a9ce65ca5a13634a P e64761f43e4240319c41117cf886d22f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.734158  7164 tablet_replica.cc:333] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.734881  7164 raft_consensus.cc:2243] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.736151  7164 raft_consensus.cc:2272] T 82abcf8a2a574841972243a029be749d P e64761f43e4240319c41117cf886d22f [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:27.736445  8974 consensus_peers.cc:597] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e -> Peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Couldn't send request to peer e64761f43e4240319c41117cf886d22f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:27.739154  7164 tablet_replica.cc:333] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.740074  7164 raft_consensus.cc:2243] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.740641  7164 raft_consensus.cc:2272] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.743474  7164 tablet_replica.cc:333] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.744256  7164 raft_consensus.cc:2243] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.745324  7164 raft_consensus.cc:2272] T 954d2ad09dcb4697adccd661625a841b P e64761f43e4240319c41117cf886d22f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.747962  7164 tablet_replica.cc:333] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.748741  7164 raft_consensus.cc:2243] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.750098  7164 raft_consensus.cc:2272] T 20b7fa75b43141c0ba1e5e08972a3a8f P e64761f43e4240319c41117cf886d22f [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.752777  7164 tablet_replica.cc:333] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.753584  7164 raft_consensus.cc:2243] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.754160  7164 raft_consensus.cc:2272] T 007e904f46ef4a1a978f97d557d2a65b P e64761f43e4240319c41117cf886d22f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.756855  7164 tablet_replica.cc:333] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.757596  7164 raft_consensus.cc:2243] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.758352  7164 raft_consensus.cc:2272] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.761195  7164 tablet_replica.cc:333] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.762059  7164 raft_consensus.cc:2243] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.763234  7164 raft_consensus.cc:2272] T fade5d45b9804cc98377ac6ab69be94b P e64761f43e4240319c41117cf886d22f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.767009  7164 tablet_replica.cc:333] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.767729  7164 raft_consensus.cc:2243] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.768743  7164 raft_consensus.cc:2272] T 6c6a6afa04ab437cbbd6011a11681763 P e64761f43e4240319c41117cf886d22f [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.771381  7164 tablet_replica.cc:333] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.772104  7164 raft_consensus.cc:2243] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.772827  7164 raft_consensus.cc:2272] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:27.775066  8974 consensus_peers.cc:597] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e -> Peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Couldn't send request to peer e64761f43e4240319c41117cf886d22f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:27.775534  7164 tablet_replica.cc:333] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f: stopping tablet replica
I20260709 13:05:27.776279  7164 raft_consensus.cc:2243] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 29 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.777257  7164 raft_consensus.cc:2272] T bf584dd56b3149a0b92539bc7115815b P e64761f43e4240319c41117cf886d22f [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:27.790673  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.817986  8972 consensus_peers.cc:597] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:27.824872  8371 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.881975  8369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.889660  8369 leader_election.cc:336] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Service unavailable: Unable to start connection negotiation thread: The pool has been shut down.
I20260709 13:05:27.890183  8369 leader_election.cc:304] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:05:27.890904  8369 raft_consensus.cc:2686] T 9d36c3e19b8c47bc9d26b071486fdec9 P e64761f43e4240319c41117cf886d22f: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:05:27.893652  8369 leader_election.cc:336] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Service unavailable: Unable to start connection negotiation thread: The pool has been shut down.
I20260709 13:05:27.894135  8369 leader_election.cc:304] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:05:27.894840  8369 raft_consensus.cc:2686] T f66b0119c0d84d8cb95b2252fcb6d390 P e64761f43e4240319c41117cf886d22f: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:05:27.898293  8369 leader_election.cc:336] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Service unavailable: Unable to start connection negotiation thread: The pool has been shut down.
I20260709 13:05:27.898763  8369 leader_election.cc:304] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:05:27.899471  8369 raft_consensus.cc:2686] T 3c1d459f4b98433682d7d19a825eb4ef P e64761f43e4240319c41117cf886d22f: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:05:27.906652  8369 leader_election.cc:336] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Service unavailable: Unable to start connection negotiation thread: The pool has been shut down.
I20260709 13:05:27.907106  8369 leader_election.cc:304] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:05:27.907701  8369 raft_consensus.cc:2686] T 6efcdb47a4a24d3680780fb6deb13911 P e64761f43e4240319c41117cf886d22f: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:05:27.911553  7164 leader_election.cc:336] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Aborted: reactor is shutting down (error 108)
I20260709 13:05:27.911994  7164 leader_election.cc:304] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:05:27.912612  7164 raft_consensus.cc:2686] T 757eb9e36091440581fbfc9a6b000e74 P e64761f43e4240319c41117cf886d22f: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:05:27.915746  7164 leader_election.cc:336] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Aborted: reactor is shutting down (error 108)
I20260709 13:05:27.916117  7164 leader_election.cc:304] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:05:27.916652  7164 raft_consensus.cc:2686] T 338bba401dcd4d8c9642cd7b255dea63 P e64761f43e4240319c41117cf886d22f: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:05:27.919111  8972 consensus_peers.cc:597] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:27.919231  7164 leader_election.cc:336] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Aborted: reactor is shutting down (error 108)
I20260709 13:05:27.919778  7164 leader_election.cc:304] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e64761f43e4240319c41117cf886d22f; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
W20260709 13:05:27.920327  7164 raft_consensus.cc:2686] T 5c7b40c72696450b884fa3a4659ccd84 P e64761f43e4240319c41117cf886d22f: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:05:28.115799  8144 maintenance_manager.cc:419] P b3bd0e8d7a084e649e387e729c0adabc: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033397
W20260709 13:05:28.199112  8972 consensus_peers.cc:597] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:28.200388  8972 consensus_peers.cc:597] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:28.202064  8143 maintenance_manager.cc:643] P b3bd0e8d7a084e649e387e729c0adabc: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.065s	user 0.039s	sys 0.006s Metrics: {"bytes_written":6924,"delete_count":0,"lbm_write_time_us":998,"lbm_writes_lt_1ms":6,"reinsert_count":0,"thread_start_us":8316,"threads_started":1,"update_count":283,"wal-append.queue_time_us":8148}
I20260709 13:05:28.204089  8144 maintenance_manager.cc:419] P b3bd0e8d7a084e649e387e729c0adabc: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.350794
W20260709 13:05:28.223428  8972 consensus_peers.cc:597] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:28.223924  8972 consensus_peers.cc:597] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:28.317477  8972 consensus_peers.cc:597] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:28.361451  8143 maintenance_manager.cc:643] P b3bd0e8d7a084e649e387e729c0adabc: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.156s	user 0.097s	sys 0.016s Metrics: {"cfile_cache_miss":9,"cfile_cache_miss_bytes":28208,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":20809,"dirs.run_cpu_time_us":572,"dirs.run_wall_time_us":1101,"lbm_read_time_us":1823,"lbm_reads_lt_1ms":25,"lbm_write_time_us":1645,"lbm_writes_lt_1ms":12,"peak_mem_usage":306358,"reinsert_count":0,"thread_start_us":55547,"threads_started":2,"update_count":303}
I20260709 13:05:28.365568  8144 maintenance_manager.cc:419] P b3bd0e8d7a084e649e387e729c0adabc: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260709 13:05:28.366413  8143 maintenance_manager.cc:643] P b3bd0e8d7a084e649e387e729c0adabc: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {}
W20260709 13:05:28.419461  8972 consensus_peers.cc:597] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:05:28.437476 16599 raft_consensus.cc:493] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:28.438835 16599 raft_consensus.cc:515] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:28.442209 16599 leader_election.cc:290] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:28.451081  8972 leader_election.cc:336] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:28.451932  8972 leader_election.cc:336] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:28.452318  8972 leader_election.cc:304] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:28.453114 16599 raft_consensus.cc:2749] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:28.460696  8972 consensus_peers.cc:597] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:28.525883 16599 raft_consensus.cc:493] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:05:28.526697 16599 raft_consensus.cc:515] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:28.528950 16599 leader_election.cc:290] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:05:28.533844  8974 leader_election.cc:336] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Network error: Client connection negotiation failed: client connection to 127.6.255.3:37367: connect: Connection refused (error 111)
W20260709 13:05:28.535435  8972 leader_election.cc:336] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:28.535924  8972 leader_election.cc:304] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:28.536621 16599 raft_consensus.cc:2749] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:28.553176  8972 consensus_peers.cc:597] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:28.611677 16601 raft_consensus.cc:493] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:05:28.612687 16601 raft_consensus.cc:515] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:28.615427 16601 leader_election.cc:290] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:05:28.621114  8972 leader_election.cc:336] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:28.622160 16601 raft_consensus.cc:493] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
W20260709 13:05:28.623072  8974 leader_election.cc:336] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Network error: Client connection negotiation failed: client connection to 127.6.255.3:37367: connect: Connection refused (error 111)
I20260709 13:05:28.623004 16601 raft_consensus.cc:515] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:28.623634  8974 leader_election.cc:304] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:28.624598 16599 raft_consensus.cc:2749] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:28.626205 16601 leader_election.cc:290] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:05:28.630839  8972 leader_election.cc:336] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:28.631345  8974 leader_election.cc:336] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Network error: Client connection negotiation failed: client connection to 127.6.255.3:37367: connect: Connection refused (error 111)
I20260709 13:05:28.631780  8974 leader_election.cc:304] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:05:28.632539 16601 raft_consensus.cc:2749] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:28.672472  8972 consensus_peers.cc:597] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:28.754109  8972 consensus_peers.cc:597] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e -> Peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Couldn't send request to peer ff7a6fe533e94c1bae555b4cae427728. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:28.767969  8972 consensus_peers.cc:597] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:05:28.793769 16601 raft_consensus.cc:493] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:05:28.794589 16601 raft_consensus.cc:515] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:28.799315 16601 leader_election.cc:290] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:05:28.801769  8974 leader_election.cc:336] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Network error: Client connection negotiation failed: client connection to 127.6.255.3:37367: connect: Connection refused (error 111)
W20260709 13:05:28.802290  8972 leader_election.cc:336] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:28.802734  8972 leader_election.cc:304] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:05:28.803385 16601 raft_consensus.cc:2749] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:28.825407 16601 raft_consensus.cc:493] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:05:28.826066 16601 raft_consensus.cc:515] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:28.828279 16601 leader_election.cc:290] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:05:28.834268  8974 leader_election.cc:336] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Network error: Client connection negotiation failed: client connection to 127.6.255.3:37367: connect: Connection refused (error 111)
W20260709 13:05:28.836366  8972 leader_election.cc:336] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:28.836768  8972 leader_election.cc:304] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:05:28.837390 16601 raft_consensus.cc:2749] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:28.856822 16599 raft_consensus.cc:493] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:05:28.857641 16599 raft_consensus.cc:515] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:28.860064 16599 leader_election.cc:290] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:05:28.868628  8972 leader_election.cc:336] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
W20260709 13:05:28.875705  8974 leader_election.cc:336] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Network error: Client connection negotiation failed: client connection to 127.6.255.3:37367: connect: Connection refused (error 111)
I20260709 13:05:28.876138  8974 leader_election.cc:304] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:28.876794 16599 raft_consensus.cc:2749] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:28.878445 16599 raft_consensus.cc:493] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:05:28.879220 16599 raft_consensus.cc:515] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:28.881743 16599 leader_election.cc:290] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:05:28.887559  8972 leader_election.cc:336] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
W20260709 13:05:28.888437  8974 leader_election.cc:336] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Network error: Client connection negotiation failed: client connection to 127.6.255.3:37367: connect: Connection refused (error 111)
I20260709 13:05:28.888907  8974 leader_election.cc:304] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:28.889779 16599 raft_consensus.cc:2749] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:28.984936 16599 raft_consensus.cc:493] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:05:28.985668 16599 raft_consensus.cc:515] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:28.987275 16599 leader_election.cc:290] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:05:28.992172  8974 leader_election.cc:336] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Network error: Client connection negotiation failed: client connection to 127.6.255.3:37367: connect: Connection refused (error 111)
W20260709 13:05:28.992617  8972 leader_election.cc:336] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:28.993062  8972 leader_election.cc:304] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:05:28.993659 16599 raft_consensus.cc:2749] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:28.998603 16599 raft_consensus.cc:493] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:05:28.999176 16599 raft_consensus.cc:515] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:29.001125 16599 leader_election.cc:290] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:05:29.005415  8972 leader_election.cc:336] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
W20260709 13:05:29.005975  8974 leader_election.cc:336] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Network error: Client connection negotiation failed: client connection to 127.6.255.3:37367: connect: Connection refused (error 111)
I20260709 13:05:29.006412  8974 leader_election.cc:304] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:29.007215 16601 raft_consensus.cc:2749] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:29.043401 16599 raft_consensus.cc:493] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:05:29.044066 16599 raft_consensus.cc:515] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:29.046203 16599 leader_election.cc:290] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:05:29.050719  8972 leader_election.cc:336] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
W20260709 13:05:29.051206  8974 leader_election.cc:336] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Network error: Client connection negotiation failed: client connection to 127.6.255.3:37367: connect: Connection refused (error 111)
I20260709 13:05:29.051601  8974 leader_election.cc:304] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:29.052177 16599 raft_consensus.cc:2749] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:29.076124  8186 debug-util.cc:398] Leaking SignalData structure 0x7b08000a75a0 after lost signal to thread 7167
W20260709 13:05:29.077945  8186 debug-util.cc:398] Leaking SignalData structure 0x7b08003d1180 after lost signal to thread 8189
W20260709 13:05:29.079953  8186 debug-util.cc:398] Leaking SignalData structure 0x7b08003e9040 after lost signal to thread 9034
I20260709 13:05:29.083981 16599 raft_consensus.cc:493] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:05:29.084885 16599 raft_consensus.cc:515] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:29.087548 16599 leader_election.cc:290] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
I20260709 13:05:29.089275 16599 raft_consensus.cc:493] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e64761f43e4240319c41117cf886d22f)
I20260709 13:05:29.090097 16599 raft_consensus.cc:515] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "e64761f43e4240319c41117cf886d22f" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 37367 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:29.093367 16599 leader_election.cc:290] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), e64761f43e4240319c41117cf886d22f (127.6.255.3:37367)
W20260709 13:05:29.097688  8974 leader_election.cc:336] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Network error: Client connection negotiation failed: client connection to 127.6.255.3:37367: connect: Connection refused (error 111)
W20260709 13:05:29.099727  8972 leader_election.cc:336] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
W20260709 13:05:29.101974  8974 leader_election.cc:336] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Network error: Client connection negotiation failed: client connection to 127.6.255.3:37367: connect: Connection refused (error 111)
I20260709 13:05:29.100159  8972 leader_election.cc:304] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: e64761f43e4240319c41117cf886d22f, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:29.103276 16599 raft_consensus.cc:2749] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:29.106232  8972 leader_election.cc:336] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:29.106647  8972 leader_election.cc:304] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, e64761f43e4240319c41117cf886d22f
I20260709 13:05:29.107378 16599 raft_consensus.cc:2749] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:05:29.175066 16599 raft_consensus.cc:493] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:29.175921 16599 raft_consensus.cc:515] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:29.178268 16599 leader_election.cc:290] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:29.190320  8972 leader_election.cc:336] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
W20260709 13:05:29.190999  8972 leader_election.cc:336] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:29.191361  8972 leader_election.cc:304] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:29.192085 16599 raft_consensus.cc:2749] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:29.250270  8972 consensus_peers.cc:597] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:29.386759  8972 consensus_peers.cc:597] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:29.483404  8972 consensus_peers.cc:597] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:05:29.606213 16601 raft_consensus.cc:493] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:29.607167 16601 raft_consensus.cc:515] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
I20260709 13:05:29.609668 16601 leader_election.cc:290] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:29.622611  8972 leader_election.cc:336] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
W20260709 13:05:29.624269  8972 leader_election.cc:336] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:29.624691  8972 leader_election.cc:304] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:29.625415 16601 raft_consensus.cc:2749] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:05:29.678782  7164 tablet_server.cc:196] TabletServer@127.6.255.3:0 shutdown complete.
W20260709 13:05:29.715340  8974 consensus_peers.cc:597] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e -> Peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Couldn't send request to peer e64761f43e4240319c41117cf886d22f. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.3:37367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:29.797123  8972 consensus_peers.cc:597] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:29.839605  8972 consensus_peers.cc:597] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:05:29.851472  7164 tablet_server.cc:179] TabletServer@127.6.255.4:0 shutting down...
W20260709 13:05:29.852173  8972 consensus_peers.cc:597] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:29.875208  8974 consensus_peers.cc:597] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e -> Peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Couldn't send request to peer e64761f43e4240319c41117cf886d22f. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.3:37367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:29.907179 16601 raft_consensus.cc:493] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ff7a6fe533e94c1bae555b4cae427728)
I20260709 13:05:29.908010 16601 raft_consensus.cc:515] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c42259d7c4b463a9f485f42c320c81b" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44679 } } peers { permanent_uuid: "ff7a6fe533e94c1bae555b4cae427728" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 38599 } } peers { permanent_uuid: "f865c4d761f5446eb12b3a029234772e" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 33825 } attrs { promote: false } }
W20260709 13:05:29.910346  8974 consensus_peers.cc:597] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e -> Peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Couldn't send request to peer e64761f43e4240319c41117cf886d22f. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.3:37367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:29.910462 16601 leader_election.cc:290] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679), ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599)
W20260709 13:05:29.922840  8972 leader_election.cc:336] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
W20260709 13:05:29.924273  8972 leader_election.cc:336] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ff7a6fe533e94c1bae555b4cae427728 (127.6.255.2:38599): Network error: Client connection negotiation failed: client connection to 127.6.255.2:38599: connect: Connection refused (error 111)
I20260709 13:05:29.924633  8972 leader_election.cc:304] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f865c4d761f5446eb12b3a029234772e; no voters: 8c42259d7c4b463a9f485f42c320c81b, ff7a6fe533e94c1bae555b4cae427728
I20260709 13:05:29.925549 16601 raft_consensus.cc:2749] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:29.940516  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:29.941773  7164 tablet_replica.cc:333] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:29.943140  7164 raft_consensus.cc:2243] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 29 LEADER]: Raft consensus shutting down.
I20260709 13:05:29.945731  7164 raft_consensus.cc:2272] T bac7f1e5192d48a98a850cb0ece17ad5 P f865c4d761f5446eb12b3a029234772e [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:29.950078  7164 tablet_replica.cc:333] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:29.951180  7164 raft_consensus.cc:2243] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:29.952672  7164 raft_consensus.cc:2272] T 09943d7e76a54d4e80eb181cac80bf22 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:29.956789  7164 tablet_replica.cc:333] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:29.957981  7164 raft_consensus.cc:2243] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:05:29.959740  7164 raft_consensus.cc:2272] T e94bcf9755aa4f0abddcd5b9fbc2939f P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:29.963433  7164 tablet_replica.cc:333] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:29.964392  7164 raft_consensus.cc:2243] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:05:29.966073  7164 raft_consensus.cc:2272] T 007e904f46ef4a1a978f97d557d2a65b P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:29.966955  8974 consensus_peers.cc:597] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e -> Peer e64761f43e4240319c41117cf886d22f (127.6.255.3:37367): Couldn't send request to peer e64761f43e4240319c41117cf886d22f. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.3:37367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:29.970229  7164 tablet_replica.cc:333] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:29.971290  7164 raft_consensus.cc:2243] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:29.972086  7164 raft_consensus.cc:2272] T 767eba28679649519167576023555b60 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:29.974016  8972 consensus_peers.cc:597] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e -> Peer 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): Couldn't send request to peer 8c42259d7c4b463a9f485f42c320c81b. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:05:29.975572  7164 tablet_replica.cc:333] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:29.976708  7164 raft_consensus.cc:2243] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:29.977869  7164 raft_consensus.cc:2272] T 60c80a6b33dc431eac0fe35b37d4ea23 P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:29.982008  7164 tablet_replica.cc:333] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:29.982951  7164 raft_consensus.cc:2243] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:29.983883  7164 raft_consensus.cc:2272] T b7c15354a2a84f868942919f011af448 P f865c4d761f5446eb12b3a029234772e [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:29.987871  7164 tablet_replica.cc:333] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:29.988956  7164 raft_consensus.cc:2243] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:29.989804  7164 raft_consensus.cc:2272] T ae82ced6ff014453bbcfd5f9f14b1f8d P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:29.993921  7164 tablet_replica.cc:333] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:29.994977  7164 raft_consensus.cc:2243] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:05:29.996538  7164 raft_consensus.cc:2272] T 1263b3742fd541f8bda5a6b6c10a8596 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:29.999548  7164 tablet_replica.cc:333] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:30.000417  7164 raft_consensus.cc:2243] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:30.001199  7164 raft_consensus.cc:2272] T 2323b22f124b407c95caf5525949dd6c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:30.004911  7164 tablet_replica.cc:333] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:30.006031  7164 raft_consensus.cc:2243] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:05:30.007434  7164 raft_consensus.cc:2272] T 430fa172910841a49c0c80f2cbf76ae5 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:30.010596  7164 tablet_replica.cc:333] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:30.011456  7164 raft_consensus.cc:2243] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:30.012300  7164 raft_consensus.cc:2272] T 10cffa7c6411440ca77cee6731faca67 P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:30.015599  7164 tablet_replica.cc:333] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:30.016381  7164 raft_consensus.cc:2243] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:30.017148  7164 raft_consensus.cc:2272] T 1112bcf4c4ea44e1933ee6287bc869f8 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:30.020314  7164 tablet_replica.cc:333] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:30.021091  7164 raft_consensus.cc:2243] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:30.022819  7164 raft_consensus.cc:2272] T eeecf7303d634c0d8983b810bb97c33c P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:30.025936  7164 tablet_replica.cc:333] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:30.028267  7164 raft_consensus.cc:2243] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:05:30.029726  7164 raft_consensus.cc:2272] T 39384c6d69ad4f658d2e777dfdcbb49c P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:30.032874  7164 tablet_replica.cc:333] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:30.033684  7164 raft_consensus.cc:2243] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:05:30.035122  7164 raft_consensus.cc:2272] T cb79894cecb541e3b7dd164271cadc16 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:30.038179  7164 tablet_replica.cc:333] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:30.038952  7164 raft_consensus.cc:2243] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:30.039640  7164 raft_consensus.cc:2272] T 954d2ad09dcb4697adccd661625a841b P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:30.042919  7164 tablet_replica.cc:333] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:30.043696  7164 raft_consensus.cc:2243] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:30.044404  7164 raft_consensus.cc:2272] T a0493b0dbbc746408a5db9e698678d83 P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:30.047444  7164 tablet_replica.cc:333] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:30.048161  7164 raft_consensus.cc:2243] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:05:30.050011  7164 raft_consensus.cc:2272] T 4b9b125a55cf4a5d9db580313045edde P f865c4d761f5446eb12b3a029234772e [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:30.052913  7164 tablet_replica.cc:333] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:30.053681  7164 raft_consensus.cc:2243] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:30.054522  7164 raft_consensus.cc:2272] T 56b5f14332d64504837ae4e2df77e409 P f865c4d761f5446eb12b3a029234772e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:30.057636  7164 tablet_replica.cc:333] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:30.058344  7164 raft_consensus.cc:2243] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:05:30.059907  7164 raft_consensus.cc:2272] T 038cd0877a674c2996410c9f327df911 P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:30.062873  7164 tablet_replica.cc:333] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:30.063567  7164 raft_consensus.cc:2243] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:05:30.064960  7164 raft_consensus.cc:2272] T b3d99c80602e45da9c43a49cf83b8211 P f865c4d761f5446eb12b3a029234772e [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:30.068077  7164 tablet_replica.cc:333] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:30.068815  7164 raft_consensus.cc:2243] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:30.069537  7164 raft_consensus.cc:2272] T b6e0ee9efb7d40309bc7e4d2450bb1b5 P f865c4d761f5446eb12b3a029234772e [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:30.072463  7164 tablet_replica.cc:333] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:30.073153  7164 raft_consensus.cc:2243] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:05:30.074575  7164 raft_consensus.cc:2272] T d32240ced15f4c36a9ce65ca5a13634a P f865c4d761f5446eb12b3a029234772e [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:30.077632  7164 tablet_replica.cc:333] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:30.078357  7164 raft_consensus.cc:2243] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:05:30.079761  7164 raft_consensus.cc:2272] T fdf12882ac9e4f7d9121a5fcb9d4efaf P f865c4d761f5446eb12b3a029234772e [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:30.082793  7164 tablet_replica.cc:333] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:30.083529  7164 raft_consensus.cc:2243] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:30.084290  7164 raft_consensus.cc:2272] T 07268be9b33649e8b0e8bddf7a3bcabe P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:30.087167  7164 tablet_replica.cc:333] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:30.087911  7164 raft_consensus.cc:2243] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:30.088584  7164 raft_consensus.cc:2272] T f19e818631f543b28a8bec9ba4547527 P f865c4d761f5446eb12b3a029234772e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:30.091495  7164 tablet_replica.cc:333] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:30.092185  7164 raft_consensus.cc:2243] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:30.093113  7164 raft_consensus.cc:2272] T bf42cb9d00c6491eb6b0cf8b896f7c5b P f865c4d761f5446eb12b3a029234772e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:30.095767  7164 tablet_replica.cc:333] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:30.096451  7164 raft_consensus.cc:2243] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:30.097077  7164 raft_consensus.cc:2272] T 4516247b88e34a1d89ec9eb29e0ecb7f P f865c4d761f5446eb12b3a029234772e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:30.099908  7164 tablet_replica.cc:333] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:30.100574  7164 raft_consensus.cc:2243] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:05:30.101974  7164 raft_consensus.cc:2272] T 83839ff3ce80447bb72b3c8d15b630ff P f865c4d761f5446eb12b3a029234772e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:30.104890  7164 tablet_replica.cc:333] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:30.105551  7164 raft_consensus.cc:2243] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:05:30.107057  7164 raft_consensus.cc:2272] T 81804f59e3124d27998ea80fc1c36663 P f865c4d761f5446eb12b3a029234772e [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:30.110186  7164 tablet_replica.cc:333] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e: stopping tablet replica
I20260709 13:05:30.110803  7164 raft_consensus.cc:2243] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:05:30.112249  7164 raft_consensus.cc:2272] T 766ac43473084aa6a9fdce530520947c P f865c4d761f5446eb12b3a029234772e [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:30.452502  7164 tablet_server.cc:196] TabletServer@127.6.255.4:0 shutdown complete.
I20260709 13:05:30.532111  7164 master.cc:562] Master@127.6.255.62:40337 shutting down...
W20260709 13:05:30.574988  8203 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 13:05:30.575253  8203 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:05:30.575440  8203 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:05:33.349823  8139 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111) [suppressed 208 similar messages]
W20260709 13:05:33.353622  8139 catalog_manager.cc:4755] TS 8c42259d7c4b463a9f485f42c320c81b (127.6.255.1:44679): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 81804f59e3124d27998ea80fc1c36663: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44679: connect: Connection refused (error 111)
I20260709 13:05:33.458878  7164 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:33.459684  7164 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:33.460047  7164 tablet_replica.cc:333] T 00000000000000000000000000000000 P b3bd0e8d7a084e649e387e729c0adabc: stopping tablet replica
I20260709 13:05:33.677654  7164 master.cc:584] Master@127.6.255.62:40337 shutdown complete.
I20260709 13:05:33.721982  7164 test_util.cc:182] -----------------------------------------------
I20260709 13:05:33.722230  7164 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602046064212-7164-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (264728 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:05:33.727207  7164 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.255.62:33261
I20260709 13:05:33.728420  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:33.734440 16610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:33.734787 16611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:33.735311 16613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:33.737325  7164 server_base.cc:1061] running on GCE node
I20260709 13:05:33.740978  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:33.741168  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:33.741353  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602333741320 us; error 0 us; skew 500 ppm
I20260709 13:05:33.741930  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:33.744436  7164 webserver.cc:533] Webserver started at http://127.6.255.62:37447/ using document root <none> and password file <none>
I20260709 13:05:33.744930  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:33.745112  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:33.745409  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:33.746747  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/master-0-root/instance:
uuid: "b17e9282f0de497b861afbf2b8adc5d1"
format_stamp: "Formatted at 2026-07-09 13:05:33 on dist-test-slave-rnv4"
I20260709 13:05:33.751121  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:05:33.754575 16618 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:33.755995  7164 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:05:33.756240  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/master-0-root
uuid: "b17e9282f0de497b861afbf2b8adc5d1"
format_stamp: "Formatted at 2026-07-09 13:05:33 on dist-test-slave-rnv4"
I20260709 13:05:33.756505  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:33.768638  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:33.769937  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:33.808470  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.62:33261
I20260709 13:05:33.808593 16669 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.62:33261 every 8 connection(s)
I20260709 13:05:33.812714 16670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.823313 16670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17e9282f0de497b861afbf2b8adc5d1: Bootstrap starting.
I20260709 13:05:33.828126 16670 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b17e9282f0de497b861afbf2b8adc5d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:33.832352 16670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17e9282f0de497b861afbf2b8adc5d1: No bootstrap required, opened a new log
I20260709 13:05:33.834529 16670 raft_consensus.cc:359] T 00000000000000000000000000000000 P b17e9282f0de497b861afbf2b8adc5d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17e9282f0de497b861afbf2b8adc5d1" member_type: VOTER }
I20260709 13:05:33.834898 16670 raft_consensus.cc:385] T 00000000000000000000000000000000 P b17e9282f0de497b861afbf2b8adc5d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:33.835089 16670 raft_consensus.cc:740] T 00000000000000000000000000000000 P b17e9282f0de497b861afbf2b8adc5d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17e9282f0de497b861afbf2b8adc5d1, State: Initialized, Role: FOLLOWER
I20260709 13:05:33.835587 16670 consensus_queue.cc:260] T 00000000000000000000000000000000 P b17e9282f0de497b861afbf2b8adc5d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17e9282f0de497b861afbf2b8adc5d1" member_type: VOTER }
I20260709 13:05:33.836004 16670 raft_consensus.cc:399] T 00000000000000000000000000000000 P b17e9282f0de497b861afbf2b8adc5d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:05:33.836192 16670 raft_consensus.cc:493] T 00000000000000000000000000000000 P b17e9282f0de497b861afbf2b8adc5d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:05:33.836395 16670 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b17e9282f0de497b861afbf2b8adc5d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:33.840767 16670 raft_consensus.cc:515] T 00000000000000000000000000000000 P b17e9282f0de497b861afbf2b8adc5d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17e9282f0de497b861afbf2b8adc5d1" member_type: VOTER }
I20260709 13:05:33.841281 16670 leader_election.cc:304] T 00000000000000000000000000000000 P b17e9282f0de497b861afbf2b8adc5d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b17e9282f0de497b861afbf2b8adc5d1; no voters: 
I20260709 13:05:33.842417 16670 leader_election.cc:290] T 00000000000000000000000000000000 P b17e9282f0de497b861afbf2b8adc5d1 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:05:33.842873 16673 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b17e9282f0de497b861afbf2b8adc5d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:33.844257 16673 raft_consensus.cc:697] T 00000000000000000000000000000000 P b17e9282f0de497b861afbf2b8adc5d1 [term 1 LEADER]: Becoming Leader. State: Replica: b17e9282f0de497b861afbf2b8adc5d1, State: Running, Role: LEADER
I20260709 13:05:33.844995 16673 consensus_queue.cc:237] T 00000000000000000000000000000000 P b17e9282f0de497b861afbf2b8adc5d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17e9282f0de497b861afbf2b8adc5d1" member_type: VOTER }
I20260709 13:05:33.845580 16670 sys_catalog.cc:565] T 00000000000000000000000000000000 P b17e9282f0de497b861afbf2b8adc5d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:05:33.850587 16675 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17e9282f0de497b861afbf2b8adc5d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b17e9282f0de497b861afbf2b8adc5d1. Latest consensus state: current_term: 1 leader_uuid: "b17e9282f0de497b861afbf2b8adc5d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17e9282f0de497b861afbf2b8adc5d1" member_type: VOTER } }
I20260709 13:05:33.851159 16674 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17e9282f0de497b861afbf2b8adc5d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b17e9282f0de497b861afbf2b8adc5d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17e9282f0de497b861afbf2b8adc5d1" member_type: VOTER } }
I20260709 13:05:33.851496 16675 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17e9282f0de497b861afbf2b8adc5d1 [sys.catalog]: This master's current role is: LEADER
I20260709 13:05:33.851836 16674 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17e9282f0de497b861afbf2b8adc5d1 [sys.catalog]: This master's current role is: LEADER
I20260709 13:05:33.853641 16681 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:05:33.858763 16681 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:05:33.861343  7164 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:05:33.868836 16681 catalog_manager.cc:1383] Generated new cluster ID: 2ad83b2cd6dc492586f92d69eb5de240
I20260709 13:05:33.869191 16681 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:05:33.890467 16681 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:05:33.892201 16681 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:05:33.920797 16681 catalog_manager.cc:6070] T 00000000000000000000000000000000 P b17e9282f0de497b861afbf2b8adc5d1: Generated new TSK 0
I20260709 13:05:33.921532 16681 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:05:33.932240  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:33.938453 16691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:33.939538 16692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:33.942632 16694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:33.943902  7164 server_base.cc:1061] running on GCE node
I20260709 13:05:33.945973  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:33.946187  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:33.946362  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602333946325 us; error 0 us; skew 500 ppm
I20260709 13:05:33.947006  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:33.949589  7164 webserver.cc:533] Webserver started at http://127.6.255.1:46089/ using document root <none> and password file <none>
I20260709 13:05:33.950098  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:33.950304  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:33.950593  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:33.951747  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/ts-0-root/instance:
uuid: "e2b87453fe654318abccdd1512b949c4"
format_stamp: "Formatted at 2026-07-09 13:05:33 on dist-test-slave-rnv4"
I20260709 13:05:33.956401  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:05:33.959996 16699 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:33.961048  7164 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.002s
I20260709 13:05:33.961711  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/ts-0-root
uuid: "e2b87453fe654318abccdd1512b949c4"
format_stamp: "Formatted at 2026-07-09 13:05:33 on dist-test-slave-rnv4"
I20260709 13:05:33.961982  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:33.974464  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:33.975625  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:33.977003  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:33.979326  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:33.979532  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:33.979830  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:33.979983  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:34.023502  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.1:40785
I20260709 13:05:34.023626 16761 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.1:40785 every 8 connection(s)
I20260709 13:05:34.029235  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:34.036516 16766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:34.038820 16767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:34.041844 16769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:34.042129  7164 server_base.cc:1061] running on GCE node
I20260709 13:05:34.045184  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:34.045512  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:34.045764  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602334045746 us; error 0 us; skew 500 ppm
I20260709 13:05:34.046468  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:34.047324 16762 heartbeater.cc:344] Connected to a master server at 127.6.255.62:33261
I20260709 13:05:34.047811 16762 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:34.048730 16762 heartbeater.cc:507] Master 127.6.255.62:33261 requested a full tablet report, sending...
I20260709 13:05:34.049584  7164 webserver.cc:533] Webserver started at http://127.6.255.2:36749/ using document root <none> and password file <none>
I20260709 13:05:34.050220  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:34.050413  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:34.050717  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:34.051268 16635 ts_manager.cc:194] Registered new tserver with Master: e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:34.052000  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/ts-1-root/instance:
uuid: "c4b86aae3a8f4a9082e639144ec46228"
format_stamp: "Formatted at 2026-07-09 13:05:34 on dist-test-slave-rnv4"
I20260709 13:05:34.053135 16635 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58724
I20260709 13:05:34.057053  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 13:05:34.060895 16774 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:34.061825  7164 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:05:34.062228  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/ts-1-root
uuid: "c4b86aae3a8f4a9082e639144ec46228"
format_stamp: "Formatted at 2026-07-09 13:05:34 on dist-test-slave-rnv4"
I20260709 13:05:34.062538  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:34.077975  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:34.079249  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:34.080752  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:34.083123  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:34.083365  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:34.083608  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:34.083781  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:34.126420  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.2:45135
I20260709 13:05:34.126540 16836 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.2:45135 every 8 connection(s)
I20260709 13:05:34.131407  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:34.139266 16840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:34.141266 16841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:34.142930  7164 server_base.cc:1061] running on GCE node
W20260709 13:05:34.143146 16843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:34.144347  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:34.144634  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:34.144562 16837 heartbeater.cc:344] Connected to a master server at 127.6.255.62:33261
I20260709 13:05:34.145071  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602334145046 us; error 0 us; skew 500 ppm
I20260709 13:05:34.145287 16837 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:34.145864  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:34.146222 16837 heartbeater.cc:507] Master 127.6.255.62:33261 requested a full tablet report, sending...
I20260709 13:05:34.148671  7164 webserver.cc:533] Webserver started at http://127.6.255.3:46697/ using document root <none> and password file <none>
I20260709 13:05:34.148671 16635 ts_manager.cc:194] Registered new tserver with Master: c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:34.149631  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:34.149905  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:34.150230  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:34.150862 16635 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58740
I20260709 13:05:34.151818  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/ts-2-root/instance:
uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
format_stamp: "Formatted at 2026-07-09 13:05:34 on dist-test-slave-rnv4"
I20260709 13:05:34.156921  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 13:05:34.160562 16848 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:34.161589  7164 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:05:34.161846  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/ts-2-root
uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
format_stamp: "Formatted at 2026-07-09 13:05:34 on dist-test-slave-rnv4"
I20260709 13:05:34.162086  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:34.179563  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:34.180724  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:34.182284  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:34.184514  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:34.184695  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:34.184979  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:34.185127  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:34.226944  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.3:45101
I20260709 13:05:34.227115 16910 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.3:45101 every 8 connection(s)
I20260709 13:05:34.232036  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:34.240633 16914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:34.240888 16915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:34.244446 16917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:34.244885 16911 heartbeater.cc:344] Connected to a master server at 127.6.255.62:33261
I20260709 13:05:34.245378 16911 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:34.245970  7164 server_base.cc:1061] running on GCE node
I20260709 13:05:34.246480 16911 heartbeater.cc:507] Master 127.6.255.62:33261 requested a full tablet report, sending...
I20260709 13:05:34.247077  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:34.247406  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:34.247644  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602334247624 us; error 0 us; skew 500 ppm
I20260709 13:05:34.248332  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:34.248777 16635 ts_manager.cc:194] Registered new tserver with Master: d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:34.250349 16635 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58750
I20260709 13:05:34.250906  7164 webserver.cc:533] Webserver started at http://127.6.255.4:39793/ using document root <none> and password file <none>
I20260709 13:05:34.251557  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:34.251777  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:34.252043  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:34.253208  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/ts-3-root/instance:
uuid: "0086163b152644feb75caeac6f145533"
format_stamp: "Formatted at 2026-07-09 13:05:34 on dist-test-slave-rnv4"
I20260709 13:05:34.257601  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:05:34.260771 16922 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:34.261619  7164 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:05:34.261914  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/ts-3-root
uuid: "0086163b152644feb75caeac6f145533"
format_stamp: "Formatted at 2026-07-09 13:05:34 on dist-test-slave-rnv4"
I20260709 13:05:34.262214  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:34.298184  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:34.299499  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:34.301059  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:34.303355  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:34.303596  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:34.303866  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:34.304061  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:34.346664  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.4:37199
I20260709 13:05:34.346810 16984 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.4:37199 every 8 connection(s)
I20260709 13:05:34.361239 16985 heartbeater.cc:344] Connected to a master server at 127.6.255.62:33261
I20260709 13:05:34.361814 16985 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:34.362546 16985 heartbeater.cc:507] Master 127.6.255.62:33261 requested a full tablet report, sending...
I20260709 13:05:34.364552 16635 ts_manager.cc:194] Registered new tserver with Master: 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:34.365175  7164 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.015189931s
I20260709 13:05:34.366079 16635 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58766
I20260709 13:05:34.390901 16635 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58778:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:05:34.652211 16950 tablet_service.cc:1511] Processing CreateTablet for tablet 27be53bca112445d8bdc640329310d67 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:05:34.652755 16949 tablet_service.cc:1511] Processing CreateTablet for tablet 5b5161020c944d28a4334dc1d572dac7 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:05:34.653762 16950 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27be53bca112445d8bdc640329310d67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.654460 16949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b5161020c944d28a4334dc1d572dac7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.663184 16948 tablet_service.cc:1511] Processing CreateTablet for tablet 535cac5e21bf41db9c037f226ebcd94e (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:05:34.663746 16947 tablet_service.cc:1511] Processing CreateTablet for tablet 42e96e3138004b3aa2f3e745f1ddfc04 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:05:34.664664 16948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 535cac5e21bf41db9c037f226ebcd94e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.665635 16947 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42e96e3138004b3aa2f3e745f1ddfc04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.680109 16946 tablet_service.cc:1511] Processing CreateTablet for tablet f3cecfcf8ae242e1b70f628f293da197 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:05:34.682518 16946 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3cecfcf8ae242e1b70f628f293da197. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.682323 16874 tablet_service.cc:1511] Processing CreateTablet for tablet 535cac5e21bf41db9c037f226ebcd94e (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:05:34.683140 17004 tablet_bootstrap.cc:492] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:34.688201 16873 tablet_service.cc:1511] Processing CreateTablet for tablet c5b9b28c4c5d4ddaa6d1ae05ce225b11 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:05:34.690433 16872 tablet_service.cc:1511] Processing CreateTablet for tablet f3cecfcf8ae242e1b70f628f293da197 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:05:34.692806 16871 tablet_service.cc:1511] Processing CreateTablet for tablet 7f37dd0138014a75bc36f33022362b3c (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:05:34.690518 16870 tablet_service.cc:1511] Processing CreateTablet for tablet 4f95af7ac3134423b1a5956c8ae950df (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:05:34.694721 16875 tablet_service.cc:1511] Processing CreateTablet for tablet 0b8add4b17084a6189509807b7e663e5 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:05:34.683734 16874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 535cac5e21bf41db9c037f226ebcd94e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.695112 16869 tablet_service.cc:1511] Processing CreateTablet for tablet ac3ad4fe0bc4435cb1c596ad215889a3 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:05:34.697253 16868 tablet_service.cc:1511] Processing CreateTablet for tablet 58ff2290a9c9451c9a786eacf7380d90 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:05:34.697638 16867 tablet_service.cc:1511] Processing CreateTablet for tablet b9fdc34d643b4ab995042f719af511aa (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:05:34.698189 16948 tablet_service.cc:1511] Processing CreateTablet for tablet ac3ad4fe0bc4435cb1c596ad215889a3 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:05:34.698356 16943 tablet_service.cc:1511] Processing CreateTablet for tablet d82d38de8d2c47b38ca8811691c68d8f (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:05:34.698930 16947 tablet_service.cc:1511] Processing CreateTablet for tablet 6288d52217814c7393ff0ec66da3b099 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:05:34.703613 16948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac3ad4fe0bc4435cb1c596ad215889a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.704313 16943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d82d38de8d2c47b38ca8811691c68d8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.699970 16944 tablet_service.cc:1511] Processing CreateTablet for tablet 58ff2290a9c9451c9a786eacf7380d90 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:05:34.709776 16947 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6288d52217814c7393ff0ec66da3b099. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.685263 16945 tablet_service.cc:1511] Processing CreateTablet for tablet 7f37dd0138014a75bc36f33022362b3c (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:05:34.710865 16945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f37dd0138014a75bc36f33022362b3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.686435 16876 tablet_service.cc:1511] Processing CreateTablet for tablet 27be53bca112445d8bdc640329310d67 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:05:34.714458 16944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58ff2290a9c9451c9a786eacf7380d90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.702829 16875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b8add4b17084a6189509807b7e663e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.718904 16870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f95af7ac3134423b1a5956c8ae950df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.719916 16868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58ff2290a9c9451c9a786eacf7380d90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.720836 16871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f37dd0138014a75bc36f33022362b3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.722991 16867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9fdc34d643b4ab995042f719af511aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.724395 16876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27be53bca112445d8bdc640329310d67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.725196 16872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3cecfcf8ae242e1b70f628f293da197. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.726514 16869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac3ad4fe0bc4435cb1c596ad215889a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.729801 16873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5b9b28c4c5d4ddaa6d1ae05ce225b11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.695211 17004 tablet_bootstrap.cc:654] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:34.766839 16727 tablet_service.cc:1511] Processing CreateTablet for tablet 27be53bca112445d8bdc640329310d67 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:05:34.778461 16727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27be53bca112445d8bdc640329310d67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.778578 16722 tablet_service.cc:1511] Processing CreateTablet for tablet f3cecfcf8ae242e1b70f628f293da197 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:05:34.779904 16722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3cecfcf8ae242e1b70f628f293da197. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.782542 16721 tablet_service.cc:1511] Processing CreateTablet for tablet 4f95af7ac3134423b1a5956c8ae950df (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:05:34.783962 16721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f95af7ac3134423b1a5956c8ae950df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.786473 16950 tablet_service.cc:1511] Processing CreateTablet for tablet 9592a31e7ffd407a959c0a3d96c0f8f9 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:05:34.787813 16950 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9592a31e7ffd407a959c0a3d96c0f8f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.790745 16942 tablet_service.cc:1511] Processing CreateTablet for tablet 612e472fbe674788986798c65836388f (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:05:34.792239 16942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 612e472fbe674788986798c65836388f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.794766 16720 tablet_service.cc:1511] Processing CreateTablet for tablet 6288d52217814c7393ff0ec66da3b099 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:05:34.796149 16720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6288d52217814c7393ff0ec66da3b099. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.771348 16726 tablet_service.cc:1511] Processing CreateTablet for tablet 0b8add4b17084a6189509807b7e663e5 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:05:34.798622 16726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b8add4b17084a6189509807b7e663e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.773160 16725 tablet_service.cc:1511] Processing CreateTablet for tablet 5b5161020c944d28a4334dc1d572dac7 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:05:34.799467 16719 tablet_service.cc:1511] Processing CreateTablet for tablet ac3ad4fe0bc4435cb1c596ad215889a3 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:05:34.774992 16724 tablet_service.cc:1511] Processing CreateTablet for tablet c5b9b28c4c5d4ddaa6d1ae05ce225b11 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:05:34.776779 16723 tablet_service.cc:1511] Processing CreateTablet for tablet 42e96e3138004b3aa2f3e745f1ddfc04 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:05:34.800112 16870 tablet_service.cc:1511] Processing CreateTablet for tablet c9043e0bc02745d187e9ca5a5c71274f (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:05:34.800256 16725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b5161020c944d28a4334dc1d572dac7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.804185 16870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9043e0bc02745d187e9ca5a5c71274f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.804587 16941 tablet_service.cc:1511] Processing CreateTablet for tablet e1660f76b7074e2cb7542ed02dd179d9 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:05:34.808378 16941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1660f76b7074e2cb7542ed02dd179d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.805811 16723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42e96e3138004b3aa2f3e745f1ddfc04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.810534 16724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5b9b28c4c5d4ddaa6d1ae05ce225b11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.812001 16718 tablet_service.cc:1511] Processing CreateTablet for tablet b9fdc34d643b4ab995042f719af511aa (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:05:34.813532 16718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9fdc34d643b4ab995042f719af511aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.814270 16719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac3ad4fe0bc4435cb1c596ad215889a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.806720 16946 tablet_service.cc:1511] Processing CreateTablet for tablet c9043e0bc02745d187e9ca5a5c71274f (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:05:34.816033 16946 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9043e0bc02745d187e9ca5a5c71274f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.828498 16874 tablet_service.cc:1511] Processing CreateTablet for tablet 3850091ca95e47b6a1b52e600a452b0b (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:05:34.831154 16949 tablet_service.cc:1511] Processing CreateTablet for tablet e091d4e9a0044dd899e731769a94064c (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:05:34.835561 16949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e091d4e9a0044dd899e731769a94064c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.847038 16874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3850091ca95e47b6a1b52e600a452b0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.853937 16871 tablet_service.cc:1511] Processing CreateTablet for tablet 3d6e27a9a96c4121b2abc71e0d7738be (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:05:34.854391 16948 tablet_service.cc:1511] Processing CreateTablet for tablet 3850091ca95e47b6a1b52e600a452b0b (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:05:34.858930 16948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3850091ca95e47b6a1b52e600a452b0b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:34.856876 16683 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:34.856195 16873 tablet_service.cc:1511] Processing CreateTablet for tablet 612e472fbe674788986798c65836388f (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:05:34.863620 16871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d6e27a9a96c4121b2abc71e0d7738be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.868450 16799 tablet_service.cc:1511] Processing CreateTablet for tablet c5b9b28c4c5d4ddaa6d1ae05ce225b11 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:05:34.869836 16799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5b9b28c4c5d4ddaa6d1ae05ce225b11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.869732 16798 tablet_service.cc:1511] Processing CreateTablet for tablet 42e96e3138004b3aa2f3e745f1ddfc04 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:05:34.870661 16795 tablet_service.cc:1511] Processing CreateTablet for tablet 6288d52217814c7393ff0ec66da3b099 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:05:34.871191 16798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42e96e3138004b3aa2f3e745f1ddfc04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.872063 16795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6288d52217814c7393ff0ec66da3b099. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.864208 16801 tablet_service.cc:1511] Processing CreateTablet for tablet 5b5161020c944d28a4334dc1d572dac7 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:05:34.873591 16801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b5161020c944d28a4334dc1d572dac7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.874661 16794 tablet_service.cc:1511] Processing CreateTablet for tablet 58ff2290a9c9451c9a786eacf7380d90 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:05:34.876103 16794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58ff2290a9c9451c9a786eacf7380d90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.865296 16796 tablet_service.cc:1511] Processing CreateTablet for tablet 4f95af7ac3134423b1a5956c8ae950df (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:05:34.878248 16796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f95af7ac3134423b1a5956c8ae950df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.878813 16793 tablet_service.cc:1511] Processing CreateTablet for tablet b9fdc34d643b4ab995042f719af511aa (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:05:34.864212 16797 tablet_service.cc:1511] Processing CreateTablet for tablet 7f37dd0138014a75bc36f33022362b3c (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:05:34.880179 16793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9fdc34d643b4ab995042f719af511aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.880926 16797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f37dd0138014a75bc36f33022362b3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.882284 16802 tablet_service.cc:1511] Processing CreateTablet for tablet 0b8add4b17084a6189509807b7e663e5 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:05:34.883682 16802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b8add4b17084a6189509807b7e663e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.866791 16800 tablet_service.cc:1511] Processing CreateTablet for tablet 535cac5e21bf41db9c037f226ebcd94e (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:05:34.886215 16800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 535cac5e21bf41db9c037f226ebcd94e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.895614 16947 tablet_service.cc:1511] Processing CreateTablet for tablet 9e8e0f9919a9404c9001b6718f102d19 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:05:34.915508 16949 tablet_service.cc:1511] Processing CreateTablet for tablet 213e9771c53e4498b315bb43fdc0aa9a (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:05:34.917575 16949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 213e9771c53e4498b315bb43fdc0aa9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.917714 16945 tablet_service.cc:1511] Processing CreateTablet for tablet 10bbce697a734600bb083d47ac877574 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:05:34.919044 16945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10bbce697a734600bb083d47ac877574. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.923544 16867 tablet_service.cc:1511] Processing CreateTablet for tablet 5857fad96a1045d7b85190e64da77bce (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:05:34.910955 16873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 612e472fbe674788986798c65836388f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.963857 16872 tablet_service.cc:1511] Processing CreateTablet for tablet 15b6c54287fa4fdb928257d6faf572f8 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:05:34.965842 16867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5857fad96a1045d7b85190e64da77bce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.967499 16869 tablet_service.cc:1511] Processing CreateTablet for tablet e091d4e9a0044dd899e731769a94064c (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:05:34.969880 16949 tablet_service.cc:1511] Processing CreateTablet for tablet 919c5788cd97463a9cf06abb18843943 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:05:34.970816 16872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15b6c54287fa4fdb928257d6faf572f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.975203 16869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e091d4e9a0044dd899e731769a94064c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.975658 17006 tablet_bootstrap.cc:492] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:34.981592 16943 tablet_service.cc:1511] Processing CreateTablet for tablet 5857fad96a1045d7b85190e64da77bce (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:05:34.982542 16722 tablet_service.cc:1511] Processing CreateTablet for tablet 213e9771c53e4498b315bb43fdc0aa9a (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:05:34.983824 16722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 213e9771c53e4498b315bb43fdc0aa9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.989589 16947 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e8e0f9919a9404c9001b6718f102d19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.994457 16721 tablet_service.cc:1511] Processing CreateTablet for tablet d82d38de8d2c47b38ca8811691c68d8f (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:05:34.995802 16721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d82d38de8d2c47b38ca8811691c68d8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.006563 16870 tablet_service.cc:1511] Processing CreateTablet for tablet 9e8e0f9919a9404c9001b6718f102d19 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:05:35.008025 16870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e8e0f9919a9404c9001b6718f102d19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.016403 16867 tablet_service.cc:1511] Processing CreateTablet for tablet 9dd082201fe04ef89c842b6404fd522e (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:05:35.017838 16867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dd082201fe04ef89c842b6404fd522e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.023263 16945 tablet_service.cc:1511] Processing CreateTablet for tablet 6c9ca489a4d2416092d6dcfae2498136 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:05:35.024606 16945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c9ca489a4d2416092d6dcfae2498136. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.027271 16869 tablet_service.cc:1511] Processing CreateTablet for tablet 10bbce697a734600bb083d47ac877574 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:05:35.028612 16869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10bbce697a734600bb083d47ac877574. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.032148 16727 tablet_service.cc:1511] Processing CreateTablet for tablet c9043e0bc02745d187e9ca5a5c71274f (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:05:35.035866 16942 tablet_service.cc:1511] Processing CreateTablet for tablet b4d83d4d3d894b81a85b945ef97d79de (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:05:35.037214 16942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4d83d4d3d894b81a85b945ef97d79de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.038906 16722 tablet_service.cc:1511] Processing CreateTablet for tablet 3d6e27a9a96c4121b2abc71e0d7738be (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:05:35.042454 17006 tablet_bootstrap.cc:654] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:35.055867 16762 heartbeater.cc:499] Master 127.6.255.62:33261 was elected leader, sending a full tablet report...
I20260709 13:05:35.094180 16727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9043e0bc02745d187e9ca5a5c71274f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.094875 16722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d6e27a9a96c4121b2abc71e0d7738be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.101912 16943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5857fad96a1045d7b85190e64da77bce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.062505 16944 tablet_service.cc:1511] Processing CreateTablet for tablet 93629337d6b8477a9cd5895a157bf918 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:05:35.063459 16721 tablet_service.cc:1511] Processing CreateTablet for tablet 9592a31e7ffd407a959c0a3d96c0f8f9 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:05:35.067406 16874 tablet_service.cc:1511] Processing CreateTablet for tablet b4d83d4d3d894b81a85b945ef97d79de (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:05:35.073544 16941 tablet_service.cc:1511] Processing CreateTablet for tablet 081841333d734eb1aa72ceb3f0ea2bf2 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:05:35.077943 16867 tablet_service.cc:1511] Processing CreateTablet for tablet 919c5788cd97463a9cf06abb18843943 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:05:35.082787 16869 tablet_service.cc:1511] Processing CreateTablet for tablet caa2367ac34d43b5b1a27410b2055005 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:05:35.083237 16868 tablet_service.cc:1511] Processing CreateTablet for tablet 93629337d6b8477a9cd5895a157bf918 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:05:35.108837 16867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 919c5788cd97463a9cf06abb18843943. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.110608 16721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9592a31e7ffd407a959c0a3d96c0f8f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.101305 16720 tablet_service.cc:1511] Processing CreateTablet for tablet 15b6c54287fa4fdb928257d6faf572f8 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:05:35.114416 16720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15b6c54287fa4fdb928257d6faf572f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.118480 16726 tablet_service.cc:1511] Processing CreateTablet for tablet e1660f76b7074e2cb7542ed02dd179d9 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:05:35.118947 17004 tablet_bootstrap.cc:492] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:35.119535 17004 ts_tablet_manager.cc:1403] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.438s	user 0.013s	sys 0.019s
I20260709 13:05:35.135813 17004 raft_consensus.cc:359] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:35.141400 16796 tablet_service.cc:1511] Processing CreateTablet for tablet d82d38de8d2c47b38ca8811691c68d8f (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:05:35.147148 16796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d82d38de8d2c47b38ca8811691c68d8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.148561 16799 tablet_service.cc:1511] Processing CreateTablet for tablet 3850091ca95e47b6a1b52e600a452b0b (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:05:35.142206 16724 tablet_service.cc:1511] Processing CreateTablet for tablet 612e472fbe674788986798c65836388f (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:05:35.150072 16799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3850091ca95e47b6a1b52e600a452b0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.145339 17004 raft_consensus.cc:385] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:35.150921 17004 raft_consensus.cc:740] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:35.151939 17004 consensus_queue.cc:260] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:35.092419 16871 tablet_service.cc:1511] Processing CreateTablet for tablet 3c19ee4785104276aec22ba772dbc6ac (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:05:35.157855 17007 tablet_bootstrap.cc:492] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:35.153879 16837 heartbeater.cc:499] Master 127.6.255.62:33261 was elected leader, sending a full tablet report...
I20260709 13:05:35.129693 16949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 919c5788cd97463a9cf06abb18843943. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.164969 16726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1660f76b7074e2cb7542ed02dd179d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.166484 16725 tablet_service.cc:1511] Processing CreateTablet for tablet 9dd082201fe04ef89c842b6404fd522e (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:05:35.167804 16725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dd082201fe04ef89c842b6404fd522e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.169984 16875 tablet_service.cc:1511] Processing CreateTablet for tablet 0e295eefc8ff46d9a31a4c0898d16c63 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:05:35.174939 16723 tablet_service.cc:1511] Processing CreateTablet for tablet 10bbce697a734600bb083d47ac877574 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:05:35.176765 16723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10bbce697a734600bb083d47ac877574. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.182510 16726 tablet_service.cc:1511] Processing CreateTablet for tablet caa2367ac34d43b5b1a27410b2055005 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:05:35.190923 16874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4d83d4d3d894b81a85b945ef97d79de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.209161 16944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93629337d6b8477a9cd5895a157bf918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.215569 17008 tablet_bootstrap.cc:492] T 6288d52217814c7393ff0ec66da3b099 P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:35.216619 16945 tablet_service.cc:1511] Processing CreateTablet for tablet e81325b638b1493999310a726811b64b (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:05:35.217978 16945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e81325b638b1493999310a726811b64b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.218765 16718 tablet_service.cc:1511] Processing CreateTablet for tablet 6c9ca489a4d2416092d6dcfae2498136 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:05:35.220523 16950 tablet_service.cc:1511] Processing CreateTablet for tablet 0e295eefc8ff46d9a31a4c0898d16c63 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:05:35.221909 16950 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e295eefc8ff46d9a31a4c0898d16c63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.223644 16941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 081841333d734eb1aa72ceb3f0ea2bf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.224264 16946 tablet_service.cc:1511] Processing CreateTablet for tablet f48e0f9f83da446a877cd06e19768929 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:05:35.226432 16873 tablet_service.cc:1511] Processing CreateTablet for tablet 9078054426a34acbb94be2b69213b6c1 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:05:35.233816 16946 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f48e0f9f83da446a877cd06e19768929. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.243095 16798 tablet_service.cc:1511] Processing CreateTablet for tablet 5857fad96a1045d7b85190e64da77bce (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:05:35.244455 16798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5857fad96a1045d7b85190e64da77bce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.235544 16948 tablet_service.cc:1511] Processing CreateTablet for tablet 7123efeaee9f4ec78a1c475eab16c4c1 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:05:35.248718 16876 tablet_service.cc:1511] Processing CreateTablet for tablet 7123efeaee9f4ec78a1c475eab16c4c1 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:05:35.250411 16948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7123efeaee9f4ec78a1c475eab16c4c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.252954 16911 heartbeater.cc:499] Master 127.6.255.62:33261 was elected leader, sending a full tablet report...
I20260709 13:05:35.262607 16872 tablet_service.cc:1511] Processing CreateTablet for tablet e81325b638b1493999310a726811b64b (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:05:35.240864 17007 tablet_bootstrap.cc:654] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:35.264467 16802 tablet_service.cc:1511] Processing CreateTablet for tablet 15b6c54287fa4fdb928257d6faf572f8 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:05:35.265967 16802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15b6c54287fa4fdb928257d6faf572f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.191201 16719 tablet_service.cc:1511] Processing CreateTablet for tablet e091d4e9a0044dd899e731769a94064c (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:05:35.236336 16871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c19ee4785104276aec22ba772dbc6ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.269747 16875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e295eefc8ff46d9a31a4c0898d16c63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.269624 16801 tablet_service.cc:1511] Processing CreateTablet for tablet e1660f76b7074e2cb7542ed02dd179d9 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:05:35.270948 16801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1660f76b7074e2cb7542ed02dd179d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.273752 16868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93629337d6b8477a9cd5895a157bf918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.268285 17008 tablet_bootstrap.cc:654] T 6288d52217814c7393ff0ec66da3b099 P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:35.272128 16794 tablet_service.cc:1511] Processing CreateTablet for tablet 9e8e0f9919a9404c9001b6718f102d19 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:05:35.274964 16869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caa2367ac34d43b5b1a27410b2055005. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.276134 16794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e8e0f9919a9404c9001b6718f102d19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.276813 16795 tablet_service.cc:1511] Processing CreateTablet for tablet 9592a31e7ffd407a959c0a3d96c0f8f9 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:05:35.278076 16873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9078054426a34acbb94be2b69213b6c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.279335 16797 tablet_service.cc:1511] Processing CreateTablet for tablet 3d6e27a9a96c4121b2abc71e0d7738be (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:05:35.282588 16872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e81325b638b1493999310a726811b64b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.285832 16726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caa2367ac34d43b5b1a27410b2055005. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.286588 16719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e091d4e9a0044dd899e731769a94064c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.287328 16876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7123efeaee9f4ec78a1c475eab16c4c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.288362 16724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 612e472fbe674788986798c65836388f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.288309 17004 ts_tablet_manager.cc:1434] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.168s	user 0.013s	sys 0.036s
I20260709 13:05:35.278112 16795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9592a31e7ffd407a959c0a3d96c0f8f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.302934 16985 heartbeater.cc:499] Master 127.6.255.62:33261 was elected leader, sending a full tablet report...
I20260709 13:05:35.303539 17004 tablet_bootstrap.cc:492] T 535cac5e21bf41db9c037f226ebcd94e P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:35.304092 16718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c9ca489a4d2416092d6dcfae2498136. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.307058 17009 raft_consensus.cc:493] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.308833 17009 raft_consensus.cc:515] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:35.308916 16870 tablet_service.cc:1511] Processing CreateTablet for tablet e5c700237eb44cefbf7bc46c1daf08cd (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:05:35.310881 16870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5c700237eb44cefbf7bc46c1daf08cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.308391 16797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d6e27a9a96c4121b2abc71e0d7738be. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:35.351517 16986 tablet.cc:2406] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:05:35.359553 17004 tablet_bootstrap.cc:654] T 535cac5e21bf41db9c037f226ebcd94e P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:35.408967 16793 tablet_service.cc:1511] Processing CreateTablet for tablet 9dd082201fe04ef89c842b6404fd522e (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:05:35.410794 16793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dd082201fe04ef89c842b6404fd522e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.412546 16800 tablet_service.cc:1511] Processing CreateTablet for tablet 213e9771c53e4498b315bb43fdc0aa9a (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:05:35.413829 16800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 213e9771c53e4498b315bb43fdc0aa9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.429976 17006 tablet_bootstrap.cc:492] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:35.430573 17006 ts_tablet_manager.cc:1403] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.473s	user 0.021s	sys 0.011s
I20260709 13:05:35.436486 17007 tablet_bootstrap.cc:492] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:35.437053 17007 ts_tablet_manager.cc:1403] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.279s	user 0.004s	sys 0.016s
I20260709 13:05:35.438012 17009 leader_election.cc:290] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:35.438479 16867 tablet_service.cc:1511] Processing CreateTablet for tablet 59de6d1228da41da938527632da08f4d (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:05:35.439810 16867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59de6d1228da41da938527632da08f4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.452518 17007 raft_consensus.cc:359] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:35.454619 16727 tablet_service.cc:1511] Processing CreateTablet for tablet b4d83d4d3d894b81a85b945ef97d79de (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:05:35.461324 16947 tablet_service.cc:1511] Processing CreateTablet for tablet e5c700237eb44cefbf7bc46c1daf08cd (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:05:35.462195 16727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4d83d4d3d894b81a85b945ef97d79de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.463660 16947 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5c700237eb44cefbf7bc46c1daf08cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.453357 17007 raft_consensus.cc:385] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:35.481333 16944 tablet_service.cc:1511] Processing CreateTablet for tablet 20cdedb225464d5e812a56ed7a7d220f (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:05:35.483583 16946 tablet_service.cc:1511] Processing CreateTablet for tablet f86d4f23f8cd4451a50b1b38155d958e (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:05:35.484921 16799 tablet_service.cc:1511] Processing CreateTablet for tablet 919c5788cd97463a9cf06abb18843943 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:05:35.485972 16943 tablet_service.cc:1511] Processing CreateTablet for tablet 6ec647c6ccd641b59808817d8dca2235 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:05:35.487720 16950 tablet_service.cc:1511] Processing CreateTablet for tablet be51f207c3bc4276b220a03e2d9d4db4 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:05:35.457842 17006 raft_consensus.cc:359] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:35.547326 16944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20cdedb225464d5e812a56ed7a7d220f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.497893 16725 tablet_service.cc:1511] Processing CreateTablet for tablet 081841333d734eb1aa72ceb3f0ea2bf2 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:05:35.505267 16720 tablet_service.cc:1511] Processing CreateTablet for tablet 0e295eefc8ff46d9a31a4c0898d16c63 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:05:35.516681 16801 tablet_service.cc:1511] Processing CreateTablet for tablet caa2367ac34d43b5b1a27410b2055005 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:05:35.494351 17007 raft_consensus.cc:740] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:35.565279 16725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 081841333d734eb1aa72ceb3f0ea2bf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.565757 17007 consensus_queue.cc:260] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:35.490594 16722 tablet_service.cc:1511] Processing CreateTablet for tablet 3c19ee4785104276aec22ba772dbc6ac (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:05:35.559943 16948 tablet_service.cc:1511] Processing CreateTablet for tablet 61f747edd7934aac93044ba370e34d17 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:05:35.547428 17006 raft_consensus.cc:385] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:35.572743 17006 raft_consensus.cc:740] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:35.574153 17006 consensus_queue.cc:260] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:35.564846 16801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caa2367ac34d43b5b1a27410b2055005. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.581954 16875 tablet_service.cc:1511] Processing CreateTablet for tablet 20cdedb225464d5e812a56ed7a7d220f (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:05:35.583734 16875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20cdedb225464d5e812a56ed7a7d220f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.572707 16720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e295eefc8ff46d9a31a4c0898d16c63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.586592 16946 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f86d4f23f8cd4451a50b1b38155d958e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.588402 16950 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be51f207c3bc4276b220a03e2d9d4db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.590672 16943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ec647c6ccd641b59808817d8dca2235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.593034 16948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61f747edd7934aac93044ba370e34d17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.598353 16722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c19ee4785104276aec22ba772dbc6ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.617273 16799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 919c5788cd97463a9cf06abb18843943. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.624275 16723 tablet_service.cc:1511] Processing CreateTablet for tablet 9078054426a34acbb94be2b69213b6c1 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:05:35.625645 16723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9078054426a34acbb94be2b69213b6c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.633458 17006 ts_tablet_manager.cc:1434] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.203s	user 0.009s	sys 0.004s
I20260709 13:05:35.671872 16874 tablet_service.cc:1511] Processing CreateTablet for tablet f86d4f23f8cd4451a50b1b38155d958e (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:05:35.673199 16874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f86d4f23f8cd4451a50b1b38155d958e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.722888 16795 tablet_service.cc:1511] Processing CreateTablet for tablet 9078054426a34acbb94be2b69213b6c1 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:05:35.679847 16942 tablet_service.cc:1511] Processing CreateTablet for tablet 02d877efcf5049a89eacf23b58ba27b9 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:05:35.676370 16794 tablet_service.cc:1511] Processing CreateTablet for tablet 6c9ca489a4d2416092d6dcfae2498136 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:05:35.680509 16872 tablet_service.cc:1511] Processing CreateTablet for tablet 0a7231efacd1463c96b35c3d03edce55 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:05:35.726241 16942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02d877efcf5049a89eacf23b58ba27b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.726850 16872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a7231efacd1463c96b35c3d03edce55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.726688 16802 tablet_service.cc:1511] Processing CreateTablet for tablet f48e0f9f83da446a877cd06e19768929 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:05:35.728421 16793 tablet_service.cc:1511] Processing CreateTablet for tablet e5c700237eb44cefbf7bc46c1daf08cd (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:05:35.684423 16949 tablet_service.cc:1511] Processing CreateTablet for tablet 86f5e278f54c419fac8c74ac63f80f74 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:05:35.686537 16798 tablet_service.cc:1511] Processing CreateTablet for tablet 93629337d6b8477a9cd5895a157bf918 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:05:35.686915 16796 tablet_service.cc:1511] Processing CreateTablet for tablet 3c19ee4785104276aec22ba772dbc6ac (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:05:35.688846 16797 tablet_service.cc:1511] Processing CreateTablet for tablet 081841333d734eb1aa72ceb3f0ea2bf2 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:05:35.692500 16870 tablet_service.cc:1511] Processing CreateTablet for tablet 61f747edd7934aac93044ba370e34d17 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:05:35.730991 16949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86f5e278f54c419fac8c74ac63f80f74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.731729 16870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61f747edd7934aac93044ba370e34d17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.693864 16719 tablet_service.cc:1511] Processing CreateTablet for tablet 7123efeaee9f4ec78a1c475eab16c4c1 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:05:35.736495 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27be53bca112445d8bdc640329310d67" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:35.739195 16719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7123efeaee9f4ec78a1c475eab16c4c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.739598 16721 tablet_service.cc:1511] Processing CreateTablet for tablet 59de6d1228da41da938527632da08f4d (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:05:35.740890 16721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59de6d1228da41da938527632da08f4d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:35.742439 16923 leader_election.cc:343] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:35.742504 16720 tablet_service.cc:1511] Processing CreateTablet for tablet 6ec647c6ccd641b59808817d8dca2235 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:05:35.743891 16720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ec647c6ccd641b59808817d8dca2235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.746006 16718 tablet_service.cc:1511] Processing CreateTablet for tablet 0a7231efacd1463c96b35c3d03edce55 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:05:35.747329 16718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a7231efacd1463c96b35c3d03edce55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.747009 16727 tablet_service.cc:1511] Processing CreateTablet for tablet be51f207c3bc4276b220a03e2d9d4db4 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:05:35.748544 16727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be51f207c3bc4276b220a03e2d9d4db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.743981 16725 tablet_service.cc:1511] Processing CreateTablet for tablet 20cdedb225464d5e812a56ed7a7d220f (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:05:35.759361 16725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20cdedb225464d5e812a56ed7a7d220f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.725349 16867 tablet_service.cc:1511] Processing CreateTablet for tablet 02d877efcf5049a89eacf23b58ba27b9 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:05:35.696599 17004 tablet_bootstrap.cc:492] T 535cac5e21bf41db9c037f226ebcd94e P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:35.766624 17004 ts_tablet_manager.cc:1403] T 535cac5e21bf41db9c037f226ebcd94e P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.463s	user 0.006s	sys 0.017s
I20260709 13:05:35.709362 16723 tablet_service.cc:1511] Processing CreateTablet for tablet f48e0f9f83da446a877cd06e19768929 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:05:35.770385 16723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f48e0f9f83da446a877cd06e19768929. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.695600 16945 tablet_service.cc:1511] Processing CreateTablet for tablet 37a5bbadcd9b48dba8fe272a18679ede (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:05:35.773696 16945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37a5bbadcd9b48dba8fe272a18679ede. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.715698 16800 tablet_service.cc:1511] Processing CreateTablet for tablet e81325b638b1493999310a726811b64b (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:05:35.778810 17004 raft_consensus.cc:359] T 535cac5e21bf41db9c037f226ebcd94e P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:35.780735 17004 raft_consensus.cc:385] T 535cac5e21bf41db9c037f226ebcd94e P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:35.781167 17004 raft_consensus.cc:740] T 535cac5e21bf41db9c037f226ebcd94e P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:35.782096 17004 consensus_queue.cc:260] T 535cac5e21bf41db9c037f226ebcd94e P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:35.784821 17004 ts_tablet_manager.cc:1434] T 535cac5e21bf41db9c037f226ebcd94e P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:05:35.786029 17004 tablet_bootstrap.cc:492] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:35.790611 16869 tablet_service.cc:1511] Processing CreateTablet for tablet 2593417c1fe4457c87cb2a01cf1f65a3 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:05:35.792075 16869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2593417c1fe4457c87cb2a01cf1f65a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.793740 17006 tablet_bootstrap.cc:492] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:35.719897 16946 tablet_service.cc:1511] Processing CreateTablet for tablet 2593417c1fe4457c87cb2a01cf1f65a3 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:05:35.780342 17008 tablet_bootstrap.cc:492] T 6288d52217814c7393ff0ec66da3b099 P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:35.794543 17008 ts_tablet_manager.cc:1403] T 6288d52217814c7393ff0ec66da3b099 P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.579s	user 0.025s	sys 0.004s
I20260709 13:05:35.798247 16946 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2593417c1fe4457c87cb2a01cf1f65a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.801929 17008 raft_consensus.cc:359] T 6288d52217814c7393ff0ec66da3b099 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:35.802872 17008 raft_consensus.cc:385] T 6288d52217814c7393ff0ec66da3b099 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:35.803330 17008 raft_consensus.cc:740] T 6288d52217814c7393ff0ec66da3b099 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:35.804260 17008 consensus_queue.cc:260] T 6288d52217814c7393ff0ec66da3b099 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:35.764021 17007 ts_tablet_manager.cc:1434] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.327s	user 0.010s	sys 0.006s
I20260709 13:05:35.760416 16724 tablet_service.cc:1511] Processing CreateTablet for tablet 61f747edd7934aac93044ba370e34d17 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:05:35.806751 17007 tablet_bootstrap.cc:492] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:35.808753 16867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02d877efcf5049a89eacf23b58ba27b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.811609 16795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9078054426a34acbb94be2b69213b6c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.812394 16796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c19ee4785104276aec22ba772dbc6ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.812156 16871 tablet_service.cc:1511] Processing CreateTablet for tablet f239a1de317547b598aeedc11994c75f (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:05:35.813266 16724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61f747edd7934aac93044ba370e34d17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.817809 16800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e81325b638b1493999310a726811b64b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.821792 16797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 081841333d734eb1aa72ceb3f0ea2bf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.825872 16793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5c700237eb44cefbf7bc46c1daf08cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.825774 16875 tablet_service.cc:1511] Processing CreateTablet for tablet 136f22096d4b41d89354924f7101d1c1 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:05:35.744998 16873 tablet_service.cc:1511] Processing CreateTablet for tablet 83ec8ffbf1044525880e6b0d7931f47e (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:05:35.829787 16802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f48e0f9f83da446a877cd06e19768929. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.784581 16876 tablet_service.cc:1511] Processing CreateTablet for tablet 37a5bbadcd9b48dba8fe272a18679ede (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:05:35.833786 16798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93629337d6b8477a9cd5895a157bf918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.836346 16870 tablet_service.cc:1511] Processing CreateTablet for tablet f9d47e8e7b784085b7079b1542dbc579 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:05:35.837759 16794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c9ca489a4d2416092d6dcfae2498136. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.838430 17008 ts_tablet_manager.cc:1434] T 6288d52217814c7393ff0ec66da3b099 P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.044s	user 0.003s	sys 0.005s
I20260709 13:05:35.838663 16872 tablet_service.cc:1511] Processing CreateTablet for tablet 7f4354282d844c7e83d760e4418512dc (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:05:35.839675 17008 tablet_bootstrap.cc:492] T 4f95af7ac3134423b1a5956c8ae950df P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:35.844339 16719 tablet_service.cc:1511] Processing CreateTablet for tablet 83ec8ffbf1044525880e6b0d7931f47e (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:05:35.850718 16719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83ec8ffbf1044525880e6b0d7931f47e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.849205 16718 tablet_service.cc:1511] Processing CreateTablet for tablet 86f5e278f54c419fac8c74ac63f80f74 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:05:35.855189 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27be53bca112445d8bdc640329310d67" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
W20260709 13:05:35.863205 16923 leader_election.cc:343] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:35.863710 16923 leader_election.cc:304] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:35.864727 17011 raft_consensus.cc:2749] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:35.865862 16718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86f5e278f54c419fac8c74ac63f80f74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.868552 17020 raft_consensus.cc:493] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.869333 17020 raft_consensus.cc:515] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:35.852234 16725 tablet_service.cc:1511] Processing CreateTablet for tablet 37a5bbadcd9b48dba8fe272a18679ede (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:05:35.841235 17006 tablet_bootstrap.cc:654] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:35.872334 16725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37a5bbadcd9b48dba8fe272a18679ede. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.873203 17020 leader_election.cc:290] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:35.875316 16726 tablet_service.cc:1511] Processing CreateTablet for tablet f239a1de317547b598aeedc11994c75f (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:05:35.925407 16800 tablet_service.cc:1511] Processing CreateTablet for tablet 59de6d1228da41da938527632da08f4d (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:05:35.894527 17008 tablet_bootstrap.cc:654] T 4f95af7ac3134423b1a5956c8ae950df P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:35.895821 16801 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43106 (request call id 21) took 1001 ms. Trace:
I20260709 13:05:35.899149 16868 tablet_service.cc:1511] Processing CreateTablet for tablet 744cc6469e3749e2aacccb45464e8198 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:05:35.902493 17004 tablet_bootstrap.cc:654] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:35.895426 16949 tablet_service.cc:1511] Processing CreateTablet for tablet f9d47e8e7b784085b7079b1542dbc579 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:05:35.905594 16871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f239a1de317547b598aeedc11994c75f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.907842 16720 tablet_service.cc:1511] Processing CreateTablet for tablet 136f22096d4b41d89354924f7101d1c1 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:05:35.895390 16941 tablet_service.cc:1511] Processing CreateTablet for tablet 136f22096d4b41d89354924f7101d1c1 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:05:35.964772 17011 raft_consensus.cc:493] T 535cac5e21bf41db9c037f226ebcd94e P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.915829 16719 tablet_service.cc:1511] Processing CreateTablet for tablet 7f4354282d844c7e83d760e4418512dc (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
W20260709 13:05:35.917734 16619 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.6.255.2:45135, user_credentials={real_user=slave}} blocked reactor thread for 64233.7us
I20260709 13:05:35.919231 16945 tablet_service.cc:1511] Processing CreateTablet for tablet 87ca116ec6174decb610df3f8f565a6b (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:05:35.890806 16723 tablet_service.cc:1511] Processing CreateTablet for tablet 87ca116ec6174decb610df3f8f565a6b (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:05:35.919243 16727 tablet_service.cc:1511] Processing CreateTablet for tablet c968ba000a4542fc9204dd9a474e364c (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:05:35.923934 17007 tablet_bootstrap.cc:654] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:35.939530 16683 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:35.950913 16726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f239a1de317547b598aeedc11994c75f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.941740 16801 rpcz_store.cc:276] 0709 13:05:34.883608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.515699 (+632091us) service_pool.cc:224] Handling call
0709 13:05:35.564240 (+ 48541us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:05:35.564341 (+   101us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:05:35.780714 (+216373us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:35.884894 (+104180us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:35.885390 (+   496us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:35.939530 16794 tablet_service.cc:1511] Processing CreateTablet for tablet 6ec647c6ccd641b59808817d8dca2235 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:05:35.983402 16875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 136f22096d4b41d89354924f7101d1c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.993321 16801 tablet_service.cc:1511] Processing CreateTablet for tablet f86d4f23f8cd4451a50b1b38155d958e (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:05:35.997996 16870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9d47e8e7b784085b7079b1542dbc579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.995548 16800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59de6d1228da41da938527632da08f4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.965351 17011 raft_consensus.cc:515] T 535cac5e21bf41db9c037f226ebcd94e P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:36.009119 17018 raft_consensus.cc:493] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.023103 17018 raft_consensus.cc:515] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:36.025002 17011 leader_election.cc:290] T 535cac5e21bf41db9c037f226ebcd94e P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:36.008841 16872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f4354282d844c7e83d760e4418512dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.016666 16941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 136f22096d4b41d89354924f7101d1c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.008018 16720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 136f22096d4b41d89354924f7101d1c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.037897 16876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37a5bbadcd9b48dba8fe272a18679ede. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.028373 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "535cac5e21bf41db9c037f226ebcd94e" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:36.008131 17021 raft_consensus.cc:493] T 6288d52217814c7393ff0ec66da3b099 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.058909 16719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f4354282d844c7e83d760e4418512dc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:36.064042 16923 leader_election.cc:343] T 535cac5e21bf41db9c037f226ebcd94e P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.059604 16722 tablet_service.cc:1511] Processing CreateTablet for tablet 744cc6469e3749e2aacccb45464e8198 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:05:36.057076 17021 raft_consensus.cc:515] T 6288d52217814c7393ff0ec66da3b099 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:36.066427 17004 tablet_bootstrap.cc:492] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:36.067029 17004 ts_tablet_manager.cc:1403] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.281s	user 0.022s	sys 0.004s
I20260709 13:05:36.067387 17021 leader_election.cc:290] T 6288d52217814c7393ff0ec66da3b099 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:36.055183 16949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9d47e8e7b784085b7079b1542dbc579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.071744 17004 raft_consensus.cc:359] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:36.072448 17004 raft_consensus.cc:385] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.072734 17004 raft_consensus.cc:740] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.073575 17004 consensus_queue.cc:260] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:36.075245 17018 leader_election.cc:290] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:36.058652 16801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f86d4f23f8cd4451a50b1b38155d958e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.096724 16873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83ec8ffbf1044525880e6b0d7931f47e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.101215 17008 tablet_bootstrap.cc:492] T 4f95af7ac3134423b1a5956c8ae950df P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:36.102098 17008 ts_tablet_manager.cc:1403] T 4f95af7ac3134423b1a5956c8ae950df P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.263s	user 0.025s	sys 0.023s
I20260709 13:05:36.091886 16945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87ca116ec6174decb610df3f8f565a6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.096724 16727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c968ba000a4542fc9204dd9a474e364c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.120671 16944 tablet_service.cc:1511] Processing CreateTablet for tablet c968ba000a4542fc9204dd9a474e364c (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:05:36.122761 16794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ec647c6ccd641b59808817d8dca2235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.123075 16868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 744cc6469e3749e2aacccb45464e8198. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.112258 16947 tablet_service.cc:1511] Processing CreateTablet for tablet 744cc6469e3749e2aacccb45464e8198 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:05:36.147222 16722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 744cc6469e3749e2aacccb45464e8198. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.149786 16944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c968ba000a4542fc9204dd9a474e364c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.158520 17004 ts_tablet_manager.cc:1434] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.091s	user 0.008s	sys 0.001s
I20260709 13:05:36.165652 16723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87ca116ec6174decb610df3f8f565a6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.169749 17008 raft_consensus.cc:359] T 4f95af7ac3134423b1a5956c8ae950df P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:36.158896 16800 tablet_service.cc:1511] Processing CreateTablet for tablet be51f207c3bc4276b220a03e2d9d4db4 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:05:36.171594 16800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be51f207c3bc4276b220a03e2d9d4db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.177614 16799 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43106 (request call id 20) took 1294 ms. Trace:
I20260709 13:05:36.178144 16799 rpcz_store.cc:276] 0709 13:05:34.883238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.483982 (+600744us) service_pool.cc:224] Handling call
0709 13:05:35.552242 (+ 68260us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:05:35.567589 (+ 15347us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:05:35.796095 (+228506us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:36.116902 (+320807us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:36.177541 (+ 60639us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13465}
I20260709 13:05:36.185617 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9fdc34d643b4ab995042f719af511aa" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:36.187041 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3cecfcf8ae242e1b70f628f293da197" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:36.188575 17004 tablet_bootstrap.cc:492] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:36.193085 16796 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43106 (request call id 24) took 1143 ms. Trace:
I20260709 13:05:36.194046 16795 tablet_service.cc:1511] Processing CreateTablet for tablet 83ec8ffbf1044525880e6b0d7931f47e (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:05:36.195354 16795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83ec8ffbf1044525880e6b0d7931f47e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.193657 16796 rpcz_store.cc:276] 0709 13:05:35.049844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.686065 (+636221us) service_pool.cc:224] Handling call
0709 13:05:35.730600 (+ 44535us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:05:35.730694 (+    94us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:05:35.885282 (+154588us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:36.147953 (+262671us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:36.192989 (+ 45036us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27666}
I20260709 13:05:36.199812 16796 tablet_service.cc:1511] Processing CreateTablet for tablet 86f5e278f54c419fac8c74ac63f80f74 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:05:36.200129 16793 tablet_service.cc:1511] Processing CreateTablet for tablet 0a7231efacd1463c96b35c3d03edce55 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
W20260709 13:05:36.201630 16849 leader_election.cc:343] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:36.188620 16700 leader_election.cc:343] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.202734 16797 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43106 (request call id 25) took 1096 ms. Trace:
I20260709 13:05:36.203410 16797 rpcz_store.cc:276] 0709 13:05:35.106442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.687974 (+581532us) service_pool.cc:224] Handling call
0709 13:05:35.731159 (+ 43185us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:05:35.731246 (+    87us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:05:35.881310 (+150064us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:36.150138 (+268828us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:36.202662 (+ 52524us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8099}
I20260709 13:05:36.181015 16799 tablet_service.cc:1511] Processing CreateTablet for tablet 02d877efcf5049a89eacf23b58ba27b9 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:05:36.206351 16797 tablet_service.cc:1511] Processing CreateTablet for tablet 2593417c1fe4457c87cb2a01cf1f65a3 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:05:36.202556 16796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86f5e278f54c419fac8c74ac63f80f74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.212407 16797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2593417c1fe4457c87cb2a01cf1f65a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.214192 16793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a7231efacd1463c96b35c3d03edce55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.170807 17008 raft_consensus.cc:385] T 4f95af7ac3134423b1a5956c8ae950df P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.218267 17008 raft_consensus.cc:740] T 4f95af7ac3134423b1a5956c8ae950df P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.219198 17008 consensus_queue.cc:260] T 4f95af7ac3134423b1a5956c8ae950df P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:36.228317 16798 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43106 (request call id 23) took 1253 ms. Trace:
I20260709 13:05:36.228698 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3cecfcf8ae242e1b70f628f293da197" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
W20260709 13:05:36.230712 16849 leader_election.cc:343] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0086163b152644feb75caeac6f145533 (127.6.255.4:37199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.231226 16849 leader_election.cc:304] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0; no voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:36.232415 17018 raft_consensus.cc:2749] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:36.238598 16947 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 744cc6469e3749e2aacccb45464e8198. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.240164 16799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02d877efcf5049a89eacf23b58ba27b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.252008 16802 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43106 (request call id 28) took 1025 ms. Trace:
I20260709 13:05:36.252502 16802 rpcz_store.cc:276] 0709 13:05:35.226444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.725558 (+499114us) service_pool.cc:224] Handling call
0709 13:05:35.727728 (+  2170us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:05:35.727811 (+    83us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:05:36.132260 (+404449us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:36.251445 (+119185us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:36.251932 (+   487us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:36.256979 16794 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43106 (request call id 31) took 1010 ms. Trace:
I20260709 13:05:36.228839 16798 rpcz_store.cc:276] 0709 13:05:34.975097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.685645 (+710548us) service_pool.cc:224] Handling call
0709 13:05:35.732741 (+ 47096us) spinlock_profiling.cc:243] Waited 892 us on lock 0x7b58006eb298. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f882569a1a2 00007f882e5dbbb9 00007f882d70cd88 00007f882d7cfb80 00007f882d77e400 00007f8826c03af4 00007f8826c03a82 00007f8826c039b1 00007f8826c0392c
0709 13:05:35.732813 (+    72us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:05:35.732900 (+    87us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:05:35.868343 (+135443us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:36.227760 (+359417us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:36.228251 (+   491us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2051968}
I20260709 13:05:36.276248 16802 tablet_service.cc:1511] Processing CreateTablet for tablet f239a1de317547b598aeedc11994c75f (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:05:36.277884 16802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f239a1de317547b598aeedc11994c75f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.274446 16794 rpcz_store.cc:276] 0709 13:05:35.246203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.938513 (+692310us) service_pool.cc:224] Handling call
0709 13:05:36.029828 (+ 91315us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:05:36.048084 (+ 18256us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:05:36.177924 (+129840us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:36.256379 (+ 78455us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:36.256895 (+   516us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:36.290786 16798 tablet_service.cc:1511] Processing CreateTablet for tablet f9d47e8e7b784085b7079b1542dbc579 (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:05:36.298219 16799 tablet_service.cc:1511] Processing CreateTablet for tablet 7f4354282d844c7e83d760e4418512dc (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:05:36.299367 16798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9d47e8e7b784085b7079b1542dbc579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.300858 17011 raft_consensus.cc:493] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.301867 17011 raft_consensus.cc:515] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:36.304047 17011 leader_election.cc:290] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:36.306011 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e96e3138004b3aa2f3e745f1ddfc04" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
W20260709 13:05:36.308218 16923 leader_election.cc:343] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.300935 17008 ts_tablet_manager.cc:1434] T 4f95af7ac3134423b1a5956c8ae950df P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.198s	user 0.010s	sys 0.000s
I20260709 13:05:36.309849 17008 tablet_bootstrap.cc:492] T 7f37dd0138014a75bc36f33022362b3c P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:36.313661 17006 tablet_bootstrap.cc:492] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:36.314237 17006 ts_tablet_manager.cc:1403] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.521s	user 0.029s	sys 0.034s
I20260709 13:05:36.288846 16794 tablet_service.cc:1511] Processing CreateTablet for tablet 87ca116ec6174decb610df3f8f565a6b (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:05:36.316622 16794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87ca116ec6174decb610df3f8f565a6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.278059 17004 tablet_bootstrap.cc:654] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.329272 17007 tablet_bootstrap.cc:492] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:36.343231 17007 ts_tablet_manager.cc:1403] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.537s	user 0.021s	sys 0.000s
I20260709 13:05:36.343979 17008 tablet_bootstrap.cc:654] T 7f37dd0138014a75bc36f33022362b3c P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.334187 17006 raft_consensus.cc:359] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:36.332064 16797 tablet_service.cc:1511] Processing CreateTablet for tablet c968ba000a4542fc9204dd9a474e364c (DEFAULT_TABLE table=test-workload [id=bfed7b8f5e3a4db4b75459147bc3b61c]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:05:36.351192 17007 raft_consensus.cc:359] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:36.351701 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6288d52217814c7393ff0ec66da3b099" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:36.351938 17007 raft_consensus.cc:385] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.352360 17007 raft_consensus.cc:740] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
W20260709 13:05:36.353544 16775 leader_election.cc:343] T 6288d52217814c7393ff0ec66da3b099 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0086163b152644feb75caeac6f145533 (127.6.255.4:37199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.353144 17007 consensus_queue.cc:260] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:36.355649 17007 ts_tablet_manager.cc:1434] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:05:36.356711 17007 tablet_bootstrap.cc:492] T ac3ad4fe0bc4435cb1c596ad215889a3 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:36.363065 17007 tablet_bootstrap.cc:654] T ac3ad4fe0bc4435cb1c596ad215889a3 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.366685 16799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f4354282d844c7e83d760e4418512dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.367761 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e96e3138004b3aa2f3e745f1ddfc04" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
W20260709 13:05:36.369637 16923 leader_election.cc:343] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.370105 16923 leader_election.cc:304] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:36.370790 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9fdc34d643b4ab995042f719af511aa" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:36.371285 17011 raft_consensus.cc:2749] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:36.370023 16811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "535cac5e21bf41db9c037f226ebcd94e" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:36.372366 16801 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43106 (request call id 32) took 1123 ms. Trace:
W20260709 13:05:36.372728 16700 leader_election.cc:343] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.373325 16700 leader_election.cc:304] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:36.375118 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6288d52217814c7393ff0ec66da3b099" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:36.376052 16793 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43106 (request call id 33) took 1052 ms. Trace:
I20260709 13:05:36.376132 17020 raft_consensus.cc:2749] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:36.372862 16923 leader_election.cc:343] T 535cac5e21bf41db9c037f226ebcd94e P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.376562 16793 rpcz_store.cc:276] 0709 13:05:35.323669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.135008 (+811339us) service_pool.cc:224] Handling call
0709 13:05:36.209144 (+ 74136us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:05:36.210312 (+  1168us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:05:36.352058 (+141746us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:36.375571 (+ 23513us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:36.375991 (+   420us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:36.377250 16923 leader_election.cc:304] T 535cac5e21bf41db9c037f226ebcd94e P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:36.378521 17011 raft_consensus.cc:2749] T 535cac5e21bf41db9c037f226ebcd94e P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:36.379690 16775 leader_election.cc:343] T 6288d52217814c7393ff0ec66da3b099 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.380175 16775 leader_election.cc:304] T 6288d52217814c7393ff0ec66da3b099 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228; no voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:36.348428 17006 raft_consensus.cc:385] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.381146 17006 raft_consensus.cc:740] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.381600 17021 raft_consensus.cc:2749] T 6288d52217814c7393ff0ec66da3b099 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:36.382035 17006 consensus_queue.cc:260] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:36.372762 16801 rpcz_store.cc:276] 0709 13:05:35.249372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.989193 (+739821us) service_pool.cc:224] Handling call
0709 13:05:35.996657 (+  7464us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:05:35.996779 (+   122us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:05:36.259845 (+263066us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:36.361369 (+101524us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:36.371972 (+ 10603us) spinlock_profiling.cc:243] Waited 9.8 ms on lock 0x7b58006eb298. stack: 00007f882569f343 00007f882569f240 00007f882502e1d4 00007f882ed6c304 00007f882ed6c299 00007f8825559db1 00007f8825699b71 00007f882569a1a2 00007f882e5dbbb9 00007f882d70cd88 00007f882d7d3973 00007f882d7cf70f 00007f882d7cfecd 00007f882d77e400 00007f8826c03af4 00007f8826c03a82
0709 13:05:36.372299 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22548224}
I20260709 13:05:36.384281 17006 ts_tablet_manager.cc:1434] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.070s	user 0.007s	sys 0.000s
I20260709 13:05:36.394017 16797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c968ba000a4542fc9204dd9a474e364c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:36.394196 17006 tablet_bootstrap.cc:492] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:36.400401 16800 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43106 (request call id 34) took 1066 ms. Trace:
I20260709 13:05:36.400951 17006 tablet_bootstrap.cc:654] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.405503 17018 raft_consensus.cc:493] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.409644 17018 raft_consensus.cc:515] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:36.410957 16800 rpcz_store.cc:276] 0709 13:05:35.334545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.157900 (+823355us) service_pool.cc:224] Handling call
0709 13:05:36.171107 (+ 13207us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:05:36.171200 (+    93us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:05:36.248777 (+ 77577us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:36.399885 (+151108us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:36.400341 (+   456us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:36.413471 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f95af7ac3134423b1a5956c8ae950df" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:36.419162 17018 leader_election.cc:290] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
W20260709 13:05:36.420149 16849 leader_election.cc:343] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.428707 16811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f95af7ac3134423b1a5956c8ae950df" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:36.429499 16811 raft_consensus.cc:2468] T 4f95af7ac3134423b1a5956c8ae950df P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
I20260709 13:05:36.430939 16849 leader_election.cc:304] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: e2b87453fe654318abccdd1512b949c4
I20260709 13:05:36.432281 17018 raft_consensus.cc:2804] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:36.432721 17018 raft_consensus.cc:493] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.433099 17018 raft_consensus.cc:3060] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.434466 17007 tablet_bootstrap.cc:492] T ac3ad4fe0bc4435cb1c596ad215889a3 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:36.434357 17020 raft_consensus.cc:493] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.435030 17020 raft_consensus.cc:515] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:36.437389 17020 leader_election.cc:290] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:36.437765 17008 tablet_bootstrap.cc:492] T 7f37dd0138014a75bc36f33022362b3c P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:36.438362 17008 ts_tablet_manager.cc:1403] T 7f37dd0138014a75bc36f33022362b3c P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.129s	user 0.010s	sys 0.003s
I20260709 13:05:36.441298 17018 raft_consensus.cc:515] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:36.445295 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f95af7ac3134423b1a5956c8ae950df" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:36.445495 16811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f95af7ac3134423b1a5956c8ae950df" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:36.446138 16811 raft_consensus.cc:3060] T 4f95af7ac3134423b1a5956c8ae950df P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:36.448405 16849 leader_election.cc:343] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.441636 17008 raft_consensus.cc:359] T 7f37dd0138014a75bc36f33022362b3c P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:36.449028 17008 raft_consensus.cc:385] T 7f37dd0138014a75bc36f33022362b3c P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.453624 17008 raft_consensus.cc:740] T 7f37dd0138014a75bc36f33022362b3c P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.441231 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3cecfcf8ae242e1b70f628f293da197" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:36.454170 16811 raft_consensus.cc:2468] T 4f95af7ac3134423b1a5956c8ae950df P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:36.454779 16886 raft_consensus.cc:2468] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 0.
I20260709 13:05:36.455443 16849 leader_election.cc:304] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: e2b87453fe654318abccdd1512b949c4
I20260709 13:05:36.455786 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3cecfcf8ae242e1b70f628f293da197" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:36.456903 17008 consensus_queue.cc:260] T 7f37dd0138014a75bc36f33022362b3c P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
W20260709 13:05:36.457348 16700 leader_election.cc:343] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0086163b152644feb75caeac6f145533 (127.6.255.4:37199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.462066 16700 leader_election.cc:304] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533
I20260709 13:05:36.463194 17020 raft_consensus.cc:2804] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:36.463581 17020 raft_consensus.cc:493] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.463940 17020 raft_consensus.cc:3060] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.458200 17007 ts_tablet_manager.cc:1403] T ac3ad4fe0bc4435cb1c596ad215889a3 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.102s	user 0.021s	sys 0.004s
I20260709 13:05:36.432292 17004 tablet_bootstrap.cc:492] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:36.460129 17018 leader_election.cc:290] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:36.466329 17004 ts_tablet_manager.cc:1403] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.278s	user 0.016s	sys 0.016s
I20260709 13:05:36.479991 17020 raft_consensus.cc:515] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:36.468317 17006 tablet_bootstrap.cc:492] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:36.480525 17004 raft_consensus.cc:359] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:36.481317 17006 ts_tablet_manager.cc:1403] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.087s	user 0.013s	sys 0.002s
I20260709 13:05:36.472009 17008 ts_tablet_manager.cc:1434] T 7f37dd0138014a75bc36f33022362b3c P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.033s	user 0.003s	sys 0.003s
I20260709 13:05:36.476226 17007 raft_consensus.cc:359] T ac3ad4fe0bc4435cb1c596ad215889a3 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:36.482442 17007 raft_consensus.cc:385] T ac3ad4fe0bc4435cb1c596ad215889a3 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.482774 17007 raft_consensus.cc:740] T ac3ad4fe0bc4435cb1c596ad215889a3 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.470000 17040 raft_consensus.cc:2804] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:36.482993 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3cecfcf8ae242e1b70f628f293da197" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:36.483606 16886 raft_consensus.cc:3060] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.481292 17004 raft_consensus.cc:385] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.483562 17007 consensus_queue.cc:260] T ac3ad4fe0bc4435cb1c596ad215889a3 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:36.495987 17021 raft_consensus.cc:493] T 7f37dd0138014a75bc36f33022362b3c P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.487630 17006 raft_consensus.cc:359] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:36.495342 17020 leader_election.cc:290] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:36.496711 17006 raft_consensus.cc:385] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.496614 17021 raft_consensus.cc:515] T 7f37dd0138014a75bc36f33022362b3c P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:36.497310 17006 raft_consensus.cc:740] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.498148 17006 consensus_queue.cc:260] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:36.499087 17021 leader_election.cc:290] T 7f37dd0138014a75bc36f33022362b3c P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:36.500590 17006 ts_tablet_manager.cc:1434] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.019s	user 0.011s	sys 0.003s
I20260709 13:05:36.486815 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3cecfcf8ae242e1b70f628f293da197" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
W20260709 13:05:36.505666 16700 leader_election.cc:343] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0086163b152644feb75caeac6f145533 (127.6.255.4:37199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.505672 17006 tablet_bootstrap.cc:492] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:36.495049 17008 tablet_bootstrap.cc:492] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:36.503423 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f37dd0138014a75bc36f33022362b3c" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
W20260709 13:05:36.511112 16775 leader_election.cc:343] T 7f37dd0138014a75bc36f33022362b3c P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0086163b152644feb75caeac6f145533 (127.6.255.4:37199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.515123 17006 tablet_bootstrap.cc:654] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.516513 16885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f37dd0138014a75bc36f33022362b3c" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:36.518837 17008 tablet_bootstrap.cc:654] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.484630 17004 raft_consensus.cc:740] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.520711 17004 consensus_queue.cc:260] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:36.523263 17004 ts_tablet_manager.cc:1434] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.045s	user 0.003s	sys 0.003s
W20260709 13:05:36.518244 16775 leader_election.cc:343] T 7f37dd0138014a75bc36f33022362b3c P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.524478 16775 leader_election.cc:304] T 7f37dd0138014a75bc36f33022362b3c P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228; no voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:36.525166 17004 tablet_bootstrap.cc:492] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:36.525619 17021 raft_consensus.cc:2749] T 7f37dd0138014a75bc36f33022362b3c P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:36.526962 16886 raft_consensus.cc:2468] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 1.
I20260709 13:05:36.528132 16700 leader_election.cc:304] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533
I20260709 13:05:36.528926 17020 raft_consensus.cc:2804] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:36.529490 17020 raft_consensus.cc:697] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:05:36.530576 17020 consensus_queue.cc:237] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:36.533576 17018 raft_consensus.cc:493] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.534140 17018 raft_consensus.cc:515] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:36.535413 17004 tablet_bootstrap.cc:654] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.536165 17018 leader_election.cc:290] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:36.538647 16811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "535cac5e21bf41db9c037f226ebcd94e" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:36.540459 17040 raft_consensus.cc:697] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Becoming Leader. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:36.540829 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "535cac5e21bf41db9c037f226ebcd94e" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:36.541558 16959 raft_consensus.cc:2468] T 535cac5e21bf41db9c037f226ebcd94e P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
W20260709 13:05:36.542511 16849 leader_election.cc:343] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.542060 17040 consensus_queue.cc:237] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:36.543643 16849 leader_election.cc:304] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: c4b86aae3a8f4a9082e639144ec46228
I20260709 13:05:36.544831 17018 raft_consensus.cc:2804] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:36.545220 17018 raft_consensus.cc:493] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.545589 17018 raft_consensus.cc:3060] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.569015 17007 ts_tablet_manager.cc:1434] T ac3ad4fe0bc4435cb1c596ad215889a3 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.103s	user 0.004s	sys 0.015s
I20260709 13:05:36.570322 17007 tablet_bootstrap.cc:492] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:36.571123 16635 catalog_manager.cc:5697] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:36.577090 17018 raft_consensus.cc:515] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:36.584764 17018 leader_election.cc:290] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:36.586778 17004 tablet_bootstrap.cc:492] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:36.587332 17004 ts_tablet_manager.cc:1403] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.062s	user 0.006s	sys 0.021s
I20260709 13:05:36.587617 17008 tablet_bootstrap.cc:492] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:36.588263 17008 ts_tablet_manager.cc:1403] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.093s	user 0.011s	sys 0.013s
I20260709 13:05:36.589373 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "535cac5e21bf41db9c037f226ebcd94e" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:36.590044 16959 raft_consensus.cc:3060] T 535cac5e21bf41db9c037f226ebcd94e P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.590550 17004 raft_consensus.cc:359] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:36.591336 17004 raft_consensus.cc:385] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.591856 17004 raft_consensus.cc:740] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.592038 17007 tablet_bootstrap.cc:654] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.593923 16811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "535cac5e21bf41db9c037f226ebcd94e" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:36.597635 17004 consensus_queue.cc:260] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
W20260709 13:05:36.598362 16849 leader_election.cc:343] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.587477 16634 catalog_manager.cc:5697] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3). New cstate: current_term: 1 leader_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:36.591337 17008 raft_consensus.cc:359] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:36.599926 17004 ts_tablet_manager.cc:1434] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.012s	user 0.011s	sys 0.000s
I20260709 13:05:36.609630 17008 raft_consensus.cc:385] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.610095 17008 raft_consensus.cc:740] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.610718 17004 tablet_bootstrap.cc:492] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:36.610925 17008 consensus_queue.cc:260] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:36.587478 17006 tablet_bootstrap.cc:492] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:36.612769 17006 ts_tablet_manager.cc:1403] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.107s	user 0.017s	sys 0.005s
I20260709 13:05:36.617053 17006 raft_consensus.cc:359] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:36.617985 17006 raft_consensus.cc:385] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.618305 17006 raft_consensus.cc:740] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.618505 17004 tablet_bootstrap.cc:654] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.619151 17006 consensus_queue.cc:260] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:36.620692 17007 tablet_bootstrap.cc:492] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:36.621251 17007 ts_tablet_manager.cc:1403] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.051s	user 0.021s	sys 0.009s
I20260709 13:05:36.623049 17008 ts_tablet_manager.cc:1434] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.006s
I20260709 13:05:36.624114 17008 tablet_bootstrap.cc:492] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:36.607988 16959 raft_consensus.cc:2468] T 535cac5e21bf41db9c037f226ebcd94e P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:36.630609 16849 leader_election.cc:304] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: c4b86aae3a8f4a9082e639144ec46228
I20260709 13:05:36.631438 17018 raft_consensus.cc:2804] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:36.631918 17018 raft_consensus.cc:697] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Becoming Leader. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:36.632006 17006 ts_tablet_manager.cc:1434] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.004s
I20260709 13:05:36.633124 17006 tablet_bootstrap.cc:492] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:36.632804 17018 consensus_queue.cc:237] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:36.641110 17008 tablet_bootstrap.cc:654] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.641999 17007 raft_consensus.cc:359] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:36.643677 17007 raft_consensus.cc:385] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.644129 17007 raft_consensus.cc:740] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.644939 17007 consensus_queue.cc:260] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:36.655887 17004 tablet_bootstrap.cc:492] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:36.656466 17004 ts_tablet_manager.cc:1403] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.046s	user 0.016s	sys 0.020s
I20260709 13:05:36.659711 17004 raft_consensus.cc:359] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:36.660449 17004 raft_consensus.cc:385] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.660742 17004 raft_consensus.cc:740] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.661496 17004 consensus_queue.cc:260] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:36.663785 17004 ts_tablet_manager.cc:1434] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 13:05:36.664880 17004 tablet_bootstrap.cc:492] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:36.666112 17007 ts_tablet_manager.cc:1434] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.045s	user 0.008s	sys 0.000s
I20260709 13:05:36.671303 17007 tablet_bootstrap.cc:492] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:36.675192 17008 tablet_bootstrap.cc:492] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:36.675773 17008 ts_tablet_manager.cc:1403] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.052s	user 0.019s	sys 0.012s
I20260709 13:05:36.675895 17006 tablet_bootstrap.cc:654] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.680382 17008 raft_consensus.cc:359] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:36.681495 17008 raft_consensus.cc:385] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.681892 17008 raft_consensus.cc:740] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.683101 17007 tablet_bootstrap.cc:654] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.682979 17008 consensus_queue.cc:260] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:36.685612 17004 tablet_bootstrap.cc:654] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.686362 17008 ts_tablet_manager.cc:1434] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.004s
I20260709 13:05:36.687853 17008 tablet_bootstrap.cc:492] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:36.692713 16634 catalog_manager.cc:5697] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3). New cstate: current_term: 1 leader_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:36.713707 17020 raft_consensus.cc:493] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.714229 17020 raft_consensus.cc:515] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:36.716415 17020 leader_election.cc:290] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:36.718735 16811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f95af7ac3134423b1a5956c8ae950df" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:36.719583 16811 raft_consensus.cc:2393] T 4f95af7ac3134423b1a5956c8ae950df P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2b87453fe654318abccdd1512b949c4 in current term 1: Already voted for candidate d4e1b05bc80e43f7941ece113fb97ab0 in this term.
I20260709 13:05:36.720467 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f95af7ac3134423b1a5956c8ae950df" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:36.722635 16700 leader_election.cc:304] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:36.723860 17020 raft_consensus.cc:3060] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.730396 17006 tablet_bootstrap.cc:492] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:36.731194 17006 ts_tablet_manager.cc:1403] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.098s	user 0.017s	sys 0.020s
I20260709 13:05:36.733980 17008 tablet_bootstrap.cc:654] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.741909 17004 tablet_bootstrap.cc:492] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:36.742309 17004 ts_tablet_manager.cc:1403] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.078s	user 0.009s	sys 0.024s
I20260709 13:05:36.742682 17008 tablet_bootstrap.cc:492] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:36.743201 17008 ts_tablet_manager.cc:1403] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.056s	user 0.011s	sys 0.001s
I20260709 13:05:36.744567 17011 raft_consensus.cc:493] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.744808 17004 raft_consensus.cc:359] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:36.745102 17011 raft_consensus.cc:515] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:36.745610 17004 raft_consensus.cc:385] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.745719 17008 raft_consensus.cc:359] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:36.747381 17011 leader_election.cc:290] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:36.747591 17008 raft_consensus.cc:385] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.747996 17008 raft_consensus.cc:740] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.748647 17008 consensus_queue.cc:260] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:36.749464 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82d38de8d2c47b38ca8811691c68d8f" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:36.749945 16811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82d38de8d2c47b38ca8811691c68d8f" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:36.750739 17008 ts_tablet_manager.cc:1434] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
W20260709 13:05:36.751528 16923 leader_election.cc:343] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.746649 17006 raft_consensus.cc:359] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:36.747202 17020 raft_consensus.cc:2749] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:36.751737 17008 tablet_bootstrap.cc:492] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:36.754458 17007 tablet_bootstrap.cc:492] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:36.755002 17007 ts_tablet_manager.cc:1403] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.088s	user 0.012s	sys 0.017s
I20260709 13:05:36.745966 17004 raft_consensus.cc:740] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.760433 17004 consensus_queue.cc:260] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:36.762789 17004 ts_tablet_manager.cc:1434] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
W20260709 13:05:36.758302 16923 leader_election.cc:343] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.766697 16923 leader_election.cc:304] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:36.767158 17004 tablet_bootstrap.cc:492] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:36.754472 17006 raft_consensus.cc:385] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.767956 17006 raft_consensus.cc:740] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.768128 17011 raft_consensus.cc:2749] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:36.768862 17006 consensus_queue.cc:260] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:36.771409 17006 ts_tablet_manager.cc:1434] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.002s
I20260709 13:05:36.772409 17006 tablet_bootstrap.cc:492] T b9fdc34d643b4ab995042f719af511aa P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:36.759284 17007 raft_consensus.cc:359] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:36.774036 17007 raft_consensus.cc:385] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.774353 17007 raft_consensus.cc:740] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.772426 17011 raft_consensus.cc:493] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.775673 17011 raft_consensus.cc:515] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:36.777874 17011 leader_election.cc:290] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:36.777992 17004 tablet_bootstrap.cc:654] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.782913 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3ad4fe0bc4435cb1c596ad215889a3" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:36.775146 17007 consensus_queue.cc:260] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:36.785763 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3ad4fe0bc4435cb1c596ad215889a3" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:36.786389 16737 raft_consensus.cc:2468] T ac3ad4fe0bc4435cb1c596ad215889a3 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 0.
I20260709 13:05:36.786810 17007 ts_tablet_manager.cc:1434] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.001s
I20260709 13:05:36.787803 16923 leader_election.cc:304] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:05:36.788097 17007 tablet_bootstrap.cc:492] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
W20260709 13:05:36.788900 16923 leader_election.cc:343] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.789390 17011 raft_consensus.cc:2804] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:36.790336 17011 raft_consensus.cc:493] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.790706 17011 raft_consensus.cc:3060] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.798421 17008 tablet_bootstrap.cc:654] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.801008 17007 tablet_bootstrap.cc:654] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.805786 17011 raft_consensus.cc:515] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:36.800462 17006 tablet_bootstrap.cc:654] T b9fdc34d643b4ab995042f719af511aa P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.813246 17007 tablet_bootstrap.cc:492] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:36.813225 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3ad4fe0bc4435cb1c596ad215889a3" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:36.813962 17007 ts_tablet_manager.cc:1403] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.011s
W20260709 13:05:36.815402 16923 leader_election.cc:343] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.817328 17007 raft_consensus.cc:359] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:36.810101 17062 raft_consensus.cc:493] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.810456 17004 tablet_bootstrap.cc:492] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:36.818146 17007 raft_consensus.cc:385] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.818504 17007 raft_consensus.cc:740] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.818738 17004 ts_tablet_manager.cc:1403] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.052s	user 0.013s	sys 0.005s
I20260709 13:05:36.819428 17007 consensus_queue.cc:260] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:36.822777 17004 raft_consensus.cc:359] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:36.823818 17004 raft_consensus.cc:385] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.824239 17004 raft_consensus.cc:740] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.825174 17004 consensus_queue.cc:260] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:36.828395 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3ad4fe0bc4435cb1c596ad215889a3" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:36.829216 16737 raft_consensus.cc:3060] T ac3ad4fe0bc4435cb1c596ad215889a3 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.830391 17007 ts_tablet_manager.cc:1434] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:05:36.837705 17062 raft_consensus.cc:515] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:36.839155 17007 tablet_bootstrap.cc:492] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:36.841882 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3cecfcf8ae242e1b70f628f293da197" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:36.843016 16886 raft_consensus.cc:2393] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0086163b152644feb75caeac6f145533 in current term 1: Already voted for candidate e2b87453fe654318abccdd1512b949c4 in this term.
I20260709 13:05:36.845975 17008 tablet_bootstrap.cc:492] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:36.846807 17008 ts_tablet_manager.cc:1403] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.095s	user 0.011s	sys 0.002s
I20260709 13:05:36.850443 16737 raft_consensus.cc:2468] T ac3ad4fe0bc4435cb1c596ad215889a3 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 1.
I20260709 13:05:36.852241 16923 leader_election.cc:304] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:36.846968 17007 tablet_bootstrap.cc:654] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.849617 17011 leader_election.cc:290] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:36.858803 17067 raft_consensus.cc:2804] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:36.859254 17067 raft_consensus.cc:697] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 [term 1 LEADER]: Becoming Leader. State: Replica: 0086163b152644feb75caeac6f145533, State: Running, Role: LEADER
I20260709 13:05:36.860064 17067 consensus_queue.cc:237] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:36.860673 17062 leader_election.cc:290] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:36.863936 17004 ts_tablet_manager.cc:1434] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.045s	user 0.008s	sys 0.000s
I20260709 13:05:36.865754 17004 tablet_bootstrap.cc:492] T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:36.867658 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3cecfcf8ae242e1b70f628f293da197" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:36.853168 17008 raft_consensus.cc:359] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:36.869406 17008 raft_consensus.cc:385] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.869836 16923 leader_election.cc:304] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:36.870644 17007 tablet_bootstrap.cc:492] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:36.871032 17062 raft_consensus.cc:3060] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.871176 17007 ts_tablet_manager.cc:1403] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.001s
I20260709 13:05:36.874197 17004 tablet_bootstrap.cc:654] T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.874572 17007 raft_consensus.cc:359] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:36.875347 17007 raft_consensus.cc:385] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.875656 17007 raft_consensus.cc:740] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.876434 17007 consensus_queue.cc:260] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:36.878991 17007 ts_tablet_manager.cc:1434] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:05:36.880260 17007 tablet_bootstrap.cc:492] T 0b8add4b17084a6189509807b7e663e5 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:36.886363 17018 raft_consensus.cc:493] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.886929 17018 raft_consensus.cc:515] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:36.889266 17062 raft_consensus.cc:2749] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:36.891639 16811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f37dd0138014a75bc36f33022362b3c" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:36.892575 16811 raft_consensus.cc:2468] T 7f37dd0138014a75bc36f33022362b3c P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
I20260709 13:05:36.894064 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f37dd0138014a75bc36f33022362b3c" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:36.892678 16635 catalog_manager.cc:5697] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0086163b152644feb75caeac6f145533 (127.6.255.4). New cstate: current_term: 1 leader_uuid: "0086163b152644feb75caeac6f145533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:36.869830 17008 raft_consensus.cc:740] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.900835 17008 consensus_queue.cc:260] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
W20260709 13:05:36.901750 16849 leader_election.cc:343] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0086163b152644feb75caeac6f145533 (127.6.255.4:37199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.902838 16849 leader_election.cc:304] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 0086163b152644feb75caeac6f145533
I20260709 13:05:36.903986 17040 raft_consensus.cc:2804] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:36.904481 17040 raft_consensus.cc:493] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.904933 17040 raft_consensus.cc:3060] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.889276 17018 leader_election.cc:290] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:36.916558 17040 raft_consensus.cc:515] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:36.919370 17020 raft_consensus.cc:493] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.919977 17020 raft_consensus.cc:515] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:36.920311 16811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f37dd0138014a75bc36f33022362b3c" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:36.921077 16811 raft_consensus.cc:3060] T 7f37dd0138014a75bc36f33022362b3c P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.921151 17021 raft_consensus.cc:493] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.921706 17021 raft_consensus.cc:515] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:36.922432 17020 leader_election.cc:290] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:36.923739 17021 leader_election.cc:290] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:36.924710 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82d38de8d2c47b38ca8811691c68d8f" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:36.925486 16959 raft_consensus.cc:2468] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 0.
I20260709 13:05:36.926185 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ff2290a9c9451c9a786eacf7380d90" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:36.927017 16700 leader_election.cc:304] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: 
W20260709 13:05:36.928182 16775 leader_election.cc:343] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0086163b152644feb75caeac6f145533 (127.6.255.4:37199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.926185 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82d38de8d2c47b38ca8811691c68d8f" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
W20260709 13:05:36.930847 16700 leader_election.cc:343] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.928172 17020 raft_consensus.cc:2804] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:36.932952 17020 raft_consensus.cc:493] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.934159 17008 ts_tablet_manager.cc:1434] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.084s	user 0.008s	sys 0.000s
I20260709 13:05:36.934413 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ff2290a9c9451c9a786eacf7380d90" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
W20260709 13:05:36.936147 16775 leader_election.cc:343] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.933382 17020 raft_consensus.cc:3060] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.936663 16775 leader_election.cc:304] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228; no voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:36.937891 17021 raft_consensus.cc:2749] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:36.941907 16811 raft_consensus.cc:2468] T 7f37dd0138014a75bc36f33022362b3c P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:36.886896 17007 tablet_bootstrap.cc:654] T 0b8add4b17084a6189509807b7e663e5 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.943075 16849 leader_election.cc:304] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:36.944098 17018 raft_consensus.cc:2804] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:36.944583 17018 raft_consensus.cc:697] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Becoming Leader. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:36.945982 17018 consensus_queue.cc:237] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:36.947268 17040 leader_election.cc:290] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:36.947829 17008 tablet_bootstrap.cc:492] T 5b5161020c944d28a4334dc1d572dac7 P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:36.949234 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f37dd0138014a75bc36f33022362b3c" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:36.938169 17006 tablet_bootstrap.cc:492] T b9fdc34d643b4ab995042f719af511aa P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:36.951534 17006 ts_tablet_manager.cc:1403] T b9fdc34d643b4ab995042f719af511aa P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.179s	user 0.045s	sys 0.050s
W20260709 13:05:36.951611 16849 leader_election.cc:343] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0086163b152644feb75caeac6f145533 (127.6.255.4:37199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.955411 17006 raft_consensus.cc:359] T b9fdc34d643b4ab995042f719af511aa P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:36.956630 17006 raft_consensus.cc:385] T b9fdc34d643b4ab995042f719af511aa P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.956990 17006 raft_consensus.cc:740] T b9fdc34d643b4ab995042f719af511aa P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.958091 17006 consensus_queue.cc:260] T b9fdc34d643b4ab995042f719af511aa P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:36.961594 17006 ts_tablet_manager.cc:1434] T b9fdc34d643b4ab995042f719af511aa P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 13:05:36.962846 17006 tablet_bootstrap.cc:492] T c9043e0bc02745d187e9ca5a5c71274f P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:36.963111 17042 consensus_queue.cc:1048] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:36.962260 17020 raft_consensus.cc:515] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:36.968259 17020 leader_election.cc:290] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:36.968895 17040 consensus_queue.cc:1048] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:05:36.979465 16683 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:36.980262 17008 tablet_bootstrap.cc:654] T 5b5161020c944d28a4334dc1d572dac7 P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.969267 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82d38de8d2c47b38ca8811691c68d8f" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:36.980989 16960 raft_consensus.cc:3060] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.982028 16635 catalog_manager.cc:5697] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3). New cstate: current_term: 1 leader_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:36.969664 17004 tablet_bootstrap.cc:492] T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:36.984951 17004 ts_tablet_manager.cc:1403] T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.119s	user 0.011s	sys 0.001s
I20260709 13:05:36.985908 17008 tablet_bootstrap.cc:492] T 5b5161020c944d28a4334dc1d572dac7 P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:36.986459 17008 ts_tablet_manager.cc:1403] T 5b5161020c944d28a4334dc1d572dac7 P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.039s	user 0.010s	sys 0.004s
I20260709 13:05:36.987800 17004 raft_consensus.cc:359] T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:36.988524 17004 raft_consensus.cc:385] T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.988792 17004 raft_consensus.cc:740] T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.971082 16811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82d38de8d2c47b38ca8811691c68d8f" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:36.989224 17008 raft_consensus.cc:359] T 5b5161020c944d28a4334dc1d572dac7 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:36.971639 17006 tablet_bootstrap.cc:654] T c9043e0bc02745d187e9ca5a5c71274f P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.992592 16960 raft_consensus.cc:2468] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 1.
I20260709 13:05:36.994616 16700 leader_election.cc:304] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:05:36.995492 17020 raft_consensus.cc:2804] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:05:36.995671 16700 leader_election.cc:343] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.996021 17020 raft_consensus.cc:697] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:05:36.996886 17020 consensus_queue.cc:237] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:37.004581 17004 consensus_queue.cc:260] T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.017951 17007 tablet_bootstrap.cc:492] T 0b8add4b17084a6189509807b7e663e5 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:37.018518 17007 ts_tablet_manager.cc:1403] T 0b8add4b17084a6189509807b7e663e5 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.138s	user 0.009s	sys 0.016s
I20260709 13:05:37.021026 17021 raft_consensus.cc:493] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.021546 17008 raft_consensus.cc:385] T 5b5161020c944d28a4334dc1d572dac7 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.022012 17008 raft_consensus.cc:740] T 5b5161020c944d28a4334dc1d572dac7 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.021845 17021 raft_consensus.cc:515] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:37.023173 17008 consensus_queue.cc:260] T 5b5161020c944d28a4334dc1d572dac7 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.026351 17004 ts_tablet_manager.cc:1434] T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
I20260709 13:05:37.027333 17062 raft_consensus.cc:493] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.027827 17062 raft_consensus.cc:515] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.027956 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b9b28c4c5d4ddaa6d1ae05ce225b11" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:37.028908 16886 raft_consensus.cc:2468] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 0.
I20260709 13:05:37.029831 17062 leader_election.cc:290] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:37.031488 16775 leader_election.cc:304] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:37.031613 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b5161020c944d28a4334dc1d572dac7" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:37.032478 16812 raft_consensus.cc:2468] T 5b5161020c944d28a4334dc1d572dac7 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 0.
I20260709 13:05:37.032826 17021 leader_election.cc:290] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:37.033802 17021 raft_consensus.cc:2804] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:37.034228 17021 raft_consensus.cc:493] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.034720 17021 raft_consensus.cc:3060] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.035305 17004 tablet_bootstrap.cc:492] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:37.037711 17062 raft_consensus.cc:493] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.038264 17062 raft_consensus.cc:515] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:37.040449 17062 leader_election.cc:290] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
W20260709 13:05:37.040958 16700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:45101, user_credentials={real_user=slave}} blocked reactor thread for 44232.6us
I20260709 13:05:37.042783 17007 raft_consensus.cc:359] T 0b8add4b17084a6189509807b7e663e5 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:37.043747 17007 raft_consensus.cc:385] T 0b8add4b17084a6189509807b7e663e5 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.044090 17007 raft_consensus.cc:740] T 0b8add4b17084a6189509807b7e663e5 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.053686 17006 tablet_bootstrap.cc:492] T c9043e0bc02745d187e9ca5a5c71274f P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:37.054606 17006 ts_tablet_manager.cc:1403] T c9043e0bc02745d187e9ca5a5c71274f P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.092s	user 0.011s	sys 0.026s
I20260709 13:05:37.062093 17007 consensus_queue.cc:260] T 0b8add4b17084a6189509807b7e663e5 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:37.064100 16923 leader_election.cc:304] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:05:37.065263 17062 raft_consensus.cc:2804] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:37.065723 17062 raft_consensus.cc:493] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.066095 17062 raft_consensus.cc:3060] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.070474 16635 catalog_manager.cc:5697] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:37.072768 16849 consensus_peers.cc:597] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Couldn't send request to peer c4b86aae3a8f4a9082e639144ec46228. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:37.073581 16811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6288d52217814c7393ff0ec66da3b099" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:37.074632 16811 raft_consensus.cc:2468] T 6288d52217814c7393ff0ec66da3b099 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 0.
I20260709 13:05:37.079192 17008 ts_tablet_manager.cc:1434] T 5b5161020c944d28a4334dc1d572dac7 P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.092s	user 0.013s	sys 0.015s
I20260709 13:05:37.078986 16923 leader_election.cc:304] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:05:37.080241 17008 tablet_bootstrap.cc:492] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:37.080359 17067 raft_consensus.cc:2804] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:37.080998 17067 raft_consensus.cc:493] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.085184 17007 ts_tablet_manager.cc:1434] T 0b8add4b17084a6189509807b7e663e5 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.066s	user 0.008s	sys 0.001s
I20260709 13:05:37.086861 17007 tablet_bootstrap.cc:492] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:37.083587 17067 raft_consensus.cc:3060] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.096156 17006 raft_consensus.cc:359] T c9043e0bc02745d187e9ca5a5c71274f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.101810 17008 tablet_bootstrap.cc:654] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.109555 17006 raft_consensus.cc:385] T c9043e0bc02745d187e9ca5a5c71274f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.110196 17006 raft_consensus.cc:740] T c9043e0bc02745d187e9ca5a5c71274f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.111248 17006 consensus_queue.cc:260] T c9043e0bc02745d187e9ca5a5c71274f P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.114809 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b5161020c944d28a4334dc1d572dac7" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:37.115366 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6288d52217814c7393ff0ec66da3b099" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:37.117163 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b9b28c4c5d4ddaa6d1ae05ce225b11" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:37.117825 17021 raft_consensus.cc:515] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:37.117161 17083 raft_consensus.cc:493] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.118921 17083 raft_consensus.cc:515] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:37.116531 17062 raft_consensus.cc:515] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.121165 17083 leader_election.cc:290] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:37.123102 16811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b5161020c944d28a4334dc1d572dac7" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:37.123821 16811 raft_consensus.cc:3060] T 5b5161020c944d28a4334dc1d572dac7 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:37.128628 16923 leader_election.cc:343] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.129982 17081 raft_consensus.cc:493] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:37.130156 16923 leader_election.cc:343] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.130673 17081 raft_consensus.cc:515] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
W20260709 13:05:37.131567 16775 leader_election.cc:343] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.133751 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213e9771c53e4498b315bb43fdc0aa9a" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:37.133143 17081 leader_election.cc:290] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:37.135341 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8add4b17084a6189509807b7e663e5" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:37.136150 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213e9771c53e4498b315bb43fdc0aa9a" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:37.137015 17006 ts_tablet_manager.cc:1434] T c9043e0bc02745d187e9ca5a5c71274f P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.082s	user 0.007s	sys 0.003s
W20260709 13:05:37.139092 16775 leader_election.cc:343] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.140002 17006 tablet_bootstrap.cc:492] T ac3ad4fe0bc4435cb1c596ad215889a3 P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:37.142436 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b9b28c4c5d4ddaa6d1ae05ce225b11" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:37.138391 17067 raft_consensus.cc:515] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:37.143599 16811 raft_consensus.cc:2468] T 5b5161020c944d28a4334dc1d572dac7 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 1.
I20260709 13:05:37.143962 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8add4b17084a6189509807b7e663e5" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:37.144608 16736 raft_consensus.cc:2468] T 0b8add4b17084a6189509807b7e663e5 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 0.
I20260709 13:05:37.144734 16923 leader_election.cc:304] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:05:37.145804 17066 raft_consensus.cc:2804] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:37.146063 17018 consensus_queue.cc:1048] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:37.146390 17066 raft_consensus.cc:697] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [term 1 LEADER]: Becoming Leader. State: Replica: 0086163b152644feb75caeac6f145533, State: Running, Role: LEADER
I20260709 13:05:37.147254 17066 consensus_queue.cc:237] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.148636 17008 tablet_bootstrap.cc:492] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:37.149192 17008 ts_tablet_manager.cc:1403] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.069s	user 0.010s	sys 0.005s
W20260709 13:05:37.149317 16700 leader_election.cc:343] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:37.150840 16775 leader_election.cc:343] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.150924 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6288d52217814c7393ff0ec66da3b099" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:37.151649 16812 raft_consensus.cc:3060] T 6288d52217814c7393ff0ec66da3b099 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.151727 16775 leader_election.cc:304] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:37.152551 17008 raft_consensus.cc:359] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:37.153012 17081 raft_consensus.cc:2804] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:37.155349 17008 raft_consensus.cc:385] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.155382 17081 raft_consensus.cc:493] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.156132 17008 raft_consensus.cc:740] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.156191 17081 raft_consensus.cc:3060] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.159163 17006 tablet_bootstrap.cc:654] T ac3ad4fe0bc4435cb1c596ad215889a3 P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.160810 17083 raft_consensus.cc:493] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.159848 17008 consensus_queue.cc:260] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:37.145987 17067 leader_election.cc:290] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:37.161393 17083 raft_consensus.cc:515] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.163281 17008 ts_tablet_manager.cc:1434] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.004s
I20260709 13:05:37.164970 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b9b28c4c5d4ddaa6d1ae05ce225b11" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:37.165619 16886 raft_consensus.cc:3060] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.167721 17081 raft_consensus.cc:515] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:37.169497 17021 leader_election.cc:290] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:37.170650 17062 leader_election.cc:290] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:37.175307 17007 tablet_bootstrap.cc:654] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.164038 17083 leader_election.cc:290] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
W20260709 13:05:37.176849 16700 leader_election.cc:343] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0086163b152644feb75caeac6f145533 (127.6.255.4:37199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.181684 17081 leader_election.cc:290] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:37.182551 16885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8add4b17084a6189509807b7e663e5" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
W20260709 13:05:37.183943 16775 leader_election.cc:343] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.185300 17083 consensus_queue.cc:1048] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:37.186185 17007 tablet_bootstrap.cc:492] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:37.159732 16812 raft_consensus.cc:2468] T 6288d52217814c7393ff0ec66da3b099 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 1.
I20260709 13:05:37.186678 17007 ts_tablet_manager.cc:1403] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.100s	user 0.001s	sys 0.010s
I20260709 13:05:37.187561 16886 raft_consensus.cc:2468] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:37.187707 16923 leader_election.cc:304] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:05:37.177480 16700 leader_election.cc:304] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228
I20260709 13:05:37.189180 16775 leader_election.cc:304] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: e2b87453fe654318abccdd1512b949c4
I20260709 13:05:37.188572 17066 raft_consensus.cc:2804] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:37.164973 17004 tablet_bootstrap.cc:654] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.190505 17066 raft_consensus.cc:697] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 [term 1 LEADER]: Becoming Leader. State: Replica: 0086163b152644feb75caeac6f145533, State: Running, Role: LEADER
I20260709 13:05:37.190713 16885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27be53bca112445d8bdc640329310d67" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:37.164989 17008 tablet_bootstrap.cc:492] T 535cac5e21bf41db9c037f226ebcd94e P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:37.191416 17066 consensus_queue.cc:237] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:37.192330 17006 tablet_bootstrap.cc:492] T ac3ad4fe0bc4435cb1c596ad215889a3 P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:37.192844 17006 ts_tablet_manager.cc:1403] T ac3ad4fe0bc4435cb1c596ad215889a3 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.053s	user 0.008s	sys 0.022s
I20260709 13:05:37.193077 17083 raft_consensus.cc:2749] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:37.193475 17081 raft_consensus.cc:2804] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:37.195158 17081 raft_consensus.cc:697] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Becoming Leader. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:05:37.195107 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27be53bca112445d8bdc640329310d67" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:37.195920 16960 raft_consensus.cc:2468] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 0.
I20260709 13:05:37.196141 17006 raft_consensus.cc:359] T ac3ad4fe0bc4435cb1c596ad215889a3 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.197005 17006 raft_consensus.cc:385] T ac3ad4fe0bc4435cb1c596ad215889a3 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.197628 17006 raft_consensus.cc:740] T ac3ad4fe0bc4435cb1c596ad215889a3 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.197867 17021 raft_consensus.cc:493] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.198467 17021 raft_consensus.cc:515] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:37.198661 17006 consensus_queue.cc:260] T ac3ad4fe0bc4435cb1c596ad215889a3 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.200307 17008 tablet_bootstrap.cc:654] T 535cac5e21bf41db9c037f226ebcd94e P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.200811 17021 leader_election.cc:290] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:37.201000 17018 consensus_queue.cc:1048] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:37.203320 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e96e3138004b3aa2f3e745f1ddfc04" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:37.204082 16960 raft_consensus.cc:2468] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 0.
I20260709 13:05:37.205507 16775 leader_election.cc:304] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:05:37.206776 17021 raft_consensus.cc:2804] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:37.207237 17006 ts_tablet_manager.cc:1434] T ac3ad4fe0bc4435cb1c596ad215889a3 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:05:37.207309 17021 raft_consensus.cc:493] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.208024 17021 raft_consensus.cc:3060] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.196173 17081 consensus_queue.cc:237] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:37.216184 17006 tablet_bootstrap.cc:492] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:37.219295 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8add4b17084a6189509807b7e663e5" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:37.219965 16736 raft_consensus.cc:3060] T 0b8add4b17084a6189509807b7e663e5 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.220233 17007 raft_consensus.cc:359] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:37.220978 17007 raft_consensus.cc:385] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.223003 17008 tablet_bootstrap.cc:492] T 535cac5e21bf41db9c037f226ebcd94e P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:37.223556 17008 ts_tablet_manager.cc:1403] T 535cac5e21bf41db9c037f226ebcd94e P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.059s	user 0.015s	sys 0.000s
I20260709 13:05:37.226207 16635 catalog_manager.cc:5697] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0086163b152644feb75caeac6f145533 (127.6.255.4). New cstate: current_term: 1 leader_uuid: "0086163b152644feb75caeac6f145533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:37.227041 17008 raft_consensus.cc:359] T 535cac5e21bf41db9c037f226ebcd94e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:37.227936 17008 raft_consensus.cc:385] T 535cac5e21bf41db9c037f226ebcd94e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.228276 17008 raft_consensus.cc:740] T 535cac5e21bf41db9c037f226ebcd94e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.229120 17008 consensus_queue.cc:260] T 535cac5e21bf41db9c037f226ebcd94e P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:37.230321 16736 raft_consensus.cc:2468] T 0b8add4b17084a6189509807b7e663e5 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:37.231524 17008 ts_tablet_manager.cc:1434] T 535cac5e21bf41db9c037f226ebcd94e P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:05:37.221261 17007 raft_consensus.cc:740] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.232748 17008 tablet_bootstrap.cc:492] T 3850091ca95e47b6a1b52e600a452b0b P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:37.234900 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6288d52217814c7393ff0ec66da3b099" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:37.236104 17021 raft_consensus.cc:515] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:37.237807 17062 raft_consensus.cc:493] T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.238173 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b5161020c944d28a4334dc1d572dac7" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:37.238287 17062 raft_consensus.cc:515] T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.239683 17008 tablet_bootstrap.cc:654] T 3850091ca95e47b6a1b52e600a452b0b P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.240549 17062 leader_election.cc:290] T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:37.242889 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e091d4e9a0044dd899e731769a94064c" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
W20260709 13:05:37.244242 16923 leader_election.cc:343] T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.246153 16634 catalog_manager.cc:5697] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "c4b86aae3a8f4a9082e639144ec46228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:37.250161 16775 leader_election.cc:304] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:37.251221 17082 raft_consensus.cc:2804] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:37.251689 17082 raft_consensus.cc:697] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Becoming Leader. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:05:37.251931 17007 consensus_queue.cc:260] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:37.252561 17082 consensus_queue.cc:237] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:37.255026 17007 ts_tablet_manager.cc:1434] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.000s
I20260709 13:05:37.255956 17007 tablet_bootstrap.cc:492] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:37.266925 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e96e3138004b3aa2f3e745f1ddfc04" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:37.267782 16958 raft_consensus.cc:3060] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.269742 17004 tablet_bootstrap.cc:492] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:37.270355 17004 ts_tablet_manager.cc:1403] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.235s	user 0.008s	sys 0.007s
W20260709 13:05:37.273679 16923 leader_election.cc:343] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:37.275067 16923 leader_election.cc:343] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.266930 17021 leader_election.cc:290] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:37.263446 17007 tablet_bootstrap.cc:654] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.276593 16700 leader_election.cc:304] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:05:37.277534 17083 raft_consensus.cc:2804] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:37.277714 16958 raft_consensus.cc:2468] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:37.277949 17083 raft_consensus.cc:493] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.278339 17083 raft_consensus.cc:3060] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.279126 16775 leader_election.cc:304] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:05:37.279984 17081 raft_consensus.cc:493] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.280460 17081 raft_consensus.cc:515] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:37.282672 17081 leader_election.cc:290] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:37.283223 17081 raft_consensus.cc:2804] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:37.283671 17081 raft_consensus.cc:697] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Becoming Leader. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:05:37.284536 17081 consensus_queue.cc:237] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:37.286988 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9fdc34d643b4ab995042f719af511aa" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:37.286831 17083 raft_consensus.cc:515] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.287653 16886 raft_consensus.cc:2468] T b9fdc34d643b4ab995042f719af511aa P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 0.
I20260709 13:05:37.288926 16775 leader_election.cc:304] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:37.289089 17083 leader_election.cc:290] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:37.290349 17021 raft_consensus.cc:493] T 535cac5e21bf41db9c037f226ebcd94e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.240615 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e96e3138004b3aa2f3e745f1ddfc04" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:37.290926 17021 raft_consensus.cc:515] T 535cac5e21bf41db9c037f226ebcd94e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:37.295444 17006 tablet_bootstrap.cc:654] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.293268 17021 leader_election.cc:290] T 535cac5e21bf41db9c037f226ebcd94e P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:37.297399 17021 raft_consensus.cc:2804] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:37.297766 17021 raft_consensus.cc:493] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.298105 17021 raft_consensus.cc:3060] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:37.300866 16700 leader_election.cc:343] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.314448 17006 tablet_bootstrap.cc:492] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:37.314980 17006 ts_tablet_manager.cc:1403] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.099s	user 0.014s	sys 0.000s
I20260709 13:05:37.321229 17004 raft_consensus.cc:359] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:37.322294 17004 raft_consensus.cc:385] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.322665 17004 raft_consensus.cc:740] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.327940 17008 tablet_bootstrap.cc:492] T 3850091ca95e47b6a1b52e600a452b0b P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:37.328385 17008 ts_tablet_manager.cc:1403] T 3850091ca95e47b6a1b52e600a452b0b P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.096s	user 0.022s	sys 0.048s
I20260709 13:05:37.325968 17004 consensus_queue.cc:260] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:37.332091 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e091d4e9a0044dd899e731769a94064c" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:37.334008 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27be53bca112445d8bdc640329310d67" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:37.334954 16735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9fdc34d643b4ab995042f719af511aa" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:37.335605 16735 raft_consensus.cc:2468] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 0.
I20260709 13:05:37.335661 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e96e3138004b3aa2f3e745f1ddfc04" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
W20260709 13:05:37.336714 16923 leader_election.cc:343] T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.337154 16923 leader_election.cc:304] T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:37.338099 17007 tablet_bootstrap.cc:492] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:37.338626 17007 ts_tablet_manager.cc:1403] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.083s	user 0.013s	sys 0.000s
I20260709 13:05:37.338380 17006 raft_consensus.cc:359] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:37.339118 17006 raft_consensus.cc:385] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.339468 17006 raft_consensus.cc:740] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.340348 17004 ts_tablet_manager.cc:1434] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.069s	user 0.008s	sys 0.000s
I20260709 13:05:37.341320 17004 tablet_bootstrap.cc:492] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:37.342470 17021 raft_consensus.cc:515] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:37.340230 17006 consensus_queue.cc:260] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:37.343438 16635 catalog_manager.cc:5697] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0086163b152644feb75caeac6f145533 (127.6.255.4). New cstate: current_term: 1 leader_uuid: "0086163b152644feb75caeac6f145533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:37.345964 17006 ts_tablet_manager.cc:1434] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260709 13:05:37.347457 17006 tablet_bootstrap.cc:492] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:37.347205 17007 raft_consensus.cc:359] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:37.347944 17007 raft_consensus.cc:385] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.349140 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "535cac5e21bf41db9c037f226ebcd94e" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:37.350080 17007 raft_consensus.cc:740] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.350968 17007 consensus_queue.cc:260] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
W20260709 13:05:37.352872 16775 leader_election.cc:343] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.353209 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "535cac5e21bf41db9c037f226ebcd94e" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:37.354379 17006 tablet_bootstrap.cc:654] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.355275 16775 leader_election.cc:304] T 535cac5e21bf41db9c037f226ebcd94e P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228; no voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:37.339470 17066 raft_consensus.cc:2749] T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:37.358278 17100 raft_consensus.cc:2749] T 535cac5e21bf41db9c037f226ebcd94e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:37.359525 17021 leader_election.cc:290] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:37.357072 17007 ts_tablet_manager.cc:1434] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.003s
I20260709 13:05:37.360340 17008 raft_consensus.cc:359] T 3850091ca95e47b6a1b52e600a452b0b P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:37.361073 17008 raft_consensus.cc:385] T 3850091ca95e47b6a1b52e600a452b0b P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.361344 17008 raft_consensus.cc:740] T 3850091ca95e47b6a1b52e600a452b0b P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.362183 17008 consensus_queue.cc:260] T 3850091ca95e47b6a1b52e600a452b0b P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:37.363024 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9fdc34d643b4ab995042f719af511aa" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:37.363745 17007 tablet_bootstrap.cc:492] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:37.363873 16886 raft_consensus.cc:3060] T b9fdc34d643b4ab995042f719af511aa P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.366638 17004 tablet_bootstrap.cc:654] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.370590 17008 ts_tablet_manager.cc:1434] T 3850091ca95e47b6a1b52e600a452b0b P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.000s
I20260709 13:05:37.371592 17008 tablet_bootstrap.cc:492] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:37.373792 17040 raft_consensus.cc:493] T c9043e0bc02745d187e9ca5a5c71274f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.374333 17040 raft_consensus.cc:515] T c9043e0bc02745d187e9ca5a5c71274f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.376528 17040 leader_election.cc:290] T c9043e0bc02745d187e9ca5a5c71274f P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:37.378412 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9043e0bc02745d187e9ca5a5c71274f" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:37.379664 17007 tablet_bootstrap.cc:654] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:37.380632 16849 leader_election.cc:343] T c9043e0bc02745d187e9ca5a5c71274f P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0086163b152644feb75caeac6f145533 (127.6.255.4:37199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.382504 16886 raft_consensus.cc:2468] T b9fdc34d643b4ab995042f719af511aa P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:37.384784 17008 tablet_bootstrap.cc:654] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:37.384697 16700 leader_election.cc:343] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.386289 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27be53bca112445d8bdc640329310d67" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:37.386876 16958 raft_consensus.cc:3060] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.387686 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9043e0bc02745d187e9ca5a5c71274f" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:37.389138 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9fdc34d643b4ab995042f719af511aa" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:37.389683 16736 raft_consensus.cc:3060] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.395008 16958 raft_consensus.cc:2468] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 1.
I20260709 13:05:37.396755 16736 raft_consensus.cc:2468] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:37.398144 16700 leader_election.cc:304] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:37.398895 17006 tablet_bootstrap.cc:492] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:37.398972 17083 raft_consensus.cc:2804] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:37.399585 17006 ts_tablet_manager.cc:1403] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.052s	user 0.030s	sys 0.020s
I20260709 13:05:37.399662 17083 raft_consensus.cc:697] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:05:37.400159 16632 catalog_manager.cc:5697] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "c4b86aae3a8f4a9082e639144ec46228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:37.401058 17083 consensus_queue.cc:237] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.402446 17004 tablet_bootstrap.cc:492] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:37.399842 17066 consensus_queue.cc:1048] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:37.402911 17004 ts_tablet_manager.cc:1403] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.062s	user 0.010s	sys 0.000s
I20260709 13:05:37.402930 17006 raft_consensus.cc:359] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.403908 17006 raft_consensus.cc:385] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.404215 17006 raft_consensus.cc:740] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.405007 17006 consensus_queue.cc:260] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
W20260709 13:05:37.405447 16849 leader_election.cc:343] T c9043e0bc02745d187e9ca5a5c71274f P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.405964 16849 leader_election.cc:304] T c9043e0bc02745d187e9ca5a5c71274f P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0; no voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:37.407300 17041 raft_consensus.cc:2749] T c9043e0bc02745d187e9ca5a5c71274f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:37.407521 17006 ts_tablet_manager.cc:1434] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:05:37.408632 17006 tablet_bootstrap.cc:492] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:37.409015 17004 raft_consensus.cc:359] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:37.409686 17004 raft_consensus.cc:385] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.409938 17004 raft_consensus.cc:740] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.410567 17004 consensus_queue.cc:260] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:37.385749 16775 leader_election.cc:304] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
W20260709 13:05:37.412272 16775 leader_election.cc:343] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.415503 17006 tablet_bootstrap.cc:654] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.416410 17004 ts_tablet_manager.cc:1434] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:05:37.416586 17021 raft_consensus.cc:2804] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:37.427047 17021 raft_consensus.cc:697] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Becoming Leader. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:05:37.427290 17004 tablet_bootstrap.cc:492] T 10bbce697a734600bb083d47ac877574 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:37.427970 17021 consensus_queue.cc:237] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:37.428392 16632 catalog_manager.cc:5697] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "c4b86aae3a8f4a9082e639144ec46228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:37.417783 17007 tablet_bootstrap.cc:492] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:37.432741 17007 ts_tablet_manager.cc:1403] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.072s	user 0.011s	sys 0.000s
I20260709 13:05:37.434011 17004 tablet_bootstrap.cc:654] T 10bbce697a734600bb083d47ac877574 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.435446 17040 raft_consensus.cc:493] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.435940 17040 raft_consensus.cc:515] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.436228 17007 raft_consensus.cc:359] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:37.437366 17007 raft_consensus.cc:385] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.437706 17007 raft_consensus.cc:740] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.438416 17040 leader_election.cc:290] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:37.440968 17008 tablet_bootstrap.cc:492] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:37.441546 17008 ts_tablet_manager.cc:1403] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.070s	user 0.017s	sys 0.003s
I20260709 13:05:37.441352 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e091d4e9a0044dd899e731769a94064c" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:37.442171 16735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e091d4e9a0044dd899e731769a94064c" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:37.443270 16958 raft_consensus.cc:2468] T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
I20260709 13:05:37.444219 16960 raft_consensus.cc:3060] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.444375 17007 consensus_queue.cc:260] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:37.445562 17008 raft_consensus.cc:359] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:37.446362 17008 raft_consensus.cc:385] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.446715 17008 raft_consensus.cc:740] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.447122 17007 ts_tablet_manager.cc:1434] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:05:37.447633 17008 consensus_queue.cc:260] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:37.448303 17007 tablet_bootstrap.cc:492] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:37.450193 17008 ts_tablet_manager.cc:1434] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:05:37.451248 17008 tablet_bootstrap.cc:492] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:37.452402 16886 raft_consensus.cc:3060] T ac3ad4fe0bc4435cb1c596ad215889a3 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:37.445755 16849 leader_election.cc:343] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.455981 16849 leader_election.cc:304] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: e2b87453fe654318abccdd1512b949c4
I20260709 13:05:37.456214 17006 tablet_bootstrap.cc:492] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:37.456770 17006 ts_tablet_manager.cc:1403] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.048s	user 0.009s	sys 0.007s
I20260709 13:05:37.457108 17040 raft_consensus.cc:2804] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:37.457551 17040 raft_consensus.cc:493] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.457923 17040 raft_consensus.cc:3060] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.459870 17006 raft_consensus.cc:359] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:37.460546 17006 raft_consensus.cc:385] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.460850 17006 raft_consensus.cc:740] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.461336 16635 catalog_manager.cc:5697] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:37.461802 17006 consensus_queue.cc:260] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:37.464541 17006 ts_tablet_manager.cc:1434] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:05:37.466279 17006 tablet_bootstrap.cc:492] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:37.467283 17007 tablet_bootstrap.cc:654] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.467373 17066 consensus_queue.cc:1048] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:37.469105 17041 consensus_queue.cc:1048] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:37.468624 17008 tablet_bootstrap.cc:654] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.470181 17040 raft_consensus.cc:515] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.473515 17040 leader_election.cc:290] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:37.476392 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e091d4e9a0044dd899e731769a94064c" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:37.477044 16958 raft_consensus.cc:3060] T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.484742 16735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e091d4e9a0044dd899e731769a94064c" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:37.491678 17006 tablet_bootstrap.cc:654] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.494316 16958 raft_consensus.cc:2468] T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:37.493659 16632 catalog_manager.cc:5697] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "c4b86aae3a8f4a9082e639144ec46228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:37.494800 16849 leader_election.cc:343] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.495582 16849 leader_election.cc:304] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: e2b87453fe654318abccdd1512b949c4
I20260709 13:05:37.496863 17040 raft_consensus.cc:2804] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:37.497618 17040 raft_consensus.cc:697] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Becoming Leader. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:37.498644 17040 consensus_queue.cc:237] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.503491 17008 tablet_bootstrap.cc:492] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:37.504096 17008 ts_tablet_manager.cc:1403] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.053s	user 0.009s	sys 0.008s
I20260709 13:05:37.508605 17008 raft_consensus.cc:359] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:37.509310 17008 raft_consensus.cc:385] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.509603 17008 raft_consensus.cc:740] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.510115 17083 consensus_queue.cc:1048] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:37.515504 17041 consensus_queue.cc:1048] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:37.516183 17004 tablet_bootstrap.cc:492] T 10bbce697a734600bb083d47ac877574 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:37.516707 17004 ts_tablet_manager.cc:1403] T 10bbce697a734600bb083d47ac877574 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.090s	user 0.014s	sys 0.006s
I20260709 13:05:37.519460 17008 consensus_queue.cc:260] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:37.522848 17008 ts_tablet_manager.cc:1434] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 13:05:37.523813 17008 tablet_bootstrap.cc:492] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:37.523895 17004 raft_consensus.cc:359] T 10bbce697a734600bb083d47ac877574 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.524639 17004 raft_consensus.cc:385] T 10bbce697a734600bb083d47ac877574 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.524948 17004 raft_consensus.cc:740] T 10bbce697a734600bb083d47ac877574 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.525647 17004 consensus_queue.cc:260] T 10bbce697a734600bb083d47ac877574 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.527654 17004 ts_tablet_manager.cc:1434] T 10bbce697a734600bb083d47ac877574 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:05:37.528491 17004 tablet_bootstrap.cc:492] T 612e472fbe674788986798c65836388f P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:37.538019 17007 tablet_bootstrap.cc:492] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:37.538771 17007 ts_tablet_manager.cc:1403] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.091s	user 0.016s	sys 0.003s
I20260709 13:05:37.545509 16811 raft_consensus.cc:3060] T 535cac5e21bf41db9c037f226ebcd94e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.544505 17007 raft_consensus.cc:359] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.551018 17007 raft_consensus.cc:385] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.551292 17007 raft_consensus.cc:740] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.553395 17007 consensus_queue.cc:260] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.554013 17008 tablet_bootstrap.cc:654] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:37.556519 16923 consensus_peers.cc:597] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:37.557603 17007 ts_tablet_manager.cc:1434] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.004s
I20260709 13:05:37.558581 17007 tablet_bootstrap.cc:492] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:37.559154 17004 tablet_bootstrap.cc:654] T 612e472fbe674788986798c65836388f P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.575845 17007 tablet_bootstrap.cc:654] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:37.593277 16700 consensus_peers.cc:597] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 -> Peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Couldn't send request to peer c4b86aae3a8f4a9082e639144ec46228. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:37.597826 17066 consensus_queue.cc:1048] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:37.616943 17007 tablet_bootstrap.cc:492] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:37.617530 17007 ts_tablet_manager.cc:1403] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.059s	user 0.008s	sys 0.004s
I20260709 13:05:37.617534 17066 consensus_queue.cc:1048] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:05:37.622251 17007 raft_consensus.cc:359] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:37.622943 17007 raft_consensus.cc:385] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.623214 17007 raft_consensus.cc:740] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.624553 17008 tablet_bootstrap.cc:492] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:37.624984 17008 ts_tablet_manager.cc:1403] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.101s	user 0.010s	sys 0.003s
I20260709 13:05:37.617875 16632 catalog_manager.cc:5697] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3). New cstate: current_term: 1 leader_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:37.626607 17007 consensus_queue.cc:260] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:37.628770 17007 ts_tablet_manager.cc:1434] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:05:37.630079 17004 tablet_bootstrap.cc:492] T 612e472fbe674788986798c65836388f P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:37.631697 17006 tablet_bootstrap.cc:492] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:37.632176 17006 ts_tablet_manager.cc:1403] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.166s	user 0.012s	sys 0.030s
I20260709 13:05:37.635203 17004 ts_tablet_manager.cc:1403] T 612e472fbe674788986798c65836388f P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.107s	user 0.015s	sys 0.004s
I20260709 13:05:37.635526 17006 raft_consensus.cc:359] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:37.636423 17006 raft_consensus.cc:385] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.636736 17006 raft_consensus.cc:740] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.637601 17006 consensus_queue.cc:260] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:37.637898 17004 raft_consensus.cc:359] T 612e472fbe674788986798c65836388f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.638618 17004 raft_consensus.cc:385] T 612e472fbe674788986798c65836388f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.638911 17004 raft_consensus.cc:740] T 612e472fbe674788986798c65836388f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.630245 17007 tablet_bootstrap.cc:492] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:37.642086 17006 ts_tablet_manager.cc:1434] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:05:37.639828 17004 consensus_queue.cc:260] T 612e472fbe674788986798c65836388f P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.648718 16735 raft_consensus.cc:3060] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.655342 17006 tablet_bootstrap.cc:492] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:37.657470 17083 raft_consensus.cc:493] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.658161 17083 raft_consensus.cc:515] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:37.662621 17083 leader_election.cc:290] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:37.662868 17004 ts_tablet_manager.cc:1434] T 612e472fbe674788986798c65836388f P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:05:37.663257 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b9b28c4c5d4ddaa6d1ae05ce225b11" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:37.663475 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b9b28c4c5d4ddaa6d1ae05ce225b11" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:37.664304 16886 raft_consensus.cc:2393] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2b87453fe654318abccdd1512b949c4 in current term 1: Already voted for candidate c4b86aae3a8f4a9082e639144ec46228 in this term.
I20260709 13:05:37.664676 17004 tablet_bootstrap.cc:492] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:37.667899 16700 leader_election.cc:304] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:37.669682 17083 raft_consensus.cc:3060] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.651520 17008 raft_consensus.cc:359] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:37.679762 17008 raft_consensus.cc:385] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.680152 17008 raft_consensus.cc:740] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.680945 17008 consensus_queue.cc:260] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:37.681747 17004 tablet_bootstrap.cc:654] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.683334 17008 ts_tablet_manager.cc:1434] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.058s	user 0.007s	sys 0.000s
I20260709 13:05:37.684446 17008 tablet_bootstrap.cc:492] T 5857fad96a1045d7b85190e64da77bce P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:37.684525 17083 raft_consensus.cc:2749] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:37.687101 17006 tablet_bootstrap.cc:654] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.689254 17062 raft_consensus.cc:493] T 612e472fbe674788986798c65836388f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.689937 17062 raft_consensus.cc:515] T 612e472fbe674788986798c65836388f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.691543 17083 raft_consensus.cc:493] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.694119 17083 raft_consensus.cc:515] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:37.696269 17078 raft_consensus.cc:493] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.699385 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612e472fbe674788986798c65836388f" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:37.703769 17078 raft_consensus.cc:515] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:37.695534 17062 leader_election.cc:290] T 612e472fbe674788986798c65836388f P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:37.705276 17062 raft_consensus.cc:493] T 10bbce697a734600bb083d47ac877574 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.697893 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6288d52217814c7393ff0ec66da3b099" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:37.705850 17062 raft_consensus.cc:515] T 10bbce697a734600bb083d47ac877574 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.697863 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6288d52217814c7393ff0ec66da3b099" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
W20260709 13:05:37.708830 16923 leader_election.cc:343] T 612e472fbe674788986798c65836388f P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.709364 17078 leader_election.cc:290] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:37.710817 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10bbce697a734600bb083d47ac877574" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:37.711009 16700 leader_election.cc:304] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228
I20260709 13:05:37.711779 17008 tablet_bootstrap.cc:654] T 5857fad96a1045d7b85190e64da77bce P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.711951 17130 raft_consensus.cc:493] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.712574 17130 raft_consensus.cc:515] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
W20260709 13:05:37.714504 16923 leader_election.cc:343] T 10bbce697a734600bb083d47ac877574 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.714877 17130 leader_election.cc:290] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:37.696538 17083 leader_election.cc:290] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:37.716650 17062 leader_election.cc:290] T 10bbce697a734600bb083d47ac877574 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:37.718003 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10bbce697a734600bb083d47ac877574" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:37.719007 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1660f76b7074e2cb7542ed02dd179d9" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
W20260709 13:05:37.720082 16923 leader_election.cc:343] T 10bbce697a734600bb083d47ac877574 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.720655 16811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e96e3138004b3aa2f3e745f1ddfc04" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:37.720932 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1660f76b7074e2cb7542ed02dd179d9" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:37.721761 16812 raft_consensus.cc:2468] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 0.
I20260709 13:05:37.720819 17021 consensus_queue.cc:1048] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:37.722193 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e96e3138004b3aa2f3e745f1ddfc04" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:37.723117 16958 raft_consensus.cc:2393] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2b87453fe654318abccdd1512b949c4 in current term 1: Already voted for candidate c4b86aae3a8f4a9082e639144ec46228 in this term.
I20260709 13:05:37.715520 17130 raft_consensus.cc:2749] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:37.723265 16923 leader_election.cc:304] T 10bbce697a734600bb083d47ac877574 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4
W20260709 13:05:37.725158 16700 leader_election.cc:343] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0086163b152644feb75caeac6f145533 (127.6.255.4:37199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.729208 17007 tablet_bootstrap.cc:654] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.730149 17021 raft_consensus.cc:493] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.730754 17021 raft_consensus.cc:515] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:37.732877 17021 leader_election.cc:290] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:37.694860 17067 consensus_queue.cc:1048] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.039s
I20260709 13:05:37.734526 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e8e0f9919a9404c9001b6718f102d19" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
W20260709 13:05:37.736122 16775 leader_election.cc:343] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.736009 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e8e0f9919a9404c9001b6718f102d19" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
W20260709 13:05:37.746150 16775 consensus_peers.cc:597] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 -> Peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Couldn't send request to peer d4e1b05bc80e43f7941ece113fb97ab0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:37.738852 17067 raft_consensus.cc:2749] T 10bbce697a734600bb083d47ac877574 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:37.747634 16775 leader_election.cc:343] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0086163b152644feb75caeac6f145533 (127.6.255.4:37199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.748206 16775 leader_election.cc:304] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228; no voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:37.749297 17100 raft_consensus.cc:2749] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:37.751336 16700 leader_election.cc:304] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533
I20260709 13:05:37.753887 16735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612e472fbe674788986798c65836388f" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:37.755985 17083 raft_consensus.cc:2804] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:37.756580 17083 raft_consensus.cc:493] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.757067 17083 raft_consensus.cc:3060] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.761716 17021 consensus_queue.cc:1048] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:37.765275 16700 leader_election.cc:304] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228
I20260709 13:05:37.767092 17007 tablet_bootstrap.cc:492] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
W20260709 13:05:37.767284 16923 leader_election.cc:343] T 612e472fbe674788986798c65836388f P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.767788 17007 ts_tablet_manager.cc:1403] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.138s	user 0.013s	sys 0.000s
I20260709 13:05:37.771807 17130 raft_consensus.cc:3060] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.767879 16923 leader_election.cc:304] T 612e472fbe674788986798c65836388f P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:37.773871 17004 tablet_bootstrap.cc:492] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:37.773808 17062 raft_consensus.cc:2749] T 612e472fbe674788986798c65836388f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:37.774585 17004 ts_tablet_manager.cc:1403] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.110s	user 0.008s	sys 0.010s
I20260709 13:05:37.788347 17078 raft_consensus.cc:493] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.792296 17007 raft_consensus.cc:359] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:37.793056 17007 raft_consensus.cc:385] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.793320 17007 raft_consensus.cc:740] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.788964 17078 raft_consensus.cc:515] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.794149 17007 consensus_queue.cc:260] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:37.794371 17004 raft_consensus.cc:359] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.795018 17004 raft_consensus.cc:385] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.789136 17121 raft_consensus.cc:493] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.795315 17004 raft_consensus.cc:740] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.789121 17021 consensus_queue.cc:1048] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:05:37.796810 17004 consensus_queue.cc:260] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.802953 17004 ts_tablet_manager.cc:1434] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.002s
I20260709 13:05:37.804409 17004 tablet_bootstrap.cc:492] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:37.806303 17121 raft_consensus.cc:515] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:37.811625 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b5161020c944d28a4334dc1d572dac7" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:37.812450 17018 raft_consensus.cc:493] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.812701 17121 leader_election.cc:290] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:37.813056 17018 raft_consensus.cc:515] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:37.814065 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b5161020c944d28a4334dc1d572dac7" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:37.815044 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3850091ca95e47b6a1b52e600a452b0b" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:37.824229 17008 tablet_bootstrap.cc:492] T 5857fad96a1045d7b85190e64da77bce P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:37.824939 17008 ts_tablet_manager.cc:1403] T 5857fad96a1045d7b85190e64da77bce P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.141s	user 0.018s	sys 0.015s
I20260709 13:05:37.815294 16811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3850091ca95e47b6a1b52e600a452b0b" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:37.809160 17078 leader_election.cc:290] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:37.821161 17006 tablet_bootstrap.cc:492] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:37.829975 17006 ts_tablet_manager.cc:1403] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.175s	user 0.013s	sys 0.027s
I20260709 13:05:37.835359 17007 ts_tablet_manager.cc:1434] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.067s	user 0.027s	sys 0.012s
I20260709 13:05:37.836750 17007 tablet_bootstrap.cc:492] T c9043e0bc02745d187e9ca5a5c71274f P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
W20260709 13:05:37.836544 16849 leader_election.cc:343] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0086163b152644feb75caeac6f145533 (127.6.255.4:37199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.831632 16811 raft_consensus.cc:2468] T 3850091ca95e47b6a1b52e600a452b0b P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
I20260709 13:05:37.843923 16849 leader_election.cc:304] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 0086163b152644feb75caeac6f145533
I20260709 13:05:37.845697 17018 leader_election.cc:290] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:37.839993 16700 leader_election.cc:304] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228
I20260709 13:05:37.846355 17006 raft_consensus.cc:359] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.847352 17006 raft_consensus.cc:385] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.844306 17021 consensus_queue.cc:1048] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:05:37.850965 17008 raft_consensus.cc:359] T 5857fad96a1045d7b85190e64da77bce P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:37.851523 17083 raft_consensus.cc:515] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:37.852109 17007 tablet_bootstrap.cc:654] T c9043e0bc02745d187e9ca5a5c71274f P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.852124 17008 raft_consensus.cc:385] T 5857fad96a1045d7b85190e64da77bce P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.852908 17008 raft_consensus.cc:740] T 5857fad96a1045d7b85190e64da77bce P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.854174 17008 consensus_queue.cc:260] T 5857fad96a1045d7b85190e64da77bce P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:37.847536 17006 raft_consensus.cc:740] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.859541 17006 consensus_queue.cc:260] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.864046 17006 ts_tablet_manager.cc:1434] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.001s
I20260709 13:05:37.865264 17006 tablet_bootstrap.cc:492] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:37.846338 17121 raft_consensus.cc:2804] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:37.871402 17008 ts_tablet_manager.cc:1434] T 5857fad96a1045d7b85190e64da77bce P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.002s
I20260709 13:05:37.870200 17121 raft_consensus.cc:493] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.877831 16811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5857fad96a1045d7b85190e64da77bce" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:37.902376 17121 raft_consensus.cc:3060] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.884197 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5857fad96a1045d7b85190e64da77bce" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:37.902915 16811 raft_consensus.cc:2468] T 5857fad96a1045d7b85190e64da77bce P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
I20260709 13:05:37.884455 17006 tablet_bootstrap.cc:654] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.875145 17138 raft_consensus.cc:2749] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:37.891405 17021 consensus_queue.cc:1048] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:05:37.874993 17004 tablet_bootstrap.cc:654] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.874765 17083 leader_election.cc:290] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
W20260709 13:05:37.904112 16849 leader_election.cc:343] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0086163b152644feb75caeac6f145533 (127.6.255.4:37199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.887233 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1660f76b7074e2cb7542ed02dd179d9" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:37.902576 17008 tablet_bootstrap.cc:492] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:37.915316 16812 raft_consensus.cc:3060] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.879534 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1660f76b7074e2cb7542ed02dd179d9" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:37.875963 17130 raft_consensus.cc:2749] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:37.919608 17136 raft_consensus.cc:493] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.906755 16849 leader_election.cc:304] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 0086163b152644feb75caeac6f145533
I20260709 13:05:37.921320 17066 raft_consensus.cc:493] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.933372 17008 tablet_bootstrap.cc:654] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.934556 17018 raft_consensus.cc:2804] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:37.943588 17018 raft_consensus.cc:493] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.943989 17018 raft_consensus.cc:3060] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.935932 17066 raft_consensus.cc:515] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.938165 17007 tablet_bootstrap.cc:492] T c9043e0bc02745d187e9ca5a5c71274f P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:37.949661 17007 ts_tablet_manager.cc:1403] T c9043e0bc02745d187e9ca5a5c71274f P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.113s	user 0.012s	sys 0.007s
W20260709 13:05:37.942485 16700 leader_election.cc:343] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0086163b152644feb75caeac6f145533 (127.6.255.4:37199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.933946 17136 raft_consensus.cc:515] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:37.954888 17007 raft_consensus.cc:359] T c9043e0bc02745d187e9ca5a5c71274f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.960196 17007 raft_consensus.cc:385] T c9043e0bc02745d187e9ca5a5c71274f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.960600 17007 raft_consensus.cc:740] T c9043e0bc02745d187e9ca5a5c71274f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.961761 17136 leader_election.cc:290] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:37.961628 17007 consensus_queue.cc:260] T c9043e0bc02745d187e9ca5a5c71274f P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:37.963668 16734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213e9771c53e4498b315bb43fdc0aa9a" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:37.972203 17007 ts_tablet_manager.cc:1434] T c9043e0bc02745d187e9ca5a5c71274f P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.022s	user 0.010s	sys 0.000s
I20260709 13:05:37.975950 17066 leader_election.cc:290] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:37.979549 17007 tablet_bootstrap.cc:492] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:37.985535 16811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213e9771c53e4498b315bb43fdc0aa9a" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:37.971362 17041 raft_consensus.cc:493] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.967789 17021 consensus_queue.cc:1048] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 13:05:37.996271 17041 raft_consensus.cc:515] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:38.000157 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9043e0bc02745d187e9ca5a5c71274f" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:38.005530 16884 raft_consensus.cc:2468] T c9043e0bc02745d187e9ca5a5c71274f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 0.
I20260709 13:05:38.006366 17041 leader_election.cc:290] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:38.007364 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9043e0bc02745d187e9ca5a5c71274f" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:38.008133 16737 raft_consensus.cc:2468] T c9043e0bc02745d187e9ca5a5c71274f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 0.
I20260709 13:05:38.009012 16733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dd082201fe04ef89c842b6404fd522e" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:38.000003 17040 raft_consensus.cc:493] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.992709 16734 raft_consensus.cc:2468] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 0.
W20260709 13:05:38.017107 16923 leader_election.cc:343] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.015089 17040 raft_consensus.cc:515] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:38.031072 17040 leader_election.cc:290] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:38.033134 16923 leader_election.cc:304] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
W20260709 13:05:38.033774 16849 leader_election.cc:343] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.034643 17066 raft_consensus.cc:2804] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:38.035082 17066 raft_consensus.cc:493] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.035419 17066 raft_consensus.cc:3060] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.042608 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10bbce697a734600bb083d47ac877574" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:38.043826 16811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dd082201fe04ef89c842b6404fd522e" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:38.045239 17121 raft_consensus.cc:515] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:38.046170 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10bbce697a734600bb083d47ac877574" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:38.046821 16960 raft_consensus.cc:2468] T 10bbce697a734600bb083d47ac877574 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
I20260709 13:05:38.048125 17150 consensus_queue.cc:1048] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:05:38.050436 16849 leader_election.cc:343] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.050954 16849 leader_election.cc:304] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:38.051196 17121 leader_election.cc:290] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:38.045387 16923 leader_election.cc:304] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228
I20260709 13:05:38.052109 17136 raft_consensus.cc:2804] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:05:38.052338 16683 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:38.052520 17136 raft_consensus.cc:493] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.052332 16849 leader_election.cc:304] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:38.053088 17136 raft_consensus.cc:3060] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.047263 17018 raft_consensus.cc:515] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:38.055636 17121 raft_consensus.cc:2749] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:38.057600 17040 raft_consensus.cc:2804] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:38.062018 17040 raft_consensus.cc:493] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.062090 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3850091ca95e47b6a1b52e600a452b0b" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:38.062489 17040 raft_consensus.cc:3060] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.068117 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5857fad96a1045d7b85190e64da77bce" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:38.068940 16810 raft_consensus.cc:3060] T 5857fad96a1045d7b85190e64da77bce P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.069352 17040 raft_consensus.cc:515] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:38.071508 17040 leader_election.cc:290] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:38.076246 17066 raft_consensus.cc:515] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:38.078235 17138 consensus_queue.cc:1048] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:38.070626 17136 raft_consensus.cc:515] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:38.058290 16811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3850091ca95e47b6a1b52e600a452b0b" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:38.093451 17004 tablet_bootstrap.cc:492] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:38.061474 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5857fad96a1045d7b85190e64da77bce" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
W20260709 13:05:38.059070 16849 leader_election.cc:343] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.096629 17004 ts_tablet_manager.cc:1403] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.293s	user 0.010s	sys 0.019s
I20260709 13:05:38.058208 17018 leader_election.cc:290] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:38.095909 16812 raft_consensus.cc:2468] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 1.
I20260709 13:05:38.101752 17008 tablet_bootstrap.cc:492] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:38.102269 17008 ts_tablet_manager.cc:1403] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.200s	user 0.015s	sys 0.007s
I20260709 13:05:38.102716 16885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9043e0bc02745d187e9ca5a5c71274f" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:38.103376 16885 raft_consensus.cc:3060] T c9043e0bc02745d187e9ca5a5c71274f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.097872 16811 raft_consensus.cc:3060] T 3850091ca95e47b6a1b52e600a452b0b P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.104923 17008 raft_consensus.cc:359] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:38.111028 17008 raft_consensus.cc:385] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.111328 17008 raft_consensus.cc:740] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
W20260709 13:05:38.107664 17146 log.cc:927] Time spent T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P d4e1b05bc80e43f7941ece113fb97ab0: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:05:38.110502 16810 raft_consensus.cc:2468] T 5857fad96a1045d7b85190e64da77bce P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
W20260709 13:05:38.116575 16849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:40785, user_credentials={real_user=slave}} blocked reactor thread for 57538.3us
I20260709 13:05:38.116395 17008 consensus_queue.cc:260] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:38.086048 17066 leader_election.cc:290] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:38.107446 17004 raft_consensus.cc:359] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:38.120848 17004 raft_consensus.cc:385] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.121671 17004 raft_consensus.cc:740] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.122166 17006 tablet_bootstrap.cc:492] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:38.122713 17006 ts_tablet_manager.cc:1403] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.258s	user 0.030s	sys 0.016s
I20260709 13:05:38.125515 16885 raft_consensus.cc:2468] T c9043e0bc02745d187e9ca5a5c71274f P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 1.
I20260709 13:05:38.126394 17008 ts_tablet_manager.cc:1434] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260709 13:05:38.128280 17008 tablet_bootstrap.cc:492] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:38.122669 17004 consensus_queue.cc:260] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:38.119515 16811 raft_consensus.cc:2468] T 3850091ca95e47b6a1b52e600a452b0b P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:38.132525 17004 ts_tablet_manager.cc:1434] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
W20260709 13:05:38.127169 16700 consensus_peers.cc:597] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 -> Peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Couldn't send request to peer d4e1b05bc80e43f7941ece113fb97ab0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:38.134346 17004 tablet_bootstrap.cc:492] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:38.132084 16923 leader_election.cc:304] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:38.136354 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10bbce697a734600bb083d47ac877574" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:38.136977 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9043e0bc02745d187e9ca5a5c71274f" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:38.137707 16736 raft_consensus.cc:3060] T c9043e0bc02745d187e9ca5a5c71274f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.141242 17066 raft_consensus.cc:2804] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:38.137053 16958 raft_consensus.cc:3060] T 10bbce697a734600bb083d47ac877574 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.144574 17006 raft_consensus.cc:359] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:38.145298 17006 raft_consensus.cc:385] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.147691 17006 raft_consensus.cc:740] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.145605 17066 raft_consensus.cc:697] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 [term 1 LEADER]: Becoming Leader. State: Replica: 0086163b152644feb75caeac6f145533, State: Running, Role: LEADER
I20260709 13:05:38.147002 16731 raft_consensus.cc:3060] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.147188 16811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213e9771c53e4498b315bb43fdc0aa9a" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:38.148360 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213e9771c53e4498b315bb43fdc0aa9a" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:38.157446 17066 consensus_queue.cc:237] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:38.158393 16736 raft_consensus.cc:2468] T c9043e0bc02745d187e9ca5a5c71274f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 1.
I20260709 13:05:38.148424 17006 consensus_queue.cc:260] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:38.141693 17136 leader_election.cc:290] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:38.166608 17006 ts_tablet_manager.cc:1434] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.043s	user 0.004s	sys 0.003s
I20260709 13:05:38.164750 16700 leader_election.cc:304] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533
I20260709 13:05:38.164680 16732 raft_consensus.cc:3060] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.170174 17138 raft_consensus.cc:2804] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:05:38.170764 16923 leader_election.cc:343] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.170969 17138 raft_consensus.cc:697] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:05:38.171782 16849 leader_election.cc:304] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:38.172325 17138 consensus_queue.cc:237] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:38.173879 17151 raft_consensus.cc:2804] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:38.174096 17021 raft_consensus.cc:493] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.174500 17151 raft_consensus.cc:697] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Becoming Leader. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:38.174646 17021 raft_consensus.cc:515] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:38.175485 17151 consensus_queue.cc:237] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:38.178419 17007 tablet_bootstrap.cc:654] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.181023 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82d38de8d2c47b38ca8811691c68d8f" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:38.182144 16958 raft_consensus.cc:2468] T 10bbce697a734600bb083d47ac877574 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:38.177817 17021 leader_election.cc:290] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:38.187178 16732 raft_consensus.cc:2468] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 1.
I20260709 13:05:38.189594 16849 leader_election.cc:304] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:38.190794 17121 raft_consensus.cc:2804] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:38.191470 17121 raft_consensus.cc:697] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Becoming Leader. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:38.192390 17121 consensus_queue.cc:237] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
W20260709 13:05:38.190842 16849 leader_election.cc:343] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0086163b152644feb75caeac6f145533 (127.6.255.4:37199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.198004 17006 tablet_bootstrap.cc:492] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:38.209003 17006 tablet_bootstrap.cc:654] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.211318 16730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82d38de8d2c47b38ca8811691c68d8f" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:38.211642 16923 leader_election.cc:304] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228
I20260709 13:05:38.213303 16811 raft_consensus.cc:3060] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.214241 17008 tablet_bootstrap.cc:654] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.216454 17136 raft_consensus.cc:2804] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:38.217348 17136 raft_consensus.cc:697] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 [term 1 LEADER]: Becoming Leader. State: Replica: 0086163b152644feb75caeac6f145533, State: Running, Role: LEADER
I20260709 13:05:38.239367 16775 leader_election.cc:304] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228; no voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4
W20260709 13:05:38.236394 16849 leader_election.cc:343] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0086163b152644feb75caeac6f145533 (127.6.255.4:37199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.273906 17004 tablet_bootstrap.cc:654] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.276669 16635 catalog_manager.cc:5697] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0086163b152644feb75caeac6f145533 (127.6.255.4). New cstate: current_term: 1 leader_uuid: "0086163b152644feb75caeac6f145533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:38.283223 16849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:37199, user_credentials={real_user=slave}} blocked reactor thread for 57222.4us
I20260709 13:05:38.284065 16849 leader_election.cc:304] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:38.268837 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10bbce697a734600bb083d47ac877574" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
W20260709 13:05:38.309077 17160 log.cc:927] Time spent T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:05:38.309847 17152 raft_consensus.cc:2749] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:38.318377 17136 consensus_queue.cc:237] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
W20260709 13:05:38.327864 16849 leader_election.cc:343] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:38.329025 16849 consensus_peers.cc:597] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:38.320590 17121 raft_consensus.cc:2804] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:38.336473 17140 raft_consensus.cc:493] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.337152 17140 raft_consensus.cc:515] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:38.338831 17008 tablet_bootstrap.cc:492] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:38.339315 17008 ts_tablet_manager.cc:1403] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.211s	user 0.014s	sys 0.022s
I20260709 13:05:38.338351 17121 raft_consensus.cc:697] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Becoming Leader. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:38.347520 17121 consensus_queue.cc:237] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:38.351678 17008 raft_consensus.cc:359] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:38.352425 17008 raft_consensus.cc:385] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.352682 17008 raft_consensus.cc:740] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.350046 17151 raft_consensus.cc:493] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.353871 17151 raft_consensus.cc:515] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:38.358537 16634 catalog_manager.cc:5697] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:38.359543 17007 tablet_bootstrap.cc:492] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:38.360351 17007 ts_tablet_manager.cc:1403] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.387s	user 0.039s	sys 0.033s
I20260709 13:05:38.361742 17151 leader_election.cc:290] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:38.363910 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ff2290a9c9451c9a786eacf7380d90" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:38.364567 16810 raft_consensus.cc:2468] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
I20260709 13:05:38.366093 17008 consensus_queue.cc:260] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:38.371342 17007 raft_consensus.cc:359] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:38.371937 17008 ts_tablet_manager.cc:1434] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.001s
I20260709 13:05:38.372327 17007 raft_consensus.cc:385] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.372756 17007 raft_consensus.cc:740] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.366523 17140 leader_election.cc:290] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:38.368083 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ff2290a9c9451c9a786eacf7380d90" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:38.368481 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d6e27a9a96c4121b2abc71e0d7738be" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:38.359493 17040 consensus_queue.cc:1048] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:05:38.368083 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d6e27a9a96c4121b2abc71e0d7738be" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:38.369664 16849 leader_election.cc:304] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:38.374250 17008 tablet_bootstrap.cc:492] T 213e9771c53e4498b315bb43fdc0aa9a P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:38.376612 17007 consensus_queue.cc:260] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:38.380102 17041 raft_consensus.cc:2804] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:38.380492 17041 raft_consensus.cc:493] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:38.380785 16700 leader_election.cc:343] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.376080 16810 raft_consensus.cc:2468] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 0.
I20260709 13:05:38.375844 16960 raft_consensus.cc:2468] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
I20260709 13:05:38.383131 16700 leader_election.cc:304] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:38.384725 17140 raft_consensus.cc:2804] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:38.385146 17140 raft_consensus.cc:493] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.385538 17140 raft_consensus.cc:3060] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.385706 17007 ts_tablet_manager.cc:1434] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.000s
I20260709 13:05:38.380841 17041 raft_consensus.cc:3060] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.386894 17007 tablet_bootstrap.cc:492] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:38.387882 17006 tablet_bootstrap.cc:492] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:38.388418 17006 ts_tablet_manager.cc:1403] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.191s	user 0.025s	sys 0.023s
I20260709 13:05:38.385989 17004 tablet_bootstrap.cc:492] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:38.390496 17004 ts_tablet_manager.cc:1403] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.256s	user 0.015s	sys 0.004s
I20260709 13:05:38.400051 17007 tablet_bootstrap.cc:654] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.406870 17140 raft_consensus.cc:515] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:38.410169 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d6e27a9a96c4121b2abc71e0d7738be" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:38.410825 16810 raft_consensus.cc:3060] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.411860 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d6e27a9a96c4121b2abc71e0d7738be" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
W20260709 13:05:38.413666 16700 leader_election.cc:343] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.414285 17140 leader_election.cc:290] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:38.418222 17004 raft_consensus.cc:359] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:38.419307 17004 raft_consensus.cc:385] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.419697 17004 raft_consensus.cc:740] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.420575 17004 consensus_queue.cc:260] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:38.423342 17004 ts_tablet_manager.cc:1434] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 13:05:38.424355 17004 tablet_bootstrap.cc:492] T 9592a31e7ffd407a959c0a3d96c0f8f9 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:38.425899 17008 tablet_bootstrap.cc:654] T 213e9771c53e4498b315bb43fdc0aa9a P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.430732 17006 raft_consensus.cc:359] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:38.432241 17006 raft_consensus.cc:385] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.432550 17006 raft_consensus.cc:740] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.433275 17006 consensus_queue.cc:260] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:38.434695 16810 raft_consensus.cc:2468] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 1.
I20260709 13:05:38.435755 17006 ts_tablet_manager.cc:1434] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.047s	user 0.008s	sys 0.000s
I20260709 13:05:38.436172 17004 tablet_bootstrap.cc:654] T 9592a31e7ffd407a959c0a3d96c0f8f9 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.435985 16700 leader_election.cc:304] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:38.436887 17006 tablet_bootstrap.cc:492] T 0b8add4b17084a6189509807b7e663e5 P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:38.438059 17140 raft_consensus.cc:2804] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:38.438972 17140 raft_consensus.cc:697] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:05:38.439900 17140 consensus_queue.cc:237] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:38.446281 17008 tablet_bootstrap.cc:492] T 213e9771c53e4498b315bb43fdc0aa9a P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:38.446985 17008 ts_tablet_manager.cc:1403] T 213e9771c53e4498b315bb43fdc0aa9a P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.073s	user 0.013s	sys 0.003s
I20260709 13:05:38.500980 17004 tablet_bootstrap.cc:492] T 9592a31e7ffd407a959c0a3d96c0f8f9 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:38.503829 17004 ts_tablet_manager.cc:1403] T 9592a31e7ffd407a959c0a3d96c0f8f9 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.080s	user 0.010s	sys 0.006s
I20260709 13:05:38.446313 17041 raft_consensus.cc:515] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:38.513002 17004 raft_consensus.cc:359] T 9592a31e7ffd407a959c0a3d96c0f8f9 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:38.516131 17041 leader_election.cc:290] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:38.520609 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ff2290a9c9451c9a786eacf7380d90" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:38.524619 17006 tablet_bootstrap.cc:654] T 0b8add4b17084a6189509807b7e663e5 P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.514407 17004 raft_consensus.cc:385] T 9592a31e7ffd407a959c0a3d96c0f8f9 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.514381 17008 raft_consensus.cc:359] T 213e9771c53e4498b315bb43fdc0aa9a P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:38.526429 17008 raft_consensus.cc:385] T 213e9771c53e4498b315bb43fdc0aa9a P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.531291 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ff2290a9c9451c9a786eacf7380d90" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:38.532281 16959 raft_consensus.cc:3060] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.538547 17152 raft_consensus.cc:493] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.526264 17004 raft_consensus.cc:740] T 9592a31e7ffd407a959c0a3d96c0f8f9 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.526791 17008 raft_consensus.cc:740] T 213e9771c53e4498b315bb43fdc0aa9a P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.524490 17066 raft_consensus.cc:493] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.544799 16810 raft_consensus.cc:3060] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:38.529234 17174 log.cc:927] Time spent T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533: Append to log took a long time: real 0.080s	user 0.002s	sys 0.000s
I20260709 13:05:38.547441 17152 raft_consensus.cc:515] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:38.554280 17007 tablet_bootstrap.cc:492] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:38.553000 17004 consensus_queue.cc:260] T 9592a31e7ffd407a959c0a3d96c0f8f9 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:38.561187 17008 consensus_queue.cc:260] T 213e9771c53e4498b315bb43fdc0aa9a P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:38.569029 17007 ts_tablet_manager.cc:1403] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.182s	user 0.014s	sys 0.003s
I20260709 13:05:38.569110 17152 leader_election.cc:290] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:38.572790 17008 ts_tablet_manager.cc:1434] T 213e9771c53e4498b315bb43fdc0aa9a P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.125s	user 0.010s	sys 0.000s
I20260709 13:05:38.573307 16957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9592a31e7ffd407a959c0a3d96c0f8f9" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:38.574399 16957 raft_consensus.cc:2468] T 9592a31e7ffd407a959c0a3d96c0f8f9 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 0.
I20260709 13:05:38.574913 17007 raft_consensus.cc:359] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:38.575784 17008 tablet_bootstrap.cc:492] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:38.575984 17007 raft_consensus.cc:385] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.576426 17007 raft_consensus.cc:740] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.582665 17004 ts_tablet_manager.cc:1434] T 9592a31e7ffd407a959c0a3d96c0f8f9 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.078s	user 0.010s	sys 0.004s
I20260709 13:05:38.584476 17004 tablet_bootstrap.cc:492] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:38.584352 16775 leader_election.cc:304] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:05:38.588855 17008 tablet_bootstrap.cc:654] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.593271 17007 consensus_queue.cc:260] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:38.595263 17007 ts_tablet_manager.cc:1434] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.003s
I20260709 13:05:38.596966 17007 tablet_bootstrap.cc:492] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:38.597157 17152 raft_consensus.cc:2804] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:38.597626 17152 raft_consensus.cc:493] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.597965 17152 raft_consensus.cc:3060] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.599663 16959 raft_consensus.cc:2468] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:38.601380 16849 leader_election.cc:304] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:38.603050 17121 raft_consensus.cc:2804] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:38.605017 17121 raft_consensus.cc:697] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Becoming Leader. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:38.606972 17121 consensus_queue.cc:237] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:38.611956 16731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9592a31e7ffd407a959c0a3d96c0f8f9" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:38.561678 17066 raft_consensus.cc:515] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:38.619879 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1660f76b7074e2cb7542ed02dd179d9" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:38.621266 17066 leader_election.cc:290] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:38.621563 17007 tablet_bootstrap.cc:654] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.624733 17004 tablet_bootstrap.cc:654] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.641901 17041 consensus_queue.cc:1048] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:38.649854 17062 consensus_queue.cc:1048] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:38.651039 16958 raft_consensus.cc:3060] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.648928 16633 catalog_manager.cc:5697] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:38.632270 16632 catalog_manager.cc:5697] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3). New cstate: current_term: 1 leader_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:38.684754 17008 tablet_bootstrap.cc:492] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:38.685278 17008 ts_tablet_manager.cc:1403] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.110s	user 0.011s	sys 0.007s
I20260709 13:05:38.685982 17021 raft_consensus.cc:493] T 213e9771c53e4498b315bb43fdc0aa9a P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.702673 17006 tablet_bootstrap.cc:492] T 0b8add4b17084a6189509807b7e663e5 P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:38.705677 17006 ts_tablet_manager.cc:1403] T 0b8add4b17084a6189509807b7e663e5 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.269s	user 0.018s	sys 0.006s
I20260709 13:05:38.707108 17152 raft_consensus.cc:515] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:38.709517 17152 leader_election.cc:290] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
W20260709 13:05:38.713145 16775 leader_election.cc:343] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.713416 16809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1660f76b7074e2cb7542ed02dd179d9" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:38.714380 16809 raft_consensus.cc:2393] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0086163b152644feb75caeac6f145533 in current term 1: Already voted for candidate e2b87453fe654318abccdd1512b949c4 in this term.
I20260709 13:05:38.715677 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9592a31e7ffd407a959c0a3d96c0f8f9" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:38.716614 16960 raft_consensus.cc:3060] T 9592a31e7ffd407a959c0a3d96c0f8f9 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.728101 16923 leader_election.cc:304] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:38.712162 17021 raft_consensus.cc:515] T 213e9771c53e4498b315bb43fdc0aa9a P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:38.728915 16632 catalog_manager.cc:5697] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3). New cstate: current_term: 1 leader_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:38.731451 17021 leader_election.cc:290] T 213e9771c53e4498b315bb43fdc0aa9a P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:38.726630 17066 consensus_queue.cc:1048] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:05:38.732969 17008 raft_consensus.cc:359] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:38.734237 17008 raft_consensus.cc:385] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.734668 17008 raft_consensus.cc:740] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.741263 16810 raft_consensus.cc:2468] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:38.709241 17006 raft_consensus.cc:359] T 0b8add4b17084a6189509807b7e663e5 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:38.746737 17006 raft_consensus.cc:385] T 0b8add4b17084a6189509807b7e663e5 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.746182 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9592a31e7ffd407a959c0a3d96c0f8f9" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:38.747078 17006 raft_consensus.cc:740] T 0b8add4b17084a6189509807b7e663e5 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.755419 17006 consensus_queue.cc:260] T 0b8add4b17084a6189509807b7e663e5 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:38.757284 17008 consensus_queue.cc:260] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:38.760393 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213e9771c53e4498b315bb43fdc0aa9a" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:38.768221 16960 raft_consensus.cc:2468] T 9592a31e7ffd407a959c0a3d96c0f8f9 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:38.768795 17140 consensus_queue.cc:1048] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:38.769172 17004 tablet_bootstrap.cc:492] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:38.769771 17004 ts_tablet_manager.cc:1403] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.186s	user 0.013s	sys 0.015s
I20260709 13:05:38.768955 16632 catalog_manager.cc:5697] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3). New cstate: current_term: 1 leader_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:38.762652 17021 raft_consensus.cc:493] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.773010 17021 raft_consensus.cc:515] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:38.781754 17008 ts_tablet_manager.cc:1434] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.096s	user 0.009s	sys 0.000s
I20260709 13:05:38.786545 17140 raft_consensus.cc:493] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.787091 17140 raft_consensus.cc:515] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:38.787577 17004 raft_consensus.cc:359] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:38.788411 17004 raft_consensus.cc:385] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.788738 17004 raft_consensus.cc:740] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.789686 16731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213e9771c53e4498b315bb43fdc0aa9a" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:38.789623 17004 consensus_queue.cc:260] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:38.790609 16731 raft_consensus.cc:2393] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c4b86aae3a8f4a9082e639144ec46228 in current term 1: Already voted for candidate 0086163b152644feb75caeac6f145533 in this term.
I20260709 13:05:38.792236 17004 ts_tablet_manager.cc:1434] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.002s
I20260709 13:05:38.780511 17021 leader_election.cc:290] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:38.794673 17007 tablet_bootstrap.cc:492] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:38.795257 17007 ts_tablet_manager.cc:1403] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.199s	user 0.024s	sys 0.025s
I20260709 13:05:38.792630 17140 leader_election.cc:290] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
W20260709 13:05:38.800554 16849 consensus_peers.cc:597] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer 0086163b152644feb75caeac6f145533 (127.6.255.4:37199): Couldn't send request to peer 0086163b152644feb75caeac6f145533. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:38.806743 17067 raft_consensus.cc:2749] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:38.809068 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:38.814008 17006 ts_tablet_manager.cc:1434] T 0b8add4b17084a6189509807b7e663e5 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.108s	user 0.007s	sys 0.001s
W20260709 13:05:38.814280 16849 consensus_peers.cc:597] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer 0086163b152644feb75caeac6f145533 (127.6.255.4:37199): Couldn't send request to peer 0086163b152644feb75caeac6f145533. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:38.815583 17006 tablet_bootstrap.cc:492] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:38.826467 17008 tablet_bootstrap.cc:492] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
W20260709 13:05:38.815452 16700 leader_election.cc:343] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.834794 17007 raft_consensus.cc:359] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:38.837703 17007 raft_consensus.cc:385] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.838019 17007 raft_consensus.cc:740] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.842054 17006 tablet_bootstrap.cc:654] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.842296 17007 consensus_queue.cc:260] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:38.829860 16811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:38.849879 17018 consensus_queue.cc:1048] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:05:38.850729 17040 raft_consensus.cc:493] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.851392 17040 raft_consensus.cc:515] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:38.852840 16811 raft_consensus.cc:2468] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 0.
I20260709 13:05:38.855221 16808 raft_consensus.cc:3060] T 213e9771c53e4498b315bb43fdc0aa9a P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.845981 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:38.867856 17040 leader_election.cc:290] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:38.868963 17004 tablet_bootstrap.cc:492] T 3850091ca95e47b6a1b52e600a452b0b P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:38.873045 16809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d6e27a9a96c4121b2abc71e0d7738be" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:38.882531 16809 raft_consensus.cc:2393] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4e1b05bc80e43f7941ece113fb97ab0 in current term 1: Already voted for candidate e2b87453fe654318abccdd1512b949c4 in this term.
I20260709 13:05:38.874828 17007 ts_tablet_manager.cc:1434] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.079s	user 0.006s	sys 0.000s
I20260709 13:05:38.875545 17008 tablet_bootstrap.cc:654] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.876893 17140 consensus_queue.cc:1048] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:38.879770 17138 raft_consensus.cc:493] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.873942 16775 leader_election.cc:304] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:05:38.900235 17021 raft_consensus.cc:2804] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:05:38.900420 16775 leader_election.cc:343] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.900866 17021 raft_consensus.cc:697] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Becoming Leader. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:05:38.901657 16775 leader_election.cc:304] T 213e9771c53e4498b315bb43fdc0aa9a P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228; no voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:38.898763 17138 raft_consensus.cc:515] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:38.902004 17021 consensus_queue.cc:237] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:38.905000 16633 catalog_manager.cc:5697] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0086163b152644feb75caeac6f145533 (127.6.255.4). New cstate: current_term: 1 leader_uuid: "0086163b152644feb75caeac6f145533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:38.905964 16775 leader_election.cc:343] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.921139 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d6e27a9a96c4121b2abc71e0d7738be" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:38.930361 16884 raft_consensus.cc:3060] T 0b8add4b17084a6189509807b7e663e5 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.930749 17138 leader_election.cc:290] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:38.935987 17150 raft_consensus.cc:2749] T 213e9771c53e4498b315bb43fdc0aa9a P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:38.936489 17007 tablet_bootstrap.cc:492] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:38.936966 17067 consensus_queue.cc:1048] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:38.945852 17041 consensus_queue.cc:1048] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:38.955636 16635 catalog_manager.cc:5697] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3). New cstate: current_term: 1 leader_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:38.974112 16732 raft_consensus.cc:3060] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.982300 17004 tablet_bootstrap.cc:654] T 3850091ca95e47b6a1b52e600a452b0b P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.984598 16731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:38.985351 16731 raft_consensus.cc:2468] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 0.
I20260709 13:05:38.993042 17007 tablet_bootstrap.cc:654] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.002985 16849 leader_election.cc:304] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:39.003887 17041 raft_consensus.cc:3060] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.005801 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dd082201fe04ef89c842b6404fd522e" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:39.012570 17041 raft_consensus.cc:2749] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:39.017875 16885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dd082201fe04ef89c842b6404fd522e" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:39.018747 16885 raft_consensus.cc:2468] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 0.
I20260709 13:05:39.029222 17018 consensus_queue.cc:1048] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:39.032737 17008 tablet_bootstrap.cc:492] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:39.033300 17008 ts_tablet_manager.cc:1403] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.207s	user 0.010s	sys 0.011s
I20260709 13:05:39.036175 17008 raft_consensus.cc:359] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:39.036859 17008 raft_consensus.cc:385] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.037111 17008 raft_consensus.cc:740] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.041893 16775 leader_election.cc:304] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:39.044916 17008 consensus_queue.cc:260] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:39.043864 17150 raft_consensus.cc:2804] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:39.059406 17150 raft_consensus.cc:493] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.063311 17150 raft_consensus.cc:3060] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.069192 17008 ts_tablet_manager.cc:1434] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260709 13:05:39.078280 17006 tablet_bootstrap.cc:492] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:39.078913 17006 ts_tablet_manager.cc:1403] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.264s	user 0.021s	sys 0.016s
I20260709 13:05:39.098047 17008 tablet_bootstrap.cc:492] T caa2367ac34d43b5b1a27410b2055005 P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:39.101142 16700 leader_election.cc:304] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:39.102409 17195 raft_consensus.cc:2804] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:39.102833 17195 raft_consensus.cc:493] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.101627 16631 catalog_manager.cc:5697] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "c4b86aae3a8f4a9082e639144ec46228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:39.103168 17195 raft_consensus.cc:3060] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.089848 17021 raft_consensus.cc:493] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.107779 17021 raft_consensus.cc:515] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:39.110749 17021 leader_election.cc:290] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:39.110788 17140 consensus_queue.cc:1048] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:39.113899 17006 raft_consensus.cc:359] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:39.114931 17006 raft_consensus.cc:385] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.115317 17006 raft_consensus.cc:740] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.115372 16885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dd082201fe04ef89c842b6404fd522e" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:39.116214 16885 raft_consensus.cc:2468] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 0.
I20260709 13:05:39.116468 17006 consensus_queue.cc:260] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:39.124536 17004 tablet_bootstrap.cc:492] T 3850091ca95e47b6a1b52e600a452b0b P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:39.125547 17062 raft_consensus.cc:493] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.126102 17062 raft_consensus.cc:515] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:39.127121 16775 leader_election.cc:304] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:39.128630 17062 leader_election.cc:290] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:39.129891 17021 raft_consensus.cc:2804] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:39.130327 17021 raft_consensus.cc:493] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.130679 17021 raft_consensus.cc:3060] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.136405 17006 ts_tablet_manager.cc:1434] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.057s	user 0.009s	sys 0.000s
I20260709 13:05:39.138103 17006 tablet_bootstrap.cc:492] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
W20260709 13:05:39.140650 17203 log.cc:927] Time spent T 0b8add4b17084a6189509807b7e663e5 P d4e1b05bc80e43f7941ece113fb97ab0: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:05:39.144308 17004 ts_tablet_manager.cc:1403] T 3850091ca95e47b6a1b52e600a452b0b P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.299s	user 0.014s	sys 0.006s
W20260709 13:05:39.110833 16700 leader_election.cc:343] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.160547 16808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9ca489a4d2416092d6dcfae2498136" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:39.165509 17040 raft_consensus.cc:493] T 0b8add4b17084a6189509807b7e663e5 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c4b86aae3a8f4a9082e639144ec46228)
W20260709 13:05:39.165918 16923 leader_election.cc:343] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.166164 17040 raft_consensus.cc:515] T 0b8add4b17084a6189509807b7e663e5 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:39.173233 17040 leader_election.cc:290] T 0b8add4b17084a6189509807b7e663e5 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:39.174975 16808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8add4b17084a6189509807b7e663e5" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:39.177006 17121 consensus_queue.cc:1048] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:39.181116 17006 tablet_bootstrap.cc:654] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.195988 17007 tablet_bootstrap.cc:492] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:39.196499 17007 ts_tablet_manager.cc:1403] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.260s	user 0.022s	sys 0.031s
I20260709 13:05:39.199399 17007 raft_consensus.cc:359] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:39.200093 17007 raft_consensus.cc:385] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.200366 17007 raft_consensus.cc:740] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.181252 17195 raft_consensus.cc:515] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:39.201088 17007 consensus_queue.cc:260] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:39.200745 17004 raft_consensus.cc:359] T 3850091ca95e47b6a1b52e600a452b0b P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.206032 17004 raft_consensus.cc:385] T 3850091ca95e47b6a1b52e600a452b0b P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.206256 17004 raft_consensus.cc:740] T 3850091ca95e47b6a1b52e600a452b0b P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
W20260709 13:05:39.158298 16700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:45135, user_credentials={real_user=slave}} blocked reactor thread for 47492.9us
I20260709 13:05:39.202051 17021 raft_consensus.cc:515] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:39.212251 17004 consensus_queue.cc:260] T 3850091ca95e47b6a1b52e600a452b0b P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.214728 17136 consensus_queue.cc:1048] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:39.215132 17004 ts_tablet_manager.cc:1434] T 3850091ca95e47b6a1b52e600a452b0b P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.070s	user 0.007s	sys 0.001s
I20260709 13:05:39.216231 17140 raft_consensus.cc:493] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.216385 17004 tablet_bootstrap.cc:492] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:39.216528 16731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8add4b17084a6189509807b7e663e5" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:39.216815 17140 raft_consensus.cc:515] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:39.219115 17140 leader_election.cc:290] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:39.221711 17006 tablet_bootstrap.cc:492] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:39.222332 17006 ts_tablet_manager.cc:1403] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.085s	user 0.015s	sys 0.006s
I20260709 13:05:39.223194 17041 consensus_queue.cc:1048] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:39.223694 17004 tablet_bootstrap.cc:654] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.224512 17195 leader_election.cc:290] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:39.231089 17007 ts_tablet_manager.cc:1434] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 13:05:39.228897 17021 leader_election.cc:290] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:39.233385 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dd082201fe04ef89c842b6404fd522e" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:39.249405 16884 raft_consensus.cc:3060] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.252179 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dd082201fe04ef89c842b6404fd522e" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:39.252846 16736 raft_consensus.cc:3060] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.253728 17004 tablet_bootstrap.cc:492] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:39.254217 17004 ts_tablet_manager.cc:1403] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.006s
I20260709 13:05:39.209044 17150 raft_consensus.cc:515] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:39.260119 17004 raft_consensus.cc:359] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.260833 17004 raft_consensus.cc:385] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.260932 16736 raft_consensus.cc:2468] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:39.261168 17004 raft_consensus.cc:740] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.232425 17008 tablet_bootstrap.cc:654] T caa2367ac34d43b5b1a27410b2055005 P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.247363 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dd082201fe04ef89c842b6404fd522e" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:39.263008 16732 raft_consensus.cc:2376] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1. Re-sending same reply.
W20260709 13:05:39.268013 16683 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:39.225149 17006 raft_consensus.cc:359] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:39.268652 17006 raft_consensus.cc:385] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.262116 17004 consensus_queue.cc:260] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.269006 17006 raft_consensus.cc:740] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.269178 16700 leader_election.cc:304] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228
I20260709 13:05:39.270081 17195 raft_consensus.cc:2764] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:05:39.270843 17150 leader_election.cc:290] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:39.271457 17008 tablet_bootstrap.cc:492] T caa2367ac34d43b5b1a27410b2055005 P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:39.271898 16885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:39.272688 17008 ts_tablet_manager.cc:1403] T caa2367ac34d43b5b1a27410b2055005 P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.175s	user 0.013s	sys 0.000s
I20260709 13:05:39.272575 16885 raft_consensus.cc:3060] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.274796 17004 ts_tablet_manager.cc:1434] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.020s	user 0.003s	sys 0.004s
I20260709 13:05:39.274621 17195 consensus_queue.cc:1048] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:39.275965 17004 tablet_bootstrap.cc:492] T 5857fad96a1045d7b85190e64da77bce P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:39.276490 16884 raft_consensus.cc:2468] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:39.278250 17006 consensus_queue.cc:260] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:39.278872 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9ca489a4d2416092d6dcfae2498136" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:39.281375 17006 ts_tablet_manager.cc:1434] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.059s	user 0.006s	sys 0.000s
I20260709 13:05:39.240379 17007 tablet_bootstrap.cc:492] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:39.282665 17006 tablet_bootstrap.cc:492] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:39.289065 17007 tablet_bootstrap.cc:654] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.288805 16775 leader_election.cc:304] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:39.289880 17006 tablet_bootstrap.cc:654] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.291651 17150 raft_consensus.cc:2804] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:39.292250 17150 raft_consensus.cc:697] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Becoming Leader. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:05:39.293088 17150 consensus_queue.cc:237] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:39.294850 17066 raft_consensus.cc:493] T 3850091ca95e47b6a1b52e600a452b0b P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.295390 17066 raft_consensus.cc:515] T 3850091ca95e47b6a1b52e600a452b0b P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.302127 17066 leader_election.cc:290] T 3850091ca95e47b6a1b52e600a452b0b P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:39.304108 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3850091ca95e47b6a1b52e600a452b0b" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:39.308121 17004 tablet_bootstrap.cc:654] T 5857fad96a1045d7b85190e64da77bce P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.310631 16885 raft_consensus.cc:2468] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:39.314349 17195 raft_consensus.cc:493] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.317469 17007 tablet_bootstrap.cc:492] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:39.318116 17195 raft_consensus.cc:515] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:39.318922 17007 ts_tablet_manager.cc:1403] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.079s	user 0.012s	sys 0.002s
I20260709 13:05:39.320182 17195 leader_election.cc:290] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:39.321832 17008 raft_consensus.cc:359] T caa2367ac34d43b5b1a27410b2055005 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:39.322433 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3850091ca95e47b6a1b52e600a452b0b" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:39.323702 17004 tablet_bootstrap.cc:492] T 5857fad96a1045d7b85190e64da77bce P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:39.323928 17008 raft_consensus.cc:385] T caa2367ac34d43b5b1a27410b2055005 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.324296 17004 ts_tablet_manager.cc:1403] T 5857fad96a1045d7b85190e64da77bce P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.049s	user 0.009s	sys 0.005s
I20260709 13:05:39.324353 17008 raft_consensus.cc:740] T caa2367ac34d43b5b1a27410b2055005 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.325598 17008 consensus_queue.cc:260] T caa2367ac34d43b5b1a27410b2055005 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:39.328152 16923 leader_election.cc:304] T 3850091ca95e47b6a1b52e600a452b0b P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:39.328406 17008 ts_tablet_manager.cc:1434] T caa2367ac34d43b5b1a27410b2055005 P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 13:05:39.329486 17066 raft_consensus.cc:2749] T 3850091ca95e47b6a1b52e600a452b0b P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:39.329649 17008 tablet_bootstrap.cc:492] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:39.329013 17007 raft_consensus.cc:359] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:39.332029 17007 raft_consensus.cc:385] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.332428 17007 raft_consensus.cc:740] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.330755 17004 raft_consensus.cc:359] T 5857fad96a1045d7b85190e64da77bce P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.333822 17004 raft_consensus.cc:385] T 5857fad96a1045d7b85190e64da77bce P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.334189 17004 raft_consensus.cc:740] T 5857fad96a1045d7b85190e64da77bce P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.334878 16775 leader_election.cc:304] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:39.333325 17007 consensus_queue.cc:260] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:39.336736 17150 raft_consensus.cc:2804] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:39.335112 17004 consensus_queue.cc:260] T 5857fad96a1045d7b85190e64da77bce P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.338022 17150 raft_consensus.cc:697] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Becoming Leader. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:05:39.339217 17150 consensus_queue.cc:237] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:39.339768 17007 ts_tablet_manager.cc:1434] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:05:39.341348 17007 tablet_bootstrap.cc:492] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:39.339695 16634 catalog_manager.cc:5697] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "c4b86aae3a8f4a9082e639144ec46228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:39.343511 17004 ts_tablet_manager.cc:1434] T 5857fad96a1045d7b85190e64da77bce P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.006s
I20260709 13:05:39.353859 17004 tablet_bootstrap.cc:492] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:39.355150 17008 tablet_bootstrap.cc:654] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.361259 17004 tablet_bootstrap.cc:654] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.362494 16849 leader_election.cc:304] T 0b8add4b17084a6189509807b7e663e5 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:39.363930 17121 raft_consensus.cc:2749] T 0b8add4b17084a6189509807b7e663e5 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:39.364259 17007 tablet_bootstrap.cc:654] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.373955 16885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10bbce697a734600bb083d47ac877574" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:39.380257 17004 tablet_bootstrap.cc:492] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:39.380139 16957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10bbce697a734600bb083d47ac877574" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:39.380753 17008 tablet_bootstrap.cc:492] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:39.380823 17004 ts_tablet_manager.cc:1403] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.000s
I20260709 13:05:39.381732 17008 ts_tablet_manager.cc:1403] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.052s	user 0.009s	sys 0.010s
I20260709 13:05:39.386770 17004 raft_consensus.cc:359] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.386981 17008 raft_consensus.cc:359] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.387508 17004 raft_consensus.cc:385] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.387813 17008 raft_consensus.cc:385] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.387818 17004 raft_consensus.cc:740] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.388373 17008 raft_consensus.cc:740] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
W20260709 13:05:39.388818 16923 leader_election.cc:343] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.388906 17004 consensus_queue.cc:260] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.389238 17008 consensus_queue.cc:260] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.389413 16923 leader_election.cc:304] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:39.391342 17066 raft_consensus.cc:2749] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:39.391590 17004 ts_tablet_manager.cc:1434] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:05:39.392341 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9078054426a34acbb94be2b69213b6c1" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:39.382740 16885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:39.393450 16885 raft_consensus.cc:2393] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e2b87453fe654318abccdd1512b949c4 in current term 1: Already voted for candidate c4b86aae3a8f4a9082e639144ec46228 in this term.
I20260709 13:05:39.394341 17004 tablet_bootstrap.cc:492] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:39.397890 17006 tablet_bootstrap.cc:492] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:39.398469 17006 ts_tablet_manager.cc:1403] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.116s	user 0.029s	sys 0.034s
I20260709 13:05:39.401684 17004 tablet_bootstrap.cc:654] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.404978 17006 raft_consensus.cc:359] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:39.409538 17006 raft_consensus.cc:385] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.410006 17006 raft_consensus.cc:740] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.411010 17007 tablet_bootstrap.cc:492] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:39.411548 17007 ts_tablet_manager.cc:1403] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.071s	user 0.010s	sys 0.007s
I20260709 13:05:39.415937 16808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:39.417938 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9078054426a34acbb94be2b69213b6c1" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:39.419109 17121 raft_consensus.cc:493] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.419695 17121 raft_consensus.cc:515] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:39.422420 17006 consensus_queue.cc:260] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:39.423350 17121 leader_election.cc:290] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:39.425405 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612e472fbe674788986798c65836388f" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:39.426122 16958 raft_consensus.cc:2468] T 612e472fbe674788986798c65836388f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
I20260709 13:05:39.426180 17006 ts_tablet_manager.cc:1434] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.000s
I20260709 13:05:39.427609 17006 tablet_bootstrap.cc:492] T 9e8e0f9919a9404c9001b6718f102d19 P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:39.427743 16849 leader_election.cc:304] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:39.428294 16736 raft_consensus.cc:3060] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.429483 17018 raft_consensus.cc:2804] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:39.430001 17018 raft_consensus.cc:493] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.430320 17018 raft_consensus.cc:3060] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.431237 17008 ts_tablet_manager.cc:1434] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.048s	user 0.005s	sys 0.001s
I20260709 13:05:39.429391 16731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:39.441128 17006 tablet_bootstrap.cc:654] T 9e8e0f9919a9404c9001b6718f102d19 P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.451958 16731 raft_consensus.cc:2393] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c4b86aae3a8f4a9082e639144ec46228 in current term 1: Already voted for candidate e2b87453fe654318abccdd1512b949c4 in this term.
I20260709 13:05:39.453680 16958 raft_consensus.cc:3060] T 5857fad96a1045d7b85190e64da77bce P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.461138 16634 catalog_manager.cc:5697] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "c4b86aae3a8f4a9082e639144ec46228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:39.463207 17018 raft_consensus.cc:515] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:39.469007 16957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612e472fbe674788986798c65836388f" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:39.469756 16957 raft_consensus.cc:3060] T 612e472fbe674788986798c65836388f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.465370 17018 leader_election.cc:290] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:39.473242 17007 raft_consensus.cc:359] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:39.474056 17007 raft_consensus.cc:385] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.474402 17007 raft_consensus.cc:740] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.475123 17004 tablet_bootstrap.cc:492] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:39.475800 17004 ts_tablet_manager.cc:1403] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.083s	user 0.014s	sys 0.005s
I20260709 13:05:39.478480 17007 consensus_queue.cc:260] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:39.480465 16700 leader_election.cc:304] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:39.481539 17140 raft_consensus.cc:2749] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:39.482012 16700 leader_election.cc:343] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.482949 17008 tablet_bootstrap.cc:492] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:39.483480 16700 leader_election.cc:304] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:39.484616 17140 raft_consensus.cc:2749] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:05:39.484790 16700 leader_election.cc:343] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.485299 16700 leader_election.cc:304] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:39.486246 17138 raft_consensus.cc:2749] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:39.487468 16731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612e472fbe674788986798c65836388f" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:39.487955 16730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612e472fbe674788986798c65836388f" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:39.489471 17150 raft_consensus.cc:493] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.490147 17150 raft_consensus.cc:515] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.491966 17007 ts_tablet_manager.cc:1434] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.080s	user 0.003s	sys 0.004s
I20260709 13:05:39.491778 17004 raft_consensus.cc:359] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:39.492442 17150 leader_election.cc:290] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:39.492579 17004 raft_consensus.cc:385] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.493377 17004 raft_consensus.cc:740] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.496176 17007 tablet_bootstrap.cc:492] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:39.497314 17004 consensus_queue.cc:260] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:39.498124 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e8e0f9919a9404c9001b6718f102d19" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:39.499643 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e8e0f9919a9404c9001b6718f102d19" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:39.500407 16960 raft_consensus.cc:2468] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 0.
I20260709 13:05:39.499662 17004 ts_tablet_manager.cc:1434] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
W20260709 13:05:39.502180 16849 leader_election.cc:343] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.502875 17004 tablet_bootstrap.cc:492] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:39.503268 17008 tablet_bootstrap.cc:654] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:39.503400 16849 leader_election.cc:343] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.504951 16957 raft_consensus.cc:2468] T 612e472fbe674788986798c65836388f P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:39.505266 16775 leader_election.cc:304] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
W20260709 13:05:39.506294 16775 leader_election.cc:343] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.507843 17006 tablet_bootstrap.cc:492] T 9e8e0f9919a9404c9001b6718f102d19 P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:39.508337 17006 ts_tablet_manager.cc:1403] T 9e8e0f9919a9404c9001b6718f102d19 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.081s	user 0.009s	sys 0.006s
I20260709 13:05:39.509279 16849 leader_election.cc:304] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: e2b87453fe654318abccdd1512b949c4
I20260709 13:05:39.509543 17140 raft_consensus.cc:493] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.510514 17062 raft_consensus.cc:493] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.511089 17018 raft_consensus.cc:2804] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:39.511536 17062 raft_consensus.cc:515] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.513073 17018 raft_consensus.cc:697] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Becoming Leader. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:39.513911 17062 leader_election.cc:290] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:39.514269 17018 consensus_queue.cc:237] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:39.515455 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93629337d6b8477a9cd5895a157bf918" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:39.516660 17021 raft_consensus.cc:2804] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:39.517189 17021 raft_consensus.cc:493] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.517554 17021 raft_consensus.cc:3060] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.517984 16885 raft_consensus.cc:3060] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.514730 17140 raft_consensus.cc:515] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:39.522650 17007 tablet_bootstrap.cc:654] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:39.524232 16923 leader_election.cc:343] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.524533 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93629337d6b8477a9cd5895a157bf918" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:39.539415 16734 raft_consensus.cc:3060] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.521589 17006 raft_consensus.cc:359] T 9e8e0f9919a9404c9001b6718f102d19 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.546723 17006 raft_consensus.cc:385] T 9e8e0f9919a9404c9001b6718f102d19 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.547669 17006 raft_consensus.cc:740] T 9e8e0f9919a9404c9001b6718f102d19 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.548552 17008 tablet_bootstrap.cc:492] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:39.549043 17008 ts_tablet_manager.cc:1403] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.066s	user 0.010s	sys 0.004s
W20260709 13:05:39.549796 16923 leader_election.cc:343] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.550457 16923 leader_election.cc:304] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:39.535363 17021 raft_consensus.cc:515] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.551476 17062 raft_consensus.cc:2749] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:39.551828 17008 raft_consensus.cc:359] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:39.552613 17008 raft_consensus.cc:385] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.543473 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9592a31e7ffd407a959c0a3d96c0f8f9" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:39.553297 16960 raft_consensus.cc:2393] T 9592a31e7ffd407a959c0a3d96c0f8f9 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2b87453fe654318abccdd1512b949c4 in current term 1: Already voted for candidate c4b86aae3a8f4a9082e639144ec46228 in this term.
I20260709 13:05:39.548671 17006 consensus_queue.cc:260] T 9e8e0f9919a9404c9001b6718f102d19 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.557945 17007 tablet_bootstrap.cc:492] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:39.560415 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e8e0f9919a9404c9001b6718f102d19" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:39.561177 16886 raft_consensus.cc:3060] T 9e8e0f9919a9404c9001b6718f102d19 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.558604 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e8e0f9919a9404c9001b6718f102d19" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:39.571296 16886 raft_consensus.cc:2468] T 9e8e0f9919a9404c9001b6718f102d19 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:39.571785 16960 raft_consensus.cc:3060] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.573623 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9592a31e7ffd407a959c0a3d96c0f8f9" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:39.575934 16775 leader_election.cc:304] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:39.576936 17152 consensus_queue.cc:1048] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:39.578274 17152 raft_consensus.cc:2804] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:39.578749 17152 raft_consensus.cc:697] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Becoming Leader. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:05:39.579751 17152 consensus_queue.cc:237] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.580596 17007 ts_tablet_manager.cc:1403] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.088s	user 0.007s	sys 0.005s
I20260709 13:05:39.581936 16957 raft_consensus.cc:3060] T 3850091ca95e47b6a1b52e600a452b0b P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.582880 17121 raft_consensus.cc:493] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:39.583436 17121 raft_consensus.cc:515] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:39.544199 17140 leader_election.cc:290] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:39.586804 17007 raft_consensus.cc:359] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:39.587420 17121 leader_election.cc:290] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:39.587574 17007 raft_consensus.cc:385] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.588047 17007 raft_consensus.cc:740] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.589195 16956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27be53bca112445d8bdc640329310d67" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:39.589639 17008 raft_consensus.cc:740] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.590363 17008 consensus_queue.cc:260] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:39.595450 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27be53bca112445d8bdc640329310d67" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:39.597648 16700 leader_election.cc:304] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228
I20260709 13:05:39.597698 17008 ts_tablet_manager.cc:1434] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.001s
I20260709 13:05:39.516506 17004 tablet_bootstrap.cc:654] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.600577 17008 tablet_bootstrap.cc:492] T e5c700237eb44cefbf7bc46c1daf08cd P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:39.588918 17007 consensus_queue.cc:260] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:39.606675 17140 raft_consensus.cc:2749] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:39.607806 17007 ts_tablet_manager.cc:1434] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 13:05:39.608903 17007 tablet_bootstrap.cc:492] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:39.613538 17150 consensus_queue.cc:1048] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:39.595537 16960 raft_consensus.cc:2468] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:39.617894 16849 leader_election.cc:304] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0; no voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:39.619190 17040 raft_consensus.cc:2749] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:39.621877 17006 ts_tablet_manager.cc:1434] T 9e8e0f9919a9404c9001b6718f102d19 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.113s	user 0.009s	sys 0.000s
I20260709 13:05:39.655076 17066 raft_consensus.cc:493] T 5857fad96a1045d7b85190e64da77bce P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d4e1b05bc80e43f7941ece113fb97ab0)
I20260709 13:05:39.658802 17066 raft_consensus.cc:515] T 5857fad96a1045d7b85190e64da77bce P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.658766 17006 tablet_bootstrap.cc:492] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:39.667657 17008 tablet_bootstrap.cc:654] T e5c700237eb44cefbf7bc46c1daf08cd P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.668084 17067 raft_consensus.cc:493] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.668604 17067 raft_consensus.cc:515] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:39.670763 17067 leader_election.cc:290] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:39.677608 17006 tablet_bootstrap.cc:654] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.701622 17150 raft_consensus.cc:493] T caa2367ac34d43b5b1a27410b2055005 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.702380 17150 raft_consensus.cc:515] T caa2367ac34d43b5b1a27410b2055005 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:39.704380 17066 leader_election.cc:290] T 5857fad96a1045d7b85190e64da77bce P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:39.704818 17150 leader_election.cc:290] T caa2367ac34d43b5b1a27410b2055005 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:39.646911 16631 catalog_manager.cc:5697] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3). New cstate: current_term: 1 leader_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:39.706681 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5857fad96a1045d7b85190e64da77bce" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:39.708971 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f48e0f9f83da446a877cd06e19768929" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:39.715883 16634 catalog_manager.cc:5697] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "c4b86aae3a8f4a9082e639144ec46228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:39.556559 17021 leader_election.cc:290] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:39.721788 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f48e0f9f83da446a877cd06e19768929" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
W20260709 13:05:39.726346 16923 leader_election.cc:343] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.731105 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5857fad96a1045d7b85190e64da77bce" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:39.734793 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caa2367ac34d43b5b1a27410b2055005" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:39.736159 17008 tablet_bootstrap.cc:492] T e5c700237eb44cefbf7bc46c1daf08cd P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:39.736384 17007 tablet_bootstrap.cc:654] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:39.736464 16923 leader_election.cc:343] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.736825 17008 ts_tablet_manager.cc:1403] T e5c700237eb44cefbf7bc46c1daf08cd P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.136s	user 0.017s	sys 0.022s
I20260709 13:05:39.737061 17004 tablet_bootstrap.cc:492] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:39.737056 16923 leader_election.cc:304] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:39.737793 17004 ts_tablet_manager.cc:1403] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.235s	user 0.028s	sys 0.055s
I20260709 13:05:39.738536 17067 raft_consensus.cc:2749] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:39.740550 17006 tablet_bootstrap.cc:492] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:39.741039 17006 ts_tablet_manager.cc:1403] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.083s	user 0.018s	sys 0.016s
I20260709 13:05:39.743659 17006 raft_consensus.cc:359] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.744354 17006 raft_consensus.cc:385] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.744627 17006 raft_consensus.cc:740] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.742835 17004 raft_consensus.cc:359] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:39.745244 17004 raft_consensus.cc:385] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.745545 17004 raft_consensus.cc:740] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.746232 17004 consensus_queue.cc:260] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:39.748332 17004 ts_tablet_manager.cc:1434] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:05:39.749313 17004 tablet_bootstrap.cc:492] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:39.749217 17008 raft_consensus.cc:359] T e5c700237eb44cefbf7bc46c1daf08cd P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.750545 17008 raft_consensus.cc:385] T e5c700237eb44cefbf7bc46c1daf08cd P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.750811 17008 raft_consensus.cc:740] T e5c700237eb44cefbf7bc46c1daf08cd P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.752458 16923 leader_election.cc:304] T 5857fad96a1045d7b85190e64da77bce P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:39.753860 17067 raft_consensus.cc:2749] T 5857fad96a1045d7b85190e64da77bce P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:39.751586 17008 consensus_queue.cc:260] T e5c700237eb44cefbf7bc46c1daf08cd P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.755424 17152 raft_consensus.cc:493] T e5c700237eb44cefbf7bc46c1daf08cd P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.755903 17152 raft_consensus.cc:515] T e5c700237eb44cefbf7bc46c1daf08cd P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.757653 17004 tablet_bootstrap.cc:654] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.757695 17007 tablet_bootstrap.cc:492] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:39.758549 17007 ts_tablet_manager.cc:1403] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.150s	user 0.013s	sys 0.005s
I20260709 13:05:39.761683 17007 raft_consensus.cc:359] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:39.762557 17007 raft_consensus.cc:385] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.762902 17007 raft_consensus.cc:740] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.757920 17152 leader_election.cc:290] T e5c700237eb44cefbf7bc46c1daf08cd P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:39.763733 17007 consensus_queue.cc:260] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:39.766340 17008 ts_tablet_manager.cc:1434] T e5c700237eb44cefbf7bc46c1daf08cd P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.029s	user 0.002s	sys 0.004s
I20260709 13:05:39.767035 17006 consensus_queue.cc:260] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.768560 17008 tablet_bootstrap.cc:492] T 59de6d1228da41da938527632da08f4d P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:39.783504 17008 tablet_bootstrap.cc:654] T 59de6d1228da41da938527632da08f4d P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.783766 17004 tablet_bootstrap.cc:492] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:39.784276 17004 ts_tablet_manager.cc:1403] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.006s
I20260709 13:05:39.786540 17007 ts_tablet_manager.cc:1434] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.028s	user 0.016s	sys 0.003s
I20260709 13:05:39.786796 17006 ts_tablet_manager.cc:1434] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.045s	user 0.003s	sys 0.006s
I20260709 13:05:39.787101 17004 raft_consensus.cc:359] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:39.787704 17007 tablet_bootstrap.cc:492] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:39.788093 17006 tablet_bootstrap.cc:492] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:39.788280 17004 raft_consensus.cc:385] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.788882 17004 raft_consensus.cc:740] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.789666 17004 consensus_queue.cc:260] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:39.791738 17004 ts_tablet_manager.cc:1434] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:05:39.792889 17004 tablet_bootstrap.cc:492] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:39.795347 17007 tablet_bootstrap.cc:654] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.797698 17152 raft_consensus.cc:493] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.798229 17152 raft_consensus.cc:515] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.799469 17004 tablet_bootstrap.cc:654] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.800670 17152 leader_election.cc:290] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:39.805859 17006 tablet_bootstrap.cc:654] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.811661 17004 tablet_bootstrap.cc:492] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:39.812347 17004 ts_tablet_manager.cc:1403] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.004s
I20260709 13:05:39.815299 17004 raft_consensus.cc:359] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:39.816552 17004 raft_consensus.cc:385] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.816907 17004 raft_consensus.cc:740] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.816713 17006 tablet_bootstrap.cc:492] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:39.817891 17006 ts_tablet_manager.cc:1403] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.030s	user 0.006s	sys 0.008s
I20260709 13:05:39.817838 17004 consensus_queue.cc:260] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:39.820828 17004 ts_tablet_manager.cc:1434] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 13:05:39.821241 17006 raft_consensus.cc:359] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:39.824832 17006 raft_consensus.cc:385] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.825181 17006 raft_consensus.cc:740] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.826126 17006 consensus_queue.cc:260] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:39.830595 17006 ts_tablet_manager.cc:1434] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:05:39.835342 17004 tablet_bootstrap.cc:492] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:39.841619 17006 tablet_bootstrap.cc:492] T 7123efeaee9f4ec78a1c475eab16c4c1 P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:39.854786 17004 tablet_bootstrap.cc:654] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.861352 17006 tablet_bootstrap.cc:654] T 7123efeaee9f4ec78a1c475eab16c4c1 P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.894004 17004 tablet_bootstrap.cc:492] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:39.894713 17004 ts_tablet_manager.cc:1403] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.060s	user 0.015s	sys 0.014s
I20260709 13:05:39.898838 17004 raft_consensus.cc:359] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.899840 17004 raft_consensus.cc:385] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.900185 17004 raft_consensus.cc:740] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.903353 17004 consensus_queue.cc:260] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.907936 17004 ts_tablet_manager.cc:1434] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.001s
I20260709 13:05:39.909415 17004 tablet_bootstrap.cc:492] T e81325b638b1493999310a726811b64b P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:39.930171 17004 tablet_bootstrap.cc:654] T e81325b638b1493999310a726811b64b P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.934437 17008 tablet_bootstrap.cc:492] T 59de6d1228da41da938527632da08f4d P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:39.935038 17008 ts_tablet_manager.cc:1403] T 59de6d1228da41da938527632da08f4d P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.167s	user 0.017s	sys 0.052s
I20260709 13:05:39.946772 17004 tablet_bootstrap.cc:492] T e81325b638b1493999310a726811b64b P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:39.947523 17004 ts_tablet_manager.cc:1403] T e81325b638b1493999310a726811b64b P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.038s	user 0.013s	sys 0.002s
I20260709 13:05:39.951331 17004 raft_consensus.cc:359] T e81325b638b1493999310a726811b64b P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.952231 17004 raft_consensus.cc:385] T e81325b638b1493999310a726811b64b P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.952557 17004 raft_consensus.cc:740] T e81325b638b1493999310a726811b64b P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.953399 17004 consensus_queue.cc:260] T e81325b638b1493999310a726811b64b P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:39.964022 17004 ts_tablet_manager.cc:1434] T e81325b638b1493999310a726811b64b P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.003s
I20260709 13:05:39.965266 17004 tablet_bootstrap.cc:492] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:39.979966 17004 tablet_bootstrap.cc:654] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.060993 17138 raft_consensus.cc:493] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.061786 17138 raft_consensus.cc:515] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:40.065083 17138 leader_election.cc:290] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:40.143769 17067 raft_consensus.cc:493] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.144775 17067 raft_consensus.cc:515] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:40.171062 17067 leader_election.cc:290] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:40.199533 17136 raft_consensus.cc:493] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.208276 17004 tablet_bootstrap.cc:492] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:40.273388 17004 ts_tablet_manager.cc:1403] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.308s	user 0.036s	sys 0.075s
I20260709 13:05:40.284052 17067 raft_consensus.cc:493] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.273120 17136 raft_consensus.cc:515] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:40.295295 17004 raft_consensus.cc:359] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:40.299340 17004 raft_consensus.cc:385] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.299695 17004 raft_consensus.cc:740] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.300805 17136 leader_election.cc:290] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:40.300477 17004 consensus_queue.cc:260] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:40.297681 17067 raft_consensus.cc:515] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:40.303349 17004 ts_tablet_manager.cc:1434] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 13:05:40.305073 17006 tablet_bootstrap.cc:492] T 7123efeaee9f4ec78a1c475eab16c4c1 P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:40.305773 17006 ts_tablet_manager.cc:1403] T 7123efeaee9f4ec78a1c475eab16c4c1 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.474s	user 0.074s	sys 0.114s
I20260709 13:05:40.308524 17067 leader_election.cc:290] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:40.310622 17006 raft_consensus.cc:359] T 7123efeaee9f4ec78a1c475eab16c4c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:40.311584 17004 tablet_bootstrap.cc:492] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:40.311640 17006 raft_consensus.cc:385] T 7123efeaee9f4ec78a1c475eab16c4c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.315120 17140 raft_consensus.cc:493] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.315610 17006 raft_consensus.cc:740] T 7123efeaee9f4ec78a1c475eab16c4c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.315683 17140 raft_consensus.cc:515] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:40.318296 17140 leader_election.cc:290] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:40.318265 17006 consensus_queue.cc:260] T 7123efeaee9f4ec78a1c475eab16c4c1 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:40.319777 17004 tablet_bootstrap.cc:654] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.320376 17006 ts_tablet_manager.cc:1434] T 7123efeaee9f4ec78a1c475eab16c4c1 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:05:40.321640 17006 tablet_bootstrap.cc:492] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:40.323490 17007 tablet_bootstrap.cc:492] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:40.323988 17007 ts_tablet_manager.cc:1403] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.537s	user 0.052s	sys 0.099s
I20260709 13:05:40.326715 16730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e295eefc8ff46d9a31a4c0898d16c63" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:40.327488 16730 raft_consensus.cc:2468] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 0.
I20260709 13:05:40.329324 16730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caa2367ac34d43b5b1a27410b2055005" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:40.358755 17006 tablet_bootstrap.cc:654] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.362859 17040 raft_consensus.cc:493] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.363518 17040 raft_consensus.cc:515] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:40.364404 17007 raft_consensus.cc:359] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:40.366061 17007 raft_consensus.cc:385] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.366407 17007 raft_consensus.cc:740] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.367156 17007 consensus_queue.cc:260] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:40.621743 17007 ts_tablet_manager.cc:1434] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.297s	user 0.015s	sys 0.002s
I20260709 13:05:40.623174 17007 tablet_bootstrap.cc:492] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
W20260709 13:05:40.526557 16775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:37199, user_credentials={real_user=slave}} blocked reactor thread for 711578us
I20260709 13:05:40.611771 17256 raft_consensus.cc:493] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.625280 17256 raft_consensus.cc:515] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:40.627687 17256 leader_election.cc:290] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:40.631011 17253 raft_consensus.cc:493] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.631565 17253 raft_consensus.cc:515] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:40.633792 17253 leader_election.cc:290] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:40.635643 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e295eefc8ff46d9a31a4c0898d16c63" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:40.636430 16884 raft_consensus.cc:2468] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 0.
I20260709 13:05:40.643162 17007 tablet_bootstrap.cc:654] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.644085 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "081841333d734eb1aa72ceb3f0ea2bf2" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:40.640422 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e295eefc8ff46d9a31a4c0898d16c63" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:40.672796 16960 raft_consensus.cc:2468] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 0.
I20260709 13:05:40.674391 17006 tablet_bootstrap.cc:492] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:40.675058 17006 ts_tablet_manager.cc:1403] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.354s	user 0.018s	sys 0.004s
I20260709 13:05:40.679448 17262 raft_consensus.cc:493] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.680006 17262 raft_consensus.cc:515] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:40.682279 17262 leader_election.cc:290] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:40.681154 17040 leader_election.cc:290] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:40.682792 16885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e295eefc8ff46d9a31a4c0898d16c63" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:40.685284 16885 raft_consensus.cc:2468] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 0.
I20260709 13:05:40.686457 17006 raft_consensus.cc:359] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:40.687182 17006 raft_consensus.cc:385] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.687469 17006 raft_consensus.cc:740] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.688144 17006 consensus_queue.cc:260] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:40.781565 16885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c700237eb44cefbf7bc46c1daf08cd" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:40.784828 17004 tablet_bootstrap.cc:492] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:40.785447 17004 ts_tablet_manager.cc:1403] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.479s	user 0.027s	sys 0.047s
I20260709 13:05:40.786098 16885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81325b638b1493999310a726811b64b" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:40.787999 17253 raft_consensus.cc:493] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c4b86aae3a8f4a9082e639144ec46228)
I20260709 13:05:40.788754 17253 raft_consensus.cc:515] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:40.789915 17004 raft_consensus.cc:359] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:40.790661 17004 raft_consensus.cc:385] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.791074 17006 ts_tablet_manager.cc:1434] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.116s	user 0.007s	sys 0.002s
I20260709 13:05:40.791005 17004 raft_consensus.cc:740] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.791229 17253 leader_election.cc:290] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:40.792166 17006 tablet_bootstrap.cc:492] T e81325b638b1493999310a726811b64b P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
W20260709 13:05:40.793637 16700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:37199, user_credentials={real_user=slave}} blocked reactor thread for 151374us
I20260709 13:05:40.792999 16885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d83d4d3d894b81a85b945ef97d79de" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:40.798906 16734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d83d4d3d894b81a85b945ef97d79de" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:40.799373 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7123efeaee9f4ec78a1c475eab16c4c1" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:40.799973 16884 raft_consensus.cc:2468] T 7123efeaee9f4ec78a1c475eab16c4c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 0.
I20260709 13:05:40.800163 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e295eefc8ff46d9a31a4c0898d16c63" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:40.800776 16736 raft_consensus.cc:2468] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
I20260709 13:05:40.801395 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9ca489a4d2416092d6dcfae2498136" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:40.802037 16960 raft_consensus.cc:2468] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 0.
I20260709 13:05:40.798115 16700 leader_election.cc:304] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:05:40.793270 17004 consensus_queue.cc:260] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:40.807132 17004 ts_tablet_manager.cc:1434] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.002s
I20260709 13:05:40.808161 17004 tablet_bootstrap.cc:492] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:40.814028 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "919c5788cd97463a9cf06abb18843943" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:40.814730 16884 raft_consensus.cc:2468] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 0.
I20260709 13:05:40.803947 16885 raft_consensus.cc:2468] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 0.
I20260709 13:05:40.815828 17006 tablet_bootstrap.cc:654] T e81325b638b1493999310a726811b64b P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.826047 17263 raft_consensus.cc:2804] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:40.826516 17263 raft_consensus.cc:493] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.826857 17263 raft_consensus.cc:3060] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.827368 17004 tablet_bootstrap.cc:654] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.836707 17008 raft_consensus.cc:359] T 59de6d1228da41da938527632da08f4d P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:40.837945 17008 raft_consensus.cc:385] T 59de6d1228da41da938527632da08f4d P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.840267 17008 raft_consensus.cc:740] T 59de6d1228da41da938527632da08f4d P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.841068 17008 consensus_queue.cc:260] T 59de6d1228da41da938527632da08f4d P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:40.839970 17007 tablet_bootstrap.cc:492] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:40.842374 17007 ts_tablet_manager.cc:1403] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.220s	user 0.020s	sys 0.014s
I20260709 13:05:40.842830 16633 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58766 (request call id 39) took 1143 ms. Trace:
I20260709 13:05:40.843277 16633 rpcz_store.cc:276] 0709 13:05:39.699891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.775400 (+ 75509us) service_pool.cc:224] Handling call
0709 13:05:40.842744 (+1067344us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":272811,"reported_tablets":5}
I20260709 13:05:40.850196 17007 raft_consensus.cc:359] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:40.851399 17007 raft_consensus.cc:385] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.851826 17007 raft_consensus.cc:740] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
W20260709 13:05:40.854954 16700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:45101, user_credentials={real_user=slave}} blocked reactor thread for 56947.3us
W20260709 13:05:40.856880 16700 leader_election.cc:343] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0086163b152644feb75caeac6f145533 (127.6.255.4:37199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.858352 16700 leader_election.cc:304] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:05:40.859071 17138 raft_consensus.cc:2804] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:40.859438 17138 raft_consensus.cc:493] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.859767 17138 raft_consensus.cc:3060] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.860491 16956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e295eefc8ff46d9a31a4c0898d16c63" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:40.861191 16956 raft_consensus.cc:2468] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
I20260709 13:05:40.865602 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "919c5788cd97463a9cf06abb18843943" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:40.866307 16959 raft_consensus.cc:2468] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
I20260709 13:05:40.868979 17007 consensus_queue.cc:260] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:40.870374 17258 consensus_queue.cc:1048] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:05:40.874310 16683 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:40.877218 17007 ts_tablet_manager.cc:1434] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
I20260709 13:05:40.878674 17258 raft_consensus.cc:493] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.879235 17258 raft_consensus.cc:515] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:40.881330 17258 leader_election.cc:290] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
W20260709 13:05:40.883630 16775 leader_election.cc:343] T caa2367ac34d43b5b1a27410b2055005 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:40.884408 16775 leader_election.cc:343] T e5c700237eb44cefbf7bc46c1daf08cd P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:40.885006 16775 leader_election.cc:343] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.890334 17007 tablet_bootstrap.cc:492] T 612e472fbe674788986798c65836388f P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:40.894935 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7123efeaee9f4ec78a1c475eab16c4c1" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:40.896530 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e96e3138004b3aa2f3e745f1ddfc04" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:40.899721 17265 raft_consensus.cc:493] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.901276 17006 tablet_bootstrap.cc:492] T e81325b638b1493999310a726811b64b P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:40.900295 17265 raft_consensus.cc:515] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:40.901897 17006 ts_tablet_manager.cc:1403] T e81325b638b1493999310a726811b64b P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.110s	user 0.012s	sys 0.011s
I20260709 13:05:40.903676 17265 leader_election.cc:290] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:40.906548 17008 ts_tablet_manager.cc:1434] T 59de6d1228da41da938527632da08f4d P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.971s	user 0.011s	sys 0.000s
I20260709 13:05:40.906313 17263 raft_consensus.cc:515] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
W20260709 13:05:40.906930 17008 ts_tablet_manager.cc:1467] T 59de6d1228da41da938527632da08f4d P c4b86aae3a8f4a9082e639144ec46228: Tablet startup took 971ms
I20260709 13:05:40.908381 17004 tablet_bootstrap.cc:492] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:40.909089 17004 ts_tablet_manager.cc:1403] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.101s	user 0.010s	sys 0.024s
I20260709 13:05:40.912569 17004 raft_consensus.cc:359] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:40.908995 17263 leader_election.cc:290] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:40.905067 17006 raft_consensus.cc:359] T e81325b638b1493999310a726811b64b P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:40.891101 16809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e96e3138004b3aa2f3e745f1ddfc04" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:40.926234 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c700237eb44cefbf7bc46c1daf08cd" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:40.913391 17004 raft_consensus.cc:385] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.927515 17004 raft_consensus.cc:740] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.927637 16803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "919c5788cd97463a9cf06abb18843943" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:40.928421 17004 consensus_queue.cc:260] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:40.890759 16807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "919c5788cd97463a9cf06abb18843943" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:40.915274 16885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e295eefc8ff46d9a31a4c0898d16c63" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:40.930917 16885 raft_consensus.cc:3060] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:40.932586 16849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:37199, user_credentials={real_user=slave}} blocked reactor thread for 61459us
I20260709 13:05:40.917850 17006 raft_consensus.cc:385] T e81325b638b1493999310a726811b64b P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.934059 17006 raft_consensus.cc:740] T e81325b638b1493999310a726811b64b P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.915603 16957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e295eefc8ff46d9a31a4c0898d16c63" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:40.935148 16957 raft_consensus.cc:3060] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.936697 16809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9ca489a4d2416092d6dcfae2498136" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:40.937212 16803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "081841333d734eb1aa72ceb3f0ea2bf2" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:40.937491 16809 raft_consensus.cc:2468] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 0.
W20260709 13:05:40.907291 17008 ts_tablet_manager.cc:1469] T 59de6d1228da41da938527632da08f4d P c4b86aae3a8f4a9082e639144ec46228: Trace:
0709 13:05:35.243020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.913227 (+670207us) service_pool.cc:224] Handling call
0709 13:05:35.966669 (+ 53442us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:05:35.975135 (+  8466us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:05:36.035425 (+ 60290us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:36.115782 (+ 80357us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:36.156563 (+ 40781us) inbound_call.cc:177] Queueing success response
0709 13:05:39.768149 (+3611586us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:05:39.935337 (+167188us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:05:39.935842 (+   505us) tablet_replica.cc:261] Starting instrumentation
0709 13:05:39.936499 (+   657us) tablet_replica.cc:280] Starting consensus
Metrics: {"mutex_wait_us":972190,"raft.queue_time_us":66686,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":16052,"tablet-open.queue_time_us":3617820}
I20260709 13:05:40.938570 17006 consensus_queue.cc:260] T e81325b638b1493999310a726811b64b P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:40.917197 17007 tablet_bootstrap.cc:654] T 612e472fbe674788986798c65836388f P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.939981 17008 tablet_bootstrap.cc:492] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:40.921582 17121 raft_consensus.cc:493] T 0b8add4b17084a6189509807b7e663e5 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c4b86aae3a8f4a9082e639144ec46228)
I20260709 13:05:40.942096 17121 raft_consensus.cc:515] T 0b8add4b17084a6189509807b7e663e5 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:40.891296 17138 raft_consensus.cc:515] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:40.920588 16956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81325b638b1493999310a726811b64b" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:40.950996 17121 leader_election.cc:290] T 0b8add4b17084a6189509807b7e663e5 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:40.951704 17268 raft_consensus.cc:493] T e81325b638b1493999310a726811b64b P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.952248 17268 raft_consensus.cc:515] T e81325b638b1493999310a726811b64b P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:40.953272 16956 raft_consensus.cc:2468] T e81325b638b1493999310a726811b64b P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 0.
I20260709 13:05:40.954663 17268 leader_election.cc:290] T e81325b638b1493999310a726811b64b P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:40.957736 16957 raft_consensus.cc:2468] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 1.
I20260709 13:05:40.959031 16700 leader_election.cc:304] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: 
W20260709 13:05:40.936537 16775 leader_election.cc:343] T caa2367ac34d43b5b1a27410b2055005 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.959761 17263 raft_consensus.cc:2804] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:40.960336 17263 raft_consensus.cc:697] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:05:40.959941 16775 leader_election.cc:304] T caa2367ac34d43b5b1a27410b2055005 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228; no voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4
W20260709 13:05:40.961500 16923 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:45101, user_credentials={real_user=slave}} blocked reactor thread for 133354us
I20260709 13:05:40.961290 17263 consensus_queue.cc:237] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:40.962497 16923 leader_election.cc:304] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:40.962865 17021 raft_consensus.cc:2749] T caa2367ac34d43b5b1a27410b2055005 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:40.963778 16923 leader_election.cc:304] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:40.965322 17253 raft_consensus.cc:2764] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:05:40.965552 16923 leader_election.cc:304] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:40.966125 17253 raft_consensus.cc:2804] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:40.966557 17274 raft_consensus.cc:2804] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:40.966612 17253 raft_consensus.cc:493] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.967110 17274 raft_consensus.cc:493] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.967322 17253 raft_consensus.cc:3060] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.967571 17274 raft_consensus.cc:3060] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.968055 16885 raft_consensus.cc:2468] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 1.
I20260709 13:05:40.970652 17138 leader_election.cc:290] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:40.973615 17006 ts_tablet_manager.cc:1434] T e81325b638b1493999310a726811b64b P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.071s	user 0.009s	sys 0.000s
I20260709 13:05:40.961139 16849 leader_election.cc:304] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:40.976639 16957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9ca489a4d2416092d6dcfae2498136" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:40.976804 16849 leader_election.cc:304] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:40.977447 17121 raft_consensus.cc:2764] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:05:40.983341 17008 tablet_bootstrap.cc:654] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.990932 17004 ts_tablet_manager.cc:1434] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.081s	user 0.008s	sys 0.000s
I20260709 13:05:40.993619 17256 raft_consensus.cc:2804] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:40.994007 17004 tablet_bootstrap.cc:492] T e5c700237eb44cefbf7bc46c1daf08cd P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:40.994182 17256 raft_consensus.cc:493] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.994660 17256 raft_consensus.cc:3060] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.993804 17006 tablet_bootstrap.cc:492] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:41.002957 17138 raft_consensus.cc:493] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.003789 17138 raft_consensus.cc:515] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:41.007790 17253 raft_consensus.cc:515] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:41.008980 16885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d83d4d3d894b81a85b945ef97d79de" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:41.010035 16885 raft_consensus.cc:2468] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 0.
I20260709 13:05:41.012964 16809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dd082201fe04ef89c842b6404fd522e" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:41.014667 16803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8add4b17084a6189509807b7e663e5" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:40.990942 16957 raft_consensus.cc:3060] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.027372 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8add4b17084a6189509807b7e663e5" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:41.029181 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9ca489a4d2416092d6dcfae2498136" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:41.035930 17007 tablet_bootstrap.cc:492] T 612e472fbe674788986798c65836388f P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:41.069669 17007 ts_tablet_manager.cc:1403] T 612e472fbe674788986798c65836388f P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.180s	user 0.015s	sys 0.015s
I20260709 13:05:41.042665 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dd082201fe04ef89c842b6404fd522e" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:41.070755 16732 raft_consensus.cc:2468] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:41.022382 16700 leader_election.cc:304] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4; no voters: 
W20260709 13:05:41.071743 16700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:45101, user_credentials={real_user=slave}} blocked reactor thread for 49502.7us
I20260709 13:05:41.072654 17007 raft_consensus.cc:359] T 612e472fbe674788986798c65836388f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:41.074450 17007 raft_consensus.cc:385] T 612e472fbe674788986798c65836388f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.976078 17274 raft_consensus.cc:515] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:40.966703 16923 leader_election.cc:304] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:41.075613 17268 raft_consensus.cc:2804] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:05:41.066671 16775 leader_election.cc:343] T e5c700237eb44cefbf7bc46c1daf08cd P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0086163b152644feb75caeac6f145533 (127.6.255.4:37199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.073504 17140 raft_consensus.cc:2804] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:05:41.073872 16700 leader_election.cc:343] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.074744 17007 raft_consensus.cc:740] T 612e472fbe674788986798c65836388f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.080166 16700 leader_election.cc:304] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228
I20260709 13:05:41.066965 17253 leader_election.cc:290] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:41.076076 17268 raft_consensus.cc:493] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.084959 17268 raft_consensus.cc:3060] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.084774 17007 consensus_queue.cc:260] T 612e472fbe674788986798c65836388f P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:41.085948 17274 leader_election.cc:290] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:41.086561 17274 raft_consensus.cc:493] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.086977 17274 raft_consensus.cc:515] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:41.087528 17007 ts_tablet_manager.cc:1434] T 612e472fbe674788986798c65836388f P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:05:41.089051 17007 tablet_bootstrap.cc:492] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:41.088953 17274 leader_election.cc:290] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:41.077050 17140 raft_consensus.cc:493] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:41.076432 16923 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:45101, user_credentials={real_user=slave}} blocked reactor thread for 109809us
I20260709 13:05:41.089706 17140 raft_consensus.cc:3060] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.077921 16775 leader_election.cc:304] T e5c700237eb44cefbf7bc46c1daf08cd P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228; no voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:41.095086 17152 raft_consensus.cc:2749] T e5c700237eb44cefbf7bc46c1daf08cd P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.095649 17004 tablet_bootstrap.cc:654] T e5c700237eb44cefbf7bc46c1daf08cd P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.095561 16775 leader_election.cc:304] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:41.096731 17271 raft_consensus.cc:2804] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:41.097126 17271 raft_consensus.cc:493] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.097488 17271 raft_consensus.cc:3060] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.098311 17138 leader_election.cc:290] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:41.104987 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d83d4d3d894b81a85b945ef97d79de" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:41.109609 17196 raft_consensus.cc:2749] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.111029 17268 raft_consensus.cc:515] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:41.111723 16803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9ca489a4d2416092d6dcfae2498136" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:41.112403 16803 raft_consensus.cc:3060] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.112663 16959 raft_consensus.cc:2393] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2b87453fe654318abccdd1512b949c4 in current term 1: Already voted for candidate 0086163b152644feb75caeac6f145533 in this term.
I20260709 13:05:41.112926 17213 consensus_queue.cc:1048] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:41.102911 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9ca489a4d2416092d6dcfae2498136" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:41.114434 17213 consensus_queue.cc:1048] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:41.114841 16732 raft_consensus.cc:2393] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c4b86aae3a8f4a9082e639144ec46228 in current term 1: Already voted for candidate e2b87453fe654318abccdd1512b949c4 in this term.
I20260709 13:05:41.121799 17006 tablet_bootstrap.cc:654] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:41.091390 16849 leader_election.cc:343] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.124653 17253 raft_consensus.cc:493] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.125545 16775 leader_election.cc:304] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228; no voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:41.125839 17256 raft_consensus.cc:515] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:41.125293 17253 raft_consensus.cc:515] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:41.127995 17256 leader_election.cc:290] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:41.142117 17253 leader_election.cc:290] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:41.145604 17263 raft_consensus.cc:493] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.147949 16635 catalog_manager.cc:5697] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:41.151247 17140 raft_consensus.cc:515] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:41.154119 17268 leader_election.cc:290] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:41.138895 17213 consensus_queue.cc:1048] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
W20260709 13:05:41.137089 16923 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:40785, user_credentials={real_user=slave}} blocked reactor thread for 46473.9us
I20260709 13:05:41.146227 17263 raft_consensus.cc:515] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:41.157899 17263 leader_election.cc:290] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
W20260709 13:05:41.137378 16849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:45135, user_credentials={real_user=slave}} blocked reactor thread for 53165.4us
I20260709 13:05:41.170393 17271 raft_consensus.cc:515] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:41.173007 17271 leader_election.cc:290] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:41.181545 17008 tablet_bootstrap.cc:492] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:41.181640 16809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:41.229501 17008 ts_tablet_manager.cc:1403] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.290s	user 0.062s	sys 0.059s
W20260709 13:05:41.178957 16849 consensus_peers.cc:597] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:41.174465 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81325b638b1493999310a726811b64b" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:41.203179 16803 raft_consensus.cc:2468] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 1.
I20260709 13:05:41.204546 17140 leader_election.cc:290] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:41.233769 17152 raft_consensus.cc:2749] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.234079 17008 raft_consensus.cc:359] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:41.234941 17008 raft_consensus.cc:385] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.177035 16957 raft_consensus.cc:2468] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 1.
I20260709 13:05:41.206033 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d83d4d3d894b81a85b945ef97d79de" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:41.207813 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d83d4d3d894b81a85b945ef97d79de" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:41.238441 16958 raft_consensus.cc:2393] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e2b87453fe654318abccdd1512b949c4 in current term 1: Already voted for candidate 0086163b152644feb75caeac6f145533 in this term.
I20260709 13:05:41.239392 16884 raft_consensus.cc:3060] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:41.157919 16923 leader_election.cc:343] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:41.241976 16923 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:40785, user_credentials={real_user=slave}} blocked reactor thread for 84082.7us
I20260709 13:05:41.198748 17007 tablet_bootstrap.cc:654] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:41.230397 16849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:40785, user_credentials={real_user=slave}} blocked reactor thread for 51536.5us
I20260709 13:05:41.232594 16959 raft_consensus.cc:3060] T e81325b638b1493999310a726811b64b P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.232991 17004 tablet_bootstrap.cc:492] T e5c700237eb44cefbf7bc46c1daf08cd P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:41.235282 17008 raft_consensus.cc:740] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.238248 16700 leader_election.cc:304] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:05:41.246403 17004 ts_tablet_manager.cc:1403] T e5c700237eb44cefbf7bc46c1daf08cd P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.253s	user 0.018s	sys 0.006s
I20260709 13:05:41.246685 16849 leader_election.cc:304] T 0b8add4b17084a6189509807b7e663e5 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:41.176291 16885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:41.247850 16885 raft_consensus.cc:2468] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 1.
I20260709 13:05:41.248705 16849 leader_election.cc:304] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228
W20260709 13:05:41.247867 16923 leader_election.cc:343] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.249933 17261 raft_consensus.cc:2749] T 0b8add4b17084a6189509807b7e663e5 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:41.250635 17261 raft_consensus.cc:2804] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:41.250974 17261 raft_consensus.cc:493] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.251284 17261 raft_consensus.cc:3060] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.252874 16700 leader_election.cc:304] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228
I20260709 13:05:41.248075 17140 raft_consensus.cc:2804] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:41.255039 17140 raft_consensus.cc:697] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:05:41.255776 17263 raft_consensus.cc:2804] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:41.256263 17263 raft_consensus.cc:493] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c4b86aae3a8f4a9082e639144ec46228)
I20260709 13:05:41.255897 17140 consensus_queue.cc:237] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:41.256429 17213 consensus_queue.cc:1048] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:41.255803 17266 consensus_queue.cc:1048] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:05:41.256670 17263 raft_consensus.cc:3060] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.260066 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "919c5788cd97463a9cf06abb18843943" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:41.260886 16958 raft_consensus.cc:2393] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d4e1b05bc80e43f7941ece113fb97ab0 in current term 1: Already voted for candidate 0086163b152644feb75caeac6f145533 in this term.
I20260709 13:05:41.261750 17004 raft_consensus.cc:359] T e5c700237eb44cefbf7bc46c1daf08cd P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:41.261878 16803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "919c5788cd97463a9cf06abb18843943" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:41.262580 17004 raft_consensus.cc:385] T e5c700237eb44cefbf7bc46c1daf08cd P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.264451 17004 raft_consensus.cc:740] T e5c700237eb44cefbf7bc46c1daf08cd P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.277500 16959 raft_consensus.cc:2468] T e81325b638b1493999310a726811b64b P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:41.275818 17004 consensus_queue.cc:260] T e5c700237eb44cefbf7bc46c1daf08cd P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:41.271906 17008 consensus_queue.cc:260] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:41.286811 16885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81325b638b1493999310a726811b64b" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:41.287489 16885 raft_consensus.cc:3060] T e81325b638b1493999310a726811b64b P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.285851 16803 raft_consensus.cc:3060] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.296828 17008 ts_tablet_manager.cc:1434] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.067s	user 0.004s	sys 0.003s
I20260709 13:05:41.297925 17008 tablet_bootstrap.cc:492] T 081841333d734eb1aa72ceb3f0ea2bf2 P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:41.301550 17281 raft_consensus.cc:493] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.301780 17274 raft_consensus.cc:493] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.302155 17281 raft_consensus.cc:515] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:41.302628 17274 raft_consensus.cc:515] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:41.304539 17281 leader_election.cc:290] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:41.305311 17274 leader_election.cc:290] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:41.310111 16885 raft_consensus.cc:2468] T e81325b638b1493999310a726811b64b P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:41.294332 16884 raft_consensus.cc:2468] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 1.
I20260709 13:05:41.313365 17196 raft_consensus.cc:493] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.313361 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "081841333d734eb1aa72ceb3f0ea2bf2" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:41.314069 16883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7123efeaee9f4ec78a1c475eab16c4c1" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:41.315155 16882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "919c5788cd97463a9cf06abb18843943" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:41.315485 17006 tablet_bootstrap.cc:492] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:41.334537 17006 ts_tablet_manager.cc:1403] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.345s	user 0.027s	sys 0.013s
I20260709 13:05:41.316224 16881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f86d4f23f8cd4451a50b1b38155d958e" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:41.317513 16730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7123efeaee9f4ec78a1c475eab16c4c1" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:41.318562 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d83d4d3d894b81a85b945ef97d79de" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:41.313056 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81325b638b1493999310a726811b64b" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:41.334896 17004 ts_tablet_manager.cc:1434] T e5c700237eb44cefbf7bc46c1daf08cd P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.088s	user 0.006s	sys 0.001s
I20260709 13:05:41.338001 16882 raft_consensus.cc:2393] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0086163b152644feb75caeac6f145533 in current term 1: Already voted for candidate d4e1b05bc80e43f7941ece113fb97ab0 in this term.
I20260709 13:05:41.339143 16732 raft_consensus.cc:2393] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0086163b152644feb75caeac6f145533 in current term 1: Already voted for candidate e2b87453fe654318abccdd1512b949c4 in this term.
I20260709 13:05:41.340481 16886 raft_consensus.cc:2393] T e81325b638b1493999310a726811b64b P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0086163b152644feb75caeac6f145533 in current term 1: Already voted for candidate c4b86aae3a8f4a9082e639144ec46228 in this term.
I20260709 13:05:41.320412 16803 raft_consensus.cc:2468] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:41.320500 16880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d83d4d3d894b81a85b945ef97d79de" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:41.283177 17261 raft_consensus.cc:515] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:41.324651 16923 leader_election.cc:304] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:41.348273 17274 raft_consensus.cc:2749] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:41.341419 17196 raft_consensus.cc:515] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:41.349572 17261 leader_election.cc:290] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:41.350716 16730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dd082201fe04ef89c842b6404fd522e" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:41.350973 17004 tablet_bootstrap.cc:492] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:41.351287 17196 leader_election.cc:290] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:41.342455 16883 raft_consensus.cc:3060] T 7123efeaee9f4ec78a1c475eab16c4c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:41.349892 16923 leader_election.cc:343] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.344668 16731 raft_consensus.cc:3060] T 612e472fbe674788986798c65836388f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:41.330556 16775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:45101, user_credentials={real_user=slave}} blocked reactor thread for 78289.4us
I20260709 13:05:41.345060 16736 raft_consensus.cc:2468] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 0.
I20260709 13:05:41.347693 16880 raft_consensus.cc:2393] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0086163b152644feb75caeac6f145533 in current term 1: Already voted for candidate e2b87453fe654318abccdd1512b949c4 in this term.
I20260709 13:05:41.355664 16700 leader_election.cc:304] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533
I20260709 13:05:41.342131 17281 raft_consensus.cc:493] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.361665 17196 raft_consensus.cc:2804] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:41.361737 17281 raft_consensus.cc:515] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:41.362200 17196 raft_consensus.cc:697] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:05:41.363854 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a5bbadcd9b48dba8fe272a18679ede" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:41.365805 16730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a5bbadcd9b48dba8fe272a18679ede" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:41.367025 17006 raft_consensus.cc:359] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:41.367831 17006 raft_consensus.cc:385] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.368165 17006 raft_consensus.cc:740] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.363219 17196 consensus_queue.cc:237] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:41.369726 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93629337d6b8477a9cd5895a157bf918" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:41.369764 16809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dd082201fe04ef89c842b6404fd522e" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:41.371733 16849 leader_election.cc:304] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:41.373121 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9078054426a34acbb94be2b69213b6c1" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:41.373804 17121 raft_consensus.cc:2749] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:05:41.375622 16700 leader_election.cc:343] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.376641 17006 consensus_queue.cc:260] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
W20260709 13:05:41.376547 16923 leader_election.cc:343] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.378816 16923 leader_election.cc:304] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:05:41.379280 17006 ts_tablet_manager.cc:1434] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 13:05:41.379892 17274 raft_consensus.cc:2804] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:41.381098 17274 raft_consensus.cc:493] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.381512 17274 raft_consensus.cc:3060] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.381894 17008 tablet_bootstrap.cc:654] T 081841333d734eb1aa72ceb3f0ea2bf2 P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.382182 16883 raft_consensus.cc:2468] T 7123efeaee9f4ec78a1c475eab16c4c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 1.
I20260709 13:05:41.384749 17281 leader_election.cc:290] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:41.390682 16849 leader_election.cc:304] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 0086163b152644feb75caeac6f145533
I20260709 13:05:41.335271 17263 raft_consensus.cc:515] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:41.397349 17004 tablet_bootstrap.cc:654] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.393611 17121 raft_consensus.cc:2804] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:41.402987 17121 raft_consensus.cc:697] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Becoming Leader. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:41.407037 17006 tablet_bootstrap.cc:492] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:41.412504 17150 raft_consensus.cc:493] T 59de6d1228da41da938527632da08f4d P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.413183 17150 raft_consensus.cc:515] T 59de6d1228da41da938527632da08f4d P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:41.414896 16809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9078054426a34acbb94be2b69213b6c1" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:41.415407 17150 leader_election.cc:290] T 59de6d1228da41da938527632da08f4d P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:41.417374 16775 leader_election.cc:304] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:05:41.418669 17007 tablet_bootstrap.cc:492] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:41.419224 17007 ts_tablet_manager.cc:1403] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.330s	user 0.019s	sys 0.019s
W20260709 13:05:41.405625 16923 leader_election.cc:343] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.404556 17121 consensus_queue.cc:237] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:41.423435 16803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81325b638b1493999310a726811b64b" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:41.420305 16883 raft_consensus.cc:1240] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Rejecting Update request from peer c4b86aae3a8f4a9082e639144ec46228 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:05:41.425931 16809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "919c5788cd97463a9cf06abb18843943" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:41.427076 16809 raft_consensus.cc:2393] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0086163b152644feb75caeac6f145533 in current term 1: Already voted for candidate d4e1b05bc80e43f7941ece113fb97ab0 in this term.
I20260709 13:05:41.420373 17150 raft_consensus.cc:2804] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:41.430241 17150 raft_consensus.cc:697] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Becoming Leader. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:05:41.431221 17150 consensus_queue.cc:237] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:41.435281 16807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "081841333d734eb1aa72ceb3f0ea2bf2" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:41.441155 17294 consensus_queue.cc:1048] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:41.441840 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93629337d6b8477a9cd5895a157bf918" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:41.446149 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59de6d1228da41da938527632da08f4d" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
W20260709 13:05:41.447956 16700 leader_election.cc:343] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.448432 16700 leader_election.cc:304] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:41.450124 17267 raft_consensus.cc:2749] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.454278 16883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59de6d1228da41da938527632da08f4d" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
W20260709 13:05:41.460440 16775 leader_election.cc:343] T 59de6d1228da41da938527632da08f4d P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.464977 17274 raft_consensus.cc:515] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:41.474995 17007 raft_consensus.cc:359] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:41.476094 17007 raft_consensus.cc:385] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.476900 17007 raft_consensus.cc:740] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.476037 17274 leader_election.cc:290] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:41.482311 17008 tablet_bootstrap.cc:492] T 081841333d734eb1aa72ceb3f0ea2bf2 P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
W20260709 13:05:41.485140 16923 leader_election.cc:343] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.508102 16923 leader_election.cc:304] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:41.509665 17281 raft_consensus.cc:2749] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:05:41.514851 16923 leader_election.cc:343] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.491282 17006 tablet_bootstrap.cc:654] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.494051 16809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f86d4f23f8cd4451a50b1b38155d958e" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:41.461869 17294 consensus_queue.cc:1059] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 13:05:41.518110 17008 ts_tablet_manager.cc:1403] T 081841333d734eb1aa72ceb3f0ea2bf2 P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.220s	user 0.008s	sys 0.011s
W20260709 13:05:41.501439 16775 leader_election.cc:343] T 59de6d1228da41da938527632da08f4d P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.484009 17007 consensus_queue.cc:260] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:41.515560 16923 leader_election.cc:304] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:41.518573 17138 consensus_queue.cc:1048] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:05:41.524564 17294 raft_consensus.cc:3055] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:41.525116 17294 raft_consensus.cc:740] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:05:41.526331 17294 consensus_queue.cc:260] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:41.528091 17294 raft_consensus.cc:3060] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.523468 16880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:41.532775 17007 ts_tablet_manager.cc:1434] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.113s	user 0.008s	sys 0.001s
I20260709 13:05:41.534917 17007 tablet_bootstrap.cc:492] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:41.519016 16775 leader_election.cc:304] T 59de6d1228da41da938527632da08f4d P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228; no voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:41.521919 17274 raft_consensus.cc:2749] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.542068 17150 raft_consensus.cc:2749] T 59de6d1228da41da938527632da08f4d P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.544072 17263 leader_election.cc:290] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
W20260709 13:05:41.521770 16923 leader_election.cc:343] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.558177 16809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:41.566524 16700 leader_election.cc:304] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:41.569391 17008 raft_consensus.cc:359] T 081841333d734eb1aa72ceb3f0ea2bf2 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:41.570912 17008 raft_consensus.cc:385] T 081841333d734eb1aa72ceb3f0ea2bf2 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.571156 17008 raft_consensus.cc:740] T 081841333d734eb1aa72ceb3f0ea2bf2 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.571233 17138 raft_consensus.cc:2749] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:41.580649 17138 consensus_queue.cc:1048] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:41.571720 17008 consensus_queue.cc:260] T 081841333d734eb1aa72ceb3f0ea2bf2 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:41.588691 17008 ts_tablet_manager.cc:1434] T 081841333d734eb1aa72ceb3f0ea2bf2 P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.070s	user 0.005s	sys 0.001s
I20260709 13:05:41.591670 16923 leader_election.cc:304] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: e2b87453fe654318abccdd1512b949c4
I20260709 13:05:41.592067 17008 tablet_bootstrap.cc:492] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:41.593801 17274 raft_consensus.cc:2804] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:41.594372 17274 raft_consensus.cc:697] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [term 1 LEADER]: Becoming Leader. State: Replica: 0086163b152644feb75caeac6f145533, State: Running, Role: LEADER
I20260709 13:05:41.591737 17004 tablet_bootstrap.cc:492] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:41.595782 17004 ts_tablet_manager.cc:1403] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.245s	user 0.029s	sys 0.041s
I20260709 13:05:41.597463 16923 leader_election.cc:304] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:41.598584 17253 raft_consensus.cc:2749] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:05:41.598942 16923 leader_election.cc:343] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.598865 17004 raft_consensus.cc:359] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:41.599383 17253 raft_consensus.cc:493] T e5c700237eb44cefbf7bc46c1daf08cd P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.599812 17004 raft_consensus.cc:385] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.600231 17004 raft_consensus.cc:740] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.600090 17253 raft_consensus.cc:515] T e5c700237eb44cefbf7bc46c1daf08cd P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:41.595273 17274 consensus_queue.cc:237] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
W20260709 13:05:41.603240 16666 debug-util.cc:398] Leaking SignalData structure 0x7b0800323740 after lost signal to thread 7167
W20260709 13:05:41.604679 16666 debug-util.cc:398] Leaking SignalData structure 0x7b0800245ec0 after lost signal to thread 16669
W20260709 13:05:41.606403 16666 debug-util.cc:398] Leaking SignalData structure 0x7b08003b0d20 after lost signal to thread 16761
W20260709 13:05:41.610009 16666 debug-util.cc:398] Leaking SignalData structure 0x7b08002468e0 after lost signal to thread 16836
I20260709 13:05:41.611344 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "081841333d734eb1aa72ceb3f0ea2bf2" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:41.611966 17253 leader_election.cc:290] T e5c700237eb44cefbf7bc46c1daf08cd P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:41.612119 16810 raft_consensus.cc:3060] T 081841333d734eb1aa72ceb3f0ea2bf2 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.614073 16730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "081841333d734eb1aa72ceb3f0ea2bf2" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:41.614871 16730 raft_consensus.cc:3060] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.615278 16809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f48e0f9f83da446a877cd06e19768929" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:41.614938 16633 catalog_manager.cc:5697] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3). New cstate: current_term: 1 leader_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:41.616974 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f48e0f9f83da446a877cd06e19768929" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:41.601162 17004 consensus_queue.cc:260] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
W20260709 13:05:41.611546 16666 debug-util.cc:398] Leaking SignalData structure 0x7b0800208b00 after lost signal to thread 16910
I20260709 13:05:41.632354 17007 tablet_bootstrap.cc:654] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.633065 17004 ts_tablet_manager.cc:1434] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 13:05:41.634161 16810 raft_consensus.cc:2468] T 081841333d734eb1aa72ceb3f0ea2bf2 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 1.
I20260709 13:05:41.637255 16631 catalog_manager.cc:5697] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "c4b86aae3a8f4a9082e639144ec46228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:41.649698 17004 tablet_bootstrap.cc:492] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533: Bootstrap starting.
W20260709 13:05:41.623405 16923 leader_election.cc:343] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:41.652778 16666 debug-util.cc:398] Leaking SignalData structure 0x7b08000f8000 after lost signal to thread 16984
I20260709 13:05:41.653447 16923 leader_election.cc:304] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:41.658560 17274 raft_consensus.cc:2749] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.660161 16923 leader_election.cc:304] T e81325b638b1493999310a726811b64b P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:41.662602 17274 raft_consensus.cc:2749] T e81325b638b1493999310a726811b64b P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.666266 17008 tablet_bootstrap.cc:654] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:41.662235 16923 leader_election.cc:343] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.667125 16923 leader_election.cc:304] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:41.666369 17006 tablet_bootstrap.cc:492] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:41.668350 17006 ts_tablet_manager.cc:1403] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.274s	user 0.030s	sys 0.011s
I20260709 13:05:41.671504 17006 raft_consensus.cc:359] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:41.672302 17006 raft_consensus.cc:385] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.672626 17006 raft_consensus.cc:740] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.673470 17006 consensus_queue.cc:260] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:41.676569 17006 ts_tablet_manager.cc:1434] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:05:41.676998 17274 raft_consensus.cc:2749] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.678498 17263 consensus_queue.cc:1048] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:41.679419 17121 raft_consensus.cc:493] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:41.654915 16666 debug-util.cc:398] Leaking SignalData structure 0x7b0800203300 after lost signal to thread 17235
I20260709 13:05:41.679916 17121 raft_consensus.cc:515] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:41.687765 17121 leader_election.cc:290] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:41.692593 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c700237eb44cefbf7bc46c1daf08cd" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:41.694142 17006 tablet_bootstrap.cc:492] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:41.694305 16810 raft_consensus.cc:2468] T e5c700237eb44cefbf7bc46c1daf08cd P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
I20260709 13:05:41.694777 16632 catalog_manager.cc:5697] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0086163b152644feb75caeac6f145533 (127.6.255.4). New cstate: current_term: 1 leader_uuid: "0086163b152644feb75caeac6f145533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:41.697997 16849 leader_election.cc:304] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
W20260709 13:05:41.678525 16923 leader_election.cc:343] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.702536 17261 raft_consensus.cc:2804] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:41.705864 17261 raft_consensus.cc:493] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.706202 17261 raft_consensus.cc:3060] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:41.725381 16923 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:45135, user_credentials={real_user=slave}} blocked reactor thread for 46884.5us
I20260709 13:05:41.709671 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c700237eb44cefbf7bc46c1daf08cd" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:41.753340 16730 raft_consensus.cc:2468] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 1.
I20260709 13:05:41.758155 17261 raft_consensus.cc:515] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:41.774415 17261 leader_election.cc:290] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:41.750592 16923 leader_election.cc:304] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:05:41.779503 17263 consensus_queue.cc:1048] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:41.784125 17004 tablet_bootstrap.cc:654] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.786306 17006 tablet_bootstrap.cc:654] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.755626 16960 raft_consensus.cc:2468] T e5c700237eb44cefbf7bc46c1daf08cd P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
I20260709 13:05:41.785635 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c700237eb44cefbf7bc46c1daf08cd" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:41.791020 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c700237eb44cefbf7bc46c1daf08cd" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:41.791661 16810 raft_consensus.cc:3060] T e5c700237eb44cefbf7bc46c1daf08cd P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.794714 17274 raft_consensus.cc:493] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.795593 17274 raft_consensus.cc:515] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:41.799003 17274 leader_election.cc:290] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
W20260709 13:05:41.799523 16923 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.2:45135, user_credentials={real_user=slave}} blocked reactor thread for 71080.1us
I20260709 13:05:41.806154 16880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c700237eb44cefbf7bc46c1daf08cd" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:41.806991 16880 raft_consensus.cc:2393] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0086163b152644feb75caeac6f145533 in current term 1: Already voted for candidate d4e1b05bc80e43f7941ece113fb97ab0 in this term.
I20260709 13:05:41.807785 17281 raft_consensus.cc:2804] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:41.808314 17281 raft_consensus.cc:697] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 [term 1 LEADER]: Becoming Leader. State: Replica: 0086163b152644feb75caeac6f145533, State: Running, Role: LEADER
I20260709 13:05:41.808825 17008 tablet_bootstrap.cc:492] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:41.809334 17008 ts_tablet_manager.cc:1403] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.217s	user 0.018s	sys 0.007s
I20260709 13:05:41.809122 17281 consensus_queue.cc:237] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:41.804873 16633 catalog_manager.cc:5697] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:41.817864 16633 catalog_manager.cc:5697] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:41.827135 17008 raft_consensus.cc:359] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:41.827915 17008 raft_consensus.cc:385] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.828167 17008 raft_consensus.cc:740] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.828946 17007 tablet_bootstrap.cc:492] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:41.828826 17008 consensus_queue.cc:260] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:41.829551 16803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86f5e278f54c419fac8c74ac63f80f74" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:41.830991 17008 ts_tablet_manager.cc:1434] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 13:05:41.838184 17007 ts_tablet_manager.cc:1403] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.304s	user 0.024s	sys 0.016s
I20260709 13:05:41.818117 16810 raft_consensus.cc:2468] T e5c700237eb44cefbf7bc46c1daf08cd P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:41.858482 16849 leader_election.cc:304] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 0086163b152644feb75caeac6f145533
I20260709 13:05:41.859248 17267 raft_consensus.cc:493] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.859711 17256 raft_consensus.cc:2804] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:41.859787 17267 raft_consensus.cc:515] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:41.860337 17256 raft_consensus.cc:697] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Becoming Leader. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:41.861397 17006 tablet_bootstrap.cc:492] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:41.861236 17256 consensus_queue.cc:237] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:41.861951 17006 ts_tablet_manager.cc:1403] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.169s	user 0.009s	sys 0.011s
I20260709 13:05:41.862437 17267 leader_election.cc:290] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:41.866086 17267 consensus_queue.cc:1048] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:41.866466 16734 raft_consensus.cc:1240] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Rejecting Update request from peer c4b86aae3a8f4a9082e639144ec46228 for earlier term 1. Current term is 2. Ops: []
W20260709 13:05:41.854266 16923 leader_election.cc:343] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.877825 17008 tablet_bootstrap.cc:492] T 9078054426a34acbb94be2b69213b6c1 P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:41.868079 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86f5e278f54c419fac8c74ac63f80f74" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:41.882999 17004 tablet_bootstrap.cc:492] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:41.868412 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c700237eb44cefbf7bc46c1daf08cd" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:41.876394 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7123efeaee9f4ec78a1c475eab16c4c1" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:41.883728 17004 ts_tablet_manager.cc:1403] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.234s	user 0.017s	sys 0.035s
I20260709 13:05:41.887362 17006 raft_consensus.cc:359] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:41.891772 17006 raft_consensus.cc:385] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.892150 17006 raft_consensus.cc:740] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.896725 17007 raft_consensus.cc:359] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:41.900184 17007 raft_consensus.cc:385] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.900465 17007 raft_consensus.cc:740] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.901247 17007 consensus_queue.cc:260] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:41.904060 17007 ts_tablet_manager.cc:1434] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.066s	user 0.005s	sys 0.001s
I20260709 13:05:41.905237 17007 tablet_bootstrap.cc:492] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:41.906560 16883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7123efeaee9f4ec78a1c475eab16c4c1" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:41.909927 16883 raft_consensus.cc:2393] T 7123efeaee9f4ec78a1c475eab16c4c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2b87453fe654318abccdd1512b949c4 in current term 1: Already voted for candidate 0086163b152644feb75caeac6f145533 in this term.
I20260709 13:05:41.910667 17150 consensus_queue.cc:1059] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:41.911664 17271 raft_consensus.cc:3055] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:41.912030 17271 raft_consensus.cc:740] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:05:41.912842 17271 consensus_queue.cc:260] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:41.881469 16923 leader_election.cc:304] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
W20260709 13:05:41.915488 16923 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.1:40785, user_credentials={real_user=slave}} blocked reactor thread for 63180.5us
I20260709 13:05:41.916486 17253 raft_consensus.cc:2749] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.918191 16700 leader_election.cc:304] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:41.919838 17140 consensus_queue.cc:1048] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:41.920543 17263 raft_consensus.cc:3060] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.920871 17004 raft_consensus.cc:359] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:41.921665 17004 raft_consensus.cc:385] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.921934 17004 raft_consensus.cc:740] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.910725 16812 raft_consensus.cc:2393] T e5c700237eb44cefbf7bc46c1daf08cd P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0086163b152644feb75caeac6f145533 in current term 1: Already voted for candidate d4e1b05bc80e43f7941ece113fb97ab0 in this term.
W20260709 13:05:41.918473 16923 leader_election.cc:343] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.926599 17004 consensus_queue.cc:260] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
W20260709 13:05:41.928853 16923 leader_election.cc:343] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.926641 17006 consensus_queue.cc:260] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:41.929304 16923 leader_election.cc:304] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:41.930943 16923 leader_election.cc:304] T e5c700237eb44cefbf7bc46c1daf08cd P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:41.933122 17007 tablet_bootstrap.cc:654] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:41.940603 16849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:45135, user_credentials={real_user=slave}} blocked reactor thread for 76618.7us
I20260709 13:05:41.946287 17006 ts_tablet_manager.cc:1434] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.084s	user 0.008s	sys 0.000s
I20260709 13:05:41.947613 17004 ts_tablet_manager.cc:1434] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.060s	user 0.008s	sys 0.000s
I20260709 13:05:41.932011 17253 raft_consensus.cc:2749] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.937373 17274 raft_consensus.cc:3060] T e5c700237eb44cefbf7bc46c1daf08cd P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.949658 17138 raft_consensus.cc:493] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.966143 17006 tablet_bootstrap.cc:492] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:41.972434 17138 raft_consensus.cc:515] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:41.974675 17138 leader_election.cc:290] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:41.978178 17006 tablet_bootstrap.cc:654] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.978758 17271 raft_consensus.cc:3060] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.993821 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20cdedb225464d5e812a56ed7a7d220f" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:41.997516 16958 raft_consensus.cc:2468] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 0.
I20260709 13:05:41.996646 17004 tablet_bootstrap.cc:492] T 61f747edd7934aac93044ba370e34d17 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:42.000677 16635 catalog_manager.cc:5697] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3). New cstate: current_term: 1 leader_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:42.014207 17324 raft_consensus.cc:493] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.014920 17324 raft_consensus.cc:515] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.024875 16880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20cdedb225464d5e812a56ed7a7d220f" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:42.031011 17150 consensus_queue.cc:1048] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:42.036542 17274 raft_consensus.cc:2749] T e5c700237eb44cefbf7bc46c1daf08cd P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:42.036994 17253 consensus_queue.cc:1048] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:42.037333 17324 leader_election.cc:290] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:42.042469 17261 consensus_queue.cc:1048] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:42.043908 17258 consensus_queue.cc:1048] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:42.053637 16809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caa2367ac34d43b5b1a27410b2055005" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:42.054418 16809 raft_consensus.cc:2468] T caa2367ac34d43b5b1a27410b2055005 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
I20260709 13:05:42.057391 16849 leader_election.cc:304] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:42.066931 17256 raft_consensus.cc:2804] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:42.067355 17256 raft_consensus.cc:493] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.067689 17256 raft_consensus.cc:3060] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:42.079169 16700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:37199, user_credentials={real_user=slave}} blocked reactor thread for 79657.1us
I20260709 13:05:42.088120 17006 tablet_bootstrap.cc:492] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:42.093321 17006 ts_tablet_manager.cc:1403] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.127s	user 0.016s	sys 0.018s
I20260709 13:05:42.093919 17004 tablet_bootstrap.cc:654] T 61f747edd7934aac93044ba370e34d17 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.093386 17008 tablet_bootstrap.cc:654] T 9078054426a34acbb94be2b69213b6c1 P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:42.095898 16700 leader_election.cc:343] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.084177 16632 catalog_manager.cc:5697] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0086163b152644feb75caeac6f145533 (127.6.255.4). New cstate: current_term: 1 leader_uuid: "0086163b152644feb75caeac6f145533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:42.102609 17007 tablet_bootstrap.cc:492] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:42.103178 17007 ts_tablet_manager.cc:1403] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.198s	user 0.017s	sys 0.001s
I20260709 13:05:42.082212 17263 raft_consensus.cc:2749] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:42.105785 16731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caa2367ac34d43b5b1a27410b2055005" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:42.098258 17271 consensus_queue.cc:1048] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:05:42.116941 16700 leader_election.cc:304] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:42.152194 17006 raft_consensus.cc:359] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.153275 17006 raft_consensus.cc:385] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.153688 17006 raft_consensus.cc:740] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.154561 17006 consensus_queue.cc:260] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.158906 17007 raft_consensus.cc:359] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.159837 17007 raft_consensus.cc:385] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.160476 17007 raft_consensus.cc:740] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.161293 17006 ts_tablet_manager.cc:1434] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.001s
I20260709 13:05:42.162322 17331 raft_consensus.cc:2804] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:05:42.163626 16849 leader_election.cc:343] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.164238 17331 raft_consensus.cc:493] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.164690 17331 raft_consensus.cc:3060] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.168185 17007 consensus_queue.cc:260] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.172530 17006 tablet_bootstrap.cc:492] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:42.174755 17256 raft_consensus.cc:515] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.176971 17256 leader_election.cc:290] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:42.178186 16803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caa2367ac34d43b5b1a27410b2055005" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:42.178853 16803 raft_consensus.cc:3060] T caa2367ac34d43b5b1a27410b2055005 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.192713 17007 ts_tablet_manager.cc:1434] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.089s	user 0.006s	sys 0.001s
I20260709 13:05:42.194146 17007 tablet_bootstrap.cc:492] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:42.189136 16803 raft_consensus.cc:2468] T caa2367ac34d43b5b1a27410b2055005 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:42.196565 17008 tablet_bootstrap.cc:492] T 9078054426a34acbb94be2b69213b6c1 P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:42.196091 16849 leader_election.cc:304] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:42.197052 17008 ts_tablet_manager.cc:1403] T 9078054426a34acbb94be2b69213b6c1 P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.320s	user 0.017s	sys 0.015s
I20260709 13:05:42.197647 17258 raft_consensus.cc:2804] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:42.198388 17258 raft_consensus.cc:697] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Becoming Leader. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:42.199287 17258 consensus_queue.cc:237] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.200078 17008 raft_consensus.cc:359] T 9078054426a34acbb94be2b69213b6c1 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.200893 17008 raft_consensus.cc:385] T 9078054426a34acbb94be2b69213b6c1 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.201167 17008 raft_consensus.cc:740] T 9078054426a34acbb94be2b69213b6c1 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.201849 17008 consensus_queue.cc:260] T 9078054426a34acbb94be2b69213b6c1 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.204289 17008 ts_tablet_manager.cc:1434] T 9078054426a34acbb94be2b69213b6c1 P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:05:42.205276 17008 tablet_bootstrap.cc:492] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:42.213171 17280 consensus_queue.cc:1048] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:42.214576 17004 tablet_bootstrap.cc:492] T 61f747edd7934aac93044ba370e34d17 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:42.215080 17004 ts_tablet_manager.cc:1403] T 61f747edd7934aac93044ba370e34d17 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.250s	user 0.018s	sys 0.000s
I20260709 13:05:42.217644 17008 tablet_bootstrap.cc:654] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.218925 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caa2367ac34d43b5b1a27410b2055005" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:42.220681 17004 raft_consensus.cc:359] T 61f747edd7934aac93044ba370e34d17 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:42.221361 17004 raft_consensus.cc:385] T 61f747edd7934aac93044ba370e34d17 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.221647 17004 raft_consensus.cc:740] T 61f747edd7934aac93044ba370e34d17 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.222325 17004 consensus_queue.cc:260] T 61f747edd7934aac93044ba370e34d17 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
W20260709 13:05:42.223744 16849 leader_election.cc:343] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.227139 17004 ts_tablet_manager.cc:1434] T 61f747edd7934aac93044ba370e34d17 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:05:42.230263 17004 tablet_bootstrap.cc:492] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:42.240343 17006 tablet_bootstrap.cc:654] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.247282 17004 tablet_bootstrap.cc:654] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.284118 16635 catalog_manager.cc:5697] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3). New cstate: current_term: 1 leader_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:42.277071 17150 raft_consensus.cc:493] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.291419 17280 raft_consensus.cc:493] T 61f747edd7934aac93044ba370e34d17 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:42.325551 16775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:45101, user_credentials={real_user=slave}} blocked reactor thread for 47371us
I20260709 13:05:42.317483 17258 raft_consensus.cc:493] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.296352 17331 raft_consensus.cc:515] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:42.312547 17150 raft_consensus.cc:515] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
W20260709 13:05:42.340955 16683 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:42.343802 17004 tablet_bootstrap.cc:492] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:42.344498 17004 ts_tablet_manager.cc:1403] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.115s	user 0.020s	sys 0.015s
I20260709 13:05:42.319867 17280 raft_consensus.cc:515] T 61f747edd7934aac93044ba370e34d17 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:42.363559 17280 leader_election.cc:290] T 61f747edd7934aac93044ba370e34d17 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:42.352070 17274 consensus_queue.cc:1048] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 13:05:42.375918 17150 leader_election.cc:290] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:42.330781 17258 raft_consensus.cc:515] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.383057 17008 tablet_bootstrap.cc:492] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:42.383548 17008 ts_tablet_manager.cc:1403] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.179s	user 0.022s	sys 0.048s
I20260709 13:05:42.385490 16734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f747edd7934aac93044ba370e34d17" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:42.388265 17331 leader_election.cc:290] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
W20260709 13:05:42.389946 16923 leader_election.cc:343] T 61f747edd7934aac93044ba370e34d17 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.391139 17006 tablet_bootstrap.cc:492] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:42.391801 17006 ts_tablet_manager.cc:1403] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.224s	user 0.011s	sys 0.004s
I20260709 13:05:42.394496 17007 tablet_bootstrap.cc:654] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.395246 16734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c19ee4785104276aec22ba772dbc6ac" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:42.396508 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20cdedb225464d5e812a56ed7a7d220f" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:42.397157 16958 raft_consensus.cc:3060] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.398278 17258 leader_election.cc:290] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:42.400300 16881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c19ee4785104276aec22ba772dbc6ac" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:42.400377 16880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f747edd7934aac93044ba370e34d17" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:42.401057 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59de6d1228da41da938527632da08f4d" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:42.401731 16812 raft_consensus.cc:2468] T 59de6d1228da41da938527632da08f4d P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
W20260709 13:05:42.402187 16923 leader_election.cc:343] T 61f747edd7934aac93044ba370e34d17 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.402616 16923 leader_election.cc:304] T 61f747edd7934aac93044ba370e34d17 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:42.403646 17281 raft_consensus.cc:2749] T 61f747edd7934aac93044ba370e34d17 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:42.405144 16849 leader_election.cc:304] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:42.405967 17256 consensus_queue.cc:1048] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:42.406401 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20cdedb225464d5e812a56ed7a7d220f" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:42.407382 17256 raft_consensus.cc:2804] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:42.407759 17256 raft_consensus.cc:493] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.408087 17256 raft_consensus.cc:3060] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.408198 17004 raft_consensus.cc:359] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:42.408919 17004 raft_consensus.cc:385] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.409183 17004 raft_consensus.cc:740] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.409837 17004 consensus_queue.cc:260] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:42.418766 16734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59de6d1228da41da938527632da08f4d" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:42.420807 17006 raft_consensus.cc:359] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.421602 17006 raft_consensus.cc:385] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.421185 17256 raft_consensus.cc:515] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.421913 17006 raft_consensus.cc:740] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.423607 17256 leader_election.cc:290] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:42.424398 17006 consensus_queue.cc:260] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.425921 17004 ts_tablet_manager.cc:1434] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.081s	user 0.005s	sys 0.001s
I20260709 13:05:42.427460 17004 tablet_bootstrap.cc:492] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
W20260709 13:05:42.415401 16700 leader_election.cc:343] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.433938 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59de6d1228da41da938527632da08f4d" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:42.435837 17274 consensus_queue.cc:1048] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:05:42.442345 16849 leader_election.cc:343] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.442430 17256 raft_consensus.cc:493] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.442955 17256 raft_consensus.cc:515] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.444906 17006 ts_tablet_manager.cc:1434] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.053s	user 0.008s	sys 0.000s
I20260709 13:05:42.445251 17256 leader_election.cc:290] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:42.445873 17006 tablet_bootstrap.cc:492] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:42.449707 16809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9078054426a34acbb94be2b69213b6c1" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:42.450379 16809 raft_consensus.cc:2468] T 9078054426a34acbb94be2b69213b6c1 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
I20260709 13:05:42.451821 16958 raft_consensus.cc:2468] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 1.
I20260709 13:05:42.451949 16849 leader_election.cc:304] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:42.452785 17258 raft_consensus.cc:2804] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:42.453152 17258 raft_consensus.cc:493] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.453482 17258 raft_consensus.cc:3060] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:42.458042 16775 leader_election.cc:343] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:42.459859 16775 leader_election.cc:343] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.460264 16775 leader_election.cc:304] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228; no voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:42.462459 16700 leader_election.cc:304] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: d4e1b05bc80e43f7941ece113fb97ab0
W20260709 13:05:42.463938 16849 leader_election.cc:343] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.465803 16731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9078054426a34acbb94be2b69213b6c1" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:42.466498 16731 raft_consensus.cc:2468] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
I20260709 13:05:42.466804 17267 raft_consensus.cc:2804] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:42.467463 17267 raft_consensus.cc:697] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:05:42.468461 17267 consensus_queue.cc:237] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:42.472119 17338 raft_consensus.cc:2749] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:42.484189 17256 consensus_queue.cc:1048] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:42.496246 17007 tablet_bootstrap.cc:492] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:42.496834 17007 ts_tablet_manager.cc:1403] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.303s	user 0.016s	sys 0.006s
I20260709 13:05:42.497296 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59de6d1228da41da938527632da08f4d" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:42.498029 16810 raft_consensus.cc:3060] T 59de6d1228da41da938527632da08f4d P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.499428 17007 raft_consensus.cc:359] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.500124 17007 raft_consensus.cc:385] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.500370 17007 raft_consensus.cc:740] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.501011 17007 consensus_queue.cc:260] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.501726 17008 raft_consensus.cc:359] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:42.502449 17008 raft_consensus.cc:385] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.502705 17008 raft_consensus.cc:740] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.503404 17007 ts_tablet_manager.cc:1434] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:05:42.503513 17008 consensus_queue.cc:260] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:42.504509 17007 tablet_bootstrap.cc:492] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:42.510941 17258 raft_consensus.cc:515] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.517817 17007 tablet_bootstrap.cc:654] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.524637 17008 ts_tablet_manager.cc:1434] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.141s	user 0.005s	sys 0.001s
I20260709 13:05:42.527078 16730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9078054426a34acbb94be2b69213b6c1" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:42.527787 16730 raft_consensus.cc:3060] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.534983 17004 tablet_bootstrap.cc:654] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.535107 16730 raft_consensus.cc:2468] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:42.540081 17008 tablet_bootstrap.cc:492] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:42.540992 17258 leader_election.cc:290] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:42.546334 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9078054426a34acbb94be2b69213b6c1" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:42.547235 16812 raft_consensus.cc:3060] T 9078054426a34acbb94be2b69213b6c1 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.551486 16849 leader_election.cc:304] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:05:42.560636 17322 raft_consensus.cc:2804] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:42.561165 17322 raft_consensus.cc:697] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Becoming Leader. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:42.562023 17322 consensus_queue.cc:237] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.576493 16631 catalog_manager.cc:5697] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:42.578626 17331 raft_consensus.cc:493] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.579159 17331 raft_consensus.cc:515] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.590417 16803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caa2367ac34d43b5b1a27410b2055005" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:42.591306 16803 raft_consensus.cc:2393] T caa2367ac34d43b5b1a27410b2055005 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2b87453fe654318abccdd1512b949c4 in current term 1: Already voted for candidate d4e1b05bc80e43f7941ece113fb97ab0 in this term.
I20260709 13:05:42.591590 16810 raft_consensus.cc:2468] T 59de6d1228da41da938527632da08f4d P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:42.601704 16849 leader_election.cc:304] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: e2b87453fe654318abccdd1512b949c4
I20260709 13:05:42.604993 17258 raft_consensus.cc:2804] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:42.605726 17258 raft_consensus.cc:697] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Becoming Leader. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:42.606930 16812 raft_consensus.cc:2468] T 9078054426a34acbb94be2b69213b6c1 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:42.608985 17258 consensus_queue.cc:237] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.610140 17331 leader_election.cc:290] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:42.607582 16632 catalog_manager.cc:5697] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3). New cstate: current_term: 1 leader_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:42.611209 17007 tablet_bootstrap.cc:492] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:42.611685 17007 ts_tablet_manager.cc:1403] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.107s	user 0.025s	sys 0.010s
I20260709 13:05:42.614159 17006 tablet_bootstrap.cc:654] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.615976 17331 raft_consensus.cc:493] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.616532 17331 raft_consensus.cc:515] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.618909 17331 leader_election.cc:290] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:42.621078 16881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caa2367ac34d43b5b1a27410b2055005" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:42.622771 17007 raft_consensus.cc:359] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:42.623526 17007 raft_consensus.cc:385] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.623786 17007 raft_consensus.cc:740] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.624431 17007 consensus_queue.cc:260] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:42.627625 16803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a7231efacd1463c96b35c3d03edce55" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:42.630333 17007 ts_tablet_manager.cc:1434] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.003s
I20260709 13:05:42.629698 17256 consensus_queue.cc:1048] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:42.633603 17007 tablet_bootstrap.cc:492] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:42.635965 17008 tablet_bootstrap.cc:654] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:42.644461 16700 leader_election.cc:343] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.654867 17280 raft_consensus.cc:493] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.655506 17280 raft_consensus.cc:515] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:42.659085 17280 leader_election.cc:290] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:42.660028 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be51f207c3bc4276b220a03e2d9d4db4" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:42.643846 16881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a7231efacd1463c96b35c3d03edce55" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:42.662201 17007 tablet_bootstrap.cc:654] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:42.662339 16923 leader_election.cc:343] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.664780 16700 leader_election.cc:304] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:42.665624 17263 raft_consensus.cc:3060] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:42.665910 16700 leader_election.cc:343] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.666345 16700 leader_election.cc:304] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:42.669302 17350 raft_consensus.cc:493] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.670011 17350 raft_consensus.cc:515] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.676832 17350 leader_election.cc:290] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:42.683044 17263 raft_consensus.cc:2749] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:42.688285 16730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be51f207c3bc4276b220a03e2d9d4db4" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
W20260709 13:05:42.695575 16923 leader_election.cc:343] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.696106 16923 leader_election.cc:304] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:42.697014 17004 tablet_bootstrap.cc:492] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:42.697810 17004 ts_tablet_manager.cc:1403] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.271s	user 0.008s	sys 0.007s
I20260709 13:05:42.693696 16881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dd082201fe04ef89c842b6404fd522e" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:42.695454 17331 raft_consensus.cc:2749] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:42.702138 17280 raft_consensus.cc:2749] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:42.709482 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dd082201fe04ef89c842b6404fd522e" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:42.706233 16881 raft_consensus.cc:2468] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 2.
I20260709 13:05:42.738910 17006 tablet_bootstrap.cc:492] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:42.739398 17006 ts_tablet_manager.cc:1403] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.294s	user 0.011s	sys 0.002s
I20260709 13:05:42.741721 16775 leader_election.cc:304] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:42.735939 17324 raft_consensus.cc:493] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.742563 17324 raft_consensus.cc:515] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.731009 17267 raft_consensus.cc:493] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c4b86aae3a8f4a9082e639144ec46228)
I20260709 13:05:42.746384 17267 raft_consensus.cc:515] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.748346 17267 leader_election.cc:290] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:42.749897 16736 raft_consensus.cc:2468] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:42.756266 17006 raft_consensus.cc:359] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:42.756795 17006 raft_consensus.cc:385] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.757035 17006 raft_consensus.cc:740] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.749846 16730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c19ee4785104276aec22ba772dbc6ac" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:42.757804 17006 consensus_queue.cc:260] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:42.763125 16803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dd082201fe04ef89c842b6404fd522e" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:42.765396 17006 ts_tablet_manager.cc:1434] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 13:05:42.766472 17006 tablet_bootstrap.cc:492] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:42.766979 17008 tablet_bootstrap.cc:492] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:42.767973 17008 ts_tablet_manager.cc:1403] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.228s	user 0.020s	sys 0.040s
I20260709 13:05:42.770967 16632 catalog_manager.cc:5697] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3). New cstate: current_term: 1 leader_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:42.772462 17008 raft_consensus.cc:359] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
W20260709 13:05:42.773854 16849 leader_election.cc:343] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.774114 17008 raft_consensus.cc:385] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.774571 17008 raft_consensus.cc:740] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.776654 17008 consensus_queue.cc:260] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:42.753091 17350 raft_consensus.cc:2804] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:42.779290 17350 raft_consensus.cc:493] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.779644 17350 raft_consensus.cc:3060] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.780577 17007 tablet_bootstrap.cc:492] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:42.781096 17007 ts_tablet_manager.cc:1403] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.148s	user 0.044s	sys 0.045s
I20260709 13:05:42.783924 17007 raft_consensus.cc:359] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:42.784638 17007 raft_consensus.cc:385] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.784920 17007 raft_consensus.cc:740] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.785739 17007 consensus_queue.cc:260] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:42.788463 17007 ts_tablet_manager.cc:1434] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:05:42.788614 17324 leader_election.cc:290] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:42.788919 17008 ts_tablet_manager.cc:1434] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.004s
I20260709 13:05:42.789705 17007 tablet_bootstrap.cc:492] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:42.790069 17008 tablet_bootstrap.cc:492] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:42.790264 17004 raft_consensus.cc:359] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:42.790918 17004 raft_consensus.cc:385] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.791182 17004 raft_consensus.cc:740] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.797061 17008 tablet_bootstrap.cc:654] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.797979 17007 tablet_bootstrap.cc:654] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.802997 17004 consensus_queue.cc:260] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:42.819084 17004 ts_tablet_manager.cc:1434] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.121s	user 0.008s	sys 0.000s
I20260709 13:05:42.819962 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c19ee4785104276aec22ba772dbc6ac" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:42.820802 16812 raft_consensus.cc:2468] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
I20260709 13:05:42.822101 16849 leader_election.cc:304] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: e2b87453fe654318abccdd1512b949c4
I20260709 13:05:42.823360 17331 raft_consensus.cc:493] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.823588 17121 raft_consensus.cc:2804] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:42.824142 17121 raft_consensus.cc:493] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.823973 17331 raft_consensus.cc:515] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:42.824590 17121 raft_consensus.cc:3060] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.826382 17331 leader_election.cc:290] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:42.827554 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dd082201fe04ef89c842b6404fd522e" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:42.828461 16884 raft_consensus.cc:2393] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2b87453fe654318abccdd1512b949c4 in current term 2: Already voted for candidate d4e1b05bc80e43f7941ece113fb97ab0 in this term.
I20260709 13:05:42.829182 17006 tablet_bootstrap.cc:654] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.829746 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec647c6ccd641b59808817d8dca2235" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
W20260709 13:05:42.832305 16700 leader_election.cc:343] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.869518 16956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec647c6ccd641b59808817d8dca2235" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:42.882987 17350 raft_consensus.cc:515] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.885032 17350 leader_election.cc:290] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 3 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
W20260709 13:05:42.885139 16700 leader_election.cc:343] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0086163b152644feb75caeac6f145533 (127.6.255.4:37199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.885600 16700 leader_election.cc:304] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228
I20260709 13:05:42.886535 16803 raft_consensus.cc:2368] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2b87453fe654318abccdd1512b949c4 for earlier term 2. Current term is 3.
I20260709 13:05:42.887223 17331 raft_consensus.cc:2749] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:42.892699 16700 leader_election.cc:400] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Vote denied by peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135) with higher term. Message: Invalid argument: T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2b87453fe654318abccdd1512b949c4 for earlier term 2. Current term is 3.
I20260709 13:05:42.893098 16700 leader_election.cc:403] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:42.894923 17121 raft_consensus.cc:515] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.896764 17331 raft_consensus.cc:3060] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.899544 17121 leader_election.cc:290] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:42.906216 17004 tablet_bootstrap.cc:492] T 6ec647c6ccd641b59808817d8dca2235 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:42.906054 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dd082201fe04ef89c842b6404fd522e" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:42.909998 17350 raft_consensus.cc:493] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.910534 17350 raft_consensus.cc:515] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:42.917338 17331 raft_consensus.cc:2749] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135) with higher term. Message: Invalid argument: T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2b87453fe654318abccdd1512b949c4 for earlier term 2. Current term is 3.
I20260709 13:05:42.920892 17004 tablet_bootstrap.cc:654] T 6ec647c6ccd641b59808817d8dca2235 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:42.926848 16700 consensus_peers.cc:597] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 -> Peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Couldn't send request to peer d4e1b05bc80e43f7941ece113fb97ab0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:42.929154 17350 leader_election.cc:290] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
W20260709 13:05:42.933646 16849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:45135, user_credentials={real_user=slave}} blocked reactor thread for 106759us
I20260709 13:05:42.925177 16736 raft_consensus.cc:2468] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 3.
I20260709 13:05:42.931934 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93629337d6b8477a9cd5895a157bf918" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:42.944965 16734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c19ee4785104276aec22ba772dbc6ac" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:42.952621 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c19ee4785104276aec22ba772dbc6ac" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:42.953294 16812 raft_consensus.cc:3060] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.939436 16960 raft_consensus.cc:2468] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 0.
I20260709 13:05:42.957852 16775 leader_election.cc:304] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:05:42.958441 17350 raft_consensus.cc:2804] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:42.958803 17350 raft_consensus.cc:493] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.958880 17006 tablet_bootstrap.cc:492] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:42.959190 17350 raft_consensus.cc:3060] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.959527 17006 ts_tablet_manager.cc:1403] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.193s	user 0.009s	sys 0.041s
I20260709 13:05:42.961673 17007 tablet_bootstrap.cc:492] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:42.962196 17007 ts_tablet_manager.cc:1403] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.173s	user 0.016s	sys 0.003s
I20260709 13:05:42.967015 17008 tablet_bootstrap.cc:492] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:42.967557 17008 ts_tablet_manager.cc:1403] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.178s	user 0.020s	sys 0.014s
I20260709 13:05:42.970567 17008 raft_consensus.cc:359] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:42.971225 17008 raft_consensus.cc:385] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.962833 16812 raft_consensus.cc:2468] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:42.973402 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93629337d6b8477a9cd5895a157bf918" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:42.971578 17004 tablet_bootstrap.cc:492] T 6ec647c6ccd641b59808817d8dca2235 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:42.974650 17004 ts_tablet_manager.cc:1403] T 6ec647c6ccd641b59808817d8dca2235 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.086s	user 0.006s	sys 0.014s
I20260709 13:05:42.975814 17006 raft_consensus.cc:359] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:42.976536 17006 raft_consensus.cc:385] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.971570 17008 raft_consensus.cc:740] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.972204 16881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dd082201fe04ef89c842b6404fd522e" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:42.977790 16881 raft_consensus.cc:3060] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.978135 17007 raft_consensus.cc:359] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.978842 17007 raft_consensus.cc:385] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.979096 17007 raft_consensus.cc:740] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.990324 17008 consensus_queue.cc:260] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:42.993257 17004 raft_consensus.cc:359] T 6ec647c6ccd641b59808817d8dca2235 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:42.994163 17004 raft_consensus.cc:385] T 6ec647c6ccd641b59808817d8dca2235 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.994486 17004 raft_consensus.cc:740] T 6ec647c6ccd641b59808817d8dca2235 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.995241 16775 leader_election.cc:304] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:05:42.996482 17351 raft_consensus.cc:2804] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:42.996982 17351 raft_consensus.cc:697] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 3 LEADER]: Becoming Leader. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:05:42.997859 17351 consensus_queue.cc:237] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:42.976809 17006 raft_consensus.cc:740] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.005980 17007 consensus_queue.cc:260] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:43.008493 17331 consensus_queue.cc:1048] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:42.993080 17008 ts_tablet_manager.cc:1434] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.004s
I20260709 13:05:43.013671 17008 tablet_bootstrap.cc:492] T 02d877efcf5049a89eacf23b58ba27b9 P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:43.020100 17258 raft_consensus.cc:493] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c4b86aae3a8f4a9082e639144ec46228)
W20260709 13:05:43.020838 16775 leader_election.cc:343] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.020697 17258 raft_consensus.cc:515] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:43.008543 17006 consensus_queue.cc:260] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:43.016157 17004 consensus_queue.cc:260] T 6ec647c6ccd641b59808817d8dca2235 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:43.007293 16881 raft_consensus.cc:2468] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 3.
I20260709 13:05:43.024834 17258 leader_election.cc:290] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:43.026274 16849 leader_election.cc:304] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:43.027299 17004 ts_tablet_manager.cc:1434] T 6ec647c6ccd641b59808817d8dca2235 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.052s	user 0.007s	sys 0.000s
I20260709 13:05:43.037596 17322 raft_consensus.cc:2804] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:05:43.037699 16849 leader_election.cc:343] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.038283 17004 tablet_bootstrap.cc:492] T 2593417c1fe4457c87cb2a01cf1f65a3 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:43.039155 17367 consensus_queue.cc:1048] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:43.039631 17350 raft_consensus.cc:515] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:43.041724 17350 leader_election.cc:290] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:43.043709 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93629337d6b8477a9cd5895a157bf918" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:43.044400 16958 raft_consensus.cc:3060] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.033766 17007 ts_tablet_manager.cc:1434] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.071s	user 0.008s	sys 0.000s
I20260709 13:05:43.038225 17322 raft_consensus.cc:697] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Becoming Leader. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:43.046209 17322 consensus_queue.cc:237] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:43.049705 16734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:43.050532 16734 raft_consensus.cc:2393] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4e1b05bc80e43f7941ece113fb97ab0 in current term 2: Already voted for candidate e2b87453fe654318abccdd1512b949c4 in this term.
I20260709 13:05:43.052678 16880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93629337d6b8477a9cd5895a157bf918" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:43.054445 17007 tablet_bootstrap.cc:492] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:43.031551 17008 tablet_bootstrap.cc:654] T 02d877efcf5049a89eacf23b58ba27b9 P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:43.057722 16775 leader_election.cc:343] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.069173 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:43.070022 16812 raft_consensus.cc:2468] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 2.
I20260709 13:05:43.070796 17004 tablet_bootstrap.cc:654] T 2593417c1fe4457c87cb2a01cf1f65a3 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.043191 17006 ts_tablet_manager.cc:1434] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.083s	user 0.006s	sys 0.002s
I20260709 13:05:43.082324 17008 tablet_bootstrap.cc:492] T 02d877efcf5049a89eacf23b58ba27b9 P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:43.085860 17007 tablet_bootstrap.cc:654] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.085876 17008 ts_tablet_manager.cc:1403] T 02d877efcf5049a89eacf23b58ba27b9 P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.072s	user 0.013s	sys 0.002s
I20260709 13:05:43.088092 16631 catalog_manager.cc:5697] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "c4b86aae3a8f4a9082e639144ec46228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:43.099154 16849 leader_election.cc:304] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:43.100057 17324 raft_consensus.cc:2804] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:43.100493 17324 raft_consensus.cc:493] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c4b86aae3a8f4a9082e639144ec46228)
I20260709 13:05:43.100929 17324 raft_consensus.cc:3060] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:43.103217 17008 raft_consensus.cc:359] T 02d877efcf5049a89eacf23b58ba27b9 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:43.080374 16958 raft_consensus.cc:2468] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:43.103972 17008 raft_consensus.cc:385] T 02d877efcf5049a89eacf23b58ba27b9 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.104813 16775 leader_election.cc:304] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:43.105913 17350 raft_consensus.cc:2804] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:43.106359 17350 raft_consensus.cc:697] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Becoming Leader. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:05:43.107107 17350 consensus_queue.cc:237] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:43.109975 17006 tablet_bootstrap.cc:492] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:43.104416 17008 raft_consensus.cc:740] T 02d877efcf5049a89eacf23b58ba27b9 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.124202 17370 consensus_queue.cc:1048] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:43.119951 17331 raft_consensus.cc:493] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.126824 17331 raft_consensus.cc:515] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:43.130429 17331 leader_election.cc:290] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:43.132485 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f48e0f9f83da446a877cd06e19768929" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:43.132897 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f48e0f9f83da446a877cd06e19768929" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:43.133247 16812 raft_consensus.cc:2468] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 0.
I20260709 13:05:43.133689 16958 raft_consensus.cc:2468] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 0.
I20260709 13:05:43.134465 17324 raft_consensus.cc:515] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:43.136183 16700 leader_election.cc:304] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:05:43.122706 17008 consensus_queue.cc:260] T 02d877efcf5049a89eacf23b58ba27b9 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:43.149737 16730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:43.150672 16730 raft_consensus.cc:2393] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d4e1b05bc80e43f7941ece113fb97ab0 in current term 2: Already voted for candidate e2b87453fe654318abccdd1512b949c4 in this term.
I20260709 13:05:43.145709 17263 raft_consensus.cc:2804] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:43.151926 17263 raft_consensus.cc:493] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.152285 17263 raft_consensus.cc:3060] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.155862 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:43.155563 16635 catalog_manager.cc:5697] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3). New cstate: current_term: 1 leader_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:43.160248 17006 tablet_bootstrap.cc:654] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.174011 16812 raft_consensus.cc:2468] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 2.
I20260709 13:05:43.171641 17263 raft_consensus.cc:515] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
W20260709 13:05:43.175361 16849 consensus_peers.cc:597] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:43.176600 16849 leader_election.cc:304] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: e2b87453fe654318abccdd1512b949c4
I20260709 13:05:43.177553 17370 raft_consensus.cc:2804] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:43.178073 17370 raft_consensus.cc:697] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 LEADER]: Becoming Leader. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:43.178893 17370 consensus_queue.cc:237] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:43.182155 17263 leader_election.cc:290] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:43.184435 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f48e0f9f83da446a877cd06e19768929" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:43.187629 17351 raft_consensus.cc:493] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.188278 17351 raft_consensus.cc:515] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:43.189483 17324 leader_election.cc:290] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:43.187649 16958 raft_consensus.cc:3060] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.190850 17351 leader_election.cc:290] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:43.193465 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec647c6ccd641b59808817d8dca2235" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:43.195534 16960 raft_consensus.cc:2468] T 6ec647c6ccd641b59808817d8dca2235 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 0.
I20260709 13:05:43.197170 16775 leader_election.cc:304] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:05:43.199893 17008 ts_tablet_manager.cc:1434] T 02d877efcf5049a89eacf23b58ba27b9 P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.113s	user 0.007s	sys 0.000s
I20260709 13:05:43.201215 17008 tablet_bootstrap.cc:492] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:43.202188 17351 raft_consensus.cc:2804] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:43.203557 17004 tablet_bootstrap.cc:492] T 2593417c1fe4457c87cb2a01cf1f65a3 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:43.245419 17004 ts_tablet_manager.cc:1403] T 2593417c1fe4457c87cb2a01cf1f65a3 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.207s	user 0.029s	sys 0.010s
I20260709 13:05:43.247978 17004 raft_consensus.cc:359] T 2593417c1fe4457c87cb2a01cf1f65a3 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:43.248546 17004 raft_consensus.cc:385] T 2593417c1fe4457c87cb2a01cf1f65a3 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.248837 17004 raft_consensus.cc:740] T 2593417c1fe4457c87cb2a01cf1f65a3 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.212072 17007 tablet_bootstrap.cc:492] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:43.211858 17322 consensus_queue.cc:1048] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:43.249768 17007 ts_tablet_manager.cc:1403] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.196s	user 0.018s	sys 0.000s
I20260709 13:05:43.249565 17004 consensus_queue.cc:260] T 2593417c1fe4457c87cb2a01cf1f65a3 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:43.212823 17274 raft_consensus.cc:493] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.221889 17367 raft_consensus.cc:493] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.250711 17274 raft_consensus.cc:515] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:43.250942 17367 raft_consensus.cc:515] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:43.226329 17008 tablet_bootstrap.cc:654] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.253073 17274 leader_election.cc:290] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:43.252822 17007 raft_consensus.cc:359] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:43.253703 17007 raft_consensus.cc:385] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.254041 17007 raft_consensus.cc:740] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.254755 17007 consensus_queue.cc:260] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:43.231604 17006 tablet_bootstrap.cc:492] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:43.255831 17006 ts_tablet_manager.cc:1403] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.146s	user 0.010s	sys 0.006s
I20260709 13:05:43.252887 17367 leader_election.cc:290] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:43.257167 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d877efcf5049a89eacf23b58ba27b9" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:43.257789 16880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d877efcf5049a89eacf23b58ba27b9" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:43.238525 16734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec647c6ccd641b59808817d8dca2235" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:43.259110 16734 raft_consensus.cc:2468] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 0.
I20260709 13:05:43.259572 17274 raft_consensus.cc:493] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.260418 17007 ts_tablet_manager.cc:1434] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:05:43.244342 17324 consensus_queue.cc:1048] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:43.229873 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f48e0f9f83da446a877cd06e19768929" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:43.261512 17007 tablet_bootstrap.cc:492] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
W20260709 13:05:43.261768 16923 leader_election.cc:343] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.260061 17274 raft_consensus.cc:515] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:43.261864 16810 raft_consensus.cc:3060] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.267530 16880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a5bbadcd9b48dba8fe272a18679ede" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:43.267221 16635 catalog_manager.cc:5697] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 reported cstate change: term changed from 1 to 2, leader changed from c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2) to d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3). New cstate: current_term: 2 leader_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:43.268637 17007 tablet_bootstrap.cc:654] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:43.269022 16923 leader_election.cc:343] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.258704 17006 raft_consensus.cc:359] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:43.270165 17006 raft_consensus.cc:385] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.270452 17006 raft_consensus.cc:740] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.270783 17274 leader_election.cc:290] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:43.271194 17006 consensus_queue.cc:260] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:43.240832 16958 raft_consensus.cc:2468] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 1.
I20260709 13:05:43.273690 17006 ts_tablet_manager.cc:1434] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:05:43.274415 16734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a5bbadcd9b48dba8fe272a18679ede" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:43.275295 17006 tablet_bootstrap.cc:492] T 93629337d6b8477a9cd5895a157bf918 P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:43.279222 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f747edd7934aac93044ba370e34d17" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
W20260709 13:05:43.280179 16923 leader_election.cc:343] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.281415 16923 leader_election.cc:304] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:43.275352 17004 ts_tablet_manager.cc:1434] T 2593417c1fe4457c87cb2a01cf1f65a3 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 13:05:43.258169 16812 raft_consensus.cc:2468] T 02d877efcf5049a89eacf23b58ba27b9 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 0.
I20260709 13:05:43.284425 17351 raft_consensus.cc:493] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.283308 16734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f747edd7934aac93044ba370e34d17" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:43.282980 17006 tablet_bootstrap.cc:654] T 93629337d6b8477a9cd5895a157bf918 P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.283910 16700 leader_election.cc:304] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:05:43.283911 17004 tablet_bootstrap.cc:492] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:43.282688 17274 raft_consensus.cc:2749] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:43.291368 16923 leader_election.cc:304] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:43.287451 16810 raft_consensus.cc:2468] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 1.
I20260709 13:05:43.292141 17281 raft_consensus.cc:2804] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:43.293092 17281 raft_consensus.cc:493] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.293488 17281 raft_consensus.cc:3060] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.290408 17263 raft_consensus.cc:2804] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:43.289223 17351 raft_consensus.cc:3060] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.282905 16958 raft_consensus.cc:2468] T 61f747edd7934aac93044ba370e34d17 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
I20260709 13:05:43.302469 17263 raft_consensus.cc:697] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:05:43.303750 17263 consensus_queue.cc:237] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:43.307425 16849 leader_election.cc:304] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:43.279395 16631 catalog_manager.cc:5697] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "c4b86aae3a8f4a9082e639144ec46228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:43.322031 17281 raft_consensus.cc:515] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:43.329214 17324 raft_consensus.cc:493] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.329794 17324 raft_consensus.cc:515] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:43.330394 17007 tablet_bootstrap.cc:492] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:43.330899 17007 ts_tablet_manager.cc:1403] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.070s	user 0.012s	sys 0.018s
I20260709 13:05:43.331864 17324 leader_election.cc:290] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:43.330857 17351 raft_consensus.cc:515] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:43.333935 17007 raft_consensus.cc:359] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:43.334623 17007 raft_consensus.cc:385] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.334905 17007 raft_consensus.cc:740] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.335515 17324 raft_consensus.cc:493] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.337471 17324 raft_consensus.cc:515] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:43.337625 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec647c6ccd641b59808817d8dca2235" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:43.338406 16958 raft_consensus.cc:3060] T 6ec647c6ccd641b59808817d8dca2235 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.339570 17324 leader_election.cc:290] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:43.340057 17008 tablet_bootstrap.cc:492] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:43.340601 17008 ts_tablet_manager.cc:1403] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.140s	user 0.005s	sys 0.019s
I20260709 13:05:43.345232 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec647c6ccd641b59808817d8dca2235" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:43.345919 16736 raft_consensus.cc:3060] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.335600 17007 consensus_queue.cc:260] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:43.347555 16958 raft_consensus.cc:2468] T 6ec647c6ccd641b59808817d8dca2235 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:43.348670 16775 leader_election.cc:304] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:05:43.349205 17007 ts_tablet_manager.cc:1434] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:05:43.349514 17350 raft_consensus.cc:2804] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:43.319222 17258 raft_consensus.cc:2804] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:43.350235 17007 tablet_bootstrap.cc:492] T 61f747edd7934aac93044ba370e34d17 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:43.350410 17258 raft_consensus.cc:493] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.338433 16803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d877efcf5049a89eacf23b58ba27b9" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:43.351320 16803 raft_consensus.cc:3060] T 02d877efcf5049a89eacf23b58ba27b9 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.350868 17258 raft_consensus.cc:3060] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.336266 17331 raft_consensus.cc:493] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.354780 17008 raft_consensus.cc:359] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:43.355787 17008 raft_consensus.cc:385] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.355589 17331 raft_consensus.cc:515] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:43.356087 17008 raft_consensus.cc:740] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.357286 17007 tablet_bootstrap.cc:654] T 61f747edd7934aac93044ba370e34d17 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.359236 17008 consensus_queue.cc:260] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
W20260709 13:05:43.361420 16849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.4:37199, user_credentials={real_user=slave}} blocked reactor thread for 54141us
I20260709 13:05:43.335901 17351 leader_election.cc:290] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:43.363080 16803 raft_consensus.cc:2468] T 02d877efcf5049a89eacf23b58ba27b9 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 1.
I20260709 13:05:43.350250 17350 raft_consensus.cc:697] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Becoming Leader. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:05:43.424644 17006 tablet_bootstrap.cc:492] T 93629337d6b8477a9cd5895a157bf918 P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:43.425254 17006 ts_tablet_manager.cc:1403] T 93629337d6b8477a9cd5895a157bf918 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.150s	user 0.011s	sys 0.002s
I20260709 13:05:43.428710 17281 leader_election.cc:290] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:43.430938 17008 ts_tablet_manager.cc:1434] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.090s	user 0.005s	sys 0.002s
I20260709 13:05:43.431999 17008 tablet_bootstrap.cc:492] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:43.360671 17258 raft_consensus.cc:515] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:43.432451 17004 tablet_bootstrap.cc:654] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.433281 16923 leader_election.cc:304] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
W20260709 13:05:43.451936 16849 leader_election.cc:343] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.434154 17331 leader_election.cc:290] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:43.479534 17006 raft_consensus.cc:359] T 93629337d6b8477a9cd5895a157bf918 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:43.485910 17006 raft_consensus.cc:385] T 93629337d6b8477a9cd5895a157bf918 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.442384 16731 raft_consensus.cc:1275] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Refusing update from remote peer c4b86aae3a8f4a9082e639144ec46228: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:43.437767 16736 raft_consensus.cc:2468] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:43.486325 17006 raft_consensus.cc:740] T 93629337d6b8477a9cd5895a157bf918 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.434157 16803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ec8ffbf1044525880e6b0d7931f47e" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:43.478575 17274 raft_consensus.cc:2804] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:05:43.495301 16683 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:43.499213 17006 consensus_queue.cc:260] T 93629337d6b8477a9cd5895a157bf918 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:43.438401 17350 consensus_queue.cc:237] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:43.506208 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d877efcf5049a89eacf23b58ba27b9" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:43.511296 17274 raft_consensus.cc:697] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 [term 1 LEADER]: Becoming Leader. State: Replica: 0086163b152644feb75caeac6f145533, State: Running, Role: LEADER
I20260709 13:05:43.532356 17274 consensus_queue.cc:237] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:43.535465 16883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ec8ffbf1044525880e6b0d7931f47e" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:43.521402 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a7231efacd1463c96b35c3d03edce55" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:43.529784 16733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20cdedb225464d5e812a56ed7a7d220f" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:43.541579 16735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a7231efacd1463c96b35c3d03edce55" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:43.542629 16735 raft_consensus.cc:2468] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
I20260709 13:05:43.544535 17007 tablet_bootstrap.cc:492] T 61f747edd7934aac93044ba370e34d17 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:43.545115 17007 ts_tablet_manager.cc:1403] T 61f747edd7934aac93044ba370e34d17 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.195s	user 0.015s	sys 0.020s
I20260709 13:05:43.548408 17007 raft_consensus.cc:359] T 61f747edd7934aac93044ba370e34d17 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:43.549095 17007 raft_consensus.cc:385] T 61f747edd7934aac93044ba370e34d17 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.549372 17007 raft_consensus.cc:740] T 61f747edd7934aac93044ba370e34d17 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
W20260709 13:05:43.549919 16923 leader_election.cc:343] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.550225 17007 consensus_queue.cc:260] T 61f747edd7934aac93044ba370e34d17 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:43.568651 17008 tablet_bootstrap.cc:654] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:43.570631 16849 leader_election.cc:343] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.573942 17394 raft_consensus.cc:493] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.588503 17006 ts_tablet_manager.cc:1434] T 93629337d6b8477a9cd5895a157bf918 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.163s	user 0.013s	sys 0.041s
I20260709 13:05:43.588646 17007 ts_tablet_manager.cc:1434] T 61f747edd7934aac93044ba370e34d17 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260709 13:05:43.597647 17007 tablet_bootstrap.cc:492] T 59de6d1228da41da938527632da08f4d P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:43.591543 17338 raft_consensus.cc:493] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.591614 17395 raft_consensus.cc:493] T 2593417c1fe4457c87cb2a01cf1f65a3 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.599399 17394 raft_consensus.cc:515] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:43.633519 17007 tablet_bootstrap.cc:654] T 59de6d1228da41da938527632da08f4d P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.602957 17395 raft_consensus.cc:515] T 2593417c1fe4457c87cb2a01cf1f65a3 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:43.601643 17338 raft_consensus.cc:515] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:43.641743 17394 leader_election.cc:290] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:43.644054 17004 tablet_bootstrap.cc:492] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:43.644596 17004 ts_tablet_manager.cc:1403] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.361s	user 0.036s	sys 0.019s
I20260709 13:05:43.647140 16881 raft_consensus.cc:3060] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.648501 17338 leader_election.cc:290] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:43.653090 17395 leader_election.cc:290] T 2593417c1fe4457c87cb2a01cf1f65a3 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:43.670850 17394 raft_consensus.cc:493] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.671651 17394 raft_consensus.cc:515] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:43.685361 17258 leader_election.cc:290] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:43.686949 17394 leader_election.cc:290] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:43.707818 17008 tablet_bootstrap.cc:492] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:43.708376 17008 ts_tablet_manager.cc:1403] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.277s	user 0.011s	sys 0.004s
I20260709 13:05:43.709463 16735 raft_consensus.cc:3060] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.716357 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2593417c1fe4457c87cb2a01cf1f65a3" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:43.718586 16809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:43.719230 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2593417c1fe4457c87cb2a01cf1f65a3" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:43.719472 16803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be51f207c3bc4276b220a03e2d9d4db4" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:43.719887 16958 raft_consensus.cc:2468] T 2593417c1fe4457c87cb2a01cf1f65a3 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 0.
I20260709 13:05:43.722813 17008 raft_consensus.cc:359] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:43.723551 17008 raft_consensus.cc:385] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.723829 17008 raft_consensus.cc:740] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.724537 17008 consensus_queue.cc:260] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:43.750180 16775 leader_election.cc:304] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:05:43.753638 17008 ts_tablet_manager.cc:1434] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.045s	user 0.009s	sys 0.000s
I20260709 13:05:43.743120 16849 leader_election.cc:304] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228
I20260709 13:05:43.753552 16635 catalog_manager.cc:5697] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:43.756875 17004 raft_consensus.cc:359] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:43.759961 17004 raft_consensus.cc:385] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.760236 17004 raft_consensus.cc:740] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.719269 16809 raft_consensus.cc:2468] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 2.
W20260709 13:05:43.756974 16700 leader_election.cc:343] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.763041 17004 consensus_queue.cc:260] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:43.754616 17350 raft_consensus.cc:2804] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:43.766659 17006 tablet_bootstrap.cc:492] T 2593417c1fe4457c87cb2a01cf1f65a3 P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:43.768333 16808 raft_consensus.cc:1275] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Refusing update from remote peer d4e1b05bc80e43f7941ece113fb97ab0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:43.761590 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2593417c1fe4457c87cb2a01cf1f65a3" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:43.777973 17007 tablet_bootstrap.cc:492] T 59de6d1228da41da938527632da08f4d P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:43.781813 17007 ts_tablet_manager.cc:1403] T 59de6d1228da41da938527632da08f4d P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.187s	user 0.048s	sys 0.037s
I20260709 13:05:43.783294 16956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f747edd7934aac93044ba370e34d17" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:43.784005 16956 raft_consensus.cc:3060] T 61f747edd7934aac93044ba370e34d17 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.770347 16883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2593417c1fe4457c87cb2a01cf1f65a3" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:43.779999 16882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:43.785878 17007 raft_consensus.cc:359] T 59de6d1228da41da938527632da08f4d P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:43.786294 17394 raft_consensus.cc:493] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.786657 17007 raft_consensus.cc:385] T 59de6d1228da41da938527632da08f4d P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.787024 17007 raft_consensus.cc:740] T 59de6d1228da41da938527632da08f4d P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.786800 17394 raft_consensus.cc:515] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:43.787751 17007 consensus_queue.cc:260] T 59de6d1228da41da938527632da08f4d P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:43.789026 17004 ts_tablet_manager.cc:1434] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.144s	user 0.003s	sys 0.010s
I20260709 13:05:43.790212 17004 tablet_bootstrap.cc:492] T f9d47e8e7b784085b7079b1542dbc579 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:43.728127 17351 consensus_queue.cc:1048] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:05:43.781026 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20cdedb225464d5e812a56ed7a7d220f" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:43.773448 16886 raft_consensus.cc:3060] T 93629337d6b8477a9cd5895a157bf918 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.793255 17394 leader_election.cc:290] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:43.772145 17350 raft_consensus.cc:493] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.795506 17350 raft_consensus.cc:3060] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.797776 16957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be51f207c3bc4276b220a03e2d9d4db4" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:43.798447 16957 raft_consensus.cc:2468] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 0.
I20260709 13:05:43.784068 17008 tablet_bootstrap.cc:492] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:43.779273 17399 raft_consensus.cc:2804] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:43.799367 17399 raft_consensus.cc:493] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.799790 17399 raft_consensus.cc:3060] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.802306 17004 tablet_bootstrap.cc:654] T f9d47e8e7b784085b7079b1542dbc579 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.806169 17007 ts_tablet_manager.cc:1434] T 59de6d1228da41da938527632da08f4d P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 13:05:43.807296 17007 tablet_bootstrap.cc:492] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:43.807332 17006 tablet_bootstrap.cc:654] T 2593417c1fe4457c87cb2a01cf1f65a3 P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.809085 16632 catalog_manager.cc:5697] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "c4b86aae3a8f4a9082e639144ec46228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:43.814236 16700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:45101, user_credentials={real_user=slave}} blocked reactor thread for 49197.7us
W20260709 13:05:43.815825 16923 leader_election.cc:343] T 2593417c1fe4457c87cb2a01cf1f65a3 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:43.820111 16700 leader_election.cc:343] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.824172 16700 leader_election.cc:304] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
W20260709 13:05:43.830017 16849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:45135, user_credentials={real_user=slave}} blocked reactor thread for 58149.6us
W20260709 13:05:43.828750 16700 leader_election.cc:343] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.831321 17007 tablet_bootstrap.cc:654] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.833626 16700 leader_election.cc:304] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:43.829622 17393 raft_consensus.cc:2749] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:43.839382 17008 tablet_bootstrap.cc:654] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.829692 17281 raft_consensus.cc:493] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.846133 16700 leader_election.cc:304] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228
I20260709 13:05:43.840909 17394 raft_consensus.cc:2804] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:43.847949 17406 consensus_queue.cc:1048] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 13:05:43.829312 17404 consensus_queue.cc:1048] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 13:05:43.861851 16882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c19ee4785104276aec22ba772dbc6ac" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:43.866109 17394 raft_consensus.cc:493] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.837618 17121 consensus_queue.cc:1048] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 13:05:43.873294 16631 catalog_manager.cc:5697] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0086163b152644feb75caeac6f145533 (127.6.255.4). New cstate: current_term: 1 leader_uuid: "0086163b152644feb75caeac6f145533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:43.876745 16849 leader_election.cc:304] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0; no voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:43.866047 16735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f747edd7934aac93044ba370e34d17" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:43.868855 17331 raft_consensus.cc:2804] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:43.845463 17399 raft_consensus.cc:515] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:43.873292 17394 raft_consensus.cc:3060] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:43.852625 17281 raft_consensus.cc:515] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:43.900635 16886 raft_consensus.cc:1275] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Refusing update from remote peer c4b86aae3a8f4a9082e639144ec46228: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:05:43.907537 16775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:40785, user_credentials={real_user=slave}} blocked reactor thread for 63641.7us
W20260709 13:05:43.909953 16700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:37199, user_credentials={real_user=slave}} blocked reactor thread for 59960.2us
I20260709 13:05:43.891490 17331 raft_consensus.cc:493] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.894776 16735 raft_consensus.cc:3060] T 61f747edd7934aac93044ba370e34d17 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:43.923094 16775 leader_election.cc:343] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.897372 17367 raft_consensus.cc:2749] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:43.929903 17006 tablet_bootstrap.cc:492] T 2593417c1fe4457c87cb2a01cf1f65a3 P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:43.930619 17006 ts_tablet_manager.cc:1403] T 2593417c1fe4457c87cb2a01cf1f65a3 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.168s	user 0.007s	sys 0.011s
I20260709 13:05:43.913897 17406 consensus_queue.cc:1048] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:05:43.935808 17004 tablet_bootstrap.cc:492] T f9d47e8e7b784085b7079b1542dbc579 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:43.936455 17004 ts_tablet_manager.cc:1403] T f9d47e8e7b784085b7079b1542dbc579 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.146s	user 0.018s	sys 0.009s
W20260709 13:05:43.936827 16923 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:40785, user_credentials={real_user=slave}} blocked reactor thread for 98889.5us
I20260709 13:05:43.925050 17331 raft_consensus.cc:3060] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.942520 17274 raft_consensus.cc:493] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.951052 17397 consensus_queue.cc:1048] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:43.908215 17350 raft_consensus.cc:515] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:43.958781 16730 raft_consensus.cc:3060] T 59de6d1228da41da938527632da08f4d P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.959165 17420 raft_consensus.cc:493] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.959686 17420 raft_consensus.cc:515] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:43.962720 17399 leader_election.cc:290] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:43.963788 17006 raft_consensus.cc:359] T 2593417c1fe4457c87cb2a01cf1f65a3 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:43.964612 17006 raft_consensus.cc:385] T 2593417c1fe4457c87cb2a01cf1f65a3 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.964529 17263 consensus_queue.cc:1048] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:43.965049 17006 raft_consensus.cc:740] T 2593417c1fe4457c87cb2a01cf1f65a3 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.967905 17004 raft_consensus.cc:359] T f9d47e8e7b784085b7079b1542dbc579 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:43.968618 17004 raft_consensus.cc:385] T f9d47e8e7b784085b7079b1542dbc579 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.968897 17004 raft_consensus.cc:740] T f9d47e8e7b784085b7079b1542dbc579 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.972206 17267 consensus_queue.cc:1048] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:43.972625 17350 leader_election.cc:290] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:43.949270 16956 raft_consensus.cc:2468] T 61f747edd7934aac93044ba370e34d17 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:43.978332 17007 tablet_bootstrap.cc:492] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:43.978873 17007 ts_tablet_manager.cc:1403] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.172s	user 0.027s	sys 0.014s
I20260709 13:05:43.982822 16810 raft_consensus.cc:2393] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0086163b152644feb75caeac6f145533 in current term 1: Already voted for candidate c4b86aae3a8f4a9082e639144ec46228 in this term.
W20260709 13:05:43.989306 16849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:45135, user_credentials={real_user=slave}} blocked reactor thread for 68268us
I20260709 13:05:43.991390 17007 raft_consensus.cc:359] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:43.992309 17007 raft_consensus.cc:385] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.995308 17007 raft_consensus.cc:740] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.934825 17351 consensus_queue.cc:1048] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 13:05:44.005609 17004 consensus_queue.cc:260] T f9d47e8e7b784085b7079b1542dbc579 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:43.956197 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c19ee4785104276aec22ba772dbc6ac" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:44.011579 17007 consensus_queue.cc:260] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
W20260709 13:05:44.013118 16923 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:40785, user_credentials={real_user=slave}} blocked reactor thread for 71751us
I20260709 13:05:43.953518 17274 raft_consensus.cc:515] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:44.029234 17006 consensus_queue.cc:260] T 2593417c1fe4457c87cb2a01cf1f65a3 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:44.035291 17007 ts_tablet_manager.cc:1434] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.056s	user 0.009s	sys 0.000s
I20260709 13:05:44.038317 17004 ts_tablet_manager.cc:1434] T f9d47e8e7b784085b7079b1542dbc579 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.102s	user 0.004s	sys 0.004s
I20260709 13:05:44.018322 16812 raft_consensus.cc:2468] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 1.
I20260709 13:05:44.024036 16731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a7231efacd1463c96b35c3d03edce55" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:44.048031 17006 ts_tablet_manager.cc:1434] T 2593417c1fe4457c87cb2a01cf1f65a3 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.117s	user 0.011s	sys 0.005s
I20260709 13:05:44.059476 16731 raft_consensus.cc:3060] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:44.066437 17396 raft_consensus.cc:493] T f9d47e8e7b784085b7079b1542dbc579 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.072306 17004 tablet_bootstrap.cc:492] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:44.061733 17281 leader_election.cc:290] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:44.067056 17396 raft_consensus.cc:515] T f9d47e8e7b784085b7079b1542dbc579 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:44.112648 17007 tablet_bootstrap.cc:492] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:44.060783 17404 raft_consensus.cc:493] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.043434 17331 raft_consensus.cc:515] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:44.068066 17420 leader_election.cc:290] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:44.125201 16700 leader_election.cc:304] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:44.047325 17394 raft_consensus.cc:515] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:44.139787 16953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2593417c1fe4457c87cb2a01cf1f65a3" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:44.140529 16953 raft_consensus.cc:3060] T 2593417c1fe4457c87cb2a01cf1f65a3 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:44.142817 17368 raft_consensus.cc:493] T 93629337d6b8477a9cd5895a157bf918 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.143365 17368 raft_consensus.cc:515] T 93629337d6b8477a9cd5895a157bf918 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
W20260709 13:05:44.146840 17430 log.cc:927] Time spent T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:05:44.146905 16734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86f5e278f54c419fac8c74ac63f80f74" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:44.147718 16734 raft_consensus.cc:2468] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 0.
I20260709 13:05:44.148509 16953 raft_consensus.cc:2468] T 2593417c1fe4457c87cb2a01cf1f65a3 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:44.170976 16775 leader_election.cc:304] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:05:44.122126 17404 raft_consensus.cc:515] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:44.208731 17400 raft_consensus.cc:2804] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:44.196997 17274 leader_election.cc:290] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:44.220871 17008 tablet_bootstrap.cc:492] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:44.221709 17008 ts_tablet_manager.cc:1403] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.438s	user 0.017s	sys 0.014s
I20260709 13:05:44.233565 16737 raft_consensus.cc:1240] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Rejecting Update request from peer d4e1b05bc80e43f7941ece113fb97ab0 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:44.252140 16731 raft_consensus.cc:2468] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:44.219115 16803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f86d4f23f8cd4451a50b1b38155d958e" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:44.253762 16849 leader_election.cc:304] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:05:44.214310 17263 raft_consensus.cc:2804] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:44.264339 16849 leader_election.cc:304] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:44.203994 17397 consensus_queue.cc:1048] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.051s
W20260709 13:05:44.270291 17424 log.cc:927] Time spent T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:05:44.271230 17396 leader_election.cc:290] T f9d47e8e7b784085b7079b1542dbc579 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:44.272751 16735 raft_consensus.cc:2468] T 61f747edd7934aac93044ba370e34d17 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:44.274212 16883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2593417c1fe4457c87cb2a01cf1f65a3" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:44.274888 16883 raft_consensus.cc:3060] T 2593417c1fe4457c87cb2a01cf1f65a3 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:44.281416 16806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86f5e278f54c419fac8c74ac63f80f74" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:44.282192 16806 raft_consensus.cc:2468] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 0.
I20260709 13:05:44.283995 17004 tablet_bootstrap.cc:654] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:44.281654 17404 leader_election.cc:290] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:44.298620 17427 consensus_queue.cc:1048] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:44.287599 17370 raft_consensus.cc:2804] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:44.264336 17400 raft_consensus.cc:697] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Becoming Leader. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:05:44.289158 17367 raft_consensus.cc:2804] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:44.358700 17007 tablet_bootstrap.cc:654] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:44.360112 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86f5e278f54c419fac8c74ac63f80f74" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:44.371161 16736 raft_consensus.cc:2468] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 0.
I20260709 13:05:44.358186 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
W20260709 13:05:44.372560 17431 log.cc:927] Time spent T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:05:44.376106 17331 leader_election.cc:290] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:44.376549 17394 leader_election.cc:290] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 3 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:44.376782 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86f5e278f54c419fac8c74ac63f80f74" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:44.377658 16959 raft_consensus.cc:2468] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 0.
I20260709 13:05:44.379828 16883 raft_consensus.cc:2468] T 2593417c1fe4457c87cb2a01cf1f65a3 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:44.380244 17368 leader_election.cc:290] T 93629337d6b8477a9cd5895a157bf918 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:44.382512 17008 raft_consensus.cc:359] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:44.383232 17008 raft_consensus.cc:385] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:44.383520 17008 raft_consensus.cc:740] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.384131 17008 consensus_queue.cc:260] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:44.387673 16775 leader_election.cc:304] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:05:44.392068 17008 ts_tablet_manager.cc:1434] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.170s	user 0.004s	sys 0.003s
I20260709 13:05:44.402061 17417 raft_consensus.cc:2804] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:44.402547 17417 raft_consensus.cc:493] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.402856 17417 raft_consensus.cc:3060] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:44.360467 17370 raft_consensus.cc:697] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Becoming Leader. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:44.364924 17367 raft_consensus.cc:697] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Becoming Leader. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:44.417842 17400 consensus_queue.cc:237] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:44.426717 16806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a7231efacd1463c96b35c3d03edce55" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:44.432137 16954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93629337d6b8477a9cd5895a157bf918" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:44.434110 16923 leader_election.cc:304] T 2593417c1fe4457c87cb2a01cf1f65a3 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
W20260709 13:05:44.435863 16923 leader_election.cc:343] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.436518 16923 leader_election.cc:304] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:05:44.439136 17417 raft_consensus.cc:515] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:44.457490 17395 raft_consensus.cc:2804] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:44.458029 17395 raft_consensus.cc:493] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.458370 17395 raft_consensus.cc:3060] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:44.458997 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93629337d6b8477a9cd5895a157bf918" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:44.461666 17281 raft_consensus.cc:2749] T 2593417c1fe4457c87cb2a01cf1f65a3 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:44.486542 16700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:45135, user_credentials={real_user=slave}} blocked reactor thread for 121393us
W20260709 13:05:44.504837 16923 consensus_peers.cc:597] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 -> Peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Couldn't send request to peer d4e1b05bc80e43f7941ece113fb97ab0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:44.507782 17417 leader_election.cc:290] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
W20260709 13:05:44.517292 16849 leader_election.cc:343] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:44.543087 16775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:45101, user_credentials={real_user=slave}} blocked reactor thread for 154235us
I20260709 13:05:44.520982 17367 consensus_queue.cc:237] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:44.518751 17370 consensus_queue.cc:237] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:44.552928 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "136f22096d4b41d89354924f7101d1c1" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:44.571648 17004 tablet_bootstrap.cc:492] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:44.575389 17004 ts_tablet_manager.cc:1403] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.515s	user 0.015s	sys 0.018s
I20260709 13:05:44.568785 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d47e8e7b784085b7079b1542dbc579" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:44.591286 16885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d47e8e7b784085b7079b1542dbc579" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:44.594583 17445 raft_consensus.cc:493] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.595250 17445 raft_consensus.cc:515] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
W20260709 13:05:44.596740 16923 leader_election.cc:343] T f9d47e8e7b784085b7079b1542dbc579 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:44.614696 16923 leader_election.cc:343] T f9d47e8e7b784085b7079b1542dbc579 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.615279 16923 leader_election.cc:304] T f9d47e8e7b784085b7079b1542dbc579 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:44.597605 17445 leader_election.cc:290] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:44.630676 16881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ec8ffbf1044525880e6b0d7931f47e" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:44.600687 16728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86f5e278f54c419fac8c74ac63f80f74" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:44.632018 16728 raft_consensus.cc:3060] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:44.628106 17425 raft_consensus.cc:2749] T f9d47e8e7b784085b7079b1542dbc579 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:44.639833 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be51f207c3bc4276b220a03e2d9d4db4" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:44.640588 16959 raft_consensus.cc:3060] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:44.628756 16923 leader_election.cc:343] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.603668 16880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f86d4f23f8cd4451a50b1b38155d958e" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:44.670111 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be51f207c3bc4276b220a03e2d9d4db4" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:44.677845 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ec8ffbf1044525880e6b0d7931f47e" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:44.678689 16736 raft_consensus.cc:2468] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 0.
I20260709 13:05:44.665972 17324 consensus_queue.cc:1059] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
W20260709 13:05:44.690805 16775 leader_election.cc:343] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.694599 16775 leader_election.cc:304] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:44.665988 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86f5e278f54c419fac8c74ac63f80f74" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
W20260709 13:05:44.693907 16700 leader_election.cc:343] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:44.709606 16923 leader_election.cc:343] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.712320 16923 leader_election.cc:304] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:44.698509 17271 raft_consensus.cc:2804] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:44.716128 16849 leader_election.cc:304] T 93629337d6b8477a9cd5895a157bf918 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0; no voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228
I20260709 13:05:44.719141 17263 raft_consensus.cc:493] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d4e1b05bc80e43f7941ece113fb97ab0)
I20260709 13:05:44.719910 17418 raft_consensus.cc:2749] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:44.720630 17271 raft_consensus.cc:493] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.727227 17256 raft_consensus.cc:2749] T 93629337d6b8477a9cd5895a157bf918 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:44.726353 17263 raft_consensus.cc:3060] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.738266 17271 raft_consensus.cc:3060] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:44.754966 16728 raft_consensus.cc:2468] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:44.771780 16736 raft_consensus.cc:1240] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Rejecting Update request from peer d4e1b05bc80e43f7941ece113fb97ab0 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:05:44.777273 17007 tablet_bootstrap.cc:492] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:44.777945 17007 ts_tablet_manager.cc:1403] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.714s	user 0.012s	sys 0.009s
W20260709 13:05:44.784461 17455 log.cc:927] Time spent T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:05:44.780989 17007 raft_consensus.cc:359] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:44.787680 17007 raft_consensus.cc:385] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:44.787920 17007 raft_consensus.cc:740] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
W20260709 13:05:44.799818 17451 log.cc:927] Time spent T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:05:44.794968 17420 consensus_queue.cc:1048] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:05:44.802191 16880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:44.798138 17004 raft_consensus.cc:359] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:44.805269 17004 raft_consensus.cc:385] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:44.805617 17004 raft_consensus.cc:740] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.805925 17271 raft_consensus.cc:515] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:44.806324 17004 consensus_queue.cc:260] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:44.810041 17271 leader_election.cc:290] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:44.812698 16775 leader_election.cc:304] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:05:44.815186 17351 raft_consensus.cc:2804] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:44.796790 17405 consensus_queue.cc:1059] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:05:44.813198 16735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "136f22096d4b41d89354924f7101d1c1" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:44.796559 17007 consensus_queue.cc:260] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:44.763975 17395 raft_consensus.cc:515] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:44.815768 17351 raft_consensus.cc:697] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Becoming Leader. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:05:44.821808 16880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ec8ffbf1044525880e6b0d7931f47e" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:44.822170 17351 consensus_queue.cc:237] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:44.827507 17274 raft_consensus.cc:493] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.828083 17274 raft_consensus.cc:515] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:44.828549 17004 ts_tablet_manager.cc:1434] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.253s	user 0.004s	sys 0.004s
I20260709 13:05:44.829694 17004 tablet_bootstrap.cc:492] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:44.830380 17274 leader_election.cc:290] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:44.838424 17006 tablet_bootstrap.cc:492] T 02d877efcf5049a89eacf23b58ba27b9 P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:44.841068 17008 tablet_bootstrap.cc:492] T f86d4f23f8cd4451a50b1b38155d958e P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:44.842489 17263 raft_consensus.cc:515] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:44.843672 16730 raft_consensus.cc:1240] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Rejecting Update request from peer d4e1b05bc80e43f7941ece113fb97ab0 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:44.845486 17263 leader_election.cc:290] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:44.845913 16960 raft_consensus.cc:2393] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c4b86aae3a8f4a9082e639144ec46228 in current term 1: Already voted for candidate 0086163b152644feb75caeac6f145533 in this term.
I20260709 13:05:44.848104 17008 tablet_bootstrap.cc:654] T f86d4f23f8cd4451a50b1b38155d958e P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:44.848487 17395 leader_election.cc:290] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:44.850945 16959 raft_consensus.cc:2468] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 1.
I20260709 13:05:44.854535 16700 leader_election.cc:304] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228
W20260709 13:05:44.855299 16923 leader_election.cc:343] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.857353 16923 leader_election.cc:304] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:44.861054 17395 raft_consensus.cc:2749] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:44.866099 17007 ts_tablet_manager.cc:1434] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.088s	user 0.007s	sys 0.000s
W20260709 13:05:44.869825 16775 leader_election.cc:343] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.874116 16700 leader_election.cc:304] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:44.876677 17393 raft_consensus.cc:2749] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:44.877110 17442 raft_consensus.cc:2804] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:44.877802 17442 raft_consensus.cc:697] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:05:44.878729 17442 consensus_queue.cc:237] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:44.880204 16730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ca116ec6174decb610df3f8f565a6b" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:44.880559 16809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ca116ec6174decb610df3f8f565a6b" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:44.883679 16804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c19ee4785104276aec22ba772dbc6ac" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:44.885417 16880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c19ee4785104276aec22ba772dbc6ac" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:44.859560 16731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ec8ffbf1044525880e6b0d7931f47e" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:44.889986 16731 raft_consensus.cc:3060] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:44.897871 16923 leader_election.cc:343] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:44.903160 16923 leader_election.cc:343] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.903865 16923 leader_election.cc:304] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:44.904136 16700 leader_election.cc:304] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:44.906504 16734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86f5e278f54c419fac8c74ac63f80f74" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:44.915764 16734 raft_consensus.cc:2393] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0086163b152644feb75caeac6f145533 in current term 1: Already voted for candidate c4b86aae3a8f4a9082e639144ec46228 in this term.
I20260709 13:05:44.916560 17414 raft_consensus.cc:2749] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:44.919041 17462 consensus_queue.cc:1059] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:05:44.920739 17004 tablet_bootstrap.cc:654] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:44.921746 17006 tablet_bootstrap.cc:654] T 02d877efcf5049a89eacf23b58ba27b9 P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:44.901865 17263 raft_consensus.cc:493] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d4e1b05bc80e43f7941ece113fb97ab0)
I20260709 13:05:44.926304 17263 raft_consensus.cc:515] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:44.922904 17420 raft_consensus.cc:2749] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:44.939909 16804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86f5e278f54c419fac8c74ac63f80f74" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:44.940851 17263 leader_election.cc:290] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:44.946867 16881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9078054426a34acbb94be2b69213b6c1" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:44.947556 17007 tablet_bootstrap.cc:492] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:44.951150 16923 leader_election.cc:304] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:44.951773 17409 raft_consensus.cc:3055] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:44.957999 17409 raft_consensus.cc:740] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:44.959064 17409 consensus_queue.cc:260] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:44.960503 17008 tablet_bootstrap.cc:492] T f86d4f23f8cd4451a50b1b38155d958e P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:44.961192 17008 ts_tablet_manager.cc:1403] T f86d4f23f8cd4451a50b1b38155d958e P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.120s	user 0.025s	sys 0.000s
W20260709 13:05:44.969739 16683 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:44.952042 17420 raft_consensus.cc:2749] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:44.950738 17271 consensus_queue.cc:1048] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.035s
I20260709 13:05:44.984278 17409 raft_consensus.cc:3060] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:44.988716 16804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9078054426a34acbb94be2b69213b6c1" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:45.016723 16731 raft_consensus.cc:2468] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:45.018368 16700 leader_election.cc:304] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:45.020484 17432 raft_consensus.cc:2749] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:45.021126 17006 tablet_bootstrap.cc:492] T 02d877efcf5049a89eacf23b58ba27b9 P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:45.023283 17007 tablet_bootstrap.cc:654] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.021673 17006 ts_tablet_manager.cc:1403] T 02d877efcf5049a89eacf23b58ba27b9 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.184s	user 0.019s	sys 0.004s
I20260709 13:05:45.025804 16775 leader_election.cc:304] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:45.026244 17006 raft_consensus.cc:359] T 02d877efcf5049a89eacf23b58ba27b9 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:45.026947 17350 raft_consensus.cc:2804] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:45.027616 17350 raft_consensus.cc:697] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Becoming Leader. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:05:45.028110 17004 tablet_bootstrap.cc:492] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:45.028435 17350 consensus_queue.cc:237] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:45.026968 17006 raft_consensus.cc:385] T 02d877efcf5049a89eacf23b58ba27b9 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.029088 17004 ts_tablet_manager.cc:1403] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.200s	user 0.020s	sys 0.004s
I20260709 13:05:45.029656 17006 raft_consensus.cc:740] T 02d877efcf5049a89eacf23b58ba27b9 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.030809 17006 consensus_queue.cc:260] T 02d877efcf5049a89eacf23b58ba27b9 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:45.033334 17006 ts_tablet_manager.cc:1434] T 02d877efcf5049a89eacf23b58ba27b9 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:05:45.034946 17004 raft_consensus.cc:359] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:45.035861 17004 raft_consensus.cc:385] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.036307 17006 tablet_bootstrap.cc:492] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:45.037417 17409 raft_consensus.cc:3055] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:45.037837 17409 raft_consensus.cc:740] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:45.039129 17008 raft_consensus.cc:359] T f86d4f23f8cd4451a50b1b38155d958e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:45.039973 17008 raft_consensus.cc:385] T f86d4f23f8cd4451a50b1b38155d958e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.040268 17008 raft_consensus.cc:740] T f86d4f23f8cd4451a50b1b38155d958e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.041821 17008 consensus_queue.cc:260] T f86d4f23f8cd4451a50b1b38155d958e P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:45.045320 17008 ts_tablet_manager.cc:1434] T f86d4f23f8cd4451a50b1b38155d958e P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.084s	user 0.006s	sys 0.000s
I20260709 13:05:45.046414 17008 tablet_bootstrap.cc:492] T 0a7231efacd1463c96b35c3d03edce55 P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:45.040248 17409 consensus_queue.cc:260] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:45.036162 17004 raft_consensus.cc:740] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.059348 17004 consensus_queue.cc:260] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:45.060171 17008 tablet_bootstrap.cc:654] T 0a7231efacd1463c96b35c3d03edce55 P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.061887 17007 tablet_bootstrap.cc:492] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:45.062470 17007 ts_tablet_manager.cc:1403] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.115s	user 0.016s	sys 0.002s
I20260709 13:05:45.066942 17006 tablet_bootstrap.cc:654] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.067492 16880 raft_consensus.cc:3060] T 02d877efcf5049a89eacf23b58ba27b9 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:45.069442 16849 consensus_peers.cc:597] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Couldn't send request to peer c4b86aae3a8f4a9082e639144ec46228. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.070113 17007 raft_consensus.cc:359] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:45.070881 17007 raft_consensus.cc:385] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.073689 17004 ts_tablet_manager.cc:1434] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.001s
I20260709 13:05:45.074999 17004 tablet_bootstrap.cc:492] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533: Bootstrap starting.
I20260709 13:05:45.081085 17006 tablet_bootstrap.cc:492] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:45.081667 17006 ts_tablet_manager.cc:1403] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.047s	user 0.014s	sys 0.000s
I20260709 13:05:45.083076 17368 consensus_queue.cc:1048] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:45.084445 17006 raft_consensus.cc:359] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:45.086913 17006 raft_consensus.cc:385] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.087311 17006 raft_consensus.cc:740] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.080374 16632 catalog_manager.cc:5697] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "c4b86aae3a8f4a9082e639144ec46228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:45.092162 17006 consensus_queue.cc:260] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:45.095517 16632 catalog_manager.cc:5697] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "c4b86aae3a8f4a9082e639144ec46228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:45.097715 17006 ts_tablet_manager.cc:1434] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:05:45.099215 17006 tablet_bootstrap.cc:492] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:45.071182 17007 raft_consensus.cc:740] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.102182 17007 consensus_queue.cc:260] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:45.106973 17006 tablet_bootstrap.cc:654] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.118403 17394 raft_consensus.cc:493] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.119061 17394 raft_consensus.cc:515] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:45.122290 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744cc6469e3749e2aacccb45464e8198" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:45.125613 17008 tablet_bootstrap.cc:492] T 0a7231efacd1463c96b35c3d03edce55 P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:45.126603 17008 ts_tablet_manager.cc:1403] T 0a7231efacd1463c96b35c3d03edce55 P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.080s	user 0.013s	sys 0.000s
W20260709 13:05:45.128290 16700 leader_election.cc:343] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.129989 17007 ts_tablet_manager.cc:1434] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.067s	user 0.007s	sys 0.000s
I20260709 13:05:45.131568 17007 tablet_bootstrap.cc:492] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:45.133082 17409 raft_consensus.cc:3060] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:45.139966 17004 tablet_bootstrap.cc:654] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.139846 16954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744cc6469e3749e2aacccb45464e8198" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:45.140489 17394 leader_election.cc:290] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
W20260709 13:05:45.143527 16700 leader_election.cc:343] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0086163b152644feb75caeac6f145533 (127.6.255.4:37199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.145144 16700 leader_election.cc:304] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:45.146715 17394 raft_consensus.cc:2749] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:45.187789 17258 consensus_queue.cc:1048] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:45.187928 17427 consensus_queue.cc:1048] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:45.195873 17258 consensus_queue.cc:1048] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:05:45.208650 17008 raft_consensus.cc:359] T 0a7231efacd1463c96b35c3d03edce55 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:45.209753 17008 raft_consensus.cc:385] T 0a7231efacd1463c96b35c3d03edce55 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.210105 17008 raft_consensus.cc:740] T 0a7231efacd1463c96b35c3d03edce55 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.217475 17440 raft_consensus.cc:493] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.218122 17440 raft_consensus.cc:515] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:45.219920 17008 consensus_queue.cc:260] T 0a7231efacd1463c96b35c3d03edce55 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:45.232429 17007 tablet_bootstrap.cc:654] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.235802 17008 ts_tablet_manager.cc:1434] T 0a7231efacd1463c96b35c3d03edce55 P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.109s	user 0.010s	sys 0.004s
I20260709 13:05:45.240803 17006 tablet_bootstrap.cc:492] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:45.241348 17006 ts_tablet_manager.cc:1403] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.142s	user 0.038s	sys 0.025s
I20260709 13:05:45.248330 17004 tablet_bootstrap.cc:492] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533: No bootstrap required, opened a new log
I20260709 13:05:45.248940 17004 ts_tablet_manager.cc:1403] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533: Time spent bootstrapping tablet: real 0.174s	user 0.015s	sys 0.001s
I20260709 13:05:45.250407 16880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a5bbadcd9b48dba8fe272a18679ede" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:45.250684 17006 raft_consensus.cc:359] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:45.251441 17006 raft_consensus.cc:385] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.251725 17006 raft_consensus.cc:740] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.252467 17006 consensus_queue.cc:260] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:45.254004 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a5bbadcd9b48dba8fe272a18679ede" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:45.254949 16960 raft_consensus.cc:2468] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 0.
I20260709 13:05:45.255497 17395 raft_consensus.cc:493] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.256075 17395 raft_consensus.cc:515] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:45.258308 17395 leader_election.cc:290] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:45.258795 17006 ts_tablet_manager.cc:1434] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.001s
I20260709 13:05:45.260769 17006 tablet_bootstrap.cc:492] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:45.260743 16804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c968ba000a4542fc9204dd9a474e364c" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
W20260709 13:05:45.261013 16700 leader_election.cc:343] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.261821 16700 leader_election.cc:304] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: d4e1b05bc80e43f7941ece113fb97ab0
W20260709 13:05:45.262197 16923 leader_election.cc:343] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.264621 17394 raft_consensus.cc:2804] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:45.265007 17394 raft_consensus.cc:493] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.265367 17394 raft_consensus.cc:3060] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.221350 17440 leader_election.cc:290] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:45.265260 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c968ba000a4542fc9204dd9a474e364c" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
W20260709 13:05:45.271289 16923 leader_election.cc:343] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.271754 16923 leader_election.cc:304] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:45.272830 17395 raft_consensus.cc:2749] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:45.273524 17394 raft_consensus.cc:515] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:45.275789 17394 leader_election.cc:290] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:45.279454 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a5bbadcd9b48dba8fe272a18679ede" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:45.287065 16953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a5bbadcd9b48dba8fe272a18679ede" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:45.287770 16953 raft_consensus.cc:3060] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:45.299988 16700 leader_election.cc:343] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.303774 17008 tablet_bootstrap.cc:492] T f9d47e8e7b784085b7079b1542dbc579 P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:45.309453 17006 tablet_bootstrap.cc:654] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.318454 17440 consensus_queue.cc:1048] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:45.328943 17007 tablet_bootstrap.cc:492] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:45.329497 17007 ts_tablet_manager.cc:1403] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.198s	user 0.025s	sys 0.001s
I20260709 13:05:45.330147 17004 raft_consensus.cc:359] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:45.330345 17368 raft_consensus.cc:493] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.330874 17004 raft_consensus.cc:385] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.331224 17004 raft_consensus.cc:740] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.331151 17368 raft_consensus.cc:515] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:45.332163 17004 consensus_queue.cc:260] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:45.332360 17007 raft_consensus.cc:359] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:45.332974 17007 raft_consensus.cc:385] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.333228 17007 raft_consensus.cc:740] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.334331 17007 consensus_queue.cc:260] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:45.338351 17004 ts_tablet_manager.cc:1434] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533: Time spent starting tablet: real 0.089s	user 0.006s	sys 0.001s
I20260709 13:05:45.337616 17368 leader_election.cc:290] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:45.343562 17007 ts_tablet_manager.cc:1434] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 13:05:45.348945 16804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f86d4f23f8cd4451a50b1b38155d958e" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:45.349725 16804 raft_consensus.cc:2468] T f86d4f23f8cd4451a50b1b38155d958e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
I20260709 13:05:45.351258 17008 tablet_bootstrap.cc:654] T f9d47e8e7b784085b7079b1542dbc579 P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.351468 16849 leader_election.cc:304] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:45.351292 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f86d4f23f8cd4451a50b1b38155d958e" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:45.354489 16960 raft_consensus.cc:2468] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
I20260709 13:05:45.355968 16953 raft_consensus.cc:2468] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 1.
I20260709 13:05:45.356211 17368 raft_consensus.cc:2804] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:45.356935 17368 raft_consensus.cc:493] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.358419 16700 leader_election.cc:304] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:45.362511 17006 tablet_bootstrap.cc:492] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:45.363175 17006 ts_tablet_manager.cc:1403] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.103s	user 0.018s	sys 0.004s
I20260709 13:05:45.363629 17007 tablet_bootstrap.cc:492] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:45.359009 17368 raft_consensus.cc:3060] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.361827 17440 raft_consensus.cc:2804] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:45.366777 17440 raft_consensus.cc:697] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:05:45.367738 17440 consensus_queue.cc:237] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:45.374018 17006 raft_consensus.cc:359] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:45.374594 17007 tablet_bootstrap.cc:654] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.374989 17006 raft_consensus.cc:385] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.376765 17006 raft_consensus.cc:740] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
W20260709 13:05:45.376816 16700 consensus_peers.cc:597] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 -> Peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Couldn't send request to peer c4b86aae3a8f4a9082e639144ec46228. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.377746 17006 consensus_queue.cc:260] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:45.378228 17351 consensus_queue.cc:1048] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:45.380398 17006 ts_tablet_manager.cc:1434] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:05:45.381994 17006 tablet_bootstrap.cc:492] T 83ec8ffbf1044525880e6b0d7931f47e P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:45.389024 17006 tablet_bootstrap.cc:654] T 83ec8ffbf1044525880e6b0d7931f47e P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.402882 17008 tablet_bootstrap.cc:492] T f9d47e8e7b784085b7079b1542dbc579 P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:45.403596 17008 ts_tablet_manager.cc:1403] T f9d47e8e7b784085b7079b1542dbc579 P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.100s	user 0.016s	sys 0.010s
I20260709 13:05:45.407089 17008 raft_consensus.cc:359] T f9d47e8e7b784085b7079b1542dbc579 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:45.408097 17008 raft_consensus.cc:385] T f9d47e8e7b784085b7079b1542dbc579 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.408818 17008 raft_consensus.cc:740] T f9d47e8e7b784085b7079b1542dbc579 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.410192 17008 consensus_queue.cc:260] T f9d47e8e7b784085b7079b1542dbc579 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:45.402722 17368 raft_consensus.cc:515] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:45.421974 17409 raft_consensus.cc:493] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.435758 17008 ts_tablet_manager.cc:1434] T f9d47e8e7b784085b7079b1542dbc579 P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.001s
I20260709 13:05:45.428399 17409 raft_consensus.cc:515] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:45.445574 16804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f86d4f23f8cd4451a50b1b38155d958e" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:45.446265 16804 raft_consensus.cc:3060] T f86d4f23f8cd4451a50b1b38155d958e P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.446362 17409 leader_election.cc:290] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:45.449710 16730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f239a1de317547b598aeedc11994c75f" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:45.451646 16809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f239a1de317547b598aeedc11994c75f" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:45.450990 17008 tablet_bootstrap.cc:492] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
W20260709 13:05:45.454276 16849 leader_election.cc:343] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.459175 17350 consensus_queue.cc:1048] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:45.460927 16804 raft_consensus.cc:2468] T f86d4f23f8cd4451a50b1b38155d958e P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
W20260709 13:05:45.461320 16849 leader_election.cc:343] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.461911 16849 leader_election.cc:304] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:45.462579 17368 leader_election.cc:290] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:45.462657 17258 raft_consensus.cc:2749] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:45.466642 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f86d4f23f8cd4451a50b1b38155d958e" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:45.461277 17008 tablet_bootstrap.cc:654] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.467295 16960 raft_consensus.cc:3060] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.473291 16849 leader_election.cc:304] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:45.474253 17258 raft_consensus.cc:2804] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:45.474804 17258 raft_consensus.cc:697] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Becoming Leader. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:45.475672 17258 consensus_queue.cc:237] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:45.480480 17007 tablet_bootstrap.cc:492] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:45.481364 17007 ts_tablet_manager.cc:1403] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.121s	user 0.022s	sys 0.017s
I20260709 13:05:45.482264 16960 raft_consensus.cc:2468] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:45.485994 17006 tablet_bootstrap.cc:492] T 83ec8ffbf1044525880e6b0d7931f47e P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:45.486553 17006 ts_tablet_manager.cc:1403] T 83ec8ffbf1044525880e6b0d7931f47e P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.105s	user 0.010s	sys 0.015s
I20260709 13:05:45.486245 17007 raft_consensus.cc:359] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:45.487064 17007 raft_consensus.cc:385] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.487380 17007 raft_consensus.cc:740] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.489648 17006 raft_consensus.cc:359] T 83ec8ffbf1044525880e6b0d7931f47e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:45.490360 17006 raft_consensus.cc:385] T 83ec8ffbf1044525880e6b0d7931f47e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.490660 17006 raft_consensus.cc:740] T 83ec8ffbf1044525880e6b0d7931f47e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.491252 17007 consensus_queue.cc:260] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:45.491330 17006 consensus_queue.cc:260] T 83ec8ffbf1044525880e6b0d7931f47e P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:45.494884 17006 ts_tablet_manager.cc:1434] T 83ec8ffbf1044525880e6b0d7931f47e P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:05:45.495730 16884 raft_consensus.cc:3060] T 83ec8ffbf1044525880e6b0d7931f47e P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.495894 17007 ts_tablet_manager.cc:1434] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 13:05:45.497061 17007 tablet_bootstrap.cc:492] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:45.495939 17006 tablet_bootstrap.cc:492] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:45.513820 17007 tablet_bootstrap.cc:654] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.514976 17397 consensus_queue.cc:1048] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:45.525620 17445 consensus_queue.cc:1048] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:45.526018 16631 catalog_manager.cc:5697] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:45.568692 17446 raft_consensus.cc:493] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.569226 17446 raft_consensus.cc:515] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:45.571274 17446 leader_election.cc:290] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:45.575795 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "136f22096d4b41d89354924f7101d1c1" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:45.584437 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "136f22096d4b41d89354924f7101d1c1" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:45.587250 16960 raft_consensus.cc:2468] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 0.
I20260709 13:05:45.584659 16804 raft_consensus.cc:3060] T 0a7231efacd1463c96b35c3d03edce55 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.589628 16700 leader_election.cc:304] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: 
W20260709 13:05:45.594528 16700 leader_election.cc:343] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.593271 17440 raft_consensus.cc:2804] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:45.602094 17440 raft_consensus.cc:493] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.602499 17440 raft_consensus.cc:3060] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.603169 17008 tablet_bootstrap.cc:492] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:45.603811 17008 ts_tablet_manager.cc:1403] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.159s	user 0.019s	sys 0.023s
I20260709 13:05:45.653077 17006 tablet_bootstrap.cc:654] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.652727 17440 raft_consensus.cc:515] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:45.657696 16632 catalog_manager.cc:5697] T 61f747edd7934aac93044ba370e34d17 P 0086163b152644feb75caeac6f145533 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3). New cstate: current_term: 1 leader_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } }
I20260709 13:05:45.660244 16881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "136f22096d4b41d89354924f7101d1c1" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:45.661731 16954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "136f22096d4b41d89354924f7101d1c1" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:45.662434 16954 raft_consensus.cc:3060] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:45.666517 16700 leader_election.cc:343] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.667438 17007 tablet_bootstrap.cc:492] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:45.668310 17007 ts_tablet_manager.cc:1403] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.171s	user 0.013s	sys 0.005s
I20260709 13:05:45.660976 17440 leader_election.cc:290] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:45.674548 17007 raft_consensus.cc:359] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:45.675730 17008 raft_consensus.cc:359] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:45.676414 17423 raft_consensus.cc:493] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.679512 17008 raft_consensus.cc:385] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.679898 17008 raft_consensus.cc:740] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.679944 16954 raft_consensus.cc:2468] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 1.
I20260709 13:05:45.675333 17007 raft_consensus.cc:385] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.679756 17423 raft_consensus.cc:515] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:45.680989 17007 raft_consensus.cc:740] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.680877 17008 consensus_queue.cc:260] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:45.681910 17007 consensus_queue.cc:260] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:45.682914 17423 leader_election.cc:290] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:45.683422 17008 ts_tablet_manager.cc:1434] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.079s	user 0.008s	sys 0.000s
I20260709 13:05:45.684425 17008 tablet_bootstrap.cc:492] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:45.684202 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d47e8e7b784085b7079b1542dbc579" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:45.684928 16960 raft_consensus.cc:2468] T f9d47e8e7b784085b7079b1542dbc579 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
I20260709 13:05:45.684813 16806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d47e8e7b784085b7079b1542dbc579" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:45.686276 16849 leader_election.cc:304] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:45.688761 16806 raft_consensus.cc:2468] T f9d47e8e7b784085b7079b1542dbc579 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 0.
I20260709 13:05:45.689388 17423 raft_consensus.cc:2804] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:45.689846 17423 raft_consensus.cc:493] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.690219 17423 raft_consensus.cc:3060] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.687731 16635 catalog_manager.cc:5697] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3). New cstate: current_term: 1 leader_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:45.688454 17006 tablet_bootstrap.cc:492] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:45.684199 17007 ts_tablet_manager.cc:1434] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:05:45.691390 17006 ts_tablet_manager.cc:1403] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.196s	user 0.011s	sys 0.008s
I20260709 13:05:45.698010 17423 raft_consensus.cc:515] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:45.699155 17006 raft_consensus.cc:359] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:45.699900 17006 raft_consensus.cc:385] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.700178 17006 raft_consensus.cc:740] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.701218 16806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d47e8e7b784085b7079b1542dbc579" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:45.698031 17008 tablet_bootstrap.cc:654] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.701912 16806 raft_consensus.cc:3060] T f9d47e8e7b784085b7079b1542dbc579 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.705400 17006 consensus_queue.cc:260] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:45.708199 17006 ts_tablet_manager.cc:1434] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 13:05:45.709224 17006 tablet_bootstrap.cc:492] T 7f4354282d844c7e83d760e4418512dc P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:45.710155 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d47e8e7b784085b7079b1542dbc579" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:45.710723 16960 raft_consensus.cc:3060] T f9d47e8e7b784085b7079b1542dbc579 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.710726 17258 raft_consensus.cc:493] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.711341 17258 raft_consensus.cc:515] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:45.713316 17258 leader_election.cc:290] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:45.714368 17446 raft_consensus.cc:493] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.714668 16700 leader_election.cc:304] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:45.715060 16806 raft_consensus.cc:2468] T f9d47e8e7b784085b7079b1542dbc579 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:45.716248 16849 leader_election.cc:304] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:45.714843 17446 raft_consensus.cc:515] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:45.717907 17440 raft_consensus.cc:2804] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:45.718504 17440 raft_consensus.cc:697] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:05:45.718828 17446 leader_election.cc:290] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:45.719468 17440 consensus_queue.cc:237] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:45.720347 17258 raft_consensus.cc:2804] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:45.721400 17258 raft_consensus.cc:697] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Becoming Leader. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:45.719688 17007 tablet_bootstrap.cc:492] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:45.722235 17258 consensus_queue.cc:237] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:05:45.727515 16960 raft_consensus.cc:2468] T f9d47e8e7b784085b7079b1542dbc579 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:45.732077 17007 tablet_bootstrap.cc:654] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.731938 16953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ca116ec6174decb610df3f8f565a6b" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:45.732600 16953 raft_consensus.cc:2468] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 0.
I20260709 13:05:45.734087 16730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "136f22096d4b41d89354924f7101d1c1" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:45.737483 16804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ca116ec6174decb610df3f8f565a6b" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:45.738257 16954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "136f22096d4b41d89354924f7101d1c1" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:45.739148 16954 raft_consensus.cc:2393] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4e1b05bc80e43f7941ece113fb97ab0 in current term 1: Already voted for candidate e2b87453fe654318abccdd1512b949c4 in this term.
I20260709 13:05:45.741459 16633 catalog_manager.cc:5697] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "c4b86aae3a8f4a9082e639144ec46228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:45.747362 16849 leader_election.cc:304] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0; no voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:45.748394 17368 raft_consensus.cc:3060] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:45.741798 16700 leader_election.cc:343] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.749689 16700 leader_election.cc:304] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228
I20260709 13:05:45.751639 17263 raft_consensus.cc:2804] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:45.752027 17263 raft_consensus.cc:493] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.752341 17263 raft_consensus.cc:3060] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.707122 17423 leader_election.cc:290] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 1 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:45.762140 16631 catalog_manager.cc:5697] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:45.763616 17008 tablet_bootstrap.cc:492] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:45.764252 17008 ts_tablet_manager.cc:1403] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.080s	user 0.022s	sys 0.018s
I20260709 13:05:45.766044 17263 raft_consensus.cc:515] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
W20260709 13:05:45.766827 16700 consensus_peers.cc:597] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 -> Peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Couldn't send request to peer d4e1b05bc80e43f7941ece113fb97ab0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.770618 17008 raft_consensus.cc:359] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:45.771452 17008 raft_consensus.cc:385] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.771765 17008 raft_consensus.cc:740] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.772541 17008 consensus_queue.cc:260] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:45.777290 17445 raft_consensus.cc:493] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.777819 17445 raft_consensus.cc:515] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:45.779820 17445 leader_election.cc:290] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:45.780754 16953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be51f207c3bc4276b220a03e2d9d4db4" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:45.776202 17008 ts_tablet_manager.cc:1434] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.001s
I20260709 13:05:45.784801 17008 tablet_bootstrap.cc:492] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:45.785837 17505 raft_consensus.cc:493] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.786464 17505 raft_consensus.cc:515] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:45.789458 16881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744cc6469e3749e2aacccb45464e8198" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:45.789721 17505 leader_election.cc:290] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
W20260709 13:05:45.790860 16923 leader_election.cc:343] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.788515 16954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ca116ec6174decb610df3f8f565a6b" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:45.791875 16954 raft_consensus.cc:3060] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.797080 17008 tablet_bootstrap.cc:654] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.874331 17263 leader_election.cc:290] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:45.875698 17422 consensus_queue.cc:1048] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:45.889523 16804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ca116ec6174decb610df3f8f565a6b" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:45.891219 16806 raft_consensus.cc:3060] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.899868 16731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744cc6469e3749e2aacccb45464e8198" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:45.900305 16730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be51f207c3bc4276b220a03e2d9d4db4" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:45.900795 16731 raft_consensus.cc:2468] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 0.
I20260709 13:05:45.941560 17368 raft_consensus.cc:2749] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:45.951329 16923 leader_election.cc:304] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:45.954273 16775 leader_election.cc:304] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228; no voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4
W20260709 13:05:45.956259 16700 leader_election.cc:343] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.957258 17420 raft_consensus.cc:2804] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:45.972754 17420 raft_consensus.cc:493] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.971998 17440 raft_consensus.cc:493] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.984175 17006 tablet_bootstrap.cc:654] T 7f4354282d844c7e83d760e4418512dc P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.986567 17420 raft_consensus.cc:3060] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.999711 17007 tablet_bootstrap.cc:492] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:46.000236 17007 ts_tablet_manager.cc:1403] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.281s	user 0.012s	sys 0.011s
I20260709 13:05:46.006628 16954 raft_consensus.cc:2468] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 1.
I20260709 13:05:46.017782 17007 raft_consensus.cc:359] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:46.025804 17007 raft_consensus.cc:385] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.026221 17007 raft_consensus.cc:740] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.027060 17007 consensus_queue.cc:260] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:46.036952 17445 raft_consensus.cc:493] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.037568 17445 raft_consensus.cc:515] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:46.039568 17445 leader_election.cc:290] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:46.042169 17008 tablet_bootstrap.cc:492] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:46.042865 17008 ts_tablet_manager.cc:1403] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.258s	user 0.049s	sys 0.039s
I20260709 13:05:45.990312 17440 raft_consensus.cc:515] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:46.002775 17432 consensus_queue.cc:1048] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.040s
I20260709 13:05:46.050154 16735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4354282d844c7e83d760e4418512dc" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:46.050581 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4354282d844c7e83d760e4418512dc" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:46.051580 16700 leader_election.cc:304] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228
I20260709 13:05:46.052958 17263 raft_consensus.cc:2804] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:46.053422 17263 raft_consensus.cc:697] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:05:46.054255 17263 consensus_queue.cc:237] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:46.062199 16735 raft_consensus.cc:2468] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 0.
I20260709 13:05:46.064141 17440 leader_election.cc:290] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:46.063803 17420 raft_consensus.cc:515] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:46.065325 17007 ts_tablet_manager.cc:1434] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.065s	user 0.007s	sys 0.000s
I20260709 13:05:46.066792 17007 tablet_bootstrap.cc:492] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
W20260709 13:05:46.069765 16775 leader_election.cc:343] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.090499 17007 tablet_bootstrap.cc:654] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.095017 16635 catalog_manager.cc:5697] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3). New cstate: current_term: 1 leader_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:46.098503 16775 leader_election.cc:304] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:46.100391 16635 catalog_manager.cc:5697] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3). New cstate: current_term: 1 leader_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:46.102402 17351 raft_consensus.cc:2804] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:46.106328 17351 raft_consensus.cc:493] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.106691 17351 raft_consensus.cc:3060] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.109706 17008 raft_consensus.cc:359] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:46.110499 17008 raft_consensus.cc:385] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.110790 17008 raft_consensus.cc:740] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.111578 17008 consensus_queue.cc:260] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:46.100812 17427 raft_consensus.cc:493] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d4e1b05bc80e43f7941ece113fb97ab0)
I20260709 13:05:46.137007 17008 ts_tablet_manager.cc:1434] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.094s	user 0.004s	sys 0.003s
I20260709 13:05:46.130350 16803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4354282d844c7e83d760e4418512dc" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:46.141582 17427 raft_consensus.cc:515] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:46.143846 17427 leader_election.cc:290] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:46.144816 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:46.145531 16884 raft_consensus.cc:2468] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 3.
I20260709 13:05:46.148895 17420 leader_election.cc:290] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:46.116086 17423 consensus_queue.cc:1048] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.035s
I20260709 13:05:46.155818 16775 leader_election.cc:304] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:46.157361 17350 raft_consensus.cc:2804] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:46.157819 17350 raft_consensus.cc:493] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d4e1b05bc80e43f7941ece113fb97ab0)
I20260709 13:05:46.158213 17350 raft_consensus.cc:3060] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:46.158412 16735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744cc6469e3749e2aacccb45464e8198" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:46.159216 16735 raft_consensus.cc:3060] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.191010 17008 tablet_bootstrap.cc:492] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:46.192979 16633 catalog_manager.cc:5697] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:46.195550 16633 catalog_manager.cc:5697] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:46.157100 17351 raft_consensus.cc:515] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:46.216367 17351 leader_election.cc:290] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:46.202831 16734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
W20260709 13:05:46.230064 16683 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:46.255879 17006 tablet_bootstrap.cc:492] T 7f4354282d844c7e83d760e4418512dc P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:46.256596 17006 ts_tablet_manager.cc:1403] T 7f4354282d844c7e83d760e4418512dc P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.548s	user 0.019s	sys 0.019s
I20260709 13:05:46.259428 16803 raft_consensus.cc:2393] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2b87453fe654318abccdd1512b949c4 in current term 1: Already voted for candidate c4b86aae3a8f4a9082e639144ec46228 in this term.
I20260709 13:05:46.273665 16735 raft_consensus.cc:2468] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 1.
I20260709 13:05:46.287665 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744cc6469e3749e2aacccb45464e8198" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:46.254616 17440 raft_consensus.cc:493] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.281666 16734 raft_consensus.cc:2393] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c4b86aae3a8f4a9082e639144ec46228 in current term 3: Already voted for candidate e2b87453fe654318abccdd1512b949c4 in this term.
I20260709 13:05:46.291541 16923 leader_election.cc:304] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:05:46.292506 17423 raft_consensus.cc:493] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.300455 17281 raft_consensus.cc:2804] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:46.299993 17514 raft_consensus.cc:493] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.306716 16735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4354282d844c7e83d760e4418512dc" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:46.310324 17397 raft_consensus.cc:2749] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:46.296363 17440 raft_consensus.cc:515] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:46.328346 17006 raft_consensus.cc:359] T 7f4354282d844c7e83d760e4418512dc P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:46.330946 17006 raft_consensus.cc:385] T 7f4354282d844c7e83d760e4418512dc P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.331297 17006 raft_consensus.cc:740] T 7f4354282d844c7e83d760e4418512dc P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.318928 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4354282d844c7e83d760e4418512dc" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:46.309890 17423 raft_consensus.cc:515] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:46.340090 17006 consensus_queue.cc:260] T 7f4354282d844c7e83d760e4418512dc P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:46.329867 16735 raft_consensus.cc:3060] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.320596 17514 raft_consensus.cc:515] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:46.350086 17423 leader_election.cc:290] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:46.354067 17007 tablet_bootstrap.cc:492] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4: No bootstrap required, opened a new log
I20260709 13:05:46.356330 17007 ts_tablet_manager.cc:1403] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.290s	user 0.029s	sys 0.043s
I20260709 13:05:46.360616 17007 raft_consensus.cc:359] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:46.362130 16806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c19ee4785104276aec22ba772dbc6ac" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
W20260709 13:05:46.362648 17520 log.cc:927] Time spent T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:05:46.364306 17007 raft_consensus.cc:385] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:05:46.366284 16923 leader_election.cc:343] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.371517 17007 raft_consensus.cc:740] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.372035 16806 raft_consensus.cc:2468] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:46.348860 16884 raft_consensus.cc:3060] T 7f4354282d844c7e83d760e4418512dc P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.376153 16849 leader_election.cc:304] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:46.372305 17007 consensus_queue.cc:260] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:46.388866 17008 tablet_bootstrap.cc:654] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.389168 17514 leader_election.cc:290] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:46.390482 16735 raft_consensus.cc:2468] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:46.371322 17281 raft_consensus.cc:697] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [term 1 LEADER]: Becoming Leader. State: Replica: 0086163b152644feb75caeac6f145533, State: Running, Role: LEADER
I20260709 13:05:46.366878 17350 raft_consensus.cc:515] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:46.408730 17007 ts_tablet_manager.cc:1434] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.052s	user 0.009s	sys 0.000s
I20260709 13:05:46.390872 17440 leader_election.cc:290] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:46.403111 17281 consensus_queue.cc:237] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:46.419907 16881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4354282d844c7e83d760e4418512dc" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:46.386920 17258 raft_consensus.cc:2804] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:46.424655 17258 raft_consensus.cc:493] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.444808 17258 raft_consensus.cc:3060] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:46.449203 17394 consensus_queue.cc:1048] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:46.451550 17350 leader_election.cc:290] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 3 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:46.453791 16809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f239a1de317547b598aeedc11994c75f" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:46.454484 16809 raft_consensus.cc:2468] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 0.
I20260709 13:05:46.442976 17526 consensus_queue.cc:1048] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:46.458693 16730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c19ee4785104276aec22ba772dbc6ac" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:46.459385 16730 raft_consensus.cc:2468] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 2.
I20260709 13:05:46.464958 17536 consensus_queue.cc:1048] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:46.475212 16775 leader_election.cc:304] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:05:46.490554 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f239a1de317547b598aeedc11994c75f" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:46.491364 16886 raft_consensus.cc:2468] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 0.
I20260709 13:05:46.498240 16700 leader_election.cc:304] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:05:46.486359 17427 raft_consensus.cc:2804] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:46.508515 16730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f239a1de317547b598aeedc11994c75f" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:46.504140 17533 raft_consensus.cc:2804] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:46.516230 17533 raft_consensus.cc:493] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.516564 17533 raft_consensus.cc:3060] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.519841 16884 raft_consensus.cc:2468] T 7f4354282d844c7e83d760e4418512dc P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:46.543820 17006 ts_tablet_manager.cc:1434] T 7f4354282d844c7e83d760e4418512dc P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.287s	user 0.012s	sys 0.000s
I20260709 13:05:46.544899 17008 tablet_bootstrap.cc:492] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:46.534770 17427 raft_consensus.cc:697] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Becoming Leader. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:05:46.556362 17427 consensus_queue.cc:237] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:46.559056 16880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f239a1de317547b598aeedc11994c75f" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:46.559787 16880 raft_consensus.cc:2468] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 0.
I20260709 13:05:46.559914 16879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:46.564990 16731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:46.566157 16731 raft_consensus.cc:2393] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c4b86aae3a8f4a9082e639144ec46228 in current term 3: Already voted for candidate e2b87453fe654318abccdd1512b949c4 in this term.
I20260709 13:05:46.568502 17006 tablet_bootstrap.cc:492] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:46.591857 16700 leader_election.cc:304] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:46.600564 17008 ts_tablet_manager.cc:1403] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.415s	user 0.014s	sys 0.019s
I20260709 13:05:46.600759 17006 tablet_bootstrap.cc:654] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.589084 16631 catalog_manager.cc:5697] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0086163b152644feb75caeac6f145533 (127.6.255.4). New cstate: current_term: 1 leader_uuid: "0086163b152644feb75caeac6f145533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:46.586715 17446 consensus_queue.cc:1048] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
W20260709 13:05:46.620277 16775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:37199, user_credentials={real_user=slave}} blocked reactor thread for 117380us
I20260709 13:05:46.610561 17535 raft_consensus.cc:2749] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:46.626667 17432 raft_consensus.cc:493] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.617087 17258 raft_consensus.cc:515] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:46.657027 17008 raft_consensus.cc:359] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:46.659528 17008 raft_consensus.cc:385] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.659932 17008 raft_consensus.cc:740] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.662477 16775 leader_election.cc:304] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:46.660775 17008 consensus_queue.cc:260] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:46.667804 17397 raft_consensus.cc:2804] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:46.677547 17397 raft_consensus.cc:493] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.677902 17397 raft_consensus.cc:3060] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.678926 17008 ts_tablet_manager.cc:1434] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.074s	user 0.007s	sys 0.000s
I20260709 13:05:46.680333 17008 tablet_bootstrap.cc:492] T 87ca116ec6174decb610df3f8f565a6b P c4b86aae3a8f4a9082e639144ec46228: Bootstrap starting.
I20260709 13:05:46.642253 17512 consensus_queue.cc:1048] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 13:05:46.693948 16879 raft_consensus.cc:2468] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 3.
I20260709 13:05:46.652863 17432 raft_consensus.cc:515] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:46.721016 17432 leader_election.cc:290] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:46.685333 17533 raft_consensus.cc:515] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:46.737730 17258 leader_election.cc:290] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 3 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:46.767302 17397 raft_consensus.cc:515] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
W20260709 13:05:46.749125 16700 consensus_peers.cc:597] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 -> Peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Couldn't send request to peer c4b86aae3a8f4a9082e639144ec46228. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.790889 16775 leader_election.cc:304] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: e2b87453fe654318abccdd1512b949c4
I20260709 13:05:46.763221 16730 raft_consensus.cc:2393] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c4b86aae3a8f4a9082e639144ec46228 in current term 1: Already voted for candidate e2b87453fe654318abccdd1512b949c4 in this term.
I20260709 13:05:46.805482 16806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c19ee4785104276aec22ba772dbc6ac" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:46.806066 17008 tablet_bootstrap.cc:654] T 87ca116ec6174decb610df3f8f565a6b P c4b86aae3a8f4a9082e639144ec46228: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.806161 16806 raft_consensus.cc:3060] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:05:46.796270 17514 raft_consensus.cc:2804] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:46.764341 17533 leader_election.cc:290] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:46.817605 17006 tablet_bootstrap.cc:492] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:46.818166 17006 ts_tablet_manager.cc:1403] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.250s	user 0.012s	sys 0.014s
I20260709 13:05:46.811998 17514 raft_consensus.cc:697] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 3 LEADER]: Becoming Leader. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:05:46.823674 17514 consensus_queue.cc:237] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:46.840173 16805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c968ba000a4542fc9204dd9a474e364c" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:46.840952 16805 raft_consensus.cc:2468] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 0.
I20260709 13:05:46.846181 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c968ba000a4542fc9204dd9a474e364c" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:46.846197 17006 raft_consensus.cc:359] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:46.849642 16879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f239a1de317547b598aeedc11994c75f" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:46.846884 16958 raft_consensus.cc:2468] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 0.
I20260709 13:05:46.849046 17006 raft_consensus.cc:385] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.852223 17006 raft_consensus.cc:740] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.850258 16879 raft_consensus.cc:3060] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.861690 16880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f239a1de317547b598aeedc11994c75f" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:46.867538 16879 raft_consensus.cc:2468] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:46.845420 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c19ee4785104276aec22ba772dbc6ac" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:46.901890 16737 raft_consensus.cc:3060] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:46.871932 16805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f239a1de317547b598aeedc11994c75f" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:46.846422 17397 leader_election.cc:290] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:46.907620 16805 raft_consensus.cc:2393] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e2b87453fe654318abccdd1512b949c4 in current term 1: Already voted for candidate c4b86aae3a8f4a9082e639144ec46228 in this term.
I20260709 13:05:46.881920 17006 consensus_queue.cc:260] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:46.914741 16631 catalog_manager.cc:5697] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "c4b86aae3a8f4a9082e639144ec46228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:46.885553 16806 raft_consensus.cc:2468] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 3.
I20260709 13:05:46.916960 16631 catalog_manager.cc:5697] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 reported cstate change: term changed from 2 to 3, leader changed from d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3) to c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2). New cstate: current_term: 3 leader_uuid: "c4b86aae3a8f4a9082e639144ec46228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:46.912580 17006 ts_tablet_manager.cc:1434] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.094s	user 0.004s	sys 0.002s
I20260709 13:05:46.919754 17006 tablet_bootstrap.cc:492] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0: Bootstrap starting.
I20260709 13:05:46.933382 16735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f239a1de317547b598aeedc11994c75f" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:46.934299 16735 raft_consensus.cc:2393] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c4b86aae3a8f4a9082e639144ec46228 in current term 1: Already voted for candidate e2b87453fe654318abccdd1512b949c4 in this term.
I20260709 13:05:46.935654 16737 raft_consensus.cc:2468] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 3.
I20260709 13:05:46.939177 16775 leader_election.cc:304] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:46.944135 17006 tablet_bootstrap.cc:654] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.944626 17008 tablet_bootstrap.cc:492] T 87ca116ec6174decb610df3f8f565a6b P c4b86aae3a8f4a9082e639144ec46228: No bootstrap required, opened a new log
I20260709 13:05:46.945206 17008 ts_tablet_manager.cc:1403] T 87ca116ec6174decb610df3f8f565a6b P c4b86aae3a8f4a9082e639144ec46228: Time spent bootstrapping tablet: real 0.265s	user 0.011s	sys 0.008s
I20260709 13:05:46.945654 17397 raft_consensus.cc:2804] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:46.946216 17397 raft_consensus.cc:697] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Becoming Leader. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:05:46.948540 17008 raft_consensus.cc:359] T 87ca116ec6174decb610df3f8f565a6b P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:46.949358 17008 raft_consensus.cc:385] T 87ca116ec6174decb610df3f8f565a6b P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.949719 17008 raft_consensus.cc:740] T 87ca116ec6174decb610df3f8f565a6b P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.952041 17008 consensus_queue.cc:260] T 87ca116ec6174decb610df3f8f565a6b P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:46.955091 17008 ts_tablet_manager.cc:1434] T 87ca116ec6174decb610df3f8f565a6b P c4b86aae3a8f4a9082e639144ec46228: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:05:46.940824 16700 leader_election.cc:304] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:05:46.947152 17397 consensus_queue.cc:237] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:46.963343 17006 tablet_bootstrap.cc:492] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0: No bootstrap required, opened a new log
I20260709 13:05:46.963889 17006 ts_tablet_manager.cc:1403] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent bootstrapping tablet: real 0.044s	user 0.008s	sys 0.015s
I20260709 13:05:46.940094 16849 leader_election.cc:304] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:46.966604 17006 raft_consensus.cc:359] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:46.970203 17006 raft_consensus.cc:385] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.970350 17536 raft_consensus.cc:2804] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:46.970484 17006 raft_consensus.cc:740] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.971091 17536 raft_consensus.cc:697] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 LEADER]: Becoming Leader. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:46.971458 17006 consensus_queue.cc:260] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:46.971997 17536 consensus_queue.cc:237] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:46.973948 17006 ts_tablet_manager.cc:1434] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.001s
I20260709 13:05:46.965380 17512 raft_consensus.cc:2804] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:46.977113 17512 raft_consensus.cc:493] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.977511 17512 raft_consensus.cc:3060] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.988554 17512 raft_consensus.cc:515] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:46.997846 17512 leader_election.cc:290] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:46.965507 16700 leader_election.cc:304] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:47.007457 17394 raft_consensus.cc:2749] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:47.012990 16631 catalog_manager.cc:5697] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "c4b86aae3a8f4a9082e639144ec46228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:47.016453 16633 catalog_manager.cc:5697] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:47.022681 16886 raft_consensus.cc:3060] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:47.034397 17281 consensus_queue.cc:1048] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:47.036232 17258 raft_consensus.cc:493] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.036779 17258 raft_consensus.cc:515] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:47.038780 17258 leader_election.cc:290] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:47.038921 16805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c968ba000a4542fc9204dd9a474e364c" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:47.039713 16805 raft_consensus.cc:3060] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:47.042737 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c968ba000a4542fc9204dd9a474e364c" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:47.043395 16960 raft_consensus.cc:3060] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:47.048532 16805 raft_consensus.cc:2468] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 1.
I20260709 13:05:47.068048 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744cc6469e3749e2aacccb45464e8198" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:47.079208 16960 raft_consensus.cc:2468] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 1.
I20260709 13:05:47.079852 17397 consensus_queue.cc:1048] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:47.085287 16731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744cc6469e3749e2aacccb45464e8198" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:05:47.105134 16731 raft_consensus.cc:2468] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:47.094223 17531 consensus_queue.cc:1048] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:47.100256 16700 leader_election.cc:304] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:05:47.108774 17394 raft_consensus.cc:2804] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:47.109254 17394 raft_consensus.cc:697] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:05:47.110035 17394 consensus_queue.cc:237] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:47.122231 17514 consensus_queue.cc:1048] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:47.129683 16806 raft_consensus.cc:3060] T 87ca116ec6174decb610df3f8f565a6b P c4b86aae3a8f4a9082e639144ec46228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:47.159593 16849 leader_election.cc:304] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533
I20260709 13:05:47.160521 17536 raft_consensus.cc:2804] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:47.160907 17536 raft_consensus.cc:493] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0086163b152644feb75caeac6f145533)
I20260709 13:05:47.161391 17536 raft_consensus.cc:3060] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.188129 17536 raft_consensus.cc:515] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:47.189138 16879 raft_consensus.cc:1240] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Rejecting Update request from peer 0086163b152644feb75caeac6f145533 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:47.202768 17536 leader_election.cc:290] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:47.205807 16731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744cc6469e3749e2aacccb45464e8198" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4"
I20260709 13:05:47.203346 17521 consensus_queue.cc:1059] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:05:47.209022 17521 raft_consensus.cc:3055] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:47.209357 17521 raft_consensus.cc:740] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Running, Role: LEADER
I20260709 13:05:47.210603 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744cc6469e3749e2aacccb45464e8198" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:47.210922 17521 consensus_queue.cc:260] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:47.214835 17521 raft_consensus.cc:3060] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.213444 16879 raft_consensus.cc:3060] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:47.227500 16631 catalog_manager.cc:5697] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:47.273113 16959 raft_consensus.cc:2468] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 2.
I20260709 13:05:47.276245 16849 leader_election.cc:304] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: e2b87453fe654318abccdd1512b949c4
I20260709 13:05:47.277113 17536 raft_consensus.cc:2804] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:47.277608 17536 raft_consensus.cc:697] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 LEADER]: Becoming Leader. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:47.278415 17536 consensus_queue.cc:237] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:47.302014 16631 catalog_manager.cc:5697] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 reported cstate change: term changed from 1 to 2, leader changed from 0086163b152644feb75caeac6f145533 (127.6.255.4) to d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3). New cstate: current_term: 2 leader_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:47.624665 16886 raft_consensus.cc:1275] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Refusing update from remote peer c4b86aae3a8f4a9082e639144ec46228: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:47.643975 16806 raft_consensus.cc:1275] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228 [term 3 FOLLOWER]: Refusing update from remote peer d4e1b05bc80e43f7941ece113fb97ab0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:47.646191 16734 raft_consensus.cc:1275] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Refusing update from remote peer d4e1b05bc80e43f7941ece113fb97ab0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:47.648002 16734 raft_consensus.cc:1275] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Refusing update from remote peer c4b86aae3a8f4a9082e639144ec46228: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:47.657053 17564 consensus_queue.cc:1048] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:47.687474 17539 consensus_queue.cc:1048] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:47.692106 17540 consensus_queue.cc:1048] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:47.699193 17514 consensus_queue.cc:1048] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:47.710443 17539 consensus_queue.cc:1048] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:47.713076 17564 consensus_queue.cc:1048] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:47.734076 17569 consensus_queue.cc:1048] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:47.758117 17394 consensus_queue.cc:1048] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:47.819272 16731 raft_consensus.cc:3060] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.828647 16731 raft_consensus.cc:1275] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Refusing update from remote peer d4e1b05bc80e43f7941ece113fb97ab0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:47.837724 16775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:45101, user_credentials={real_user=slave}} blocked reactor thread for 61432.1us
I20260709 13:05:47.859077 17536 consensus_queue.cc:1048] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:47.870456 16959 raft_consensus.cc:1275] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Refusing update from remote peer d4e1b05bc80e43f7941ece113fb97ab0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:47.872593 17258 consensus_queue.cc:1048] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:05:48.444810 16981 debug-util.cc:398] Leaking SignalData structure 0x7b080029e520 after lost signal to thread 7167
W20260709 13:05:48.446110 16981 debug-util.cc:398] Leaking SignalData structure 0x7b080040caa0 after lost signal to thread 16669
W20260709 13:05:48.447748 16981 debug-util.cc:398] Leaking SignalData structure 0x7b08001a2920 after lost signal to thread 16761
W20260709 13:05:48.449100 16981 debug-util.cc:398] Leaking SignalData structure 0x7b0800125300 after lost signal to thread 16836
W20260709 13:05:48.450459 16981 debug-util.cc:398] Leaking SignalData structure 0x7b0800346b60 after lost signal to thread 16910
W20260709 13:05:48.451795 16981 debug-util.cc:398] Leaking SignalData structure 0x7b08000f3000 after lost signal to thread 16984
W20260709 13:05:48.761137 16700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:37199, user_credentials={real_user=slave}} blocked reactor thread for 62142.7us
I20260709 13:05:49.054332  7164 tablet_server.cc:179] TabletServer@127.6.255.1:0 shutting down...
W20260709 13:05:49.099114 16923 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:49.127624 16923 consensus_peers.cc:597] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:49.139519 16923 consensus_peers.cc:597] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:49.143872 16849 consensus_peers.cc:597] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:49.148540 16849 consensus_peers.cc:597] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:49.164444 16775 consensus_peers.cc:597] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:49.175519 16775 consensus_peers.cc:597] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:49.180723 16775 consensus_peers.cc:597] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:49.182593 16849 consensus_peers.cc:597] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:49.205410  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:49.207537  7164 tablet_replica.cc:333] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.208957  7164 raft_consensus.cc:2243] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:49.211297  7164 raft_consensus.cc:2272] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.215461  7164 tablet_replica.cc:333] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.216430  7164 raft_consensus.cc:2243] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:49.217798  7164 raft_consensus.cc:2272] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:49.218421 16775 consensus_peers.cc:597] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:49.221962  7164 tablet_replica.cc:333] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.223030  7164 raft_consensus.cc:2243] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.223698  7164 raft_consensus.cc:2272] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.227634  7164 tablet_replica.cc:333] T 59de6d1228da41da938527632da08f4d P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.228719  7164 raft_consensus.cc:2243] T 59de6d1228da41da938527632da08f4d P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.229341  7164 raft_consensus.cc:2272] T 59de6d1228da41da938527632da08f4d P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.233100  7164 tablet_replica.cc:333] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.234212  7164 raft_consensus.cc:2243] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.234910  7164 raft_consensus.cc:2272] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:49.237568 16775 consensus_peers.cc:597] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:49.238447  7164 tablet_replica.cc:333] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.239531  7164 raft_consensus.cc:2243] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.240206  7164 raft_consensus.cc:2272] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.244445  7164 tablet_replica.cc:333] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.245497  7164 raft_consensus.cc:2243] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:49.246850  7164 raft_consensus.cc:2272] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:49.247926 16775 consensus_peers.cc:597] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:49.250479  7164 tablet_replica.cc:333] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.251541  7164 raft_consensus.cc:2243] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.252261  7164 raft_consensus.cc:2272] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.255651  7164 tablet_replica.cc:333] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.256508  7164 raft_consensus.cc:2243] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.257193  7164 raft_consensus.cc:2272] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.261503  7164 tablet_replica.cc:333] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.262550  7164 raft_consensus.cc:2243] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.263299  7164 raft_consensus.cc:2272] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.267796  7164 tablet_replica.cc:333] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.268782  7164 raft_consensus.cc:2243] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.269488  7164 raft_consensus.cc:2272] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.273298  7164 tablet_replica.cc:333] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.274252  7164 raft_consensus.cc:2243] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:49.275533  7164 raft_consensus.cc:2272] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.279609  7164 tablet_replica.cc:333] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.280488  7164 raft_consensus.cc:2243] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.281150  7164 raft_consensus.cc:2272] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.285188  7164 tablet_replica.cc:333] T 61f747edd7934aac93044ba370e34d17 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.286176  7164 raft_consensus.cc:2243] T 61f747edd7934aac93044ba370e34d17 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.286886  7164 raft_consensus.cc:2272] T 61f747edd7934aac93044ba370e34d17 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.290723  7164 tablet_replica.cc:333] T 612e472fbe674788986798c65836388f P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.291631  7164 raft_consensus.cc:2243] T 612e472fbe674788986798c65836388f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.292353  7164 raft_consensus.cc:2272] T 612e472fbe674788986798c65836388f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.295961  7164 tablet_replica.cc:333] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.296976  7164 raft_consensus.cc:2243] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.297751  7164 raft_consensus.cc:2272] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.301771  7164 tablet_replica.cc:333] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.302743  7164 raft_consensus.cc:2243] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:49.304032  7164 raft_consensus.cc:2272] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.308190  7164 tablet_replica.cc:333] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.309221  7164 raft_consensus.cc:2243] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:49.310711  7164 raft_consensus.cc:2272] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.314378  7164 tablet_replica.cc:333] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.315582  7164 raft_consensus.cc:2243] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:49.316931  7164 raft_consensus.cc:2272] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.320431  7164 tablet_replica.cc:333] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.321450  7164 raft_consensus.cc:2243] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.329161  7164 raft_consensus.cc:2272] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.332825  7164 tablet_replica.cc:333] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.333792  7164 raft_consensus.cc:2243] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.334625  7164 raft_consensus.cc:2272] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.338089  7164 tablet_replica.cc:333] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.338941  7164 raft_consensus.cc:2243] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:49.340274  7164 raft_consensus.cc:2272] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.346117  7164 tablet_replica.cc:333] T 0b8add4b17084a6189509807b7e663e5 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.347003  7164 raft_consensus.cc:2243] T 0b8add4b17084a6189509807b7e663e5 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.347893  7164 raft_consensus.cc:2272] T 0b8add4b17084a6189509807b7e663e5 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.351164  7164 tablet_replica.cc:333] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.352005  7164 raft_consensus.cc:2243] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.352627  7164 raft_consensus.cc:2272] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.356527  7164 tablet_replica.cc:333] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.357504  7164 raft_consensus.cc:2243] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.358084  7164 raft_consensus.cc:2272] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.361255  7164 tablet_replica.cc:333] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.362216  7164 raft_consensus.cc:2243] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.362890  7164 raft_consensus.cc:2272] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.366161  7164 tablet_replica.cc:333] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.367036  7164 raft_consensus.cc:2243] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.367682  7164 raft_consensus.cc:2272] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.371706  7164 tablet_replica.cc:333] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.374086  7164 raft_consensus.cc:2243] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.374754  7164 raft_consensus.cc:2272] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.378762  7164 tablet_replica.cc:333] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.379612  7164 raft_consensus.cc:2243] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:49.382165  7164 raft_consensus.cc:2272] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.385540  7164 tablet_replica.cc:333] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.386394  7164 raft_consensus.cc:2243] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.387072  7164 raft_consensus.cc:2272] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.390412  7164 tablet_replica.cc:333] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.391208  7164 raft_consensus.cc:2243] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.391819  7164 raft_consensus.cc:2272] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.395304  7164 tablet_replica.cc:333] T c9043e0bc02745d187e9ca5a5c71274f P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.397873  7164 raft_consensus.cc:2243] T c9043e0bc02745d187e9ca5a5c71274f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.398584  7164 raft_consensus.cc:2272] T c9043e0bc02745d187e9ca5a5c71274f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.404140  7164 tablet_replica.cc:333] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.404942  7164 raft_consensus.cc:2243] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:49.405978  7164 raft_consensus.cc:2272] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.409281  7164 tablet_replica.cc:333] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.410142  7164 raft_consensus.cc:2243] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:49.411424  7164 raft_consensus.cc:2272] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.414624  7164 tablet_replica.cc:333] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.415553  7164 raft_consensus.cc:2243] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:49.416726  7164 raft_consensus.cc:2272] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.420248  7164 tablet_replica.cc:333] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.421294  7164 raft_consensus.cc:2243] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.422003  7164 raft_consensus.cc:2272] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.425408  7164 tablet_replica.cc:333] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.426203  7164 raft_consensus.cc:2243] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:49.441532  7164 raft_consensus.cc:2272] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.454586  7164 tablet_replica.cc:333] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.455681  7164 raft_consensus.cc:2243] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:49.456961  7164 raft_consensus.cc:2272] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:49.459954 16849 consensus_peers.cc:597] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:49.460119  7164 tablet_replica.cc:333] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
W20260709 13:05:49.469619 16849 consensus_peers.cc:597] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:49.470044  7164 raft_consensus.cc:2243] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:05:49.470525 16849 consensus_peers.cc:597] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:49.470860  7164 raft_consensus.cc:2272] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:49.468515 16775 consensus_peers.cc:597] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:49.471508 16849 consensus_peers.cc:597] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:49.473156 16923 consensus_peers.cc:597] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:49.471907 16775 consensus_peers.cc:597] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:49.475342 16923 consensus_peers.cc:597] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:49.482406  7164 tablet_replica.cc:333] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.484081  7164 raft_consensus.cc:2243] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:49.487200  7164 raft_consensus.cc:2272] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.492317  7164 tablet_replica.cc:333] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.496652  7164 raft_consensus.cc:2243] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.497308  7164 raft_consensus.cc:2272] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.504899  7164 tablet_replica.cc:333] T ac3ad4fe0bc4435cb1c596ad215889a3 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.505688  7164 raft_consensus.cc:2243] T ac3ad4fe0bc4435cb1c596ad215889a3 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.515031  7164 raft_consensus.cc:2272] T ac3ad4fe0bc4435cb1c596ad215889a3 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.522500  7164 tablet_replica.cc:333] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.523275  7164 raft_consensus.cc:2243] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.523880  7164 raft_consensus.cc:2272] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:49.528402 16923 consensus_peers.cc:597] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:49.529035  7164 tablet_replica.cc:333] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.529924  7164 raft_consensus.cc:2243] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.530604  7164 raft_consensus.cc:2272] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:49.531213 16923 consensus_peers.cc:597] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:49.542555  7164 tablet_replica.cc:333] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4: stopping tablet replica
I20260709 13:05:49.543459  7164 raft_consensus.cc:2243] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.544117  7164 raft_consensus.cc:2272] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:49.549772 16923 consensus_peers.cc:597] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:49.553962 16775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:37199, user_credentials={real_user=slave}} blocked reactor thread for 73963.7us
W20260709 13:05:49.559952 16775 consensus_peers.cc:597] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:49.560586 16775 consensus_peers.cc:597] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:49.583690 16849 consensus_peers.cc:597] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:49.587107 16849 consensus_peers.cc:597] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:49.590180 16775 consensus_peers.cc:597] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:49.607717 16849 consensus_peers.cc:597] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:49.609529 16849 consensus_peers.cc:597] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:49.613638 16775 consensus_peers.cc:597] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:49.768153  7164 tablet_server.cc:196] TabletServer@127.6.255.1:0 shutdown complete.
I20260709 13:05:49.970916  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:50.001107 17601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:50.028258 17603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:50.063285 17605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:50.072970  7164 server_base.cc:1061] running on GCE node
I20260709 13:05:50.079300  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:50.079668  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:50.079916  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602350079899 us; error 0 us; skew 500 ppm
I20260709 13:05:50.080722  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:50.118422  7164 webserver.cc:533] Webserver started at http://127.6.255.1:46089/ using document root <none> and password file <none>
I20260709 13:05:50.119268  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:50.119549  7164 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:05:50.135336  7164 fs_manager.cc:714] Time spent opening directory manager: real 0.014s	user 0.007s	sys 0.005s
I20260709 13:05:50.155411 17610 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:50.156420  7164 fs_manager.cc:730] Time spent opening block manager: real 0.010s	user 0.004s	sys 0.004s
I20260709 13:05:50.156821  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/ts-0-root
uuid: "e2b87453fe654318abccdd1512b949c4"
format_stamp: "Formatted at 2026-07-09 13:05:33 on dist-test-slave-rnv4"
I20260709 13:05:50.157516  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602046064212-7164-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:50.206481  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:50.214632  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:50.225944  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:50.249480 17617 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
I20260709 13:05:50.296196 17567 raft_consensus.cc:493] T 87ca116ec6174decb610df3f8f565a6b P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.297034 17567 raft_consensus.cc:515] T 87ca116ec6174decb610df3f8f565a6b P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:50.312417 17567 leader_election.cc:290] T 87ca116ec6174decb610df3f8f565a6b P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:50.317929 16956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ca116ec6174decb610df3f8f565a6b" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
W20260709 13:05:50.326170 16775 leader_election.cc:336] T 87ca116ec6174decb610df3f8f565a6b P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:50.326749 16775 leader_election.cc:304] T 87ca116ec6174decb610df3f8f565a6b P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228; no voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:50.327538 17567 raft_consensus.cc:2749] T 87ca116ec6174decb610df3f8f565a6b P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:50.375226 17564 raft_consensus.cc:493] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.375908 17564 raft_consensus.cc:515] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:50.378196 17564 leader_election.cc:290] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:50.379299 17591 raft_consensus.cc:493] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.379896 17591 raft_consensus.cc:515] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:50.381693 17584 raft_consensus.cc:493] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.382169 17591 leader_election.cc:290] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:50.382254 17584 raft_consensus.cc:515] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:50.384747 17584 leader_election.cc:290] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:50.385990 16956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20cdedb225464d5e812a56ed7a7d220f" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
W20260709 13:05:50.395192 16923 leader_election.cc:336] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
W20260709 13:05:50.397284 16923 leader_election.cc:336] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:50.402848 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c968ba000a4542fc9204dd9a474e364c" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:50.403036 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1660f76b7074e2cb7542ed02dd179d9" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
W20260709 13:05:50.404214 16849 leader_election.cc:336] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:50.404716 16849 leader_election.cc:304] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0; no voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:50.405848 17517 raft_consensus.cc:2749] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:50.414952 16923 leader_election.cc:304] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:50.416603 17584 raft_consensus.cc:2749] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:50.418356 16923 leader_election.cc:304] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:50.422348 17584 raft_consensus.cc:2749] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:50.428071 17517 raft_consensus.cc:493] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.428640 17517 raft_consensus.cc:515] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:50.430859 17517 leader_election.cc:290] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:50.431048 17258 raft_consensus.cc:493] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.431540 17258 raft_consensus.cc:515] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:50.432507 17517 raft_consensus.cc:493] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.433279 17258 leader_election.cc:290] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:50.435989 17584 raft_consensus.cc:493] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.436599 17584 raft_consensus.cc:515] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:50.438208 17595 raft_consensus.cc:493] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.438786 17595 raft_consensus.cc:515] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:50.441015 17595 leader_election.cc:290] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:50.442659 17584 leader_election.cc:290] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:50.433249 17517 raft_consensus.cc:515] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:50.448414 17517 leader_election.cc:290] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
W20260709 13:05:50.449862 16923 leader_election.cc:336] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:50.451102 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d6e27a9a96c4121b2abc71e0d7738be" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:50.455099 16956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e295eefc8ff46d9a31a4c0898d16c63" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:50.461333 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be51f207c3bc4276b220a03e2d9d4db4" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:50.464008 17584 raft_consensus.cc:493] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.464540 17602 raft_consensus.cc:493] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.464694 17584 raft_consensus.cc:515] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:50.465160 17602 raft_consensus.cc:515] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:50.467552 17602 leader_election.cc:290] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:50.467638 17584 leader_election.cc:290] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:50.471127 16956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1660f76b7074e2cb7542ed02dd179d9" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:50.473520 17602 raft_consensus.cc:493] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.474104 17602 raft_consensus.cc:515] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
W20260709 13:05:50.475844 16923 leader_election.cc:336] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
W20260709 13:05:50.476394 16923 leader_election.cc:336] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:50.476578 17602 leader_election.cc:290] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:50.477288 16956 raft_consensus.cc:2468] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:50.477684 16881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20cdedb225464d5e812a56ed7a7d220f" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:50.479907 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3cecfcf8ae242e1b70f628f293da197" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:50.480347 16881 raft_consensus.cc:2468] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 1.
I20260709 13:05:50.483340 17602 raft_consensus.cc:493] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.483929 17602 raft_consensus.cc:515] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:50.484506 16923 leader_election.cc:304] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: e2b87453fe654318abccdd1512b949c4
I20260709 13:05:50.485860 17591 raft_consensus.cc:2804] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:50.486296 17591 raft_consensus.cc:493] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.486676 17591 raft_consensus.cc:3060] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.495062 17591 raft_consensus.cc:515] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:50.496404 16956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "136f22096d4b41d89354924f7101d1c1" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:50.495097 17602 leader_election.cc:290] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:50.502362 16881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27be53bca112445d8bdc640329310d67" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:50.503137 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20cdedb225464d5e812a56ed7a7d220f" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:50.503726 16884 raft_consensus.cc:3060] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.509769 16775 leader_election.cc:304] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:05:50.511634 16884 raft_consensus.cc:2468] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 2.
I20260709 13:05:50.484992 17584 raft_consensus.cc:493] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.514245 17595 raft_consensus.cc:2804] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:50.514788 17595 raft_consensus.cc:493] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.514590 17584 raft_consensus.cc:515] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:50.515252 17595 raft_consensus.cc:3060] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.516898 16956 raft_consensus.cc:2468] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:50.521605 17584 leader_election.cc:290] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:50.523130 16923 leader_election.cc:304] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:50.524044 17600 raft_consensus.cc:2749] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:50.526769 16849 leader_election.cc:336] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:50.527292 16849 leader_election.cc:304] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:50.497289 17591 leader_election.cc:290] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:50.529467 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9ca489a4d2416092d6dcfae2498136" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
W20260709 13:05:50.532419 16923 leader_election.cc:336] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:50.532891 16923 leader_election.cc:304] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:50.534865 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "136f22096d4b41d89354924f7101d1c1" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:50.535472 16884 raft_consensus.cc:2468] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 1.
I20260709 13:05:50.535779 17564 raft_consensus.cc:2749] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:50.536353 17591 raft_consensus.cc:2749] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:50.538288 16849 leader_election.cc:336] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
W20260709 13:05:50.540393 16775 leader_election.cc:336] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:50.542616 17595 raft_consensus.cc:515] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:50.544507 17591 raft_consensus.cc:493] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.545082 17591 raft_consensus.cc:515] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:50.546766 16849 leader_election.cc:304] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0; no voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:50.547374 17591 leader_election.cc:290] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:50.548774 16923 leader_election.cc:304] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:50.549685 16923 leader_election.cc:304] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:50.550338 17595 leader_election.cc:290] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:50.550431 17602 raft_consensus.cc:2804] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:50.551254 17600 raft_consensus.cc:2804] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:50.551640 17600 raft_consensus.cc:493] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.552003 17600 raft_consensus.cc:3060] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.552134 17564 raft_consensus.cc:2749] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:50.553874 17587 raft_consensus.cc:493] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.554436 17587 raft_consensus.cc:515] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:50.556833 17587 leader_election.cc:290] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:50.557533 17602 raft_consensus.cc:697] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Becoming Leader. State: Replica: 0086163b152644feb75caeac6f145533, State: Running, Role: LEADER
I20260709 13:05:50.558575 17602 consensus_queue.cc:237] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:50.561866 16956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1660f76b7074e2cb7542ed02dd179d9" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:50.562539 16956 raft_consensus.cc:3060] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.564087 17600 raft_consensus.cc:515] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
W20260709 13:05:50.567651 16923 leader_election.cc:336] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:50.568080 16923 leader_election.cc:304] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:50.573029 17591 raft_consensus.cc:2749] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:50.574247 16923 leader_election.cc:336] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:50.575264 16849 leader_election.cc:304] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:50.575870 17600 leader_election.cc:290] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:50.577292 17587 raft_consensus.cc:493] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.577916 17587 raft_consensus.cc:515] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:50.580557 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a5bbadcd9b48dba8fe272a18679ede" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:50.588717 16923 leader_election.cc:304] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:50.590791 17587 leader_election.cc:290] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:50.591481 17591 raft_consensus.cc:2749] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:50.592684 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82d38de8d2c47b38ca8811691c68d8f" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:50.593580 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "136f22096d4b41d89354924f7101d1c1" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:50.594147 16884 raft_consensus.cc:3060] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.615690 16884 raft_consensus.cc:2468] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 2.
I20260709 13:05:50.616183 17564 raft_consensus.cc:2764] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:05:50.618501 17587 raft_consensus.cc:493] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.619136 17587 raft_consensus.cc:515] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:50.621729 17587 leader_election.cc:290] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
W20260709 13:05:50.628093 16923 leader_election.cc:336] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:50.628432 17602 raft_consensus.cc:493] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.629127 17602 raft_consensus.cc:515] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:50.631062 16923 leader_election.cc:304] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:50.632449 17602 leader_election.cc:290] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:50.633499 17600 raft_consensus.cc:2804] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:50.634050 17600 raft_consensus.cc:697] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Becoming Leader. State: Replica: 0086163b152644feb75caeac6f145533, State: Running, Role: LEADER
I20260709 13:05:50.634971 17600 consensus_queue.cc:237] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:50.646852 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c968ba000a4542fc9204dd9a474e364c" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:50.647619 16959 raft_consensus.cc:2468] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:50.652030 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be51f207c3bc4276b220a03e2d9d4db4" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:50.652680 16959 raft_consensus.cc:2468] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
W20260709 13:05:50.652586 16923 leader_election.cc:336] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:50.654052 16923 leader_election.cc:304] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4
W20260709 13:05:50.654457 16775 leader_election.cc:336] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
W20260709 13:05:50.654958 16923 leader_election.cc:336] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:50.657918 17602 raft_consensus.cc:2749] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:50.661639 17618 log.cc:927] Time spent T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:05:50.663654 16956 raft_consensus.cc:2468] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 2.
W20260709 13:05:50.665750 16923 leader_election.cc:336] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:50.668542 17587 raft_consensus.cc:493] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.669145 17587 raft_consensus.cc:515] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
W20260709 13:05:50.642812 16849 leader_election.cc:336] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:50.676275 17587 leader_election.cc:290] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
W20260709 13:05:50.677997 16775 leader_election.cc:336] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:50.689371 16635 catalog_manager.cc:5697] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 reported cstate change: term changed from 1 to 2, leader changed from e2b87453fe654318abccdd1512b949c4 (127.6.255.1) to 0086163b152644feb75caeac6f145533 (127.6.255.4). New cstate: current_term: 2 leader_uuid: "0086163b152644feb75caeac6f145533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:50.691404 16635 catalog_manager.cc:5697] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 reported cstate change: term changed from 1 to 2, leader changed from e2b87453fe654318abccdd1512b949c4 (127.6.255.1) to 0086163b152644feb75caeac6f145533 (127.6.255.4). New cstate: current_term: 2 leader_uuid: "0086163b152644feb75caeac6f145533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:50.707357 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ca116ec6174decb610df3f8f565a6b" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:50.707984 16812 raft_consensus.cc:2468] T 87ca116ec6174decb610df3f8f565a6b P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 1.
I20260709 13:05:50.710788 17600 raft_consensus.cc:493] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.711355 17600 raft_consensus.cc:515] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:50.712890 17517 raft_consensus.cc:493] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.713419 17600 leader_election.cc:290] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:50.713521 17517 raft_consensus.cc:515] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:50.715760 17517 leader_election.cc:290] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:50.717058 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e295eefc8ff46d9a31a4c0898d16c63" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:50.717657 16884 raft_consensus.cc:2468] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 1.
I20260709 13:05:50.720466 16923 leader_election.cc:304] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
W20260709 13:05:50.724154 16923 leader_election.cc:336] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:50.724686 17600 raft_consensus.cc:2804] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:50.725697 17600 raft_consensus.cc:493] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.726426 17600 raft_consensus.cc:3060] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.733180 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3cecfcf8ae242e1b70f628f293da197" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:50.733973 16959 raft_consensus.cc:2468] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:50.741887 17600 raft_consensus.cc:515] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:50.743494 16849 leader_election.cc:304] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:50.745136 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e295eefc8ff46d9a31a4c0898d16c63" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:50.745779 16886 raft_consensus.cc:3060] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.746681 17564 raft_consensus.cc:2804] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:50.747097 17564 raft_consensus.cc:493] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.747493 17564 raft_consensus.cc:3060] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.755559 16886 raft_consensus.cc:2468] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 2.
I20260709 13:05:50.755882 17564 raft_consensus.cc:515] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:50.757272 16923 leader_election.cc:304] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:50.759208 17602 raft_consensus.cc:2804] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:50.759745 17602 raft_consensus.cc:697] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Becoming Leader. State: Replica: 0086163b152644feb75caeac6f145533, State: Running, Role: LEADER
I20260709 13:05:50.762305 17564 leader_election.cc:290] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:50.760723 17602 consensus_queue.cc:237] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:50.764218 17517 raft_consensus.cc:493] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.773080 17517 raft_consensus.cc:515] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
W20260709 13:05:50.765232 16775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:45101, user_credentials={real_user=slave}} blocked reactor thread for 58248.6us
I20260709 13:05:50.777149 17517 leader_election.cc:290] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:50.778003 16775 leader_election.cc:304] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228; no voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:50.779023 17587 raft_consensus.cc:2749] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:50.779477 16775 leader_election.cc:304] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:05:50.780435 17587 raft_consensus.cc:2804] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:50.780846 17587 raft_consensus.cc:493] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
W20260709 13:05:50.780911 16923 leader_election.cc:336] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:50.781296 17587 raft_consensus.cc:3060] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.781967 17600 leader_election.cc:290] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:50.780576 16775 leader_election.cc:304] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:05:50.785852 17621 raft_consensus.cc:2804] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:50.786252 17621 raft_consensus.cc:493] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.786609 17621 raft_consensus.cc:3060] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.789573 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3cecfcf8ae242e1b70f628f293da197" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:50.790190 16959 raft_consensus.cc:3060] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.793135 17587 raft_consensus.cc:515] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:50.785853 16775 leader_election.cc:304] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: e2b87453fe654318abccdd1512b949c4
I20260709 13:05:50.796367 17567 raft_consensus.cc:2804] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:50.796855 17567 raft_consensus.cc:697] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [term 2 LEADER]: Becoming Leader. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:05:50.799080 17595 raft_consensus.cc:493] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.799331 17567 consensus_queue.cc:237] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:50.799791 17595 raft_consensus.cc:515] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:50.803212 17595 leader_election.cc:290] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:50.803194 16956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f48e0f9f83da446a877cd06e19768929" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:50.803277 16923 leader_election.cc:304] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: e2b87453fe654318abccdd1512b949c4
I20260709 13:05:50.797580 16959 raft_consensus.cc:2468] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 2.
I20260709 13:05:50.804911 17600 raft_consensus.cc:2804] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:50.805421 17600 raft_consensus.cc:493] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.805904 17600 raft_consensus.cc:3060] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.814320 17587 leader_election.cc:290] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:50.816607 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d83d4d3d894b81a85b945ef97d79de" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
W20260709 13:05:50.819101 16849 leader_election.cc:336] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:50.823444 16849 leader_election.cc:304] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:50.824218 17258 raft_consensus.cc:2804] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:50.824688 17258 raft_consensus.cc:697] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 LEADER]: Becoming Leader. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:50.830080 17258 consensus_queue.cc:237] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:50.832926 17564 raft_consensus.cc:493] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.834761 17564 raft_consensus.cc:515] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:50.839244 17564 leader_election.cc:290] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:50.839888 17564 raft_consensus.cc:493] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.840735 17564 raft_consensus.cc:515] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:50.841066 17600 raft_consensus.cc:515] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:50.843436 17564 leader_election.cc:290] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:50.843757 17584 raft_consensus.cc:493] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.844489 17584 raft_consensus.cc:515] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:50.847229 17584 leader_election.cc:290] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:50.849501 17621 raft_consensus.cc:515] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:50.851300 17600 leader_election.cc:290] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:50.851559 17621 leader_election.cc:290] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
W20260709 13:05:50.860648 16923 leader_election.cc:336] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
W20260709 13:05:50.861236 16923 leader_election.cc:336] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:50.884907 16803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82d38de8d2c47b38ca8811691c68d8f" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:50.885878 16803 raft_consensus.cc:2468] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 1.
I20260709 13:05:50.906816 16633 catalog_manager.cc:5697] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 reported cstate change: term changed from 1 to 2, leader changed from e2b87453fe654318abccdd1512b949c4 (127.6.255.1) to c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "c4b86aae3a8f4a9082e639144ec46228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:50.916864 16632 catalog_manager.cc:5697] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 reported cstate change: term changed from 1 to 2, leader changed from e2b87453fe654318abccdd1512b949c4 (127.6.255.1) to d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3). New cstate: current_term: 2 leader_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:50.921861 16635 catalog_manager.cc:5697] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 reported cstate change: term changed from 1 to 2, leader changed from e2b87453fe654318abccdd1512b949c4 (127.6.255.1) to 0086163b152644feb75caeac6f145533 (127.6.255.4). New cstate: current_term: 2 leader_uuid: "0086163b152644feb75caeac6f145533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:50.928740 16849 leader_election.cc:336] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
W20260709 13:05:50.929656 16849 leader_election.cc:336] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:50.930049 16849 leader_election.cc:304] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0; no voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:50.929916 16803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ca116ec6174decb610df3f8f565a6b" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:50.930868 16803 raft_consensus.cc:3060] T 87ca116ec6174decb610df3f8f565a6b P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.931020 17258 raft_consensus.cc:2749] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:50.936776 17600 raft_consensus.cc:493] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.937482 17600 raft_consensus.cc:515] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:50.939657 17600 leader_election.cc:290] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:50.946547 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a5bbadcd9b48dba8fe272a18679ede" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:50.947304 16959 raft_consensus.cc:2468] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
W20260709 13:05:50.947819 16775 leader_election.cc:336] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
W20260709 13:05:50.956800 16775 leader_election.cc:336] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
W20260709 13:05:50.951244 16923 leader_election.cc:336] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:50.957867 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27be53bca112445d8bdc640329310d67" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:50.964179 16959 raft_consensus.cc:2468] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 1.
I20260709 13:05:50.950429 16803 raft_consensus.cc:2468] T 87ca116ec6174decb610df3f8f565a6b P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 2.
I20260709 13:05:50.967007 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9ca489a4d2416092d6dcfae2498136" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:50.967653 16959 raft_consensus.cc:2468] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:50.969700 16956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c968ba000a4542fc9204dd9a474e364c" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:50.970347 16956 raft_consensus.cc:3060] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.967000 16849 leader_election.cc:304] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:50.973742 17629 raft_consensus.cc:2804] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:50.974110 16849 leader_election.cc:304] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:50.975476 17584 raft_consensus.cc:493] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.976130 17584 raft_consensus.cc:515] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:50.977622 16884 raft_consensus.cc:1275] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Refusing update from remote peer 0086163b152644feb75caeac6f145533: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:50.978578 17584 leader_election.cc:290] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:50.979251 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be51f207c3bc4276b220a03e2d9d4db4" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:50.979489 17564 raft_consensus.cc:2804] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:50.979885 17564 raft_consensus.cc:493] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.980401 17564 raft_consensus.cc:3060] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.980510 17584 consensus_queue.cc:1048] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:50.980971 17627 raft_consensus.cc:493] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.981599 17627 raft_consensus.cc:515] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:50.982070 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d83d4d3d894b81a85b945ef97d79de" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:50.982743 16884 raft_consensus.cc:2468] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 1.
I20260709 13:05:50.983872 17627 leader_election.cc:290] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:50.974125 17629 raft_consensus.cc:493] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.984972 17629 raft_consensus.cc:3060] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.985693 16923 leader_election.cc:304] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:05:50.986464 17584 raft_consensus.cc:2804] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:50.986891 17584 raft_consensus.cc:493] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.987293 17584 raft_consensus.cc:3060] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.990831 17564 raft_consensus.cc:515] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:50.979939 16960 raft_consensus.cc:3060] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:50.991153 16923 leader_election.cc:336] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:50.993306 17564 leader_election.cc:290] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:50.994925 16923 leader_election.cc:304] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: e2b87453fe654318abccdd1512b949c4
I20260709 13:05:50.995797 17591 raft_consensus.cc:2804] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:50.996219 17591 raft_consensus.cc:493] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:50.996605 17591 raft_consensus.cc:3060] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:51.006371 17591 raft_consensus.cc:515] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:51.007834 17584 raft_consensus.cc:515] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:51.008576 17591 leader_election.cc:290] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:51.009969 16956 raft_consensus.cc:2468] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 2.
I20260709 13:05:51.010206 17584 leader_election.cc:290] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:51.010996 16960 raft_consensus.cc:2468] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 2.
W20260709 13:05:51.013808 16849 leader_election.cc:336] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
W20260709 13:05:51.015604 16849 leader_election.cc:336] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:51.019001 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27be53bca112445d8bdc640329310d67" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:51.024632 16960 raft_consensus.cc:3060] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:51.020328 17629 raft_consensus.cc:515] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
W20260709 13:05:51.022533 16849 leader_election.cc:336] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:51.030550 16775 leader_election.cc:304] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:05:51.031550 17629 leader_election.cc:290] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:51.031574 17567 raft_consensus.cc:2804] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:51.032393 17567 raft_consensus.cc:493] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:51.031665 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d83d4d3d894b81a85b945ef97d79de" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:51.032896 17567 raft_consensus.cc:3060] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:51.037729 16884 raft_consensus.cc:3060] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:51.051517 17567 raft_consensus.cc:515] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:51.060518 16775 leader_election.cc:304] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:05:51.065872 16775 leader_election.cc:304] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
W20260709 13:05:51.063561 16923 leader_election.cc:336] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:51.066118 16960 raft_consensus.cc:2468] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 2.
I20260709 13:05:51.064181 16884 raft_consensus.cc:2468] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 2.
W20260709 13:05:51.066730 16923 leader_election.cc:336] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:51.098693 16809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82d38de8d2c47b38ca8811691c68d8f" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:51.099419 16809 raft_consensus.cc:3060] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:51.100232 17567 leader_election.cc:290] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:51.100633 16923 leader_election.cc:304] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: e2b87453fe654318abccdd1512b949c4
I20260709 13:05:51.101293 16956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a5bbadcd9b48dba8fe272a18679ede" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:51.101240 16886 raft_consensus.cc:1275] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Refusing update from remote peer 0086163b152644feb75caeac6f145533: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:51.068156 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f48e0f9f83da446a877cd06e19768929" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:51.103528 16812 raft_consensus.cc:2468] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 1.
I20260709 13:05:51.074247 17627 raft_consensus.cc:2804] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:51.101482 17602 raft_consensus.cc:2804] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:51.105135 17602 raft_consensus.cc:697] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Becoming Leader. State: Replica: 0086163b152644feb75caeac6f145533, State: Running, Role: LEADER
I20260709 13:05:51.105418 17627 raft_consensus.cc:697] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 [term 2 LEADER]: Becoming Leader. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:05:51.105662 16956 raft_consensus.cc:3060] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:51.106143 17602 consensus_queue.cc:237] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:51.106564 17627 consensus_queue.cc:237] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
W20260709 13:05:51.092357 16775 leader_election.cc:336] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:51.108310 16775 leader_election.cc:304] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228; no voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4
I20260709 13:05:51.109025 17587 raft_consensus.cc:2749] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:51.083755 17621 raft_consensus.cc:2804] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:51.110606 17621 raft_consensus.cc:697] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [term 2 LEADER]: Becoming Leader. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
W20260709 13:05:51.113677 16849 leader_election.cc:336] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:51.115136 16849 leader_election.cc:304] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: e2b87453fe654318abccdd1512b949c4
I20260709 13:05:51.112588 17621 consensus_queue.cc:237] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:51.117512 17629 raft_consensus.cc:2804] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:51.118026 17629 raft_consensus.cc:697] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 LEADER]: Becoming Leader. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:51.122539 17629 consensus_queue.cc:237] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:51.130875 16923 leader_election.cc:304] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: e2b87453fe654318abccdd1512b949c4
I20260709 13:05:51.136162 17584 consensus_queue.cc:1048] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:05:51.141592 16923 consensus_peers.cc:597] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:51.162758 16809 raft_consensus.cc:2468] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 2.
I20260709 13:05:51.163211 16956 raft_consensus.cc:2468] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4e1b05bc80e43f7941ece113fb97ab0 in term 2.
I20260709 13:05:51.169535 16849 leader_election.cc:304] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: e2b87453fe654318abccdd1512b949c4
I20260709 13:05:51.174789 17258 raft_consensus.cc:2804] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:51.175248 17258 raft_consensus.cc:697] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 LEADER]: Becoming Leader. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:05:51.167619 17584 raft_consensus.cc:2804] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:51.182482 17258 consensus_queue.cc:237] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
W20260709 13:05:51.191145 16775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:40785, user_credentials={real_user=slave}} blocked reactor thread for 70934.4us
I20260709 13:05:51.177289 17584 raft_consensus.cc:697] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Becoming Leader. State: Replica: 0086163b152644feb75caeac6f145533, State: Running, Role: LEADER
I20260709 13:05:51.208642 17584 consensus_queue.cc:237] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:51.224213 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9ca489a4d2416092d6dcfae2498136" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:05:51.224916 16960 raft_consensus.cc:3060] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:51.225955 16923 consensus_peers.cc:597] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:51.224699 16633 catalog_manager.cc:5697] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 reported cstate change: term changed from 1 to 2, leader changed from e2b87453fe654318abccdd1512b949c4 (127.6.255.1) to 0086163b152644feb75caeac6f145533 (127.6.255.4). New cstate: current_term: 2 leader_uuid: "0086163b152644feb75caeac6f145533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:51.212567 16635 catalog_manager.cc:5697] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 reported cstate change: term changed from 1 to 2, leader changed from e2b87453fe654318abccdd1512b949c4 (127.6.255.1) to c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "c4b86aae3a8f4a9082e639144ec46228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:51.234942 16923 leader_election.cc:304] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: e2b87453fe654318abccdd1512b949c4
I20260709 13:05:51.235728 17602 raft_consensus.cc:2804] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:51.236122 17602 raft_consensus.cc:493] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:51.236486 17602 raft_consensus.cc:3060] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:51.249066 16960 raft_consensus.cc:2468] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 2.
I20260709 13:05:51.254453 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d6e27a9a96c4121b2abc71e0d7738be" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:51.255206 16884 raft_consensus.cc:2468] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 1.
I20260709 13:05:51.257351 16633 catalog_manager.cc:5697] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 reported cstate change: term changed from 1 to 2, leader changed from e2b87453fe654318abccdd1512b949c4 (127.6.255.1) to c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "c4b86aae3a8f4a9082e639144ec46228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:51.258877 16775 leader_election.cc:336] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
W20260709 13:05:51.259713 16775 leader_election.cc:336] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
W20260709 13:05:51.260821 16923 consensus_peers.cc:597] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:51.263473 16923 leader_election.cc:304] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: e2b87453fe654318abccdd1512b949c4
I20260709 13:05:51.244165 17602 raft_consensus.cc:515] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:51.266731 17591 raft_consensus.cc:2804] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:51.267366 17591 raft_consensus.cc:697] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Becoming Leader. State: Replica: 0086163b152644feb75caeac6f145533, State: Running, Role: LEADER
I20260709 13:05:51.268414 17591 consensus_queue.cc:237] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:51.272223 16635 catalog_manager.cc:5697] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 reported cstate change: term changed from 1 to 2, leader changed from e2b87453fe654318abccdd1512b949c4 (127.6.255.1) to d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3). New cstate: current_term: 2 leader_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:51.278695 17617 ts_tablet_manager.cc:548] Loading tablet metadata (26/45 complete)
I20260709 13:05:51.282682 17602 leader_election.cc:290] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:51.284544 16632 catalog_manager.cc:5697] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 reported cstate change: term changed from 1 to 2, leader changed from e2b87453fe654318abccdd1512b949c4 (127.6.255.1) to 0086163b152644feb75caeac6f145533 (127.6.255.4). New cstate: current_term: 2 leader_uuid: "0086163b152644feb75caeac6f145533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:51.295118 16775 leader_election.cc:336] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
W20260709 13:05:51.306458 16923 leader_election.cc:336] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:51.316478 16632 catalog_manager.cc:5697] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 reported cstate change: term changed from 1 to 2, leader changed from e2b87453fe654318abccdd1512b949c4 (127.6.255.1) to 0086163b152644feb75caeac6f145533 (127.6.255.4). New cstate: current_term: 2 leader_uuid: "0086163b152644feb75caeac6f145533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:51.321915 16775 leader_election.cc:304] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:05:51.322660 17627 raft_consensus.cc:2804] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:51.323174 17627 raft_consensus.cc:697] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 2 LEADER]: Becoming Leader. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:05:51.324093 17627 consensus_queue.cc:237] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:51.324445 16635 catalog_manager.cc:5697] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 reported cstate change: term changed from 1 to 2, leader changed from e2b87453fe654318abccdd1512b949c4 (127.6.255.1) to d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3). New cstate: current_term: 2 leader_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:51.332561 16879 raft_consensus.cc:1275] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Refusing update from remote peer 0086163b152644feb75caeac6f145533: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:51.334226 17602 consensus_queue.cc:1048] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:51.343566 16809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f48e0f9f83da446a877cd06e19768929" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:05:51.344226 16809 raft_consensus.cc:3060] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:51.354928 16775 leader_election.cc:336] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
W20260709 13:05:51.355700 16775 leader_election.cc:336] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:51.370628 16809 raft_consensus.cc:2468] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 2.
I20260709 13:05:51.379803 16775 leader_election.cc:304] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: e2b87453fe654318abccdd1512b949c4
I20260709 13:05:51.382525 17626 raft_consensus.cc:2804] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:51.382906 17626 raft_consensus.cc:493] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:05:51.383255 17626 raft_consensus.cc:3060] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:51.391362 17626 raft_consensus.cc:515] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
W20260709 13:05:51.412563 16923 consensus_peers.cc:597] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:51.393229 17626 leader_election.cc:290] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 election: Requested vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:51.425617 16923 leader_election.cc:304] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: e2b87453fe654318abccdd1512b949c4
I20260709 13:05:51.424340 16632 catalog_manager.cc:5697] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 reported cstate change: term changed from 1 to 2, leader changed from e2b87453fe654318abccdd1512b949c4 (127.6.255.1) to c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "c4b86aae3a8f4a9082e639144ec46228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:51.426425 17584 raft_consensus.cc:2804] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:51.426931 17584 raft_consensus.cc:697] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Becoming Leader. State: Replica: 0086163b152644feb75caeac6f145533, State: Running, Role: LEADER
I20260709 13:05:51.427748 17584 consensus_queue.cc:237] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:51.437296 16960 raft_consensus.cc:1275] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Refusing update from remote peer c4b86aae3a8f4a9082e639144ec46228: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:51.453984 16632 catalog_manager.cc:5697] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 reported cstate change: term changed from 1 to 2, leader changed from e2b87453fe654318abccdd1512b949c4 (127.6.255.1) to 0086163b152644feb75caeac6f145533 (127.6.255.4). New cstate: current_term: 2 leader_uuid: "0086163b152644feb75caeac6f145533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:51.463109 16881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d6e27a9a96c4121b2abc71e0d7738be" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:05:51.463785 16881 raft_consensus.cc:3060] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:51.471518 17626 consensus_queue.cc:1048] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:05:51.479995 16849 consensus_peers.cc:597] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:51.488176 16775 leader_election.cc:336] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:05:51.499236 16881 raft_consensus.cc:2468] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 2.
I20260709 13:05:51.506203  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:05:51.506623  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 1.261s	user 0.010s	sys 0.000s
I20260709 13:05:51.506973  7164 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:05:51.516372 17617 tablet_bootstrap.cc:492] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:51.522357 16960 raft_consensus.cc:1275] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Refusing update from remote peer d4e1b05bc80e43f7941ece113fb97ab0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:51.522478 16775 consensus_peers.cc:597] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:51.523551 16775 leader_election.cc:304] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0; no voters: e2b87453fe654318abccdd1512b949c4
I20260709 13:05:51.539363 17258 consensus_queue.cc:1048] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:51.524812 17567 raft_consensus.cc:2804] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:51.550287 17567 raft_consensus.cc:697] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [term 2 LEADER]: Becoming Leader. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:05:51.551294 17567 consensus_queue.cc:237] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:51.567898 16956 raft_consensus.cc:1275] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Refusing update from remote peer c4b86aae3a8f4a9082e639144ec46228: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:51.568652 16632 catalog_manager.cc:5697] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 reported cstate change: term changed from 1 to 2, leader changed from e2b87453fe654318abccdd1512b949c4 (127.6.255.1) to c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "c4b86aae3a8f4a9082e639144ec46228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:51.569736 17567 consensus_queue.cc:1048] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:51.570978 16956 raft_consensus.cc:1275] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Refusing update from remote peer c4b86aae3a8f4a9082e639144ec46228: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:51.572741 17567 consensus_queue.cc:1048] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:51.581197 17617 tablet_bootstrap.cc:492] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:51.582196 17617 tablet_bootstrap.cc:492] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:51.588764 17617 ts_tablet_manager.cc:1403] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.073s	user 0.013s	sys 0.011s
W20260709 13:05:51.588755 16849 consensus_peers.cc:597] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:51.597788 17617 raft_consensus.cc:359] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:51.598562 17617 raft_consensus.cc:740] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
W20260709 13:05:51.599041 16775 consensus_peers.cc:597] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:51.601411 17617 consensus_queue.cc:260] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:51.603981 16953 raft_consensus.cc:1275] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Refusing update from remote peer d4e1b05bc80e43f7941ece113fb97ab0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:51.609921 17617 ts_tablet_manager.cc:1434] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.003s
I20260709 13:05:51.611665 17617 tablet_bootstrap.cc:492] T 61f747edd7934aac93044ba370e34d17 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:51.613240 17258 consensus_queue.cc:1048] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:05:51.616288 16849 consensus_peers.cc:597] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:51.617688 16849 consensus_peers.cc:597] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:51.654300 16775 consensus_peers.cc:597] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:51.668074 16775 consensus_peers.cc:597] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:51.668634 16849 consensus_peers.cc:597] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:51.668908 16923 consensus_peers.cc:597] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:51.669472 16923 consensus_peers.cc:597] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:51.676744 16775 consensus_peers.cc:597] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:51.685341 16849 consensus_peers.cc:597] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:51.685997 16849 consensus_peers.cc:597] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:51.687589 16923 consensus_peers.cc:597] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:51.688030 16775 consensus_peers.cc:597] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:51.692207 17617 tablet_bootstrap.cc:492] T 61f747edd7934aac93044ba370e34d17 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:51.693235 17617 tablet_bootstrap.cc:492] T 61f747edd7934aac93044ba370e34d17 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:51.693964 17617 ts_tablet_manager.cc:1403] T 61f747edd7934aac93044ba370e34d17 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.082s	user 0.038s	sys 0.019s
I20260709 13:05:51.696583 17617 raft_consensus.cc:359] T 61f747edd7934aac93044ba370e34d17 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
W20260709 13:05:51.703085 16775 consensus_peers.cc:597] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:51.704003 16809 raft_consensus.cc:1275] T 87ca116ec6174decb610df3f8f565a6b P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Refusing update from remote peer 0086163b152644feb75caeac6f145533: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:51.697235 17617 raft_consensus.cc:740] T 61f747edd7934aac93044ba370e34d17 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:51.712720 17617 consensus_queue.cc:260] T 61f747edd7934aac93044ba370e34d17 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:51.714706 17617 ts_tablet_manager.cc:1434] T 61f747edd7934aac93044ba370e34d17 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.000s
I20260709 13:05:51.715505 17617 tablet_bootstrap.cc:492] T 0b8add4b17084a6189509807b7e663e5 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:51.717070 17602 consensus_queue.cc:1048] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:05:51.731804 16849 consensus_peers.cc:597] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:51.738261 16810 raft_consensus.cc:1275] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Refusing update from remote peer 0086163b152644feb75caeac6f145533: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:51.742094 17602 consensus_queue.cc:1048] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:51.762297 16959 raft_consensus.cc:1275] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Refusing update from remote peer d4e1b05bc80e43f7941ece113fb97ab0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:51.766824 17629 consensus_queue.cc:1048] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:51.775641 16959 raft_consensus.cc:1275] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Refusing update from remote peer c4b86aae3a8f4a9082e639144ec46228: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:51.770820 16849 consensus_peers.cc:597] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:51.786381 16923 consensus_peers.cc:597] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:51.809810 16879 raft_consensus.cc:1275] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Refusing update from remote peer 0086163b152644feb75caeac6f145533: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:51.811446 17584 consensus_queue.cc:1048] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:51.814487 17567 consensus_queue.cc:1048] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:51.817025 17617 tablet_bootstrap.cc:492] T 0b8add4b17084a6189509807b7e663e5 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:51.818122 17617 tablet_bootstrap.cc:492] T 0b8add4b17084a6189509807b7e663e5 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:51.818806 17617 ts_tablet_manager.cc:1403] T 0b8add4b17084a6189509807b7e663e5 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.104s	user 0.015s	sys 0.012s
I20260709 13:05:51.829921 17617 raft_consensus.cc:359] T 0b8add4b17084a6189509807b7e663e5 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:51.830700 17617 raft_consensus.cc:740] T 0b8add4b17084a6189509807b7e663e5 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:51.831553 17617 consensus_queue.cc:260] T 0b8add4b17084a6189509807b7e663e5 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:51.833833 17617 ts_tablet_manager.cc:1434] T 0b8add4b17084a6189509807b7e663e5 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:05:51.834674 17617 tablet_bootstrap.cc:492] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
W20260709 13:05:51.842211 16923 consensus_peers.cc:597] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:51.845386 16923 consensus_peers.cc:597] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:51.857323 16775 consensus_peers.cc:597] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:51.883719 16923 consensus_peers.cc:597] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:51.894601 17617 tablet_bootstrap.cc:492] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:51.895704 17617 tablet_bootstrap.cc:492] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:51.896449 17617 ts_tablet_manager.cc:1403] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.062s	user 0.013s	sys 0.004s
I20260709 13:05:51.899353 17617 raft_consensus.cc:359] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:51.900662 17617 raft_consensus.cc:740] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
W20260709 13:05:51.902382 16775 consensus_peers.cc:597] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:51.909309 17617 consensus_queue.cc:260] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:51.915128 17617 ts_tablet_manager.cc:1434] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 13:05:51.916031 17617 tablet_bootstrap.cc:492] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
W20260709 13:05:51.916834 17665 log.cc:927] Time spent T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 13:05:51.920488 16775 consensus_peers.cc:597] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:51.921041 16775 consensus_peers.cc:597] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:51.932305 16849 consensus_peers.cc:597] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:51.956423 16849 consensus_peers.cc:597] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:51.956938 16849 consensus_peers.cc:597] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:51.965291 16775 consensus_peers.cc:597] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:51.969141 17617 tablet_bootstrap.cc:492] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:51.970137 17617 tablet_bootstrap.cc:492] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:51.970786 17617 ts_tablet_manager.cc:1403] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.055s	user 0.019s	sys 0.008s
I20260709 13:05:51.973234 17617 raft_consensus.cc:359] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:51.973935 17617 raft_consensus.cc:740] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:51.976138 16810 raft_consensus.cc:1275] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Refusing update from remote peer 0086163b152644feb75caeac6f145533: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:51.977803 17584 consensus_queue.cc:1048] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:51.985689 17617 consensus_queue.cc:260] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:51.988005 17617 ts_tablet_manager.cc:1434] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:05:51.988780 17617 tablet_bootstrap.cc:492] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:52.022847 17617 tablet_bootstrap.cc:492] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:52.023798 17617 tablet_bootstrap.cc:492] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:52.024415 17617 ts_tablet_manager.cc:1403] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.036s	user 0.008s	sys 0.008s
I20260709 13:05:52.026773 17617 raft_consensus.cc:359] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
W20260709 13:05:52.027107 16923 consensus_peers.cc:597] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:52.027432 17617 raft_consensus.cc:740] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:52.032303 17617 consensus_queue.cc:260] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:52.038746 17617 ts_tablet_manager.cc:1434] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:05:52.039539 17617 tablet_bootstrap.cc:492] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
W20260709 13:05:52.127694 16849 consensus_peers.cc:597] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:52.128015 16923 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:40785, user_credentials={real_user=slave}} blocked reactor thread for 59614.2us
W20260709 13:05:52.128187 16849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:40785, user_credentials={real_user=slave}} blocked reactor thread for 60581.3us
I20260709 13:05:52.153324 17617 tablet_bootstrap.cc:492] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:52.154326 17617 tablet_bootstrap.cc:492] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:52.154948 17617 ts_tablet_manager.cc:1403] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.116s	user 0.015s	sys 0.009s
I20260709 13:05:52.157578 17617 raft_consensus.cc:359] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:52.158231 17617 raft_consensus.cc:740] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:52.160732 17617 consensus_queue.cc:260] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:52.164760 17617 ts_tablet_manager.cc:1434] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:05:52.168069 17617 tablet_bootstrap.cc:492] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
W20260709 13:05:52.171223 16923 consensus_peers.cc:597] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:52.183243  7164 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:05:52.183794  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.677s	user 0.313s	sys 0.029s
I20260709 13:05:52.190508 17617 tablet_bootstrap.cc:492] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:52.191951 17617 tablet_bootstrap.cc:492] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:52.192896 17617 ts_tablet_manager.cc:1403] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.003s
I20260709 13:05:52.195904 17617 raft_consensus.cc:359] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:52.196799 17617 raft_consensus.cc:740] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:52.197671 17617 consensus_queue.cc:260] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:52.199205 16879 raft_consensus.cc:1275] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Refusing update from remote peer c4b86aae3a8f4a9082e639144ec46228: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:52.199975 17617 ts_tablet_manager.cc:1434] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.000s
I20260709 13:05:52.200903 17617 tablet_bootstrap.cc:492] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
W20260709 13:05:52.221031 16775 consensus_peers.cc:597] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:52.231410 17673 consensus_queue.cc:1048] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:05:52.237658 16849 consensus_peers.cc:597] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:52.251688 16775 consensus_peers.cc:597] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:52.261145 16775 consensus_peers.cc:597] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:52.275326 17617 tablet_bootstrap.cc:492] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:52.276304 17617 tablet_bootstrap.cc:492] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:52.276940 17617 ts_tablet_manager.cc:1403] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.076s	user 0.035s	sys 0.029s
I20260709 13:05:52.280623 17617 raft_consensus.cc:359] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:52.281975 17617 raft_consensus.cc:740] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:52.282646 17617 consensus_queue.cc:260] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:52.284621 17617 ts_tablet_manager.cc:1434] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:05:52.285382 17617 tablet_bootstrap.cc:492] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
W20260709 13:05:52.286768 16923 consensus_peers.cc:597] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:52.296218 16775 consensus_peers.cc:597] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:52.326436 17617 tablet_bootstrap.cc:492] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:52.327411 17617 tablet_bootstrap.cc:492] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:52.328064 17617 ts_tablet_manager.cc:1403] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.043s	user 0.016s	sys 0.020s
I20260709 13:05:52.330616 17617 raft_consensus.cc:359] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:52.331332 17617 raft_consensus.cc:740] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:52.332154 17617 consensus_queue.cc:260] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:52.336073 17617 ts_tablet_manager.cc:1434] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:05:52.336953 17617 tablet_bootstrap.cc:492] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
W20260709 13:05:52.348002 16775 consensus_peers.cc:597] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:52.366156 17617 tablet_bootstrap.cc:492] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:52.367291 17617 tablet_bootstrap.cc:492] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:52.368033 17617 ts_tablet_manager.cc:1403] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.031s	user 0.018s	sys 0.009s
I20260709 13:05:52.372634 17617 raft_consensus.cc:359] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:52.373553 17617 raft_consensus.cc:740] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:52.374440 17617 consensus_queue.cc:260] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:52.376696 17617 ts_tablet_manager.cc:1434] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:05:52.378019 17617 tablet_bootstrap.cc:492] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:52.444535 17617 tablet_bootstrap.cc:492] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:52.445457 17617 tablet_bootstrap.cc:492] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:52.446117 17617 ts_tablet_manager.cc:1403] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.069s	user 0.022s	sys 0.000s
I20260709 13:05:52.448495 17617 raft_consensus.cc:359] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:52.449090 17617 raft_consensus.cc:740] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:52.449828 17617 consensus_queue.cc:260] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:52.452021 17617 ts_tablet_manager.cc:1434] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:05:52.452750 17617 tablet_bootstrap.cc:492] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:52.489302 17617 tablet_bootstrap.cc:492] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:52.490259 17617 tablet_bootstrap.cc:492] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:52.491147 17617 ts_tablet_manager.cc:1403] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.039s	user 0.014s	sys 0.000s
I20260709 13:05:52.493768 17617 raft_consensus.cc:359] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:52.494477 17617 raft_consensus.cc:740] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:52.495203 17617 consensus_queue.cc:260] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:52.497349 17617 ts_tablet_manager.cc:1434] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:05:52.510293 17617 tablet_bootstrap.cc:492] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:52.554191 17617 tablet_bootstrap.cc:492] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:52.555114 17617 tablet_bootstrap.cc:492] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:52.559922 17617 ts_tablet_manager.cc:1403] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.050s	user 0.021s	sys 0.000s
I20260709 13:05:52.566967 17617 raft_consensus.cc:359] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:52.567689 17617 raft_consensus.cc:740] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:52.571026 17617 consensus_queue.cc:260] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:52.577617 17617 ts_tablet_manager.cc:1434] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:05:52.578509 17617 tablet_bootstrap.cc:492] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:52.644987 17617 tablet_bootstrap.cc:492] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:52.647081 17617 tablet_bootstrap.cc:492] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:52.648677 17617 ts_tablet_manager.cc:1403] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.070s	user 0.027s	sys 0.005s
I20260709 13:05:52.652086 17617 raft_consensus.cc:359] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:52.652868 17617 raft_consensus.cc:740] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:52.653815 17617 consensus_queue.cc:260] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:52.656569 17617 ts_tablet_manager.cc:1434] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:05:52.657481 17617 tablet_bootstrap.cc:492] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:52.714099 17617 tablet_bootstrap.cc:492] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:52.715258 17617 tablet_bootstrap.cc:492] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:52.716091 17617 ts_tablet_manager.cc:1403] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.059s	user 0.019s	sys 0.007s
I20260709 13:05:52.721043 17617 raft_consensus.cc:359] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:52.721998 17617 raft_consensus.cc:740] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:52.725651 17617 consensus_queue.cc:260] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:52.729868 17617 ts_tablet_manager.cc:1434] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:05:52.730901 17617 tablet_bootstrap.cc:492] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:52.783123 17617 tablet_bootstrap.cc:492] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:52.783972 17617 tablet_bootstrap.cc:492] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:52.784552 17617 ts_tablet_manager.cc:1403] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.054s	user 0.041s	sys 0.002s
I20260709 13:05:52.787441 17617 raft_consensus.cc:359] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:52.788655 17617 raft_consensus.cc:740] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:52.791663 17617 consensus_queue.cc:260] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:52.800560 17657 raft_consensus.cc:493] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.801128 17657 raft_consensus.cc:515] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:52.809535 17657 leader_election.cc:290] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:05:52.840474 17617 ts_tablet_manager.cc:1434] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.056s	user 0.008s	sys 0.004s
I20260709 13:05:52.841413 17617 tablet_bootstrap.cc:492] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:52.884841 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9fdc34d643b4ab995042f719af511aa" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:52.916536 16879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9fdc34d643b4ab995042f719af511aa" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:52.918851 17611 leader_election.cc:304] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:52.920051 17720 raft_consensus.cc:2749] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:52.929747 17617 tablet_bootstrap.cc:492] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:52.930754 17617 tablet_bootstrap.cc:492] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:52.932003 17617 ts_tablet_manager.cc:1403] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.091s	user 0.013s	sys 0.010s
I20260709 13:05:52.934356 17617 raft_consensus.cc:359] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:52.934855 17617 raft_consensus.cc:740] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:52.935686 17617 consensus_queue.cc:260] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:52.937741 17617 ts_tablet_manager.cc:1434] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:05:52.938651 17617 tablet_bootstrap.cc:492] T ac3ad4fe0bc4435cb1c596ad215889a3 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:52.962698 17617 tablet_bootstrap.cc:492] T ac3ad4fe0bc4435cb1c596ad215889a3 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:52.963732 17617 tablet_bootstrap.cc:492] T ac3ad4fe0bc4435cb1c596ad215889a3 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:52.964371 17617 ts_tablet_manager.cc:1403] T ac3ad4fe0bc4435cb1c596ad215889a3 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.026s	user 0.018s	sys 0.004s
I20260709 13:05:52.967665 17617 raft_consensus.cc:359] T ac3ad4fe0bc4435cb1c596ad215889a3 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:52.968524 17617 raft_consensus.cc:740] T ac3ad4fe0bc4435cb1c596ad215889a3 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:52.970002 17617 consensus_queue.cc:260] T ac3ad4fe0bc4435cb1c596ad215889a3 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:52.972580 17617 ts_tablet_manager.cc:1434] T ac3ad4fe0bc4435cb1c596ad215889a3 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.001s
I20260709 13:05:52.973363 17617 tablet_bootstrap.cc:492] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:52.995160 17617 tablet_bootstrap.cc:492] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:52.996356 17617 tablet_bootstrap.cc:492] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:52.997203 17617 ts_tablet_manager.cc:1403] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.004s
I20260709 13:05:53.001518 17617 raft_consensus.cc:359] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:53.003392 17617 raft_consensus.cc:740] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:53.004034 17617 consensus_queue.cc:260] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:53.008845 17617 ts_tablet_manager.cc:1434] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:05:53.009840 17617 tablet_bootstrap.cc:492] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:53.046484 17617 tablet_bootstrap.cc:492] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:53.059319 17617 tablet_bootstrap.cc:492] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:53.065622 17617 ts_tablet_manager.cc:1403] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.056s	user 0.023s	sys 0.000s
I20260709 13:05:53.068373 17617 raft_consensus.cc:359] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:53.069125 17617 raft_consensus.cc:740] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:53.070123 17617 consensus_queue.cc:260] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:53.092411 17617 ts_tablet_manager.cc:1434] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 13:05:53.093988 17617 tablet_bootstrap.cc:492] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:53.124480 17617 tablet_bootstrap.cc:492] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:53.129046 17617 tablet_bootstrap.cc:492] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:53.129751 17617 ts_tablet_manager.cc:1403] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.036s	user 0.019s	sys 0.000s
I20260709 13:05:53.132473 17617 raft_consensus.cc:359] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:53.134404 17617 raft_consensus.cc:740] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:53.135185 17617 consensus_queue.cc:260] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:53.136086 17753 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.1:40785 every 8 connection(s)
I20260709 13:05:53.136940  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.1:40785
I20260709 13:05:53.137579 17617 ts_tablet_manager.cc:1434] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:05:53.138533 17617 tablet_bootstrap.cc:492] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:53.167739 17617 tablet_bootstrap.cc:492] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:53.168772 17617 tablet_bootstrap.cc:492] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:53.169453 17617 ts_tablet_manager.cc:1403] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.031s	user 0.017s	sys 0.008s
I20260709 13:05:53.172204 17617 raft_consensus.cc:359] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:53.172901 17617 raft_consensus.cc:740] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:53.175585 17617 consensus_queue.cc:260] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:53.178444 17617 ts_tablet_manager.cc:1434] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:05:53.179267 17617 tablet_bootstrap.cc:492] T c9043e0bc02745d187e9ca5a5c71274f P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:53.207929 17718 raft_consensus.cc:3060] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:53.236927 17755 heartbeater.cc:344] Connected to a master server at 127.6.255.62:33261
I20260709 13:05:53.237407 17755 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:53.238422 17755 heartbeater.cc:507] Master 127.6.255.62:33261 requested a full tablet report, sending...
I20260709 13:05:53.242323 17718 raft_consensus.cc:3060] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:53.248677 17617 tablet_bootstrap.cc:492] T c9043e0bc02745d187e9ca5a5c71274f P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:53.249805 17617 tablet_bootstrap.cc:492] T c9043e0bc02745d187e9ca5a5c71274f P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:53.250475 17617 ts_tablet_manager.cc:1403] T c9043e0bc02745d187e9ca5a5c71274f P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.071s	user 0.015s	sys 0.011s
I20260709 13:05:53.253206 17617 raft_consensus.cc:359] T c9043e0bc02745d187e9ca5a5c71274f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:53.253932 17617 raft_consensus.cc:740] T c9043e0bc02745d187e9ca5a5c71274f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:53.254709 17617 consensus_queue.cc:260] T c9043e0bc02745d187e9ca5a5c71274f P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:53.256851 17617 ts_tablet_manager.cc:1434] T c9043e0bc02745d187e9ca5a5c71274f P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.002s
I20260709 13:05:53.261593 17617 tablet_bootstrap.cc:492] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:53.336071 17617 tablet_bootstrap.cc:492] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:53.339658 16635 ts_manager.cc:194] Re-registered known tserver with Master: e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:05:53.353317 17715 raft_consensus.cc:3060] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:53.385515 17617 tablet_bootstrap.cc:492] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:53.386914 17617 ts_tablet_manager.cc:1403] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.126s	user 0.024s	sys 0.009s
I20260709 13:05:53.402578 17617 raft_consensus.cc:359] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:53.403419 17617 raft_consensus.cc:740] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:53.404563 17617 consensus_queue.cc:260] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:53.421837 17617 ts_tablet_manager.cc:1434] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.001s
I20260709 13:05:53.422801 17617 tablet_bootstrap.cc:492] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:53.453773 17617 tablet_bootstrap.cc:492] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:53.454865 17617 tablet_bootstrap.cc:492] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:53.458251 17715 raft_consensus.cc:3060] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:53.463438 16635 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36268
I20260709 13:05:53.469664 17755 heartbeater.cc:499] Master 127.6.255.62:33261 was elected leader, sending a full tablet report...
I20260709 13:05:53.471220 17718 raft_consensus.cc:3060] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:53.478214 17617 ts_tablet_manager.cc:1403] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.056s	user 0.027s	sys 0.000s
I20260709 13:05:53.481573 17617 raft_consensus.cc:359] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:53.482417 17617 raft_consensus.cc:740] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:53.484357 17617 consensus_queue.cc:260] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:53.487390 17617 ts_tablet_manager.cc:1434] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:05:53.490103 17617 tablet_bootstrap.cc:492] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:53.490237 17716 raft_consensus.cc:3060] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:53.498027 17714 raft_consensus.cc:3060] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:53.518810 17720 raft_consensus.cc:493] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.519487 17720 raft_consensus.cc:515] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:53.531869 17617 tablet_bootstrap.cc:492] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:53.532968 17617 tablet_bootstrap.cc:492] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:53.535552 17617 ts_tablet_manager.cc:1403] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.046s	user 0.023s	sys 0.003s
I20260709 13:05:53.535620 16881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3cecfcf8ae242e1b70f628f293da197" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:53.544082 17720 leader_election.cc:290] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:05:53.559583 16956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3cecfcf8ae242e1b70f628f293da197" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:53.574719 17617 raft_consensus.cc:359] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:53.575608 17611 leader_election.cc:304] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:05:53.578744 17720 raft_consensus.cc:2749] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:53.579792 17617 raft_consensus.cc:740] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:53.581288 17617 consensus_queue.cc:260] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:53.593964 17617 ts_tablet_manager.cc:1434] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.058s	user 0.008s	sys 0.000s
I20260709 13:05:53.595263 17617 tablet_bootstrap.cc:492] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:53.611819 17712 raft_consensus.cc:3060] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:53.617831 17720 raft_consensus.cc:493] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.618605 17720 raft_consensus.cc:515] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:53.625267 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f48e0f9f83da446a877cd06e19768929" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:05:53.627614 17720 leader_election.cc:290] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:05:53.632400 16956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f48e0f9f83da446a877cd06e19768929" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:05:53.642221 17611 leader_election.cc:304] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228
I20260709 13:05:53.643988 17720 raft_consensus.cc:2749] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:53.645618 17711 raft_consensus.cc:3060] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:53.738790 17720 raft_consensus.cc:493] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.762036 17720 raft_consensus.cc:515] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:53.819373 17617 tablet_bootstrap.cc:492] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:53.843868 17617 tablet_bootstrap.cc:492] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:53.844866 17617 ts_tablet_manager.cc:1403] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.250s	user 0.042s	sys 0.021s
W20260709 13:05:53.871500 16849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:40785, user_credentials={real_user=slave}} blocked reactor thread for 148675us
I20260709 13:05:53.879714 17617 raft_consensus.cc:359] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:53.880935 17617 raft_consensus.cc:740] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:53.893736 17617 consensus_queue.cc:260] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:53.910182 17617 ts_tablet_manager.cc:1434] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.065s	user 0.006s	sys 0.004s
W20260709 13:05:53.950515 17780 log.cc:927] Time spent T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:05:53.944610 17720 leader_election.cc:290] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
W20260709 13:05:53.953907 17768 log.cc:927] Time spent T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
W20260709 13:05:53.957593 17784 log.cc:927] Time spent T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:05:53.981175 17708 raft_consensus.cc:3060] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:53.981496 17617 tablet_bootstrap.cc:492] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
W20260709 13:05:54.012218 17781 log.cc:927] Time spent T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4: Append to log took a long time: real 0.169s	user 0.003s	sys 0.000s
I20260709 13:05:53.995515 17709 raft_consensus.cc:3060] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:54.048560 16923 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:40785, user_credentials={real_user=slave}} blocked reactor thread for 45942.5us
I20260709 13:05:54.062909 16954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20cdedb225464d5e812a56ed7a7d220f" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
W20260709 13:05:54.066119 16923 consensus_peers.cc:597] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:54.066840 17617 tablet_bootstrap.cc:492] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:54.073619 17617 tablet_bootstrap.cc:492] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:54.074353 17617 ts_tablet_manager.cc:1403] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.093s	user 0.013s	sys 0.003s
I20260709 13:05:54.096031 17713 raft_consensus.cc:3060] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:54.102489 17617 raft_consensus.cc:359] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:54.103289 17617 raft_consensus.cc:740] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:54.112569 17617 consensus_queue.cc:260] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:54.114619 17617 ts_tablet_manager.cc:1434] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.000s
I20260709 13:05:54.125644 16879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20cdedb225464d5e812a56ed7a7d220f" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:05:54.135123 17611 leader_election.cc:304] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0
W20260709 13:05:54.140326 16775 consensus_peers.cc:597] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:54.141757 17617 tablet_bootstrap.cc:492] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:54.148340 17770 raft_consensus.cc:2749] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:54.175716 16849 consensus_peers.cc:597] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:54.178001 16923 consensus_peers.cc:597] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:54.208070 16849 consensus_peers.cc:597] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:54.215996 16849 consensus_peers.cc:597] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:54.242411 17617 tablet_bootstrap.cc:492] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:54.243633 17617 tablet_bootstrap.cc:492] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:54.244361 17617 ts_tablet_manager.cc:1403] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.103s	user 0.021s	sys 0.000s
I20260709 13:05:54.247218 17617 raft_consensus.cc:359] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:54.247944 17617 raft_consensus.cc:740] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:54.248802 17617 consensus_queue.cc:260] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:54.250710 17617 ts_tablet_manager.cc:1434] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:05:54.251544 17617 tablet_bootstrap.cc:492] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
W20260709 13:05:54.295511 16849 consensus_peers.cc:597] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:54.304627 17617 tablet_bootstrap.cc:492] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:54.305788 17617 tablet_bootstrap.cc:492] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:54.306483 17617 ts_tablet_manager.cc:1403] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.055s	user 0.021s	sys 0.006s
I20260709 13:05:54.309463 17617 raft_consensus.cc:359] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:54.310212 17617 raft_consensus.cc:740] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:54.310940 17617 consensus_queue.cc:260] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:54.313303 17617 ts_tablet_manager.cc:1434] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.002s
I20260709 13:05:54.314190 17617 tablet_bootstrap.cc:492] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
W20260709 13:05:54.336154 16849 consensus_peers.cc:597] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:54.349550 17617 tablet_bootstrap.cc:492] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:54.350656 17617 tablet_bootstrap.cc:492] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:54.351450 17617 ts_tablet_manager.cc:1403] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.037s	user 0.026s	sys 0.004s
I20260709 13:05:54.354249 17617 raft_consensus.cc:359] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:54.354939 17617 raft_consensus.cc:740] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:54.355626 17617 consensus_queue.cc:260] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:54.357868 17617 ts_tablet_manager.cc:1434] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:05:54.358711 17617 tablet_bootstrap.cc:492] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:54.363377 17714 raft_consensus.cc:3060] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:54.380470 16775 consensus_peers.cc:597] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:54.384505 17617 tablet_bootstrap.cc:492] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:54.386180 17617 tablet_bootstrap.cc:492] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:54.386873 17617 ts_tablet_manager.cc:1403] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.012s
I20260709 13:05:54.389598 17617 raft_consensus.cc:359] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:54.390295 17617 raft_consensus.cc:740] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:54.390983 17617 consensus_queue.cc:260] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:54.393226 17617 ts_tablet_manager.cc:1434] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:05:54.397641 17617 tablet_bootstrap.cc:492] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:54.406800 17713 raft_consensus.cc:3060] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:54.424494 16775 consensus_peers.cc:597] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:54.493351 16849 consensus_peers.cc:597] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:54.509740 17617 tablet_bootstrap.cc:492] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:54.510881 17617 tablet_bootstrap.cc:492] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:54.511649 17617 ts_tablet_manager.cc:1403] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.114s	user 0.020s	sys 0.041s
I20260709 13:05:54.517778 17617 raft_consensus.cc:359] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:54.518544 17617 raft_consensus.cc:740] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:54.519368 17617 consensus_queue.cc:260] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:54.521785 17617 ts_tablet_manager.cc:1434] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 13:05:54.522648 17617 tablet_bootstrap.cc:492] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:54.585049 17617 tablet_bootstrap.cc:492] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:54.585947 17617 tablet_bootstrap.cc:492] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:54.586541 17617 ts_tablet_manager.cc:1403] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.064s	user 0.025s	sys 0.036s
I20260709 13:05:54.588587 17617 raft_consensus.cc:359] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:54.589154 17617 raft_consensus.cc:740] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:54.589787 17617 consensus_queue.cc:260] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:54.591903 17617 ts_tablet_manager.cc:1434] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:05:54.594952 17617 tablet_bootstrap.cc:492] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
W20260709 13:05:54.677716 16849 consensus_peers.cc:597] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:54.722251 17617 tablet_bootstrap.cc:492] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:54.723839 17617 tablet_bootstrap.cc:492] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:54.724574 17617 ts_tablet_manager.cc:1403] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.130s	user 0.042s	sys 0.040s
I20260709 13:05:54.727316 17617 raft_consensus.cc:359] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:54.727995 17617 raft_consensus.cc:740] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:54.728828 17617 consensus_queue.cc:260] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:54.742810 17617 ts_tablet_manager.cc:1434] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:05:54.762395 17617 tablet_bootstrap.cc:492] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:54.817624 17617 tablet_bootstrap.cc:492] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:54.818642 17617 tablet_bootstrap.cc:492] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:54.819367 17617 ts_tablet_manager.cc:1403] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.059s	user 0.011s	sys 0.007s
I20260709 13:05:54.822191 17617 raft_consensus.cc:359] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:05:54.822904 17617 raft_consensus.cc:740] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:54.823585 17617 consensus_queue.cc:260] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
W20260709 13:05:54.833045 16775 consensus_peers.cc:597] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:54.834504 17617 ts_tablet_manager.cc:1434] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:05:54.845721 17617 tablet_bootstrap.cc:492] T 612e472fbe674788986798c65836388f P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:54.878310 17617 tablet_bootstrap.cc:492] T 612e472fbe674788986798c65836388f P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:54.880209 17617 tablet_bootstrap.cc:492] T 612e472fbe674788986798c65836388f P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:54.884284 17617 ts_tablet_manager.cc:1403] T 612e472fbe674788986798c65836388f P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.039s	user 0.015s	sys 0.017s
I20260709 13:05:54.890036 17617 raft_consensus.cc:359] T 612e472fbe674788986798c65836388f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:54.890992 17617 raft_consensus.cc:740] T 612e472fbe674788986798c65836388f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:54.892010 17617 consensus_queue.cc:260] T 612e472fbe674788986798c65836388f P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:54.911301 17617 ts_tablet_manager.cc:1434] T 612e472fbe674788986798c65836388f P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 13:05:54.926326 17617 tablet_bootstrap.cc:492] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:54.970038 17617 tablet_bootstrap.cc:492] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:54.971158 17617 tablet_bootstrap.cc:492] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:54.971930 17617 ts_tablet_manager.cc:1403] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.046s	user 0.011s	sys 0.009s
I20260709 13:05:54.975080 17617 raft_consensus.cc:359] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:54.975801 17617 raft_consensus.cc:740] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:54.976670 17617 consensus_queue.cc:260] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:54.979025 17617 ts_tablet_manager.cc:1434] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:05:55.018004 17617 tablet_bootstrap.cc:492] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:55.175844 17617 tablet_bootstrap.cc:492] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:55.177096 17617 tablet_bootstrap.cc:492] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:55.178082 17617 ts_tablet_manager.cc:1403] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.160s	user 0.054s	sys 0.021s
I20260709 13:05:55.181227 17617 raft_consensus.cc:359] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:55.182138 17617 raft_consensus.cc:740] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:55.182994 17617 consensus_queue.cc:260] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:55.194875 17617 ts_tablet_manager.cc:1434] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:05:55.195961 17617 tablet_bootstrap.cc:492] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:55.251961 17617 tablet_bootstrap.cc:492] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:55.253232 17617 tablet_bootstrap.cc:492] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:55.254086 17617 ts_tablet_manager.cc:1403] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.058s	user 0.024s	sys 0.002s
I20260709 13:05:55.257467 17617 raft_consensus.cc:359] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:55.262567 17617 raft_consensus.cc:740] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:55.263571 17617 consensus_queue.cc:260] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:55.266906 17617 ts_tablet_manager.cc:1434] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:05:55.271682 17617 tablet_bootstrap.cc:492] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:55.313467 17617 tablet_bootstrap.cc:492] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:55.318287 17617 tablet_bootstrap.cc:492] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:55.319067 17617 ts_tablet_manager.cc:1403] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.048s	user 0.012s	sys 0.015s
I20260709 13:05:55.321689 17617 raft_consensus.cc:359] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:55.322441 17617 raft_consensus.cc:740] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:55.323151 17617 consensus_queue.cc:260] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:55.335086 17617 ts_tablet_manager.cc:1434] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:05:55.335985 17617 tablet_bootstrap.cc:492] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:55.369092 17716 raft_consensus.cc:3060] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:55.372282 17617 tablet_bootstrap.cc:492] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:55.377985 17617 tablet_bootstrap.cc:492] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:55.378975 17617 ts_tablet_manager.cc:1403] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.043s	user 0.016s	sys 0.007s
I20260709 13:05:55.381604 17617 raft_consensus.cc:359] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:55.382305 17617 raft_consensus.cc:740] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:55.382982 17617 consensus_queue.cc:260] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:05:55.385401 17617 ts_tablet_manager.cc:1434] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:05:55.386255 17617 tablet_bootstrap.cc:492] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:55.447893 17617 tablet_bootstrap.cc:492] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:55.448839 17617 tablet_bootstrap.cc:492] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:55.449559 17617 ts_tablet_manager.cc:1403] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.064s	user 0.020s	sys 0.011s
I20260709 13:05:55.452136 17617 raft_consensus.cc:359] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:55.452816 17617 raft_consensus.cc:740] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:55.453461 17617 consensus_queue.cc:260] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:55.457368 17617 ts_tablet_manager.cc:1434] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:05:55.465782 17617 tablet_bootstrap.cc:492] T 59de6d1228da41da938527632da08f4d P e2b87453fe654318abccdd1512b949c4: Bootstrap starting.
I20260709 13:05:55.488477 17617 tablet_bootstrap.cc:492] T 59de6d1228da41da938527632da08f4d P e2b87453fe654318abccdd1512b949c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:55.489538 17617 tablet_bootstrap.cc:492] T 59de6d1228da41da938527632da08f4d P e2b87453fe654318abccdd1512b949c4: Bootstrap complete.
I20260709 13:05:55.490239 17617 ts_tablet_manager.cc:1403] T 59de6d1228da41da938527632da08f4d P e2b87453fe654318abccdd1512b949c4: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.010s
I20260709 13:05:55.493163 17617 raft_consensus.cc:359] T 59de6d1228da41da938527632da08f4d P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:55.493887 17617 raft_consensus.cc:740] T 59de6d1228da41da938527632da08f4d P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Initialized, Role: FOLLOWER
I20260709 13:05:55.494542 17617 consensus_queue.cc:260] T 59de6d1228da41da938527632da08f4d P e2b87453fe654318abccdd1512b949c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:05:55.496798 17617 ts_tablet_manager.cc:1434] T 59de6d1228da41da938527632da08f4d P e2b87453fe654318abccdd1512b949c4: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
W20260709 13:06:02.326568 16925 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.000s	sys 0.014s
W20260709 13:06:02.327296 16925 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.000s	sys 0.015s
I20260709 13:06:02.428079 17856 raft_consensus.cc:493] T 4f95af7ac3134423b1a5956c8ae950df P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d4e1b05bc80e43f7941ece113fb97ab0)
I20260709 13:06:02.429059 17856 raft_consensus.cc:515] T 4f95af7ac3134423b1a5956c8ae950df P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:06:02.436479 17856 leader_election.cc:290] T 4f95af7ac3134423b1a5956c8ae950df P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:06:02.468602 17849 raft_consensus.cc:493] T e5c700237eb44cefbf7bc46c1daf08cd P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d4e1b05bc80e43f7941ece113fb97ab0)
I20260709 13:06:02.468811 17858 raft_consensus.cc:493] T f86d4f23f8cd4451a50b1b38155d958e P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d4e1b05bc80e43f7941ece113fb97ab0)
I20260709 13:06:02.469393 17849 raft_consensus.cc:515] T e5c700237eb44cefbf7bc46c1daf08cd P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:06:02.469714 17858 raft_consensus.cc:515] T f86d4f23f8cd4451a50b1b38155d958e P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:06:02.472265 17849 leader_election.cc:290] T e5c700237eb44cefbf7bc46c1daf08cd P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:06:02.473901 17858 leader_election.cc:290] T f86d4f23f8cd4451a50b1b38155d958e P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:06:02.474596 17858 raft_consensus.cc:493] T 9078054426a34acbb94be2b69213b6c1 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d4e1b05bc80e43f7941ece113fb97ab0)
I20260709 13:06:02.475976 17858 raft_consensus.cc:515] T 9078054426a34acbb94be2b69213b6c1 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:06:02.478061 17850 raft_consensus.cc:493] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c4b86aae3a8f4a9082e639144ec46228)
I20260709 13:06:02.478799 17850 raft_consensus.cc:515] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:06:02.485634 17857 raft_consensus.cc:493] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c4b86aae3a8f4a9082e639144ec46228)
I20260709 13:06:02.486418 17857 raft_consensus.cc:515] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:06:02.487583 17850 leader_election.cc:290] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:06:02.488976 17857 leader_election.cc:290] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:06:02.489274 17862 raft_consensus.cc:493] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c4b86aae3a8f4a9082e639144ec46228)
I20260709 13:06:02.489979 17862 raft_consensus.cc:515] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:06:02.492475 17858 leader_election.cc:290] T 9078054426a34acbb94be2b69213b6c1 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:06:02.493062 17840 raft_consensus.cc:493] T b9fdc34d643b4ab995042f719af511aa P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c4b86aae3a8f4a9082e639144ec46228)
I20260709 13:06:02.493721 17840 raft_consensus.cc:515] T b9fdc34d643b4ab995042f719af511aa P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:06:02.495926 17869 raft_consensus.cc:493] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0086163b152644feb75caeac6f145533)
I20260709 13:06:02.496748 17840 leader_election.cc:290] T b9fdc34d643b4ab995042f719af511aa P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:06:02.496766 17869 raft_consensus.cc:515] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:06:02.499260 17856 raft_consensus.cc:493] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d4e1b05bc80e43f7941ece113fb97ab0)
I20260709 13:06:02.501864 17856 raft_consensus.cc:515] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:06:02.504545 17856 leader_election.cc:290] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:06:02.493067 17862 leader_election.cc:290] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:06:02.522114 17869 leader_election.cc:290] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:06:02.523844 17873 raft_consensus.cc:493] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d4e1b05bc80e43f7941ece113fb97ab0)
I20260709 13:06:02.524559 17873 raft_consensus.cc:515] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:02.524958 17862 raft_consensus.cc:493] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0086163b152644feb75caeac6f145533)
I20260709 13:06:02.527202 17873 leader_election.cc:290] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:06:02.529139 16881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10bbce697a734600bb083d47ac877574" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:06:02.525698 17862 raft_consensus.cc:515] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:02.572676 17861 raft_consensus.cc:493] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c4b86aae3a8f4a9082e639144ec46228)
I20260709 13:06:02.573589 17861 raft_consensus.cc:515] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:06:02.574970 17862 leader_election.cc:290] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:06:02.576423 17861 leader_election.cc:290] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:06:02.522164 17872 raft_consensus.cc:493] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c4b86aae3a8f4a9082e639144ec46228)
I20260709 13:06:02.554008 17870 raft_consensus.cc:493] T 081841333d734eb1aa72ceb3f0ea2bf2 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0086163b152644feb75caeac6f145533)
I20260709 13:06:02.582433 17872 raft_consensus.cc:515] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:06:02.582657 17870 raft_consensus.cc:515] T 081841333d734eb1aa72ceb3f0ea2bf2 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:06:02.586018 17861 raft_consensus.cc:493] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c4b86aae3a8f4a9082e639144ec46228)
I20260709 13:06:02.586889 17861 raft_consensus.cc:515] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:06:02.589795 17861 leader_election.cc:290] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:06:02.592235 17872 leader_election.cc:290] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:06:02.600442 17872 raft_consensus.cc:493] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0086163b152644feb75caeac6f145533)
I20260709 13:06:02.565287 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9fdc34d643b4ab995042f719af511aa" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:06:02.594378 17873 raft_consensus.cc:493] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0086163b152644feb75caeac6f145533)
I20260709 13:06:02.554617 17876 raft_consensus.cc:493] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0086163b152644feb75caeac6f145533)
W20260709 13:06:02.600980 17749 debug-util.cc:398] Leaking SignalData structure 0x7b0800121b60 after lost signal to thread 7167
I20260709 13:06:02.609498 17872 raft_consensus.cc:515] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
W20260709 13:06:02.610910 17749 debug-util.cc:398] Leaking SignalData structure 0x7b0800365280 after lost signal to thread 16669
I20260709 13:06:02.612226 16881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d83d4d3d894b81a85b945ef97d79de" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
W20260709 13:06:02.612800 17749 debug-util.cc:398] Leaking SignalData structure 0x7b080021d580 after lost signal to thread 16836
I20260709 13:06:02.563795 16809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:06:02.606256 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:06:02.614598 17876 raft_consensus.cc:515] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:02.616889 17876 leader_election.cc:290] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
W20260709 13:06:02.617084 17749 debug-util.cc:398] Leaking SignalData structure 0x7b0800233fc0 after lost signal to thread 16910
I20260709 13:06:02.617741 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10bbce697a734600bb083d47ac877574" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
W20260709 13:06:02.619134 17749 debug-util.cc:398] Leaking SignalData structure 0x7b0800165e00 after lost signal to thread 16984
W20260709 13:06:02.621693 17749 debug-util.cc:398] Leaking SignalData structure 0x7b08003263a0 after lost signal to thread 17753
I20260709 13:06:02.628482 16959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ff2290a9c9451c9a786eacf7380d90" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:06:02.607792 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:06:02.586469 17870 leader_election.cc:290] T 081841333d734eb1aa72ceb3f0ea2bf2 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:06:02.606887 17874 raft_consensus.cc:493] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0086163b152644feb75caeac6f145533)
I20260709 13:06:02.623246 16879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7123efeaee9f4ec78a1c475eab16c4c1" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:06:02.631083 17874 raft_consensus.cc:515] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:02.631659 16953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f86d4f23f8cd4451a50b1b38155d958e" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:06:02.628597 17716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b6c54287fa4fdb928257d6faf572f8" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:06:02.625392 16956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7123efeaee9f4ec78a1c475eab16c4c1" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:06:02.625243 17861 raft_consensus.cc:493] T 7f4354282d844c7e83d760e4418512dc P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c4b86aae3a8f4a9082e639144ec46228)
I20260709 13:06:02.633154 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82d38de8d2c47b38ca8811691c68d8f" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:06:02.636996 17861 raft_consensus.cc:515] T 7f4354282d844c7e83d760e4418512dc P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:06:02.636430 17874 leader_election.cc:290] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:06:02.640281 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f95af7ac3134423b1a5956c8ae950df" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:06:02.641366 17861 leader_election.cc:290] T 7f4354282d844c7e83d760e4418512dc P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:06:02.642299 17714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9fdc34d643b4ab995042f719af511aa" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:06:02.644989 16881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c700237eb44cefbf7bc46c1daf08cd" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:06:02.645670 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f86d4f23f8cd4451a50b1b38155d958e" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:06:02.646041 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b5161020c944d28a4334dc1d572dac7" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:06:02.646412 17873 raft_consensus.cc:515] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:02.648471 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9078054426a34acbb94be2b69213b6c1" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:06:02.650084 17873 leader_election.cc:290] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:06:02.650665 17873 raft_consensus.cc:493] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c4b86aae3a8f4a9082e639144ec46228)
I20260709 13:06:02.650840 17611 leader_election.cc:304] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:06:02.651345 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ff2290a9c9451c9a786eacf7380d90" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:06:02.654115 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c700237eb44cefbf7bc46c1daf08cd" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:06:02.668617 17611 leader_election.cc:304] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:06:02.669505 17876 raft_consensus.cc:2749] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:02.669862 17611 leader_election.cc:304] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:06:02.669162 17874 raft_consensus.cc:2749] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:02.670670 17876 raft_consensus.cc:2749] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:02.671941 16881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "136f22096d4b41d89354924f7101d1c1" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:06:02.673004 16956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "136f22096d4b41d89354924f7101d1c1" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:06:02.673395 17874 raft_consensus.cc:493] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c4b86aae3a8f4a9082e639144ec46228)
I20260709 13:06:02.673622 16953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d83d4d3d894b81a85b945ef97d79de" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:06:02.674005 17874 raft_consensus.cc:515] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:06:02.674489 17611 leader_election.cc:304] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:06:02.675632 17876 raft_consensus.cc:2749] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:02.676113 17874 leader_election.cc:290] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:06:02.677497 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f239a1de317547b598aeedc11994c75f" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:06:02.678192 16881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f239a1de317547b598aeedc11994c75f" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" is_pre_election: true
I20260709 13:06:02.654028 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b5161020c944d28a4334dc1d572dac7" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:06:02.654062 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "081841333d734eb1aa72ceb3f0ea2bf2" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:06:02.655900 17872 leader_election.cc:290] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:06:02.651228 17873 raft_consensus.cc:515] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:06:02.681465 17873 leader_election.cc:290] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:06:02.661747 17875 raft_consensus.cc:493] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c4b86aae3a8f4a9082e639144ec46228)
I20260709 13:06:02.682044 17611 leader_election.cc:304] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:06:02.682169 17875 raft_consensus.cc:515] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:06:02.683776 17611 leader_election.cc:304] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228
I20260709 13:06:02.666471 17716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e96e3138004b3aa2f3e745f1ddfc04" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:06:02.685545 17872 raft_consensus.cc:2749] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:02.666901 17714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be51f207c3bc4276b220a03e2d9d4db4" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:06:02.686261 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c968ba000a4542fc9204dd9a474e364c" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:06:02.687006 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c968ba000a4542fc9204dd9a474e364c" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:06:02.687074 17872 raft_consensus.cc:2749] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:02.689054 17611 leader_election.cc:304] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:06:02.689349 17875 leader_election.cc:290] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:06:02.690095 17873 raft_consensus.cc:2749] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:02.692315 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec647c6ccd641b59808817d8dca2235" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:06:02.693078 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec647c6ccd641b59808817d8dca2235" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:06:02.694684 17611 leader_election.cc:304] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228
I20260709 13:06:02.695551 17875 raft_consensus.cc:2749] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:02.697041 17611 leader_election.cc:304] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2b87453fe654318abccdd1512b949c4; no voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228
I20260709 13:06:02.697850 17875 raft_consensus.cc:2749] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:02.702759 17714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f95af7ac3134423b1a5956c8ae950df" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:06:02.710850 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be51f207c3bc4276b220a03e2d9d4db4" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:06:02.712054 17714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9078054426a34acbb94be2b69213b6c1" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:06:02.712239 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e96e3138004b3aa2f3e745f1ddfc04" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:06:02.717780 17712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82d38de8d2c47b38ca8811691c68d8f" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:06:02.720422 16958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d83d4d3d894b81a85b945ef97d79de" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0086163b152644feb75caeac6f145533" is_pre_election: true
I20260709 13:06:02.721232 17714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "081841333d734eb1aa72ceb3f0ea2bf2" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:06:02.723136 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d6e27a9a96c4121b2abc71e0d7738be" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:06:02.724493 16923 leader_election.cc:304] T e5c700237eb44cefbf7bc46c1daf08cd P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:06:02.724730 16808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b9b28c4c5d4ddaa6d1ae05ce225b11" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:06:02.725306 17857 raft_consensus.cc:2749] T e5c700237eb44cefbf7bc46c1daf08cd P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:02.727118 17714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d83d4d3d894b81a85b945ef97d79de" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:06:02.727478 17712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d6e27a9a96c4121b2abc71e0d7738be" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:06:02.727710 17716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b9b28c4c5d4ddaa6d1ae05ce225b11" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:06:02.729748 17712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4354282d844c7e83d760e4418512dc" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e2b87453fe654318abccdd1512b949c4" is_pre_election: true
I20260709 13:06:02.730408 16808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f4354282d844c7e83d760e4418512dc" candidate_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4b86aae3a8f4a9082e639144ec46228" is_pre_election: true
I20260709 13:06:02.730587 16923 leader_election.cc:304] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:06:02.731822 17843 raft_consensus.cc:2749] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:02.733316 16923 leader_election.cc:304] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:06:02.733945 17843 raft_consensus.cc:2749] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:02.737141 16775 leader_election.cc:304] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228; no voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4
I20260709 13:06:02.738538 17858 raft_consensus.cc:2749] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:02.739127 16775 leader_election.cc:304] T 081841333d734eb1aa72ceb3f0ea2bf2 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228; no voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4
I20260709 13:06:02.739540 16849 leader_election.cc:304] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:06:02.740190 17858 raft_consensus.cc:2749] T 081841333d734eb1aa72ceb3f0ea2bf2 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:02.740748 16849 leader_election.cc:304] T b9fdc34d643b4ab995042f719af511aa P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:06:02.741011 17861 raft_consensus.cc:2749] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:02.741631 17847 raft_consensus.cc:2749] T b9fdc34d643b4ab995042f719af511aa P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:02.745170 16849 leader_election.cc:304] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:06:02.745874 17861 raft_consensus.cc:2749] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:02.746296 16849 leader_election.cc:304] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:06:02.747056 17861 raft_consensus.cc:2749] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:02.747470 16849 leader_election.cc:304] T 7f4354282d844c7e83d760e4418512dc P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:06:02.748618 16849 leader_election.cc:304] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0; no voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4
I20260709 13:06:02.749402 17855 raft_consensus.cc:2749] T 7f4354282d844c7e83d760e4418512dc P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:02.750079 17855 raft_consensus.cc:2749] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:02.749749 16775 leader_election.cc:304] T 4f95af7ac3134423b1a5956c8ae950df P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228; no voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4
I20260709 13:06:02.751044 17870 raft_consensus.cc:2749] T 4f95af7ac3134423b1a5956c8ae950df P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:02.753166 16775 leader_election.cc:304] T f86d4f23f8cd4451a50b1b38155d958e P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228; no voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:06:02.753924 17870 raft_consensus.cc:2749] T f86d4f23f8cd4451a50b1b38155d958e P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:02.755436 16775 leader_election.cc:304] T 9078054426a34acbb94be2b69213b6c1 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228; no voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4
I20260709 13:06:02.756229 17870 raft_consensus.cc:2749] T 9078054426a34acbb94be2b69213b6c1 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:02.756755 16775 leader_election.cc:304] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228; no voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0
I20260709 13:06:02.757537 17858 raft_consensus.cc:2749] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:04.107766  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:04.176369 16879 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58ff2290a9c9451c9a786eacf7380d90"
dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
mode: GRACEFUL
new_leader_uuid: "0086163b152644feb75caeac6f145533"
 from {username='slave'} at 127.0.0.1:60318
I20260709 13:06:04.177207 16879 raft_consensus.cc:606] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Received request to transfer leadership to TS 0086163b152644feb75caeac6f145533
I20260709 13:06:04.192349 16808 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c9ca489a4d2416092d6dcfae2498136"
dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
mode: GRACEFUL
new_leader_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:59734
I20260709 13:06:04.192948 16808 raft_consensus.cc:606] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 2 LEADER]: Received request to transfer leadership to TS e2b87453fe654318abccdd1512b949c4
I20260709 13:06:04.195739 16808 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86f5e278f54c419fac8c74ac63f80f74"
dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
mode: GRACEFUL
new_leader_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:59734
I20260709 13:06:04.196384 16808 raft_consensus.cc:606] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Received request to transfer leadership to TS e2b87453fe654318abccdd1512b949c4
I20260709 13:06:04.199131 16808 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b9fdc34d643b4ab995042f719af511aa"
dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
mode: GRACEFUL
new_leader_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:59734
I20260709 13:06:04.199725 16808 raft_consensus.cc:606] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Received request to transfer leadership to TS e2b87453fe654318abccdd1512b949c4
I20260709 13:06:04.202601 16808 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5b9b28c4c5d4ddaa6d1ae05ce225b11"
dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
mode: GRACEFUL
new_leader_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:59734
I20260709 13:06:04.203199 16808 raft_consensus.cc:606] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Received request to transfer leadership to TS e2b87453fe654318abccdd1512b949c4
I20260709 13:06:04.205880 16808 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f239a1de317547b598aeedc11994c75f"
dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
mode: GRACEFUL
new_leader_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:59734
I20260709 13:06:04.206496 16808 raft_consensus.cc:606] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Received request to transfer leadership to TS e2b87453fe654318abccdd1512b949c4
I20260709 13:06:04.207741  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:06:04.209028  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:04.365674 17878 raft_consensus.cc:993] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228: : Instructing follower e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:04.366338 17870 raft_consensus.cc:1081] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Signalling peer e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:04.367897 17714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b9fdc34d643b4ab995042f719af511aa"
dest_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:33780
I20260709 13:06:04.368462 17714 raft_consensus.cc:493] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:04.368798 17714 raft_consensus.cc:3060] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:04.380884 17714 raft_consensus.cc:515] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:06:04.383090 17714 leader_election.cc:290] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:06:04.384275 16808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9fdc34d643b4ab995042f719af511aa" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:06:04.385251 16808 raft_consensus.cc:3055] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:04.385627 16808 raft_consensus.cc:740] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:06:04.386171 16879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9fdc34d643b4ab995042f719af511aa" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:06:04.386811 16879 raft_consensus.cc:3060] T b9fdc34d643b4ab995042f719af511aa P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:04.386840 16808 consensus_queue.cc:260] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:06:04.388613 16808 raft_consensus.cc:3060] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:04.395669 16879 raft_consensus.cc:2468] T b9fdc34d643b4ab995042f719af511aa P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 2.
I20260709 13:06:04.397506 16808 raft_consensus.cc:2468] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 2.
I20260709 13:06:04.400056 17611 leader_election.cc:304] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4; no voters: 
W20260709 13:06:04.443460 17611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.255.3:45101, user_credentials={real_user=slave}} blocked reactor thread for 43515.9us
I20260709 13:06:04.443738 17882 raft_consensus.cc:2804] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:04.447059 17882 raft_consensus.cc:697] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [term 2 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:06:04.448207 17882 consensus_queue.cc:237] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:06:04.481331 17885 raft_consensus.cc:993] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228: : Instructing follower e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:04.483937 17855 raft_consensus.cc:993] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0: : Instructing follower 0086163b152644feb75caeac6f145533 to start an election
I20260709 13:06:04.484457 17855 raft_consensus.cc:1081] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Signalling peer 0086163b152644feb75caeac6f145533 to start an election
I20260709 13:06:04.483355 17885 raft_consensus.cc:1081] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Signalling peer e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:04.487550 17714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f239a1de317547b598aeedc11994c75f"
dest_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:33780
I20260709 13:06:04.488359 17714 raft_consensus.cc:493] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:04.488838 17714 raft_consensus.cc:3060] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:04.520730 16849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:45135, user_credentials={real_user=slave}} blocked reactor thread for 51904.6us
I20260709 13:06:04.524937 16953 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "58ff2290a9c9451c9a786eacf7380d90"
dest_uuid: "0086163b152644feb75caeac6f145533"
 from {username='slave'} at 127.0.0.1:58426
I20260709 13:06:04.525739 16953 raft_consensus.cc:493] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:04.526172 16953 raft_consensus.cc:3060] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:04.543522 16953 raft_consensus.cc:515] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:06:04.545991 17877 raft_consensus.cc:993] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228: : Instructing follower e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:04.546567 17877 raft_consensus.cc:1081] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Signalling peer e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:04.547055 16953 leader_election.cc:290] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:06:04.574155 16881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ff2290a9c9451c9a786eacf7380d90" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:06:04.574931 16881 raft_consensus.cc:3055] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:04.575271 16881 raft_consensus.cc:740] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:06:04.576082 16881 consensus_queue.cc:260] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:06:04.577412 16881 raft_consensus.cc:3060] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:04.512280 16631 catalog_manager.cc:5697] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 1 to 2, leader changed from c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2) to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 2 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:04.587428 17714 raft_consensus.cc:515] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:06:04.590306 16881 raft_consensus.cc:2468] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 2.
I20260709 13:06:04.590320 16808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f239a1de317547b598aeedc11994c75f" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:06:04.591255 16808 raft_consensus.cc:3055] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:04.591698 16808 raft_consensus.cc:740] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:06:04.593122 16923 leader_election.cc:304] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:06:04.592641 16808 consensus_queue.cc:260] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:06:04.594063 17888 raft_consensus.cc:2804] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:04.594272 16808 raft_consensus.cc:3060] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:04.595327 17888 raft_consensus.cc:697] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Becoming Leader. State: Replica: 0086163b152644feb75caeac6f145533, State: Running, Role: LEADER
I20260709 13:06:04.599342 17712 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "86f5e278f54c419fac8c74ac63f80f74"
dest_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:33780
I20260709 13:06:04.599978 17712 raft_consensus.cc:493] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:04.600381 17712 raft_consensus.cc:3060] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:04.596273 17888 consensus_queue.cc:237] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:06:04.608572 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ff2290a9c9451c9a786eacf7380d90" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:06:04.609241 16812 raft_consensus.cc:3060] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:04.616676 16808 raft_consensus.cc:2468] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 2.
I20260709 13:06:04.618083 17611 leader_election.cc:304] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:06:04.619194 17882 raft_consensus.cc:2804] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:04.619663 17882 raft_consensus.cc:697] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 2 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:06:04.617220 16812 raft_consensus.cc:2468] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 2.
I20260709 13:06:04.620560 17882 consensus_queue.cc:237] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:06:04.624017 17885 raft_consensus.cc:993] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228: : Instructing follower e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:04.624563 17885 raft_consensus.cc:1081] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Signalling peer e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:04.630501 17878 raft_consensus.cc:993] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228: : Instructing follower e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:04.631013 17878 raft_consensus.cc:1081] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 2 LEADER]: Signalling peer e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:04.643646 17714 leader_election.cc:290] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:06:04.646818 17713 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5b9b28c4c5d4ddaa6d1ae05ce225b11"
dest_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:33780
I20260709 13:06:04.647536 17713 raft_consensus.cc:493] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:04.647930 17713 raft_consensus.cc:3060] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:04.646449 17714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c9ca489a4d2416092d6dcfae2498136"
dest_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:33780
I20260709 13:06:04.654528 17714 raft_consensus.cc:493] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:04.646891 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f239a1de317547b598aeedc11994c75f" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:06:04.655215 16884 raft_consensus.cc:3060] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:04.655277 17713 raft_consensus.cc:515] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:06:04.658048 16808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b9b28c4c5d4ddaa6d1ae05ce225b11" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:06:04.658592 16808 raft_consensus.cc:3055] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:04.661392 16808 raft_consensus.cc:740] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:06:04.660022 17714 raft_consensus.cc:3060] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:04.662310 16808 consensus_queue.cc:260] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:06:04.663501 16808 raft_consensus.cc:3060] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:04.678939 17713 leader_election.cc:290] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:06:04.681597 16881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b9b28c4c5d4ddaa6d1ae05ce225b11" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:06:04.675732 17712 raft_consensus.cc:515] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:06:04.686349 16881 raft_consensus.cc:3060] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:04.685712 16884 raft_consensus.cc:2468] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 2.
I20260709 13:06:04.704134 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86f5e278f54c419fac8c74ac63f80f74" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:06:04.703819 16632 catalog_manager.cc:5697] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533 reported cstate change: term changed from 1 to 2, leader changed from d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3) to 0086163b152644feb75caeac6f145533 (127.6.255.4). New cstate: current_term: 2 leader_uuid: "0086163b152644feb75caeac6f145533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:04.704813 16960 raft_consensus.cc:3060] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:04.717849 16808 raft_consensus.cc:2468] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 2.
I20260709 13:06:04.719118 17611 leader_election.cc:304] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:06:04.720178 17882 raft_consensus.cc:2804] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:04.720714 17882 raft_consensus.cc:697] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [term 2 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:06:04.720890 16881 raft_consensus.cc:2468] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 2.
I20260709 13:06:04.721715 17882 consensus_queue.cc:237] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:06:04.724431 16960 raft_consensus.cc:2468] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 2.
I20260709 13:06:04.725757 17712 leader_election.cc:290] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:06:04.726377 17611 leader_election.cc:304] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:06:04.728449 17883 raft_consensus.cc:2804] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:06:04.730877 17891 log.cc:927] Time spent T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:06:04.734058 16808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86f5e278f54c419fac8c74ac63f80f74" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:06:04.734778 16808 raft_consensus.cc:3055] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:04.735114 16808 raft_consensus.cc:740] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:06:04.736824 17714 raft_consensus.cc:515] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:06:04.739271 17714 leader_election.cc:290] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 3 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:06:04.740207 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9ca489a4d2416092d6dcfae2498136" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:06:04.740470 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9ca489a4d2416092d6dcfae2498136" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:06:04.740829 16960 raft_consensus.cc:3060] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:04.741086 16812 raft_consensus.cc:3055] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:04.741482 16812 raft_consensus.cc:740] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:06:04.742297 16812 consensus_queue.cc:260] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:06:04.743533 16812 raft_consensus.cc:3060] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:04.749028 16960 raft_consensus.cc:2468] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 3.
I20260709 13:06:04.753413 17611 leader_election.cc:304] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:06:04.755043 16808 consensus_queue.cc:260] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:06:04.756634 16808 raft_consensus.cc:3060] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:04.762326 16812 raft_consensus.cc:2468] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 3.
I20260709 13:06:04.764974 17896 raft_consensus.cc:2804] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:04.766781 17896 raft_consensus.cc:697] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [term 3 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:06:04.767911 17896 consensus_queue.cc:237] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:06:04.769495 17883 raft_consensus.cc:697] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [term 2 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:06:04.770685 17883 consensus_queue.cc:237] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:06:04.778991 16808 raft_consensus.cc:2468] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 2.
I20260709 13:06:04.812314 16631 catalog_manager.cc:5697] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 1 to 2, leader changed from c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2) to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 2 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:04.813676 16631 catalog_manager.cc:5697] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 1 to 2, leader changed from c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2) to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 2 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:04.815014 16631 catalog_manager.cc:5697] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 1 to 2, leader changed from c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2) to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 2 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:04.816469 16631 catalog_manager.cc:5697] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 2 to 3, leader changed from c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2) to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 3 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:04.906920 16808 raft_consensus.cc:1275] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:04.908417 17896 consensus_queue.cc:1048] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:04.917959 16881 raft_consensus.cc:1275] T b9fdc34d643b4ab995042f719af511aa P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:04.919948 17883 consensus_queue.cc:1048] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:05.044636 16812 raft_consensus.cc:1275] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:05.045877 17894 consensus_queue.cc:1048] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:05.057021 16881 raft_consensus.cc:1275] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:05.058990 17883 consensus_queue.cc:1048] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:05.103266 16812 raft_consensus.cc:1275] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Refusing update from remote peer 0086163b152644feb75caeac6f145533: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:05.104791 17888 consensus_queue.cc:1048] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:05.155983 16881 raft_consensus.cc:1275] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Refusing update from remote peer 0086163b152644feb75caeac6f145533: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:05.157488 17857 consensus_queue.cc:1048] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:05.196563 16812 raft_consensus.cc:1275] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 3 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:05.197986 17894 consensus_queue.cc:1048] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:05.208988 16960 raft_consensus.cc:1275] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:05.210922 17894 consensus_queue.cc:1048] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:05.212594  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:05.307209 16812 raft_consensus.cc:1275] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:05.310950 17896 consensus_queue.cc:1048] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:05.315562 16808 raft_consensus.cc:1275] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:05.317438 17894 consensus_queue.cc:1048] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:05.335947 16881 raft_consensus.cc:1275] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:05.341364 17882 consensus_queue.cc:1048] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:05.394501 16960 raft_consensus.cc:1275] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:06:05.395666 16849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:45135, user_credentials={real_user=slave}} blocked reactor thread for 47242.9us
I20260709 13:06:05.404824 17894 consensus_queue.cc:1048] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:06:05.531836 16849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.4:37199, user_credentials={real_user=slave}} blocked reactor thread for 66062.8us
I20260709 13:06:05.632711  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:05.687860 16884 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3c19ee4785104276aec22ba772dbc6ac"
dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
mode: GRACEFUL
new_leader_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:60318
I20260709 13:06:05.688572 16884 raft_consensus.cc:606] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 LEADER]: Received request to transfer leadership to TS e2b87453fe654318abccdd1512b949c4
I20260709 13:06:05.712231 16956 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7123efeaee9f4ec78a1c475eab16c4c1"
dest_uuid: "0086163b152644feb75caeac6f145533"
mode: GRACEFUL
new_leader_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:49794
I20260709 13:06:05.712914 16956 raft_consensus.cc:606] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [term 1 LEADER]: Received request to transfer leadership to TS e2b87453fe654318abccdd1512b949c4
I20260709 13:06:05.716013 16886 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f37dd0138014a75bc36f33022362b3c"
dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
mode: GRACEFUL
new_leader_uuid: "0086163b152644feb75caeac6f145533"
 from {username='slave'} at 127.0.0.1:60318
I20260709 13:06:05.716645 16886 raft_consensus.cc:606] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Received request to transfer leadership to TS 0086163b152644feb75caeac6f145533
I20260709 13:06:05.719736 16810 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "83ec8ffbf1044525880e6b0d7931f47e"
dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
mode: GRACEFUL
new_leader_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:59734
I20260709 13:06:05.720278 16810 raft_consensus.cc:606] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Received request to transfer leadership to TS e2b87453fe654318abccdd1512b949c4
I20260709 13:06:05.733713 16886 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f3cecfcf8ae242e1b70f628f293da197"
dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
mode: GRACEFUL
new_leader_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:60318
I20260709 13:06:05.734342 16886 raft_consensus.cc:606] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 LEADER]: Received request to transfer leadership to TS e2b87453fe654318abccdd1512b949c4
I20260709 13:06:05.736088  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:06:05.738193  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:05.929240 17916 raft_consensus.cc:993] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0: : Instructing follower e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:05.929879 17917 raft_consensus.cc:1081] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 LEADER]: Signalling peer e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:05.931473 17714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f3cecfcf8ae242e1b70f628f293da197"
dest_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:33770
I20260709 13:06:05.931934 17714 raft_consensus.cc:493] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:05.932232 17714 raft_consensus.cc:3060] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:05.937048 17714 raft_consensus.cc:515] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:05.939917 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3cecfcf8ae242e1b70f628f293da197" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:06:05.940574 16886 raft_consensus.cc:3055] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:05.940901 16886 raft_consensus.cc:740] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:06:05.940243 17714 leader_election.cc:290] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 3 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:06:05.941773 16886 consensus_queue.cc:260] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:05.943233 16886 raft_consensus.cc:3060] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:05.944178 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3cecfcf8ae242e1b70f628f293da197" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:06:05.944828 16960 raft_consensus.cc:3060] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:05.954957 16886 raft_consensus.cc:2468] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 3.
I20260709 13:06:05.956296 17611 leader_election.cc:304] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:06:05.957378 17894 raft_consensus.cc:2804] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:05.957908 17894 raft_consensus.cc:697] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 3 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:06:05.958806 17894 consensus_queue.cc:237] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:05.959852 16960 raft_consensus.cc:2468] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 3.
I20260709 13:06:05.979784 16635 catalog_manager.cc:5697] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 2 to 3, leader changed from d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3) to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 3 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:06.072479 17885 raft_consensus.cc:993] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228: : Instructing follower e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:06.072981 17885 raft_consensus.cc:1081] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Signalling peer e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:06.076403 17713 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "83ec8ffbf1044525880e6b0d7931f47e"
dest_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:33780
I20260709 13:06:06.077044 17713 raft_consensus.cc:493] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:06.078702 17713 raft_consensus.cc:3060] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:06.081583 17843 raft_consensus.cc:993] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533: : Instructing follower e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:06.083798 17843 raft_consensus.cc:1081] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [term 1 LEADER]: Signalling peer e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:06.088065 17714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7123efeaee9f4ec78a1c475eab16c4c1"
dest_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:33794
I20260709 13:06:06.088745 17714 raft_consensus.cc:493] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:06.089182 17714 raft_consensus.cc:3060] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:06.097643 17713 raft_consensus.cc:515] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:06:06.098924 17714 raft_consensus.cc:515] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:06.102725 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7123efeaee9f4ec78a1c475eab16c4c1" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:06:06.103390 16886 raft_consensus.cc:3060] T 7123efeaee9f4ec78a1c475eab16c4c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:06.103474 17713 leader_election.cc:290] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:06:06.108531 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ec8ffbf1044525880e6b0d7931f47e" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:06:06.109148 16884 raft_consensus.cc:3060] T 83ec8ffbf1044525880e6b0d7931f47e P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:06.109627 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ec8ffbf1044525880e6b0d7931f47e" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:06:06.110272 16810 raft_consensus.cc:3055] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:06.110594 16810 raft_consensus.cc:740] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:06:06.111470 16810 consensus_queue.cc:260] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:06:06.112843 16810 raft_consensus.cc:3060] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:06.116015 17714 leader_election.cc:290] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:06:06.117379 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7123efeaee9f4ec78a1c475eab16c4c1" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:06:06.118129 16960 raft_consensus.cc:3055] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:06.118456 16960 raft_consensus.cc:740] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Running, Role: LEADER
I20260709 13:06:06.119347 16960 consensus_queue.cc:260] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:06.120595 16960 raft_consensus.cc:3060] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:06.120815 16886 raft_consensus.cc:2468] T 7123efeaee9f4ec78a1c475eab16c4c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 2.
I20260709 13:06:06.122354 17611 leader_election.cc:304] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:06:06.122572 16810 raft_consensus.cc:2468] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 2.
I20260709 13:06:06.123353 17894 raft_consensus.cc:2804] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:06.123929 17894 raft_consensus.cc:697] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [term 2 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:06:06.124915 17894 consensus_queue.cc:237] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:06.141477 17611 leader_election.cc:304] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:06:06.143294 17882 raft_consensus.cc:2804] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:06.147094 16884 raft_consensus.cc:2468] T 83ec8ffbf1044525880e6b0d7931f47e P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 2.
I20260709 13:06:06.148777 16960 raft_consensus.cc:2468] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 2.
I20260709 13:06:06.150064 17882 raft_consensus.cc:697] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [term 2 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:06:06.151041 17882 consensus_queue.cc:237] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:06:06.155858 16635 catalog_manager.cc:5697] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 1 to 2, leader changed from 0086163b152644feb75caeac6f145533 (127.6.255.4) to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 2 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:06.182235 16635 catalog_manager.cc:5697] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 1 to 2, leader changed from c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2) to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 2 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:06.234160 17917 raft_consensus.cc:993] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0: : Instructing follower 0086163b152644feb75caeac6f145533 to start an election
I20260709 13:06:06.235256 17916 raft_consensus.cc:1081] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Signalling peer 0086163b152644feb75caeac6f145533 to start an election
I20260709 13:06:06.237253 16960 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7f37dd0138014a75bc36f33022362b3c"
dest_uuid: "0086163b152644feb75caeac6f145533"
 from {username='slave'} at 127.0.0.1:58426
I20260709 13:06:06.237695 17916 raft_consensus.cc:993] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0: : Instructing follower e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:06.237957 16960 raft_consensus.cc:493] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:06.238202 17916 raft_consensus.cc:1081] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 LEADER]: Signalling peer e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:06.238423 16960 raft_consensus.cc:3060] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:06.241590 17713 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3c19ee4785104276aec22ba772dbc6ac"
dest_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:33770
I20260709 13:06:06.242091 17713 raft_consensus.cc:493] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:06.242398 17713 raft_consensus.cc:3060] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:06.246958 16960 raft_consensus.cc:515] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:06:06.247226 17713 raft_consensus.cc:515] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:06:06.249094 17713 leader_election.cc:290] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 4 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:06:06.250164 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c19ee4785104276aec22ba772dbc6ac" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:06:06.250524 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f37dd0138014a75bc36f33022362b3c" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:06:06.250815 16810 raft_consensus.cc:3060] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:06.251116 16812 raft_consensus.cc:3060] T 7f37dd0138014a75bc36f33022362b3c P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:06.250676 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c19ee4785104276aec22ba772dbc6ac" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:06:06.253320 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f37dd0138014a75bc36f33022362b3c" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:06:06.254042 16886 raft_consensus.cc:3055] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:06.254439 16886 raft_consensus.cc:740] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:06:06.253831 16884 raft_consensus.cc:3055] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:06.255477 16884 raft_consensus.cc:740] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:06:06.255384 16886 consensus_queue.cc:260] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:06:06.256438 16884 consensus_queue.cc:260] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:06:06.256875 16886 raft_consensus.cc:3060] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:06.257714 16812 raft_consensus.cc:2468] T 7f37dd0138014a75bc36f33022362b3c P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 2.
I20260709 13:06:06.258761 16884 raft_consensus.cc:3060] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:06.259152 16923 leader_election.cc:304] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:06:06.263792 17857 raft_consensus.cc:2804] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:06.263994 16810 raft_consensus.cc:2468] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 4.
I20260709 13:06:06.253252 16960 leader_election.cc:290] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:06:06.265332 17611 leader_election.cc:304] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:06:06.266312 17882 raft_consensus.cc:2804] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:06.266772 17882 raft_consensus.cc:697] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 4 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:06:06.267601 17882 consensus_queue.cc:237] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } }
I20260709 13:06:06.270299 16884 raft_consensus.cc:2468] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 4.
I20260709 13:06:06.271117 17857 raft_consensus.cc:697] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Becoming Leader. State: Replica: 0086163b152644feb75caeac6f145533, State: Running, Role: LEADER
I20260709 13:06:06.272015 17857 consensus_queue.cc:237] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:06:06.274996 16886 raft_consensus.cc:2468] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 2.
I20260709 13:06:06.291946 16635 catalog_manager.cc:5697] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 3 to 4, leader changed from d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3) to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 4 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:06.300841 16633 catalog_manager.cc:5697] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533 reported cstate change: term changed from 1 to 2, leader changed from d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3) to 0086163b152644feb75caeac6f145533 (127.6.255.4). New cstate: current_term: 2 leader_uuid: "0086163b152644feb75caeac6f145533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:06.383244 16960 raft_consensus.cc:1275] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:06.384754 17882 consensus_queue.cc:1048] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:06.412041 16886 raft_consensus.cc:1275] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:06.413592 17882 consensus_queue.cc:1048] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:06.554673 16956 raft_consensus.cc:1275] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:06.556144 17882 consensus_queue.cc:1048] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:06.559160 16886 raft_consensus.cc:1275] T 7123efeaee9f4ec78a1c475eab16c4c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:06.560480 17894 consensus_queue.cc:1048] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:06.660748 16812 raft_consensus.cc:1275] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228 [term 4 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:06.662155 17894 consensus_queue.cc:1048] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:06.678721 16810 raft_consensus.cc:1275] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:06.679317 16886 raft_consensus.cc:1275] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 4 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:06.682318 17882 consensus_queue.cc:1048] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:06.687105 17931 consensus_queue.cc:1048] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:06.688923 16886 raft_consensus.cc:1275] T 83ec8ffbf1044525880e6b0d7931f47e P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:06.692231 17931 consensus_queue.cc:1048] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:06.739244  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:06.742748 16812 raft_consensus.cc:1275] T 7f37dd0138014a75bc36f33022362b3c P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Refusing update from remote peer 0086163b152644feb75caeac6f145533: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:06.744148 17843 consensus_queue.cc:1048] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:06.818132 16886 raft_consensus.cc:1275] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Refusing update from remote peer 0086163b152644feb75caeac6f145533: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:06.822551 17948 consensus_queue.cc:1048] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:06.856168  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:06.914302 16879 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "744cc6469e3749e2aacccb45464e8198"
dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
mode: GRACEFUL
new_leader_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:60318
I20260709 13:06:06.914917 16879 raft_consensus.cc:606] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 LEADER]: Received request to transfer leadership to TS e2b87453fe654318abccdd1512b949c4
I20260709 13:06:06.918275 16879 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "919c5788cd97463a9cf06abb18843943"
dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
mode: GRACEFUL
new_leader_uuid: "c4b86aae3a8f4a9082e639144ec46228"
 from {username='slave'} at 127.0.0.1:60318
I20260709 13:06:06.918959 16879 raft_consensus.cc:606] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Received request to transfer leadership to TS c4b86aae3a8f4a9082e639144ec46228
I20260709 13:06:06.921797 16879 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e091d4e9a0044dd899e731769a94064c"
dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
mode: GRACEFUL
new_leader_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:60318
I20260709 13:06:06.922344 16879 raft_consensus.cc:606] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Received request to transfer leadership to TS e2b87453fe654318abccdd1512b949c4
I20260709 13:06:06.925290 16956 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f48e0f9f83da446a877cd06e19768929"
dest_uuid: "0086163b152644feb75caeac6f145533"
mode: GRACEFUL
new_leader_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:49794
I20260709 13:06:06.925884 16956 raft_consensus.cc:606] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Received request to transfer leadership to TS e2b87453fe654318abccdd1512b949c4
I20260709 13:06:06.928653 16879 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f86d4f23f8cd4451a50b1b38155d958e"
dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
mode: GRACEFUL
new_leader_uuid: "0086163b152644feb75caeac6f145533"
 from {username='slave'} at 127.0.0.1:60318
I20260709 13:06:06.929281 16879 raft_consensus.cc:606] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Received request to transfer leadership to TS 0086163b152644feb75caeac6f145533
I20260709 13:06:06.930635  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:06:06.931969  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:07.011837 17916 raft_consensus.cc:993] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0: : Instructing follower 0086163b152644feb75caeac6f145533 to start an election
I20260709 13:06:07.012441 17886 raft_consensus.cc:1081] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Signalling peer 0086163b152644feb75caeac6f145533 to start an election
I20260709 13:06:07.014145 16960 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f86d4f23f8cd4451a50b1b38155d958e"
dest_uuid: "0086163b152644feb75caeac6f145533"
 from {username='slave'} at 127.0.0.1:58426
I20260709 13:06:07.014662 16960 raft_consensus.cc:493] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:07.015013 16960 raft_consensus.cc:3060] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:07.021346 16960 raft_consensus.cc:515] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:06:07.023131 16960 leader_election.cc:290] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:06:07.024107 16812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f86d4f23f8cd4451a50b1b38155d958e" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:06:07.024824 16812 raft_consensus.cc:3060] T f86d4f23f8cd4451a50b1b38155d958e P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:07.024750 16879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f86d4f23f8cd4451a50b1b38155d958e" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:06:07.025830 16879 raft_consensus.cc:3055] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:07.026189 16879 raft_consensus.cc:740] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:06:07.026984 16879 consensus_queue.cc:260] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:06:07.028273 16879 raft_consensus.cc:3060] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:07.031944 16812 raft_consensus.cc:2468] T f86d4f23f8cd4451a50b1b38155d958e P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 2.
I20260709 13:06:07.033839 16923 leader_election.cc:304] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:06:07.036226 17948 raft_consensus.cc:2804] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:07.037081 17948 raft_consensus.cc:697] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Becoming Leader. State: Replica: 0086163b152644feb75caeac6f145533, State: Running, Role: LEADER
I20260709 13:06:07.037899 16879 raft_consensus.cc:2468] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 2.
I20260709 13:06:07.038113 17948 consensus_queue.cc:237] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:06:07.048861 16635 catalog_manager.cc:5697] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 reported cstate change: term changed from 1 to 2, leader changed from d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3) to 0086163b152644feb75caeac6f145533 (127.6.255.4). New cstate: current_term: 2 leader_uuid: "0086163b152644feb75caeac6f145533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:07.094094 17951 raft_consensus.cc:993] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0: : Instructing follower e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:07.094568 17886 raft_consensus.cc:1081] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 LEADER]: Signalling peer e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:07.096074 17712 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "744cc6469e3749e2aacccb45464e8198"
dest_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:33770
I20260709 13:06:07.096598 17712 raft_consensus.cc:493] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:07.097004 17712 raft_consensus.cc:3060] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.105484 17886 raft_consensus.cc:993] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0: : Instructing follower c4b86aae3a8f4a9082e639144ec46228 to start an election
I20260709 13:06:07.106084 17886 raft_consensus.cc:1081] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Signalling peer c4b86aae3a8f4a9082e639144ec46228 to start an election
I20260709 13:06:07.106707 17712 raft_consensus.cc:515] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:07.107741 16812 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "919c5788cd97463a9cf06abb18843943"
dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
 from {username='slave'} at 127.0.0.1:43134
I20260709 13:06:07.108278 16812 raft_consensus.cc:493] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:07.108630 16812 raft_consensus.cc:3060] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:07.108767 17712 leader_election.cc:290] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 3 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:06:07.109843 16879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744cc6469e3749e2aacccb45464e8198" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:06:07.110209 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744cc6469e3749e2aacccb45464e8198" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:06:07.110420 16879 raft_consensus.cc:3055] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:07.110776 16960 raft_consensus.cc:3060] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.110797 16879 raft_consensus.cc:740] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:06:07.112288 16879 consensus_queue.cc:260] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:07.113493 16879 raft_consensus.cc:3060] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.115473 16812 raft_consensus.cc:515] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:06:07.117491 16812 leader_election.cc:290] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:06:07.119884 16956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "919c5788cd97463a9cf06abb18843943" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:06:07.120047 16960 raft_consensus.cc:2468] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 3.
I20260709 13:06:07.120472 16956 raft_consensus.cc:3060] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:07.121294 17611 leader_election.cc:304] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:06:07.122104 17931 raft_consensus.cc:2804] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:07.122574 17931 raft_consensus.cc:697] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [term 3 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:06:07.123219 16879 raft_consensus.cc:2468] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 3.
I20260709 13:06:07.123565 17931 consensus_queue.cc:237] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:07.124117 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "919c5788cd97463a9cf06abb18843943" candidate_uuid: "c4b86aae3a8f4a9082e639144ec46228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:06:07.124724 16884 raft_consensus.cc:3055] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:07.125015 16884 raft_consensus.cc:740] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:06:07.125814 16884 consensus_queue.cc:260] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:06:07.127012 16884 raft_consensus.cc:3060] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:07.130347 16956 raft_consensus.cc:2468] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 2.
I20260709 13:06:07.133121 16775 leader_election.cc:304] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, c4b86aae3a8f4a9082e639144ec46228; no voters: 
I20260709 13:06:07.134521 16884 raft_consensus.cc:2468] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b86aae3a8f4a9082e639144ec46228 in term 2.
I20260709 13:06:07.135169 17922 raft_consensus.cc:2804] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:07.137943 17922 raft_consensus.cc:697] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228 [term 2 LEADER]: Becoming Leader. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:06:07.138871 17922 consensus_queue.cc:237] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:06:07.148444 16635 catalog_manager.cc:5697] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 2 to 3, leader changed from d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3) to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 3 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:07.163527 16635 catalog_manager.cc:5697] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228 reported cstate change: term changed from 1 to 2, leader changed from d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3) to c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "c4b86aae3a8f4a9082e639144ec46228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:07.211192 17951 raft_consensus.cc:993] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0: : Instructing follower e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:07.211623 17886 raft_consensus.cc:1081] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Signalling peer e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:07.212950 17712 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e091d4e9a0044dd899e731769a94064c"
dest_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:33770
I20260709 13:06:07.213402 17712 raft_consensus.cc:493] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:07.213759 17712 raft_consensus.cc:3060] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:07.219151 17712 raft_consensus.cc:515] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:07.220739 17712 leader_election.cc:290] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:06:07.221540 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e091d4e9a0044dd899e731769a94064c" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:06:07.221848 16956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e091d4e9a0044dd899e731769a94064c" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:06:07.222190 16884 raft_consensus.cc:3055] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:07.222486 16956 raft_consensus.cc:3060] T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:07.222533 16884 raft_consensus.cc:740] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4e1b05bc80e43f7941ece113fb97ab0, State: Running, Role: LEADER
I20260709 13:06:07.223361 16884 consensus_queue.cc:260] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:07.224566 16884 raft_consensus.cc:3060] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:07.229545 16956 raft_consensus.cc:2468] T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 2.
I20260709 13:06:07.230935 17611 leader_election.cc:304] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:06:07.231508 16884 raft_consensus.cc:2468] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 2.
I20260709 13:06:07.231842 17931 raft_consensus.cc:2804] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:07.232522 17931 raft_consensus.cc:697] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4 [term 2 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:06:07.233284 17931 consensus_queue.cc:237] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:07.243445 16635 catalog_manager.cc:5697] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 1 to 2, leader changed from d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3) to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 2 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:07.518121 17857 raft_consensus.cc:993] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533: : Instructing follower e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:07.518739 17888 raft_consensus.cc:1081] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Signalling peer e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:07.520525 17713 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f48e0f9f83da446a877cd06e19768929"
dest_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:33794
I20260709 13:06:07.521128 17713 raft_consensus.cc:493] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:07.521633 17713 raft_consensus.cc:3060] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.528736 17713 raft_consensus.cc:515] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:06:07.531011 17713 leader_election.cc:290] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 3 election: Requested vote from peers 0086163b152644feb75caeac6f145533 (127.6.255.4:37199), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:06:07.532047 16956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f48e0f9f83da446a877cd06e19768929" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:06:07.532755 16956 raft_consensus.cc:3055] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:07.533049 16956 raft_consensus.cc:740] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Running, Role: LEADER
I20260709 13:06:07.533214 16812 raft_consensus.cc:1275] T f86d4f23f8cd4451a50b1b38155d958e P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Refusing update from remote peer 0086163b152644feb75caeac6f145533: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:07.533530 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f48e0f9f83da446a877cd06e19768929" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:06:07.534142 16810 raft_consensus.cc:3060] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.533900 16956 consensus_queue.cc:260] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:06:07.535211 16956 raft_consensus.cc:3060] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.535483 17888 consensus_queue.cc:1048] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:07.542709 16956 raft_consensus.cc:2468] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 3.
I20260709 13:06:07.542709 16810 raft_consensus.cc:2468] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 3.
I20260709 13:06:07.545126 17611 leader_election.cc:304] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:06:07.546185 17894 raft_consensus.cc:2804] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:07.546758 17894 raft_consensus.cc:697] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 3 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:06:07.547806 17894 consensus_queue.cc:237] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:06:07.568102 16635 catalog_manager.cc:5697] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 2 to 3, leader changed from 0086163b152644feb75caeac6f145533 (127.6.255.4) to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 3 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:07.574184 16884 raft_consensus.cc:1275] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Refusing update from remote peer 0086163b152644feb75caeac6f145533: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:07.576851 17888 consensus_queue.cc:1048] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:07.582955 16956 raft_consensus.cc:1275] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:07.589267 17894 consensus_queue.cc:1048] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:07.642462 16884 raft_consensus.cc:1275] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:07.645648 17894 consensus_queue.cc:1048] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:07.728988 16884 raft_consensus.cc:1275] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:07.730363 17894 consensus_queue.cc:1048] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:07.739745 16956 raft_consensus.cc:1275] T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:07.741405 17931 consensus_queue.cc:1048] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:07.746052 16879 raft_consensus.cc:1275] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Refusing update from remote peer c4b86aae3a8f4a9082e639144ec46228: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:07.747696 17885 consensus_queue.cc:1048] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:07.752985 16960 raft_consensus.cc:1275] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Refusing update from remote peer c4b86aae3a8f4a9082e639144ec46228: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:07.754478 17959 consensus_queue.cc:1048] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:07.932886  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:07.994150  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:08.038581 16960 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b5161020c944d28a4334dc1d572dac7"
dest_uuid: "0086163b152644feb75caeac6f145533"
mode: GRACEFUL
new_leader_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:49794
I20260709 13:06:08.039245 16960 raft_consensus.cc:606] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [term 1 LEADER]: Received request to transfer leadership to TS e2b87453fe654318abccdd1512b949c4
I20260709 13:06:08.043779 16810 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e8e0f9919a9404c9001b6718f102d19"
dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
mode: GRACEFUL
new_leader_uuid: "0086163b152644feb75caeac6f145533"
 from {username='slave'} at 127.0.0.1:59734
I20260709 13:06:08.044234 16956 raft_consensus.cc:1275] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:08.044340 16810 raft_consensus.cc:606] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Received request to transfer leadership to TS 0086163b152644feb75caeac6f145533
I20260709 13:06:08.045953 17882 consensus_queue.cc:1048] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:08.047919  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:06:08.049701  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:08.051209 16810 raft_consensus.cc:1275] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228 [term 3 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:08.052395 17882 consensus_queue.cc:1048] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:08.128713 17959 raft_consensus.cc:993] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228: : Instructing follower 0086163b152644feb75caeac6f145533 to start an election
I20260709 13:06:08.129267 17885 raft_consensus.cc:1081] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Signalling peer 0086163b152644feb75caeac6f145533 to start an election
I20260709 13:06:08.130815 16960 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9e8e0f9919a9404c9001b6718f102d19"
dest_uuid: "0086163b152644feb75caeac6f145533"
 from {username='slave'} at 127.0.0.1:58440
I20260709 13:06:08.131332 16960 raft_consensus.cc:493] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:08.131650 16960 raft_consensus.cc:3060] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:08.136214 16960 raft_consensus.cc:515] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:06:08.137953 16960 leader_election.cc:290] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101)
I20260709 13:06:08.138628 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e8e0f9919a9404c9001b6718f102d19" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:06:08.138859 16884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e8e0f9919a9404c9001b6718f102d19" candidate_uuid: "0086163b152644feb75caeac6f145533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:06:08.139333 16810 raft_consensus.cc:3055] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:08.139479 16884 raft_consensus.cc:3060] T 9e8e0f9919a9404c9001b6718f102d19 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:08.139736 16810 raft_consensus.cc:740] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c4b86aae3a8f4a9082e639144ec46228, State: Running, Role: LEADER
I20260709 13:06:08.140626 16810 consensus_queue.cc:260] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:06:08.141633 16810 raft_consensus.cc:3060] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:08.145388 16884 raft_consensus.cc:2468] T 9e8e0f9919a9404c9001b6718f102d19 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 2.
I20260709 13:06:08.146618 16923 leader_election.cc:304] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0086163b152644feb75caeac6f145533, d4e1b05bc80e43f7941ece113fb97ab0; no voters: 
I20260709 13:06:08.147395 17949 raft_consensus.cc:2804] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:08.147397 16810 raft_consensus.cc:2468] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0086163b152644feb75caeac6f145533 in term 2.
I20260709 13:06:08.147979 17949 raft_consensus.cc:697] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Becoming Leader. State: Replica: 0086163b152644feb75caeac6f145533, State: Running, Role: LEADER
I20260709 13:06:08.148871 17949 consensus_queue.cc:237] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } }
I20260709 13:06:08.160040 16631 catalog_manager.cc:5697] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533 reported cstate change: term changed from 1 to 2, leader changed from c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2) to 0086163b152644feb75caeac6f145533 (127.6.255.4). New cstate: current_term: 2 leader_uuid: "0086163b152644feb75caeac6f145533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:08.431542 17949 raft_consensus.cc:993] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533: : Instructing follower e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:08.432077 17888 raft_consensus.cc:1081] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [term 1 LEADER]: Signalling peer e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:08.433737 17712 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5b5161020c944d28a4334dc1d572dac7"
dest_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:33794
I20260709 13:06:08.434301 17712 raft_consensus.cc:493] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:08.434597 17712 raft_consensus.cc:3060] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:08.439783 17712 raft_consensus.cc:515] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:08.441298 17712 leader_election.cc:290] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 election: Requested vote from peers c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:06:08.442073 16810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b5161020c944d28a4334dc1d572dac7" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4b86aae3a8f4a9082e639144ec46228"
I20260709 13:06:08.442329 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b5161020c944d28a4334dc1d572dac7" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:06:08.443018 16960 raft_consensus.cc:3055] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:08.443315 16960 raft_consensus.cc:740] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Running, Role: LEADER
I20260709 13:06:08.443745 16810 raft_consensus.cc:3060] T 5b5161020c944d28a4334dc1d572dac7 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:08.444300 16960 consensus_queue.cc:260] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:08.445734 16960 raft_consensus.cc:3060] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:08.451912 16810 raft_consensus.cc:2468] T 5b5161020c944d28a4334dc1d572dac7 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 2.
I20260709 13:06:08.453299 16960 raft_consensus.cc:2468] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 2.
I20260709 13:06:08.453259 17611 leader_election.cc:304] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:06:08.454485 17882 raft_consensus.cc:2804] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:08.454937 17882 raft_consensus.cc:697] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [term 2 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:06:08.455901 17882 consensus_queue.cc:237] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:08.466952 16631 catalog_manager.cc:5697] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 1 to 2, leader changed from 0086163b152644feb75caeac6f145533 (127.6.255.4) to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 2 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:08.743698 16884 raft_consensus.cc:1275] T 9e8e0f9919a9404c9001b6718f102d19 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Refusing update from remote peer 0086163b152644feb75caeac6f145533: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:08.745110 17949 consensus_queue.cc:1048] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:08.761319 16812 raft_consensus.cc:1275] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Refusing update from remote peer 0086163b152644feb75caeac6f145533: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:08.764777 17949 consensus_queue.cc:1048] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:08.911170 16956 raft_consensus.cc:1275] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:08.915028 17882 consensus_queue.cc:1048] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:08.932529 16808 raft_consensus.cc:1275] T 5b5161020c944d28a4334dc1d572dac7 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:08.937041 17894 consensus_queue.cc:1048] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:09.051194  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:09.135283  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:09.176957 16960 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0e295eefc8ff46d9a31a4c0898d16c63"
dest_uuid: "0086163b152644feb75caeac6f145533"
mode: GRACEFUL
new_leader_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:49794
I20260709 13:06:09.177786 16960 raft_consensus.cc:606] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Received request to transfer leadership to TS e2b87453fe654318abccdd1512b949c4
I20260709 13:06:09.180027  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:06:09.181704  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:09.595521 17949 raft_consensus.cc:993] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533: : Instructing follower e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:09.596195 17888 raft_consensus.cc:1081] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Signalling peer e2b87453fe654318abccdd1512b949c4 to start an election
I20260709 13:06:09.598970 17712 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0e295eefc8ff46d9a31a4c0898d16c63"
dest_uuid: "e2b87453fe654318abccdd1512b949c4"
 from {username='slave'} at 127.0.0.1:33794
I20260709 13:06:09.599557 17712 raft_consensus.cc:493] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:09.599982 17712 raft_consensus.cc:3060] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:09.606160 17712 raft_consensus.cc:515] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:09.608476 17712 leader_election.cc:290] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 3 election: Requested vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), 0086163b152644feb75caeac6f145533 (127.6.255.4:37199)
I20260709 13:06:09.609602 16879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e295eefc8ff46d9a31a4c0898d16c63" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d4e1b05bc80e43f7941ece113fb97ab0"
I20260709 13:06:09.610335 16879 raft_consensus.cc:3060] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:09.610404 16960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e295eefc8ff46d9a31a4c0898d16c63" candidate_uuid: "e2b87453fe654318abccdd1512b949c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0086163b152644feb75caeac6f145533"
I20260709 13:06:09.611095 16960 raft_consensus.cc:3055] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:09.611441 16960 raft_consensus.cc:740] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0086163b152644feb75caeac6f145533, State: Running, Role: LEADER
I20260709 13:06:09.612214 16960 consensus_queue.cc:260] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:09.613241 16960 raft_consensus.cc:3060] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:09.617923 16879 raft_consensus.cc:2468] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 3.
I20260709 13:06:09.620797 17611 leader_election.cc:304] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4; no voters: 
I20260709 13:06:09.621779 17882 raft_consensus.cc:2804] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:09.622274 17882 raft_consensus.cc:697] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [term 3 LEADER]: Becoming Leader. State: Replica: e2b87453fe654318abccdd1512b949c4, State: Running, Role: LEADER
I20260709 13:06:09.623392 17882 consensus_queue.cc:237] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:09.626670 16960 raft_consensus.cc:2468] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b87453fe654318abccdd1512b949c4 in term 3.
I20260709 13:06:09.634819 16631 catalog_manager.cc:5697] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 reported cstate change: term changed from 2 to 3, leader changed from 0086163b152644feb75caeac6f145533 (127.6.255.4) to e2b87453fe654318abccdd1512b949c4 (127.6.255.1). New cstate: current_term: 3 leader_uuid: "e2b87453fe654318abccdd1512b949c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:10.033105 16879 raft_consensus.cc:1275] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:10.034739 17894 consensus_queue.cc:1048] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:10.048895 16960 raft_consensus.cc:1275] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Refusing update from remote peer e2b87453fe654318abccdd1512b949c4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:10.050422 17894 consensus_queue.cc:1048] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:10.182689  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:10.230939  7164 tablet_server.cc:179] TabletServer@127.6.255.1:40785 shutting down...
W20260709 13:06:10.236100 16849 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 335 similar messages]
W20260709 13:06:10.240666 16849 consensus_peers.cc:597] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.267727  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:10.269075  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.270327  7164 raft_consensus.cc:2243] T 59de6d1228da41da938527632da08f4d P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.271018  7164 raft_consensus.cc:2272] T 59de6d1228da41da938527632da08f4d P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.274976  7164 tablet_replica.cc:333] stopping tablet replica
W20260709 13:06:10.275362 16923 consensus_peers.cc:597] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.275923  7164 raft_consensus.cc:2243] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.276468  7164 raft_consensus.cc:2272] T 9078054426a34acbb94be2b69213b6c1 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.280308  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.281090  7164 raft_consensus.cc:2243] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:10.282037  7164 raft_consensus.cc:2272] T 83ec8ffbf1044525880e6b0d7931f47e P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.284241  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.285013  7164 raft_consensus.cc:2243] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.285563  7164 raft_consensus.cc:2272] T 42e96e3138004b3aa2f3e745f1ddfc04 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.288025  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.288928  7164 raft_consensus.cc:2243] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:06:10.290107  7164 raft_consensus.cc:2272] T 3c19ee4785104276aec22ba772dbc6ac P e2b87453fe654318abccdd1512b949c4 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:10.293071 16923 consensus_peers.cc:597] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.293957  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.295188  7164 raft_consensus.cc:2243] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.295684  7164 raft_consensus.cc:2272] T 6ec647c6ccd641b59808817d8dca2235 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.299129  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.300843  7164 raft_consensus.cc:2243] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.301648  7164 raft_consensus.cc:2272] T caa2367ac34d43b5b1a27410b2055005 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.304939  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.306113  7164 raft_consensus.cc:2243] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:10.307410  7164 raft_consensus.cc:2272] T 7123efeaee9f4ec78a1c475eab16c4c1 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.310966  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.311614  7164 raft_consensus.cc:2243] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:10.312479  7164 raft_consensus.cc:2272] T 86f5e278f54c419fac8c74ac63f80f74 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.316195  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.316807  7164 raft_consensus.cc:2243] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.317682  7164 raft_consensus.cc:2272] T 3d6e27a9a96c4121b2abc71e0d7738be P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.319980  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.320634  7164 raft_consensus.cc:2243] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.321056  7164 raft_consensus.cc:2272] T 87ca116ec6174decb610df3f8f565a6b P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.324817  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.325771  7164 raft_consensus.cc:2243] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.326578  7164 raft_consensus.cc:2272] T 136f22096d4b41d89354924f7101d1c1 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.330309  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.331354  7164 raft_consensus.cc:2243] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:10.332566  7164 raft_consensus.cc:2272] T 6c9ca489a4d2416092d6dcfae2498136 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.337206  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.338085  7164 raft_consensus.cc:2243] T c9043e0bc02745d187e9ca5a5c71274f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.338706  7164 raft_consensus.cc:2272] T c9043e0bc02745d187e9ca5a5c71274f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.342149  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.343228  7164 raft_consensus.cc:2243] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.344000  7164 raft_consensus.cc:2272] T 213e9771c53e4498b315bb43fdc0aa9a P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:10.348146 16849 consensus_peers.cc:597] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.348405  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.349295  7164 raft_consensus.cc:2243] T 612e472fbe674788986798c65836388f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.350009  7164 raft_consensus.cc:2272] T 612e472fbe674788986798c65836388f P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.353200  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.354209  7164 raft_consensus.cc:2243] T 61f747edd7934aac93044ba370e34d17 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.354777  7164 raft_consensus.cc:2272] T 61f747edd7934aac93044ba370e34d17 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.358959  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.359927  7164 raft_consensus.cc:2243] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.360572  7164 raft_consensus.cc:2272] T 081841333d734eb1aa72ceb3f0ea2bf2 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:10.361809 16849 consensus_peers.cc:597] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.363643  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.364322  7164 raft_consensus.cc:2243] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.364718  7164 raft_consensus.cc:2272] T 7f4354282d844c7e83d760e4418512dc P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.368059  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.368844  7164 raft_consensus.cc:2243] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.369513  7164 raft_consensus.cc:2272] T c968ba000a4542fc9204dd9a474e364c P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:10.370571 16923 consensus_peers.cc:597] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.372378  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.373101  7164 raft_consensus.cc:2243] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.373644  7164 raft_consensus.cc:2272] T 20cdedb225464d5e812a56ed7a7d220f P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.376343  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.377478  7164 raft_consensus.cc:2243] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.378049  7164 raft_consensus.cc:2272] T 9592a31e7ffd407a959c0a3d96c0f8f9 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.381171  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.381897  7164 raft_consensus.cc:2243] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:10.382718  7164 raft_consensus.cc:2272] T 0e295eefc8ff46d9a31a4c0898d16c63 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.385773  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.386351  7164 raft_consensus.cc:2243] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.386744  7164 raft_consensus.cc:2272] T 9dd082201fe04ef89c842b6404fd522e P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.389256  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.389875  7164 raft_consensus.cc:2243] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.390282  7164 raft_consensus.cc:2272] T 6288d52217814c7393ff0ec66da3b099 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.392702  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.393644  7164 raft_consensus.cc:2243] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:10.394905  7164 raft_consensus.cc:2272] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:10.396850 16923 consensus_peers.cc:597] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.398454  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.399351  7164 raft_consensus.cc:2243] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 3 LEADER]: Raft consensus shutting down.
W20260709 13:06:10.399857 16775 consensus_peers.cc:597] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.400475  7164 raft_consensus.cc:2272] T f48e0f9f83da446a877cd06e19768929 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.402853  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.403461  7164 raft_consensus.cc:2243] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.403992  7164 raft_consensus.cc:2272] T 27be53bca112445d8bdc640329310d67 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.407084  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.407917  7164 raft_consensus.cc:2243] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:10.409180  7164 raft_consensus.cc:2272] T 744cc6469e3749e2aacccb45464e8198 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.412062  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.412602  7164 raft_consensus.cc:2243] T ac3ad4fe0bc4435cb1c596ad215889a3 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.413196  7164 raft_consensus.cc:2272] T ac3ad4fe0bc4435cb1c596ad215889a3 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.416420  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.417009  7164 raft_consensus.cc:2243] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.417622  7164 raft_consensus.cc:2272] T 4f95af7ac3134423b1a5956c8ae950df P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:10.419060 16849 consensus_peers.cc:597] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.420569  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.421310  7164 raft_consensus.cc:2243] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.421803  7164 raft_consensus.cc:2272] T b4d83d4d3d894b81a85b945ef97d79de P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.424208  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.424906  7164 raft_consensus.cc:2243] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.425530  7164 raft_consensus.cc:2272] T 15b6c54287fa4fdb928257d6faf572f8 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.427798  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.428354  7164 raft_consensus.cc:2243] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.429000  7164 raft_consensus.cc:2272] T 0a7231efacd1463c96b35c3d03edce55 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.431454  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.432166  7164 raft_consensus.cc:2243] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:10.433149  7164 raft_consensus.cc:2272] T e091d4e9a0044dd899e731769a94064c P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.436275  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.436864  7164 raft_consensus.cc:2243] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.437394  7164 raft_consensus.cc:2272] T d82d38de8d2c47b38ca8811691c68d8f P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.440627  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.441370  7164 raft_consensus.cc:2243] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.441912  7164 raft_consensus.cc:2272] T 37a5bbadcd9b48dba8fe272a18679ede P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.444329  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.444880  7164 raft_consensus.cc:2243] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:10.445719  7164 raft_consensus.cc:2272] T f3cecfcf8ae242e1b70f628f293da197 P e2b87453fe654318abccdd1512b949c4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.447779  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.448421  7164 raft_consensus.cc:2243] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.449016  7164 raft_consensus.cc:2272] T e1660f76b7074e2cb7542ed02dd179d9 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.452021  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.452769  7164 raft_consensus.cc:2243] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.453336  7164 raft_consensus.cc:2272] T be51f207c3bc4276b220a03e2d9d4db4 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:10.455003 16775 consensus_peers.cc:597] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.456274  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.457064  7164 raft_consensus.cc:2243] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:06:10.457687 16775 consensus_peers.cc:597] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.458225  7164 raft_consensus.cc:2272] T f239a1de317547b598aeedc11994c75f P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.461198  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.461972  7164 raft_consensus.cc:2243] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.462517  7164 raft_consensus.cc:2272] T 10bbce697a734600bb083d47ac877574 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.465646  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.466352  7164 raft_consensus.cc:2243] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:10.467355  7164 raft_consensus.cc:2272] T 5b5161020c944d28a4334dc1d572dac7 P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.470896  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.471673  7164 raft_consensus.cc:2243] T 0b8add4b17084a6189509807b7e663e5 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.472198  7164 raft_consensus.cc:2272] T 0b8add4b17084a6189509807b7e663e5 P e2b87453fe654318abccdd1512b949c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.475755  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:10.476507  7164 raft_consensus.cc:2243] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:10.477749  7164 raft_consensus.cc:2272] T b9fdc34d643b4ab995042f719af511aa P e2b87453fe654318abccdd1512b949c4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:10.489622 16923 consensus_peers.cc:597] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.503316 16775 consensus_peers.cc:597] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.513463 16775 consensus_peers.cc:597] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.516185 16849 consensus_peers.cc:597] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.531448 16849 consensus_peers.cc:597] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.540362 16775 consensus_peers.cc:597] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.541371 16923 consensus_peers.cc:597] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.551052 17612 connection.cc:466] server connection from 127.0.0.1:33770 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33770 (request call id 451) could send its response
W20260709 13:06:10.552055 16775 connection.cc:573] client connection to 127.6.255.1:40785 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:06:10.552861 16775 consensus_peers.cc:597] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.558162 16849 consensus_peers.cc:597] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.575547 16923 consensus_peers.cc:597] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.586056 16923 consensus_peers.cc:597] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.588182 16849 consensus_peers.cc:597] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.598132  7164 tablet_server.cc:196] TabletServer@127.6.255.1:40785 shutdown complete.
W20260709 13:06:10.627908 16775 consensus_peers.cc:597] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.648658 16849 consensus_peers.cc:597] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.696254  7164 tablet_server.cc:179] TabletServer@127.6.255.2:0 shutting down...
W20260709 13:06:10.716123 16923 consensus_peers.cc:597] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 -> Peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Couldn't send request to peer c4b86aae3a8f4a9082e639144ec46228. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.723058 16775 consensus_peers.cc:597] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.734618 16923 consensus_peers.cc:597] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.745846 16775 consensus_peers.cc:597] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.747323 16923 consensus_peers.cc:597] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533 -> Peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Couldn't send request to peer c4b86aae3a8f4a9082e639144ec46228. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.751191 16923 consensus_peers.cc:597] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.759225 16775 consensus_peers.cc:597] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.763458 16849 consensus_peers.cc:597] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Couldn't send request to peer c4b86aae3a8f4a9082e639144ec46228. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.764077 16849 consensus_peers.cc:597] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Couldn't send request to peer c4b86aae3a8f4a9082e639144ec46228. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.766842 16923 consensus_peers.cc:597] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 -> Peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Couldn't send request to peer c4b86aae3a8f4a9082e639144ec46228. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.767026 16849 consensus_peers.cc:597] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Couldn't send request to peer e2b87453fe654318abccdd1512b949c4. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.768077  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:10.769179  7164 tablet_replica.cc:333] T 87ca116ec6174decb610df3f8f565a6b P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.770267  7164 raft_consensus.cc:2243] T 87ca116ec6174decb610df3f8f565a6b P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.770952  7164 raft_consensus.cc:2272] T 87ca116ec6174decb610df3f8f565a6b P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.773872  7164 tablet_replica.cc:333] T f86d4f23f8cd4451a50b1b38155d958e P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.774569  7164 raft_consensus.cc:2243] T f86d4f23f8cd4451a50b1b38155d958e P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.775045  7164 raft_consensus.cc:2272] T f86d4f23f8cd4451a50b1b38155d958e P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.777143  7164 tablet_replica.cc:333] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.777818  7164 raft_consensus.cc:2243] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.778294  7164 raft_consensus.cc:2272] T 86f5e278f54c419fac8c74ac63f80f74 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.780365  7164 tablet_replica.cc:333] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.780948  7164 raft_consensus.cc:2243] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.781484  7164 raft_consensus.cc:2272] T f239a1de317547b598aeedc11994c75f P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.783960  7164 tablet_replica.cc:333] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.784675  7164 raft_consensus.cc:2243] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:10.785840  7164 raft_consensus.cc:2272] T be51f207c3bc4276b220a03e2d9d4db4 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.789546  7164 tablet_replica.cc:333] T 02d877efcf5049a89eacf23b58ba27b9 P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.790570  7164 raft_consensus.cc:2243] T 02d877efcf5049a89eacf23b58ba27b9 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.791134  7164 raft_consensus.cc:2272] T 02d877efcf5049a89eacf23b58ba27b9 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.794337  7164 tablet_replica.cc:333] T f9d47e8e7b784085b7079b1542dbc579 P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.796259  7164 raft_consensus.cc:2243] T f9d47e8e7b784085b7079b1542dbc579 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.796829  7164 raft_consensus.cc:2272] T f9d47e8e7b784085b7079b1542dbc579 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.800083  7164 tablet_replica.cc:333] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.800947  7164 raft_consensus.cc:2243] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:10.802141  7164 raft_consensus.cc:2272] T 6ec647c6ccd641b59808817d8dca2235 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.805177  7164 tablet_replica.cc:333] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.805994  7164 raft_consensus.cc:2243] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.806540  7164 raft_consensus.cc:2272] T f48e0f9f83da446a877cd06e19768929 P c4b86aae3a8f4a9082e639144ec46228 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.809211  7164 tablet_replica.cc:333] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.809924  7164 raft_consensus.cc:2243] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:10.810735  7164 raft_consensus.cc:2272] T 93629337d6b8477a9cd5895a157bf918 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.812939  7164 tablet_replica.cc:333] T 9078054426a34acbb94be2b69213b6c1 P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.813601  7164 raft_consensus.cc:2243] T 9078054426a34acbb94be2b69213b6c1 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.814045  7164 raft_consensus.cc:2272] T 9078054426a34acbb94be2b69213b6c1 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.816159  7164 tablet_replica.cc:333] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.816725  7164 raft_consensus.cc:2243] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.817200  7164 raft_consensus.cc:2272] T 3c19ee4785104276aec22ba772dbc6ac P c4b86aae3a8f4a9082e639144ec46228 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.819208  7164 tablet_replica.cc:333] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.819806  7164 raft_consensus.cc:2243] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:10.820626  7164 raft_consensus.cc:2272] T 919c5788cd97463a9cf06abb18843943 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.822702  7164 tablet_replica.cc:333] T 59de6d1228da41da938527632da08f4d P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.823298  7164 raft_consensus.cc:2243] T 59de6d1228da41da938527632da08f4d P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.823766  7164 raft_consensus.cc:2272] T 59de6d1228da41da938527632da08f4d P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.826056  7164 tablet_replica.cc:333] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.826651  7164 raft_consensus.cc:2243] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.827256  7164 raft_consensus.cc:2272] T 6c9ca489a4d2416092d6dcfae2498136 P c4b86aae3a8f4a9082e639144ec46228 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.830102  7164 tablet_replica.cc:333] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.830677  7164 raft_consensus.cc:2243] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:10.831461  7164 raft_consensus.cc:2272] T e81325b638b1493999310a726811b64b P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.833400  7164 tablet_replica.cc:333] T 213e9771c53e4498b315bb43fdc0aa9a P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.834220  7164 raft_consensus.cc:2243] T 213e9771c53e4498b315bb43fdc0aa9a P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.834780  7164 raft_consensus.cc:2272] T 213e9771c53e4498b315bb43fdc0aa9a P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.837519  7164 tablet_replica.cc:333] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.838300  7164 raft_consensus.cc:2243] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:10.839212  7164 raft_consensus.cc:2272] T c968ba000a4542fc9204dd9a474e364c P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.841928  7164 tablet_replica.cc:333] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.842698  7164 raft_consensus.cc:2243] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:10.843865  7164 raft_consensus.cc:2272] T 9592a31e7ffd407a959c0a3d96c0f8f9 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.846345  7164 tablet_replica.cc:333] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.847018  7164 raft_consensus.cc:2243] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:10.847826  7164 raft_consensus.cc:2272] T 3d6e27a9a96c4121b2abc71e0d7738be P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.850003  7164 tablet_replica.cc:333] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.850788  7164 raft_consensus.cc:2243] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.851372  7164 raft_consensus.cc:2272] T 9e8e0f9919a9404c9001b6718f102d19 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.853693  7164 tablet_replica.cc:333] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.854274  7164 raft_consensus.cc:2243] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:10.855082  7164 raft_consensus.cc:2272] T e1660f76b7074e2cb7542ed02dd179d9 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.857147  7164 tablet_replica.cc:333] T 0a7231efacd1463c96b35c3d03edce55 P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.857787  7164 raft_consensus.cc:2243] T 0a7231efacd1463c96b35c3d03edce55 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.858443  7164 raft_consensus.cc:2272] T 0a7231efacd1463c96b35c3d03edce55 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.861747  7164 tablet_replica.cc:333] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.862624  7164 raft_consensus.cc:2243] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:06:10.862881 16923 consensus_peers.cc:597] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533 -> Peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Couldn't send request to peer c4b86aae3a8f4a9082e639144ec46228. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.864068  7164 raft_consensus.cc:2272] T 2593417c1fe4457c87cb2a01cf1f65a3 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.867394  7164 tablet_replica.cc:333] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.868250  7164 raft_consensus.cc:2243] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 3 LEADER]: Raft consensus shutting down.
W20260709 13:06:10.869485 16849 consensus_peers.cc:597] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Couldn't send request to peer c4b86aae3a8f4a9082e639144ec46228. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.869513  7164 raft_consensus.cc:2272] T 15b6c54287fa4fdb928257d6faf572f8 P c4b86aae3a8f4a9082e639144ec46228 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.874322  7164 tablet_replica.cc:333] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.875339  7164 raft_consensus.cc:2243] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.876092  7164 raft_consensus.cc:2272] T d82d38de8d2c47b38ca8811691c68d8f P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:10.879118 16849 consensus_peers.cc:597] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Couldn't send request to peer c4b86aae3a8f4a9082e639144ec46228. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.880152  7164 tablet_replica.cc:333] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.881410  7164 raft_consensus.cc:2243] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.882076  7164 raft_consensus.cc:2272] T 83ec8ffbf1044525880e6b0d7931f47e P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:10.882352 16849 consensus_peers.cc:597] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Couldn't send request to peer c4b86aae3a8f4a9082e639144ec46228. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.885421  7164 tablet_replica.cc:333] T 535cac5e21bf41db9c037f226ebcd94e P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.886116  7164 raft_consensus.cc:2243] T 535cac5e21bf41db9c037f226ebcd94e P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.886575  7164 raft_consensus.cc:2272] T 535cac5e21bf41db9c037f226ebcd94e P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.888768  7164 tablet_replica.cc:333] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.889806  7164 raft_consensus.cc:2243] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.890288  7164 raft_consensus.cc:2272] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.892552  7164 tablet_replica.cc:333] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.893138  7164 raft_consensus.cc:2243] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:10.894093  7164 raft_consensus.cc:2272] T 9dd082201fe04ef89c842b6404fd522e P c4b86aae3a8f4a9082e639144ec46228 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.896445  7164 tablet_replica.cc:333] T 6288d52217814c7393ff0ec66da3b099 P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.896940  7164 raft_consensus.cc:2243] T 6288d52217814c7393ff0ec66da3b099 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.897337  7164 raft_consensus.cc:2272] T 6288d52217814c7393ff0ec66da3b099 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.899328  7164 tablet_replica.cc:333] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.899843  7164 raft_consensus.cc:2243] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:10.900746  7164 raft_consensus.cc:2272] T 42e96e3138004b3aa2f3e745f1ddfc04 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.903031  7164 tablet_replica.cc:333] T 3850091ca95e47b6a1b52e600a452b0b P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.903650  7164 raft_consensus.cc:2243] T 3850091ca95e47b6a1b52e600a452b0b P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.904122  7164 raft_consensus.cc:2272] T 3850091ca95e47b6a1b52e600a452b0b P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.906351  7164 tablet_replica.cc:333] T 7f37dd0138014a75bc36f33022362b3c P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.907413  7164 raft_consensus.cc:2243] T 7f37dd0138014a75bc36f33022362b3c P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.907994  7164 raft_consensus.cc:2272] T 7f37dd0138014a75bc36f33022362b3c P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.910784  7164 tablet_replica.cc:333] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
W20260709 13:06:10.911437 16923 consensus_peers.cc:597] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 -> Peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Couldn't send request to peer c4b86aae3a8f4a9082e639144ec46228. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.911661  7164 raft_consensus.cc:2243] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:10.912737  7164 raft_consensus.cc:2272] T 7f4354282d844c7e83d760e4418512dc P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.915238  7164 tablet_replica.cc:333] T 081841333d734eb1aa72ceb3f0ea2bf2 P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.915863  7164 raft_consensus.cc:2243] T 081841333d734eb1aa72ceb3f0ea2bf2 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.916339  7164 raft_consensus.cc:2272] T 081841333d734eb1aa72ceb3f0ea2bf2 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.919163  7164 tablet_replica.cc:333] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.919849  7164 raft_consensus.cc:2243] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.920765  7164 raft_consensus.cc:2272] T 58ff2290a9c9451c9a786eacf7380d90 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.923019  7164 tablet_replica.cc:333] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.923545  7164 raft_consensus.cc:2243] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.923982  7164 raft_consensus.cc:2272] T b9fdc34d643b4ab995042f719af511aa P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.926560  7164 tablet_replica.cc:333] T e5c700237eb44cefbf7bc46c1daf08cd P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.927194  7164 raft_consensus.cc:2243] T e5c700237eb44cefbf7bc46c1daf08cd P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.927635  7164 raft_consensus.cc:2272] T e5c700237eb44cefbf7bc46c1daf08cd P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.932227  7164 tablet_replica.cc:333] T caa2367ac34d43b5b1a27410b2055005 P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.933302  7164 raft_consensus.cc:2243] T caa2367ac34d43b5b1a27410b2055005 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.934419  7164 raft_consensus.cc:2272] T caa2367ac34d43b5b1a27410b2055005 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.938494  7164 tablet_replica.cc:333] T 5b5161020c944d28a4334dc1d572dac7 P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.939312  7164 raft_consensus.cc:2243] T 5b5161020c944d28a4334dc1d572dac7 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.939796  7164 raft_consensus.cc:2272] T 5b5161020c944d28a4334dc1d572dac7 P c4b86aae3a8f4a9082e639144ec46228 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.942533  7164 tablet_replica.cc:333] T 5857fad96a1045d7b85190e64da77bce P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.943253  7164 raft_consensus.cc:2243] T 5857fad96a1045d7b85190e64da77bce P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.943956  7164 raft_consensus.cc:2272] T 5857fad96a1045d7b85190e64da77bce P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:10.944721 16849 consensus_peers.cc:597] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Couldn't send request to peer c4b86aae3a8f4a9082e639144ec46228. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.947539  7164 tablet_replica.cc:333] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.948632  7164 raft_consensus.cc:2243] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:10.950138  7164 raft_consensus.cc:2272] T 0b8add4b17084a6189509807b7e663e5 P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:10.954077  7164 tablet_replica.cc:333] T 4f95af7ac3134423b1a5956c8ae950df P c4b86aae3a8f4a9082e639144ec46228: stopping tablet replica
I20260709 13:06:10.954815  7164 raft_consensus.cc:2243] T 4f95af7ac3134423b1a5956c8ae950df P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:10.955718  7164 raft_consensus.cc:2272] T 4f95af7ac3134423b1a5956c8ae950df P c4b86aae3a8f4a9082e639144ec46228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.058934  7164 tablet_server.cc:196] TabletServer@127.6.255.2:0 shutdown complete.
W20260709 13:06:11.062705 16923 consensus_peers.cc:597] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 -> Peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Couldn't send request to peer c4b86aae3a8f4a9082e639144ec46228. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:45135: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.081674 16849 consensus_peers.cc:597] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Couldn't send request to peer c4b86aae3a8f4a9082e639144ec46228. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:45135: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.087550 16923 consensus_peers.cc:597] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 -> Peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Couldn't send request to peer c4b86aae3a8f4a9082e639144ec46228. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:45135: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.091754 16923 consensus_peers.cc:597] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 -> Peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Couldn't send request to peer c4b86aae3a8f4a9082e639144ec46228. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:45135: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.110286 16923 consensus_peers.cc:597] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533 -> Peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Couldn't send request to peer c4b86aae3a8f4a9082e639144ec46228. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:45135: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.112092 16849 consensus_peers.cc:597] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Couldn't send request to peer c4b86aae3a8f4a9082e639144ec46228. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:45135: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.125000 16849 consensus_peers.cc:597] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Couldn't send request to peer c4b86aae3a8f4a9082e639144ec46228. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:45135: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.172528  7164 tablet_server.cc:179] TabletServer@127.6.255.3:0 shutting down...
W20260709 13:06:11.195731 16849 consensus_peers.cc:597] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 -> Peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Couldn't send request to peer c4b86aae3a8f4a9082e639144ec46228. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:45135: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.206174 16923 consensus_peers.cc:597] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 -> Peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Couldn't send request to peer d4e1b05bc80e43f7941ece113fb97ab0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.243469  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:11.244613  7164 tablet_replica.cc:333] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.245666  7164 raft_consensus.cc:2243] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.246824  7164 raft_consensus.cc:2272] T f9d47e8e7b784085b7079b1542dbc579 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.249722  7164 tablet_replica.cc:333] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.250538  7164 raft_consensus.cc:2243] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.251134  7164 raft_consensus.cc:2272] T f86d4f23f8cd4451a50b1b38155d958e P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.253975  7164 tablet_replica.cc:333] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.254827  7164 raft_consensus.cc:2243] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.255384  7164 raft_consensus.cc:2272] T f239a1de317547b598aeedc11994c75f P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.258852  7164 tablet_replica.cc:333] T 02d877efcf5049a89eacf23b58ba27b9 P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.259915  7164 raft_consensus.cc:2243] T 02d877efcf5049a89eacf23b58ba27b9 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.260437 18003 raft_consensus.cc:493] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:06:11.260646  7164 raft_consensus.cc:2272] T 02d877efcf5049a89eacf23b58ba27b9 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.261029 18003 raft_consensus.cc:515] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:06:11.263815 18003 leader_election.cc:290] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:06:11.264036  7164 tablet_replica.cc:333] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.264957  7164 raft_consensus.cc:2243] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.265708  7164 raft_consensus.cc:2272] T 136f22096d4b41d89354924f7101d1c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.269114  7164 tablet_replica.cc:333] T 93629337d6b8477a9cd5895a157bf918 P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.270015  7164 raft_consensus.cc:2243] T 93629337d6b8477a9cd5895a157bf918 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.270527  7164 raft_consensus.cc:2272] T 93629337d6b8477a9cd5895a157bf918 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.273607  7164 tablet_replica.cc:333] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.274446  7164 raft_consensus.cc:2243] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.275012  7164 raft_consensus.cc:2272] T 20cdedb225464d5e812a56ed7a7d220f P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.277324 16923 consensus_peers.cc:597] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 -> Peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Couldn't send request to peer c4b86aae3a8f4a9082e639144ec46228. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:45135: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.278194  7164 tablet_replica.cc:333] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.279626  7164 raft_consensus.cc:2243] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.280313  7164 raft_consensus.cc:2272] T 3c19ee4785104276aec22ba772dbc6ac P d4e1b05bc80e43f7941ece113fb97ab0 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.282332 16923 consensus_peers.cc:597] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533 -> Peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Couldn't send request to peer d4e1b05bc80e43f7941ece113fb97ab0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.283375  7164 tablet_replica.cc:333] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.284266  7164 raft_consensus.cc:2243] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.285332  7164 raft_consensus.cc:2272] T 9078054426a34acbb94be2b69213b6c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.288324  7164 tablet_replica.cc:333] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.289259  7164 raft_consensus.cc:2243] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.290297  7164 raft_consensus.cc:2272] T 59de6d1228da41da938527632da08f4d P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.291359 16923 leader_election.cc:336] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Network error: Client connection negotiation failed: client connection to 127.6.255.2:45135: connect: Connection refused (error 111)
I20260709 13:06:11.293612  7164 tablet_replica.cc:333] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
W20260709 13:06:11.294516 16923 consensus_peers.cc:597] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 -> Peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Couldn't send request to peer d4e1b05bc80e43f7941ece113fb97ab0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.294639  7164 raft_consensus.cc:2243] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:06:11.295255 16923 consensus_peers.cc:597] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 -> Peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Couldn't send request to peer d4e1b05bc80e43f7941ece113fb97ab0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.295902 16923 consensus_peers.cc:597] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533 -> Peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Couldn't send request to peer d4e1b05bc80e43f7941ece113fb97ab0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.295972  7164 raft_consensus.cc:2272] T caa2367ac34d43b5b1a27410b2055005 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.296550 16923 leader_election.cc:336] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:06:11.297014 16923 leader_election.cc:304] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:06:11.297917 18012 raft_consensus.cc:2749] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:11.299710  7164 tablet_replica.cc:333] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.300668  7164 raft_consensus.cc:2243] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:06:11.301067 16923 consensus_peers.cc:597] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 -> Peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Couldn't send request to peer d4e1b05bc80e43f7941ece113fb97ab0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.301636  7164 raft_consensus.cc:2272] T e5c700237eb44cefbf7bc46c1daf08cd P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.304684  7164 tablet_replica.cc:333] T e81325b638b1493999310a726811b64b P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.305500  7164 raft_consensus.cc:2243] T e81325b638b1493999310a726811b64b P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.306071  7164 raft_consensus.cc:2272] T e81325b638b1493999310a726811b64b P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.308677  7164 tablet_replica.cc:333] T 7123efeaee9f4ec78a1c475eab16c4c1 P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.309324  7164 raft_consensus.cc:2243] T 7123efeaee9f4ec78a1c475eab16c4c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.309811  7164 raft_consensus.cc:2272] T 7123efeaee9f4ec78a1c475eab16c4c1 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.312021  7164 tablet_replica.cc:333] T 9e8e0f9919a9404c9001b6718f102d19 P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.312827  7164 raft_consensus.cc:2243] T 9e8e0f9919a9404c9001b6718f102d19 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.313406  7164 raft_consensus.cc:2272] T 9e8e0f9919a9404c9001b6718f102d19 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.315555  7164 tablet_replica.cc:333] T 7f4354282d844c7e83d760e4418512dc P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.316161  7164 raft_consensus.cc:2243] T 7f4354282d844c7e83d760e4418512dc P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.316766  7164 raft_consensus.cc:2272] T 7f4354282d844c7e83d760e4418512dc P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.319231  7164 tablet_replica.cc:333] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.319886  7164 raft_consensus.cc:2243] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.320892  7164 raft_consensus.cc:2272] T 61f747edd7934aac93044ba370e34d17 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.323666  7164 tablet_replica.cc:333] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.324292  7164 raft_consensus.cc:2243] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.324772  7164 raft_consensus.cc:2272] T 58ff2290a9c9451c9a786eacf7380d90 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.327308  7164 tablet_replica.cc:333] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.327889  7164 raft_consensus.cc:2243] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.328783  7164 raft_consensus.cc:2272] T 612e472fbe674788986798c65836388f P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.330902  7164 tablet_replica.cc:333] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.331575  7164 raft_consensus.cc:2243] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.332212  7164 raft_consensus.cc:2272] T e091d4e9a0044dd899e731769a94064c P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.335095  7164 tablet_replica.cc:333] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.335798  7164 raft_consensus.cc:2243] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.336673  7164 raft_consensus.cc:2272] T 5857fad96a1045d7b85190e64da77bce P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.339653  7164 tablet_replica.cc:333] T 0b8add4b17084a6189509807b7e663e5 P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.340415  7164 raft_consensus.cc:2243] T 0b8add4b17084a6189509807b7e663e5 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.340931  7164 raft_consensus.cc:2272] T 0b8add4b17084a6189509807b7e663e5 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.343116  7164 tablet_replica.cc:333] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.343824  7164 raft_consensus.cc:2243] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.344866  7164 raft_consensus.cc:2272] T 10bbce697a734600bb083d47ac877574 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.347554  7164 tablet_replica.cc:333] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.348145  7164 raft_consensus.cc:2243] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.348608  7164 raft_consensus.cc:2272] T b4d83d4d3d894b81a85b945ef97d79de P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.350746  7164 tablet_replica.cc:333] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.351317  7164 raft_consensus.cc:2243] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.352136  7164 raft_consensus.cc:2272] T 4f95af7ac3134423b1a5956c8ae950df P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.354933  7164 tablet_replica.cc:333] T ac3ad4fe0bc4435cb1c596ad215889a3 P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.355590  7164 raft_consensus.cc:2243] T ac3ad4fe0bc4435cb1c596ad215889a3 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.356215  7164 raft_consensus.cc:2272] T ac3ad4fe0bc4435cb1c596ad215889a3 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.359095  7164 tablet_replica.cc:333] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.359755  7164 raft_consensus.cc:2243] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.360313  7164 raft_consensus.cc:2272] T 744cc6469e3749e2aacccb45464e8198 P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.363399  7164 tablet_replica.cc:333] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.364017  7164 raft_consensus.cc:2243] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.364933  7164 raft_consensus.cc:2272] T 27be53bca112445d8bdc640329310d67 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.367189  7164 tablet_replica.cc:333] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.367849  7164 raft_consensus.cc:2243] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.368357  7164 raft_consensus.cc:2272] T 3d6e27a9a96c4121b2abc71e0d7738be P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.370419  7164 tablet_replica.cc:333] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.370952  7164 raft_consensus.cc:2243] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.371744  7164 raft_consensus.cc:2272] T 37a5bbadcd9b48dba8fe272a18679ede P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.373735  7164 tablet_replica.cc:333] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.374264  7164 raft_consensus.cc:2243] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.374701  7164 raft_consensus.cc:2272] T f3cecfcf8ae242e1b70f628f293da197 P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.377168  7164 tablet_replica.cc:333] T c9043e0bc02745d187e9ca5a5c71274f P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.377954  7164 raft_consensus.cc:2243] T c9043e0bc02745d187e9ca5a5c71274f P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.378427  7164 raft_consensus.cc:2272] T c9043e0bc02745d187e9ca5a5c71274f P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.380457  7164 tablet_replica.cc:333] T b9fdc34d643b4ab995042f719af511aa P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.381187 18012 raft_consensus.cc:493] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:06:11.381323  7164 raft_consensus.cc:2243] T b9fdc34d643b4ab995042f719af511aa P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.382078  7164 raft_consensus.cc:2272] T b9fdc34d643b4ab995042f719af511aa P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.381994 18012 raft_consensus.cc:515] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:06:11.383968 18012 leader_election.cc:290] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
I20260709 13:06:11.385001  7164 tablet_replica.cc:333] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.385905  7164 raft_consensus.cc:2243] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.386423  7164 raft_consensus.cc:2272] T 9dd082201fe04ef89c842b6404fd522e P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.389317  7164 tablet_replica.cc:333] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
W20260709 13:06:11.389340 16923 leader_election.cc:336] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
W20260709 13:06:11.390219 16923 leader_election.cc:336] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Network error: Client connection negotiation failed: client connection to 127.6.255.2:45135: connect: Connection refused (error 111)
I20260709 13:06:11.390223  7164 raft_consensus.cc:2243] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.390700 16923 leader_election.cc:304] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:06:11.390981  7164 raft_consensus.cc:2272] T c5b9b28c4c5d4ddaa6d1ae05ce225b11 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.391535 18012 raft_consensus.cc:2749] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:11.393894  7164 tablet_replica.cc:333] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.394650  7164 raft_consensus.cc:2243] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.395275  7164 raft_consensus.cc:2272] T 919c5788cd97463a9cf06abb18843943 P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.396842 16923 consensus_peers.cc:597] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 -> Peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Couldn't send request to peer d4e1b05bc80e43f7941ece113fb97ab0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.398124  7164 tablet_replica.cc:333] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.398710  7164 raft_consensus.cc:2243] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.399233  7164 raft_consensus.cc:2272] T 0e295eefc8ff46d9a31a4c0898d16c63 P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.401261  7164 tablet_replica.cc:333] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.401837  7164 raft_consensus.cc:2243] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.402585  7164 raft_consensus.cc:2272] T 3850091ca95e47b6a1b52e600a452b0b P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.404548  7164 tablet_replica.cc:333] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.404999  7164 raft_consensus.cc:2243] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.405359  7164 raft_consensus.cc:2272] T 7f37dd0138014a75bc36f33022362b3c P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.407510  7164 tablet_replica.cc:333] T 2593417c1fe4457c87cb2a01cf1f65a3 P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.407977  7164 raft_consensus.cc:2243] T 2593417c1fe4457c87cb2a01cf1f65a3 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.408373  7164 raft_consensus.cc:2272] T 2593417c1fe4457c87cb2a01cf1f65a3 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.410327  7164 tablet_replica.cc:333] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.410856  7164 raft_consensus.cc:2243] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.411602  7164 raft_consensus.cc:2272] T 0a7231efacd1463c96b35c3d03edce55 P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.412626 18012 raft_consensus.cc:493] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:06:11.413388 18012 raft_consensus.cc:515] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:11.414110  7164 tablet_replica.cc:333] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.414686  7164 raft_consensus.cc:2243] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.415261  7164 raft_consensus.cc:2272] T 15b6c54287fa4fdb928257d6faf572f8 P d4e1b05bc80e43f7941ece113fb97ab0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.415294 18012 leader_election.cc:290] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
I20260709 13:06:11.418496  7164 tablet_replica.cc:333] T 83ec8ffbf1044525880e6b0d7931f47e P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
W20260709 13:06:11.418926 16923 leader_election.cc:336] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:06:11.419216  7164 raft_consensus.cc:2243] T 83ec8ffbf1044525880e6b0d7931f47e P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:11.419538 16923 leader_election.cc:336] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:06:11.419876  7164 raft_consensus.cc:2272] T 83ec8ffbf1044525880e6b0d7931f47e P d4e1b05bc80e43f7941ece113fb97ab0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.419997 16923 leader_election.cc:304] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4
I20260709 13:06:11.420770 18012 raft_consensus.cc:2749] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:11.422502  7164 tablet_replica.cc:333] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0: stopping tablet replica
I20260709 13:06:11.423061  7164 raft_consensus.cc:2243] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.424052  7164 raft_consensus.cc:2272] T 535cac5e21bf41db9c037f226ebcd94e P d4e1b05bc80e43f7941ece113fb97ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.447309 18012 raft_consensus.cc:493] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:06:11.448166 18012 raft_consensus.cc:515] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } } peers { permanent_uuid: "c4b86aae3a8f4a9082e639144ec46228" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 45135 } }
I20260709 13:06:11.450946 18012 leader_election.cc:290] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785), c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135)
W20260709 13:06:11.454883 16923 leader_election.cc:336] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
W20260709 13:06:11.456346 16923 leader_election.cc:336] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c4b86aae3a8f4a9082e639144ec46228 (127.6.255.2:45135): Network error: Client connection negotiation failed: client connection to 127.6.255.2:45135: connect: Connection refused (error 111)
I20260709 13:06:11.456736 16923 leader_election.cc:304] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: c4b86aae3a8f4a9082e639144ec46228, e2b87453fe654318abccdd1512b949c4
I20260709 13:06:11.457340 18012 raft_consensus.cc:2749] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:06:11.472448 16923 consensus_peers.cc:597] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 -> Peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Couldn't send request to peer d4e1b05bc80e43f7941ece113fb97ab0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.507026  7164 tablet_server.cc:196] TabletServer@127.6.255.3:0 shutdown complete.
I20260709 13:06:11.554906 18012 raft_consensus.cc:493] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:06:11.555790 18012 raft_consensus.cc:515] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:11.558195 18012 leader_election.cc:290] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
W20260709 13:06:11.567237 16923 leader_election.cc:336] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Network error: Client connection negotiation failed: client connection to 127.6.255.3:45101: connect: Connection refused (error 111)
W20260709 13:06:11.569535 16923 leader_election.cc:336] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:06:11.570024 16923 leader_election.cc:304] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4
I20260709 13:06:11.571024 17888 raft_consensus.cc:2749] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:06:11.583225 16923 consensus_peers.cc:597] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 -> Peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Couldn't send request to peer d4e1b05bc80e43f7941ece113fb97ab0. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.3:45101: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.595935  7164 tablet_server.cc:179] TabletServer@127.6.255.4:0 shutting down...
I20260709 13:06:11.606842 17888 raft_consensus.cc:493] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:06:11.607487 17888 raft_consensus.cc:515] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:11.609220 17888 leader_election.cc:290] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
W20260709 13:06:11.613729 16923 leader_election.cc:336] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Network error: Client connection negotiation failed: client connection to 127.6.255.3:45101: connect: Connection refused (error 111)
W20260709 13:06:11.615471 16923 leader_election.cc:336] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:06:11.616128 16923 leader_election.cc:304] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4
I20260709 13:06:11.616989 17888 raft_consensus.cc:2749] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:06:11.650475 16923 consensus_peers.cc:597] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533 -> Peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Couldn't send request to peer d4e1b05bc80e43f7941ece113fb97ab0. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.3:45101: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.662839  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:11.663834  7164 tablet_replica.cc:333] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.664554  7164 raft_consensus.cc:2243] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.665621  7164 raft_consensus.cc:2272] T 87ca116ec6174decb610df3f8f565a6b P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.667809  7164 tablet_replica.cc:333] T 2593417c1fe4457c87cb2a01cf1f65a3 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.668421  7164 raft_consensus.cc:2243] T 2593417c1fe4457c87cb2a01cf1f65a3 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.668927  7164 raft_consensus.cc:2272] T 2593417c1fe4457c87cb2a01cf1f65a3 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.670995  7164 tablet_replica.cc:333] T f9d47e8e7b784085b7079b1542dbc579 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.671622  7164 raft_consensus.cc:2243] T f9d47e8e7b784085b7079b1542dbc579 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.672355  7164 raft_consensus.cc:2272] T f9d47e8e7b784085b7079b1542dbc579 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.675009  7164 tablet_replica.cc:333] T 6ec647c6ccd641b59808817d8dca2235 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.675658  7164 raft_consensus.cc:2243] T 6ec647c6ccd641b59808817d8dca2235 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.676131  7164 raft_consensus.cc:2272] T 6ec647c6ccd641b59808817d8dca2235 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.678280  7164 tablet_replica.cc:333] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.678913  7164 raft_consensus.cc:2243] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.679721  7164 raft_consensus.cc:2272] T 02d877efcf5049a89eacf23b58ba27b9 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.681875  7164 tablet_replica.cc:333] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.682700  7164 raft_consensus.cc:2243] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.683501  7164 raft_consensus.cc:2272] T be51f207c3bc4276b220a03e2d9d4db4 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.684013 18003 raft_consensus.cc:493] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e2b87453fe654318abccdd1512b949c4)
I20260709 13:06:11.684587 18003 raft_consensus.cc:515] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e1b05bc80e43f7941ece113fb97ab0" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 45101 } } peers { permanent_uuid: "0086163b152644feb75caeac6f145533" member_type: VOTER last_known_addr { host: "127.6.255.4" port: 37199 } } peers { permanent_uuid: "e2b87453fe654318abccdd1512b949c4" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 40785 } }
I20260709 13:06:11.686859  7164 tablet_replica.cc:333] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.687882  7164 raft_consensus.cc:2243] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.688645  7164 raft_consensus.cc:2272] T 86f5e278f54c419fac8c74ac63f80f74 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.686836 18003 leader_election.cc:290] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101), e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785)
W20260709 13:06:11.691021 16923 leader_election.cc:336] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d4e1b05bc80e43f7941ece113fb97ab0 (127.6.255.3:45101): Network error: Client connection negotiation failed: client connection to 127.6.255.3:45101: connect: Connection refused (error 111)
I20260709 13:06:11.691462  7164 tablet_replica.cc:333] T e5c700237eb44cefbf7bc46c1daf08cd P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.692296  7164 raft_consensus.cc:2243] T e5c700237eb44cefbf7bc46c1daf08cd P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.692874  7164 raft_consensus.cc:2272] T e5c700237eb44cefbf7bc46c1daf08cd P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.692986 16923 leader_election.cc:336] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e2b87453fe654318abccdd1512b949c4 (127.6.255.1:40785): Network error: Client connection negotiation failed: client connection to 127.6.255.1:40785: connect: Connection refused (error 111)
I20260709 13:06:11.693467 16923 leader_election.cc:304] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0086163b152644feb75caeac6f145533; no voters: d4e1b05bc80e43f7941ece113fb97ab0, e2b87453fe654318abccdd1512b949c4
I20260709 13:06:11.694051 18003 raft_consensus.cc:2749] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:11.695335  7164 tablet_replica.cc:333] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.696278  7164 raft_consensus.cc:2243] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.697240  7164 raft_consensus.cc:2272] T f86d4f23f8cd4451a50b1b38155d958e P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.700316  7164 tablet_replica.cc:333] T e81325b638b1493999310a726811b64b P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.701113  7164 raft_consensus.cc:2243] T e81325b638b1493999310a726811b64b P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.701697  7164 raft_consensus.cc:2272] T e81325b638b1493999310a726811b64b P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.704552  7164 tablet_replica.cc:333] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.705304  7164 raft_consensus.cc:2243] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.706256  7164 raft_consensus.cc:2272] T b4d83d4d3d894b81a85b945ef97d79de P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.708441  7164 tablet_replica.cc:333] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.709039  7164 raft_consensus.cc:2243] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.709527  7164 raft_consensus.cc:2272] T 7123efeaee9f4ec78a1c475eab16c4c1 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.711478  7164 tablet_replica.cc:333] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.711995  7164 raft_consensus.cc:2243] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.712400  7164 raft_consensus.cc:2272] T 919c5788cd97463a9cf06abb18843943 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.714251  7164 tablet_replica.cc:333] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.714758  7164 raft_consensus.cc:2243] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.715118  7164 raft_consensus.cc:2272] T 0e295eefc8ff46d9a31a4c0898d16c63 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.716841  7164 tablet_replica.cc:333] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.717384  7164 raft_consensus.cc:2243] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.718046  7164 raft_consensus.cc:2272] T 744cc6469e3749e2aacccb45464e8198 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.721086  7164 tablet_replica.cc:333] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.722087  7164 raft_consensus.cc:2243] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.722720  7164 raft_consensus.cc:2272] T 27be53bca112445d8bdc640329310d67 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.725535  7164 tablet_replica.cc:333] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.726148  7164 raft_consensus.cc:2243] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.726531  7164 raft_consensus.cc:2272] T f48e0f9f83da446a877cd06e19768929 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.729451  7164 tablet_replica.cc:333] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.730268  7164 raft_consensus.cc:2243] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.731276  7164 raft_consensus.cc:2272] T 136f22096d4b41d89354924f7101d1c1 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.734364  7164 tablet_replica.cc:333] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.735085  7164 raft_consensus.cc:2243] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.735677  7164 raft_consensus.cc:2272] T 93629337d6b8477a9cd5895a157bf918 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.738037  7164 tablet_replica.cc:333] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.738660  7164 raft_consensus.cc:2243] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.739498  7164 raft_consensus.cc:2272] T 9e8e0f9919a9404c9001b6718f102d19 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.741681  7164 tablet_replica.cc:333] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.742308  7164 raft_consensus.cc:2243] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.743083  7164 raft_consensus.cc:2272] T 7f37dd0138014a75bc36f33022362b3c P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.745162  7164 tablet_replica.cc:333] T 3850091ca95e47b6a1b52e600a452b0b P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.745738  7164 raft_consensus.cc:2243] T 3850091ca95e47b6a1b52e600a452b0b P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.746147  7164 raft_consensus.cc:2272] T 3850091ca95e47b6a1b52e600a452b0b P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.748035  7164 tablet_replica.cc:333] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.748586  7164 raft_consensus.cc:2243] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.749322  7164 raft_consensus.cc:2272] T c968ba000a4542fc9204dd9a474e364c P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.751689  7164 tablet_replica.cc:333] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.752261  7164 raft_consensus.cc:2243] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.753103  7164 raft_consensus.cc:2272] T 20cdedb225464d5e812a56ed7a7d220f P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.755244  7164 tablet_replica.cc:333] T 9592a31e7ffd407a959c0a3d96c0f8f9 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.755805  7164 raft_consensus.cc:2243] T 9592a31e7ffd407a959c0a3d96c0f8f9 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.756224  7164 raft_consensus.cc:2272] T 9592a31e7ffd407a959c0a3d96c0f8f9 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.758123  7164 tablet_replica.cc:333] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.758683  7164 raft_consensus.cc:2243] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.759097  7164 raft_consensus.cc:2272] T e1660f76b7074e2cb7542ed02dd179d9 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.761065  7164 tablet_replica.cc:333] T 61f747edd7934aac93044ba370e34d17 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.761677  7164 raft_consensus.cc:2243] T 61f747edd7934aac93044ba370e34d17 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.762102  7164 raft_consensus.cc:2272] T 61f747edd7934aac93044ba370e34d17 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.764022  7164 tablet_replica.cc:333] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.764549  7164 raft_consensus.cc:2243] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.765407  7164 raft_consensus.cc:2272] T 081841333d734eb1aa72ceb3f0ea2bf2 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.767344  7164 tablet_replica.cc:333] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.767904  7164 raft_consensus.cc:2243] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.768697  7164 raft_consensus.cc:2272] T 58ff2290a9c9451c9a786eacf7380d90 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.770623  7164 tablet_replica.cc:333] T 612e472fbe674788986798c65836388f P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.771235  7164 raft_consensus.cc:2243] T 612e472fbe674788986798c65836388f P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.771670  7164 raft_consensus.cc:2272] T 612e472fbe674788986798c65836388f P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.773521  7164 tablet_replica.cc:333] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.774065  7164 raft_consensus.cc:2243] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.774497  7164 raft_consensus.cc:2272] T 5b5161020c944d28a4334dc1d572dac7 P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.776471  7164 tablet_replica.cc:333] T 5857fad96a1045d7b85190e64da77bce P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.776986  7164 raft_consensus.cc:2243] T 5857fad96a1045d7b85190e64da77bce P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.777421  7164 raft_consensus.cc:2272] T 5857fad96a1045d7b85190e64da77bce P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.779186  7164 tablet_replica.cc:333] T 10bbce697a734600bb083d47ac877574 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.779702  7164 raft_consensus.cc:2243] T 10bbce697a734600bb083d47ac877574 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.780113  7164 raft_consensus.cc:2272] T 10bbce697a734600bb083d47ac877574 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.782123  7164 tablet_replica.cc:333] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.782743  7164 raft_consensus.cc:2243] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.783499  7164 raft_consensus.cc:2272] T d82d38de8d2c47b38ca8811691c68d8f P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.785363  7164 tablet_replica.cc:333] T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.785899  7164 raft_consensus.cc:2243] T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.786330  7164 raft_consensus.cc:2272] T e091d4e9a0044dd899e731769a94064c P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.788188  7164 tablet_replica.cc:333] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.788789  7164 raft_consensus.cc:2243] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.789712  7164 raft_consensus.cc:2272] T 213e9771c53e4498b315bb43fdc0aa9a P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.792333  7164 tablet_replica.cc:333] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.792917  7164 raft_consensus.cc:2243] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.793481  7164 raft_consensus.cc:2272] T 42e96e3138004b3aa2f3e745f1ddfc04 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.795599  7164 tablet_replica.cc:333] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.796048  7164 raft_consensus.cc:2243] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.796782  7164 raft_consensus.cc:2272] T 6288d52217814c7393ff0ec66da3b099 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.798821  7164 tablet_replica.cc:333] T 535cac5e21bf41db9c037f226ebcd94e P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.799300  7164 raft_consensus.cc:2243] T 535cac5e21bf41db9c037f226ebcd94e P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.799671  7164 raft_consensus.cc:2272] T 535cac5e21bf41db9c037f226ebcd94e P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.801621  7164 tablet_replica.cc:333] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.802068  7164 raft_consensus.cc:2243] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.802788  7164 raft_consensus.cc:2272] T c9043e0bc02745d187e9ca5a5c71274f P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.804611  7164 tablet_replica.cc:333] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.805074  7164 raft_consensus.cc:2243] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.805496  7164 raft_consensus.cc:2272] T 37a5bbadcd9b48dba8fe272a18679ede P 0086163b152644feb75caeac6f145533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.807404  7164 tablet_replica.cc:333] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.807848  7164 raft_consensus.cc:2243] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.808218  7164 raft_consensus.cc:2272] T f3cecfcf8ae242e1b70f628f293da197 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.810595  7164 tablet_replica.cc:333] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.811077  7164 raft_consensus.cc:2243] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.811462  7164 raft_consensus.cc:2272] T 6c9ca489a4d2416092d6dcfae2498136 P 0086163b152644feb75caeac6f145533 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.813241  7164 tablet_replica.cc:333] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533: stopping tablet replica
I20260709 13:06:11.813740  7164 raft_consensus.cc:2243] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.814461  7164 raft_consensus.cc:2272] T ac3ad4fe0bc4435cb1c596ad215889a3 P 0086163b152644feb75caeac6f145533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.917325  7164 tablet_server.cc:196] TabletServer@127.6.255.4:0 shutdown complete.
I20260709 13:06:11.990733  7164 master.cc:562] Master@127.6.255.62:33261 shutting down...
I20260709 13:06:12.023413  7164 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b17e9282f0de497b861afbf2b8adc5d1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.024329  7164 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b17e9282f0de497b861afbf2b8adc5d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.024691  7164 tablet_replica.cc:333] T 00000000000000000000000000000000 P b17e9282f0de497b861afbf2b8adc5d1: stopping tablet replica
I20260709 13:06:12.082758  7164 master.cc:584] Master@127.6.255.62:33261 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (38456 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:06:12.182397  7164 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.255.62:37899
I20260709 13:06:12.183629  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:12.190234 18018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:12.190949 18020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:12.190893 18017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:12.192085  7164 server_base.cc:1061] running on GCE node
I20260709 13:06:12.193540  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:12.193733  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:12.193945  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602372193924 us; error 0 us; skew 500 ppm
I20260709 13:06:12.194554  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:12.197172  7164 webserver.cc:533] Webserver started at http://127.6.255.62:45267/ using document root <none> and password file <none>
I20260709 13:06:12.197701  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:12.198123  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:12.198550  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:12.199857  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602046064212-7164-0/minicluster-data/master-0-root/instance:
uuid: "abdef627a1e54172bbbc993071d7844f"
format_stamp: "Formatted at 2026-07-09 13:06:12 on dist-test-slave-rnv4"
I20260709 13:06:12.204865  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.004s
I20260709 13:06:12.208691 18025 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:12.209964  7164 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:06:12.210382  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602046064212-7164-0/minicluster-data/master-0-root
uuid: "abdef627a1e54172bbbc993071d7844f"
format_stamp: "Formatted at 2026-07-09 13:06:12 on dist-test-slave-rnv4"
I20260709 13:06:12.210716  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602046064212-7164-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602046064212-7164-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602046064212-7164-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:12.229084  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:12.230477  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:12.277083  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.62:37899
I20260709 13:06:12.277212 18076 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.62:37899 every 8 connection(s)
I20260709 13:06:12.282369 18077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.294904 18077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abdef627a1e54172bbbc993071d7844f: Bootstrap starting.
I20260709 13:06:12.301240 18077 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abdef627a1e54172bbbc993071d7844f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:12.306313 18077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abdef627a1e54172bbbc993071d7844f: No bootstrap required, opened a new log
I20260709 13:06:12.308560 18077 raft_consensus.cc:359] T 00000000000000000000000000000000 P abdef627a1e54172bbbc993071d7844f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abdef627a1e54172bbbc993071d7844f" member_type: VOTER }
I20260709 13:06:12.308974 18077 raft_consensus.cc:385] T 00000000000000000000000000000000 P abdef627a1e54172bbbc993071d7844f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:12.309155 18077 raft_consensus.cc:740] T 00000000000000000000000000000000 P abdef627a1e54172bbbc993071d7844f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abdef627a1e54172bbbc993071d7844f, State: Initialized, Role: FOLLOWER
I20260709 13:06:12.309870 18077 consensus_queue.cc:260] T 00000000000000000000000000000000 P abdef627a1e54172bbbc993071d7844f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abdef627a1e54172bbbc993071d7844f" member_type: VOTER }
I20260709 13:06:12.310309 18077 raft_consensus.cc:399] T 00000000000000000000000000000000 P abdef627a1e54172bbbc993071d7844f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:06:12.310503 18077 raft_consensus.cc:493] T 00000000000000000000000000000000 P abdef627a1e54172bbbc993071d7844f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:06:12.310714 18077 raft_consensus.cc:3060] T 00000000000000000000000000000000 P abdef627a1e54172bbbc993071d7844f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:12.315971 18077 raft_consensus.cc:515] T 00000000000000000000000000000000 P abdef627a1e54172bbbc993071d7844f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abdef627a1e54172bbbc993071d7844f" member_type: VOTER }
I20260709 13:06:12.316509 18077 leader_election.cc:304] T 00000000000000000000000000000000 P abdef627a1e54172bbbc993071d7844f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abdef627a1e54172bbbc993071d7844f; no voters: 
I20260709 13:06:12.317869 18077 leader_election.cc:290] T 00000000000000000000000000000000 P abdef627a1e54172bbbc993071d7844f [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:06:12.318434 18080 raft_consensus.cc:2804] T 00000000000000000000000000000000 P abdef627a1e54172bbbc993071d7844f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:12.320109 18080 raft_consensus.cc:697] T 00000000000000000000000000000000 P abdef627a1e54172bbbc993071d7844f [term 1 LEADER]: Becoming Leader. State: Replica: abdef627a1e54172bbbc993071d7844f, State: Running, Role: LEADER
I20260709 13:06:12.320880 18080 consensus_queue.cc:237] T 00000000000000000000000000000000 P abdef627a1e54172bbbc993071d7844f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abdef627a1e54172bbbc993071d7844f" member_type: VOTER }
I20260709 13:06:12.321082 18077 sys_catalog.cc:565] T 00000000000000000000000000000000 P abdef627a1e54172bbbc993071d7844f [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:06:12.326859 18081 sys_catalog.cc:455] T 00000000000000000000000000000000 P abdef627a1e54172bbbc993071d7844f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abdef627a1e54172bbbc993071d7844f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abdef627a1e54172bbbc993071d7844f" member_type: VOTER } }
I20260709 13:06:12.327348 18082 sys_catalog.cc:455] T 00000000000000000000000000000000 P abdef627a1e54172bbbc993071d7844f [sys.catalog]: SysCatalogTable state changed. Reason: New leader abdef627a1e54172bbbc993071d7844f. Latest consensus state: current_term: 1 leader_uuid: "abdef627a1e54172bbbc993071d7844f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abdef627a1e54172bbbc993071d7844f" member_type: VOTER } }
I20260709 13:06:12.327809 18081 sys_catalog.cc:458] T 00000000000000000000000000000000 P abdef627a1e54172bbbc993071d7844f [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:12.328195 18082 sys_catalog.cc:458] T 00000000000000000000000000000000 P abdef627a1e54172bbbc993071d7844f [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:12.333258 18088 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:06:12.341738 18088 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:06:12.343080  7164 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:06:12.353212 18088 catalog_manager.cc:1383] Generated new cluster ID: 4c8c1ed7e22546cea47ee80e6fae907a
I20260709 13:06:12.353701 18088 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:06:12.379130 18088 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:06:12.380713 18088 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:06:12.399107 18088 catalog_manager.cc:6070] T 00000000000000000000000000000000 P abdef627a1e54172bbbc993071d7844f: Generated new TSK 0
I20260709 13:06:12.399887 18088 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:06:12.410182  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:12.416915 18098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:12.417541 18099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:12.420779  7164 server_base.cc:1061] running on GCE node
W20260709 13:06:12.421310 18101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:12.422403  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:12.422596  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:12.422822  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602372422795 us; error 0 us; skew 500 ppm
I20260709 13:06:12.423390  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:12.425938  7164 webserver.cc:533] Webserver started at http://127.6.255.1:35929/ using document root <none> and password file <none>
I20260709 13:06:12.426431  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:12.426636  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:12.426909  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:12.427994  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602046064212-7164-0/minicluster-data/ts-0-root/instance:
uuid: "500e80cda2474f66b0ad65f4bfa00258"
format_stamp: "Formatted at 2026-07-09 13:06:12 on dist-test-slave-rnv4"
I20260709 13:06:12.432792  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:06:12.437098 18106 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:12.438124  7164 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:06:12.438445  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602046064212-7164-0/minicluster-data/ts-0-root
uuid: "500e80cda2474f66b0ad65f4bfa00258"
format_stamp: "Formatted at 2026-07-09 13:06:12 on dist-test-slave-rnv4"
I20260709 13:06:12.438705  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602046064212-7164-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602046064212-7164-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602046064212-7164-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:12.457013  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:12.458844  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:12.460600  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:12.463137  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:12.463361  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:12.463650  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:12.463833  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:12.520685  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.1:44331
I20260709 13:06:12.520958 18168 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.1:44331 every 8 connection(s)
I20260709 13:06:12.528018  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:12.538056 18173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:12.539729 18176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:12.539893 18174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:12.541697  7164 server_base.cc:1061] running on GCE node
I20260709 13:06:12.543347  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:12.543576  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:12.543808  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602372543771 us; error 0 us; skew 500 ppm
I20260709 13:06:12.544696  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:12.547803 18169 heartbeater.cc:344] Connected to a master server at 127.6.255.62:37899
I20260709 13:06:12.548071  7164 webserver.cc:533] Webserver started at http://127.6.255.2:43749/ using document root <none> and password file <none>
I20260709 13:06:12.548295 18169 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:12.548915  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:12.549216  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:12.549412 18169 heartbeater.cc:507] Master 127.6.255.62:37899 requested a full tablet report, sending...
I20260709 13:06:12.549614  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:12.551180  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602046064212-7164-0/minicluster-data/ts-1-root/instance:
uuid: "3272b165a59e4467925ea233bbb5f6f8"
format_stamp: "Formatted at 2026-07-09 13:06:12 on dist-test-slave-rnv4"
I20260709 13:06:12.552345 18042 ts_manager.cc:194] Registered new tserver with Master: 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331)
I20260709 13:06:12.554981 18042 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45104
I20260709 13:06:12.556720  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:06:12.560869 18181 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:12.562216  7164 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:06:12.562498  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602046064212-7164-0/minicluster-data/ts-1-root
uuid: "3272b165a59e4467925ea233bbb5f6f8"
format_stamp: "Formatted at 2026-07-09 13:06:12 on dist-test-slave-rnv4"
I20260709 13:06:12.562767  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602046064212-7164-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602046064212-7164-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602046064212-7164-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:12.586354  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:12.587774  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:12.589377  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:12.591763  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:12.592015  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:12.592284  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:12.592460  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:12.648875  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.2:46515
I20260709 13:06:12.649021 18243 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.2:46515 every 8 connection(s)
I20260709 13:06:12.654533  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:12.663096 18247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:12.664686 18248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:12.666952 18250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:12.667709  7164 server_base.cc:1061] running on GCE node
I20260709 13:06:12.668318 18244 heartbeater.cc:344] Connected to a master server at 127.6.255.62:37899
I20260709 13:06:12.668748 18244 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:12.668869  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:12.669175  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:12.669500  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602372669356 us; error 0 us; skew 500 ppm
I20260709 13:06:12.669620 18244 heartbeater.cc:507] Master 127.6.255.62:37899 requested a full tablet report, sending...
I20260709 13:06:12.670365  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:12.672851 18042 ts_manager.cc:194] Registered new tserver with Master: 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515)
I20260709 13:06:12.674810 18042 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44946
I20260709 13:06:12.674891  7164 webserver.cc:533] Webserver started at http://127.6.255.3:43639/ using document root <none> and password file <none>
I20260709 13:06:12.675674  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:12.675920  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:12.676265  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:12.678162  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602046064212-7164-0/minicluster-data/ts-2-root/instance:
uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
format_stamp: "Formatted at 2026-07-09 13:06:12 on dist-test-slave-rnv4"
I20260709 13:06:12.684125  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:06:12.688472 18255 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:12.689581  7164 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:06:12.689976  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602046064212-7164-0/minicluster-data/ts-2-root
uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
format_stamp: "Formatted at 2026-07-09 13:06:12 on dist-test-slave-rnv4"
I20260709 13:06:12.690326  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602046064212-7164-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602046064212-7164-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602046064212-7164-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:12.712400  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:12.713838  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:12.715190  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:12.717532  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:12.717720  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:12.718022  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:12.718165  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:12.767546  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.3:42323
I20260709 13:06:12.767693 18317 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.3:42323 every 8 connection(s)
I20260709 13:06:12.784576 18318 heartbeater.cc:344] Connected to a master server at 127.6.255.62:37899
I20260709 13:06:12.785061 18318 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:12.785880 18318 heartbeater.cc:507] Master 127.6.255.62:37899 requested a full tablet report, sending...
I20260709 13:06:12.787994 18042 ts_manager.cc:194] Registered new tserver with Master: ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:12.788400  7164 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016558232s
I20260709 13:06:12.789417 18042 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44954
I20260709 13:06:12.819645 18042 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44966:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:12.822791 18042 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:12.896595 18283 tablet_service.cc:1511] Processing CreateTablet for tablet 490ba2fed8234928a81ff40ca50e58c4 (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:06:12.897053 18282 tablet_service.cc:1511] Processing CreateTablet for tablet 308eb63baa0f4f51b9116e3b240b0932 (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:06:12.898627 18283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 490ba2fed8234928a81ff40ca50e58c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.899359 18282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 308eb63baa0f4f51b9116e3b240b0932. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.903234 18281 tablet_service.cc:1511] Processing CreateTablet for tablet 5214f540f4d64609b515bb6b540791ec (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:06:12.906327 18281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5214f540f4d64609b515bb6b540791ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.922914 18280 tablet_service.cc:1511] Processing CreateTablet for tablet 24794fe9a88c4959aa0101a5e612121f (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:06:12.926528 18280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24794fe9a88c4959aa0101a5e612121f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.951537 18332 tablet_bootstrap.cc:492] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a: Bootstrap starting.
I20260709 13:06:12.951260 18282 tablet_service.cc:1511] Processing CreateTablet for tablet fd44efaf24c849de9baeeb9a97271dbd (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:06:12.955674 18282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd44efaf24c849de9baeeb9a97271dbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.957161 18280 tablet_service.cc:1511] Processing CreateTablet for tablet a1a834c742b84d70859baac3a6ff751a (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:06:12.965044 18280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1a834c742b84d70859baac3a6ff751a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.965224 18134 tablet_service.cc:1511] Processing CreateTablet for tablet 490ba2fed8234928a81ff40ca50e58c4 (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:06:12.962515 18283 tablet_service.cc:1511] Processing CreateTablet for tablet fd3970d05a344a0bb83eb2970cd735e6 (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:06:12.967005 18134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 490ba2fed8234928a81ff40ca50e58c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.966866 18281 tablet_service.cc:1511] Processing CreateTablet for tablet b45d6ca8b05141749dbacd2eaaf9eb49 (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:06:12.967465 18283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd3970d05a344a0bb83eb2970cd735e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.968271 18281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b45d6ca8b05141749dbacd2eaaf9eb49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.977905 18279 tablet_service.cc:1511] Processing CreateTablet for tablet 8befc400255448babc5d5b6432d74740 (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:06:12.963421 18332 tablet_bootstrap.cc:654] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:12.979733 18279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8befc400255448babc5d5b6432d74740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.984042 18133 tablet_service.cc:1511] Processing CreateTablet for tablet 308eb63baa0f4f51b9116e3b240b0932 (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:06:12.986025 18133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 308eb63baa0f4f51b9116e3b240b0932. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.997733 18278 tablet_service.cc:1511] Processing CreateTablet for tablet a4a89e1ce0e14a47848477495dc4bb7b (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:06:12.999300 18278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4a89e1ce0e14a47848477495dc4bb7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.999436 18132 tablet_service.cc:1511] Processing CreateTablet for tablet 5214f540f4d64609b515bb6b540791ec (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:06:13.002485 18131 tablet_service.cc:1511] Processing CreateTablet for tablet 24794fe9a88c4959aa0101a5e612121f (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:06:13.002985 18127 tablet_service.cc:1511] Processing CreateTablet for tablet b45d6ca8b05141749dbacd2eaaf9eb49 (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:06:13.005565 18126 tablet_service.cc:1511] Processing CreateTablet for tablet 8befc400255448babc5d5b6432d74740 (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:06:13.006523 18129 tablet_service.cc:1511] Processing CreateTablet for tablet a1a834c742b84d70859baac3a6ff751a (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:06:13.007642 18125 tablet_service.cc:1511] Processing CreateTablet for tablet a4a89e1ce0e14a47848477495dc4bb7b (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:06:13.008536 18130 tablet_service.cc:1511] Processing CreateTablet for tablet fd44efaf24c849de9baeeb9a97271dbd (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:06:13.005035 18128 tablet_service.cc:1511] Processing CreateTablet for tablet fd3970d05a344a0bb83eb2970cd735e6 (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:06:13.010272 18132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5214f540f4d64609b515bb6b540791ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.014153 18332 tablet_bootstrap.cc:492] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a: No bootstrap required, opened a new log
I20260709 13:06:13.014367 18125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4a89e1ce0e14a47848477495dc4bb7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.014773 18332 ts_tablet_manager.cc:1403] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a: Time spent bootstrapping tablet: real 0.064s	user 0.020s	sys 0.005s
I20260709 13:06:13.018054 18129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1a834c742b84d70859baac3a6ff751a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.017988 18332 raft_consensus.cc:359] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.018777 18332 raft_consensus.cc:385] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.019086 18332 raft_consensus.cc:740] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac455e76c74d4abd95c4d73003ca9d3a, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.020215 18332 consensus_queue.cc:260] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.022809 18209 tablet_service.cc:1511] Processing CreateTablet for tablet 490ba2fed8234928a81ff40ca50e58c4 (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:06:13.024592 18209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 490ba2fed8234928a81ff40ca50e58c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.028259 18207 tablet_service.cc:1511] Processing CreateTablet for tablet 5214f540f4d64609b515bb6b540791ec (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:06:13.030483 18207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5214f540f4d64609b515bb6b540791ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.031826 18205 tablet_service.cc:1511] Processing CreateTablet for tablet fd44efaf24c849de9baeeb9a97271dbd (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:06:13.034535 18208 tablet_service.cc:1511] Processing CreateTablet for tablet 308eb63baa0f4f51b9116e3b240b0932 (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:06:13.037693 18205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd44efaf24c849de9baeeb9a97271dbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.041621 18206 tablet_service.cc:1511] Processing CreateTablet for tablet 24794fe9a88c4959aa0101a5e612121f (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:06:13.043241 18206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24794fe9a88c4959aa0101a5e612121f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.029234 18204 tablet_service.cc:1511] Processing CreateTablet for tablet a1a834c742b84d70859baac3a6ff751a (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:06:13.049793 18208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 308eb63baa0f4f51b9116e3b240b0932. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.050897 18204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1a834c742b84d70859baac3a6ff751a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.053457 18203 tablet_service.cc:1511] Processing CreateTablet for tablet fd3970d05a344a0bb83eb2970cd735e6 (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:06:13.056022 18203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd3970d05a344a0bb83eb2970cd735e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.049242 18126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8befc400255448babc5d5b6432d74740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.058363 18130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd44efaf24c849de9baeeb9a97271dbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.076184 18131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24794fe9a88c4959aa0101a5e612121f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.077805 18200 tablet_service.cc:1511] Processing CreateTablet for tablet a4a89e1ce0e14a47848477495dc4bb7b (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:06:13.079057 18202 tablet_service.cc:1511] Processing CreateTablet for tablet b45d6ca8b05141749dbacd2eaaf9eb49 (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:06:13.077818 18201 tablet_service.cc:1511] Processing CreateTablet for tablet 8befc400255448babc5d5b6432d74740 (DEFAULT_TABLE table=test-workload [id=7279e5df137a486fad51011613f84af6]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:06:13.080967 18201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8befc400255448babc5d5b6432d74740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.083562 18202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b45d6ca8b05141749dbacd2eaaf9eb49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.085793 18127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b45d6ca8b05141749dbacd2eaaf9eb49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.087569 18128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd3970d05a344a0bb83eb2970cd735e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.102838 18200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4a89e1ce0e14a47848477495dc4bb7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.121842 18332 ts_tablet_manager.cc:1434] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a: Time spent starting tablet: real 0.107s	user 0.018s	sys 0.029s
I20260709 13:06:13.122920 18318 heartbeater.cc:499] Master 127.6.255.62:37899 was elected leader, sending a full tablet report...
I20260709 13:06:13.123391 18332 tablet_bootstrap.cc:492] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a: Bootstrap starting.
I20260709 13:06:13.130884 18332 tablet_bootstrap.cc:654] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.148689 18335 tablet_bootstrap.cc:492] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap starting.
I20260709 13:06:13.160120 18336 tablet_bootstrap.cc:492] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8: Bootstrap starting.
I20260709 13:06:13.182145 18334 raft_consensus.cc:493] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.193887 18335 tablet_bootstrap.cc:654] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.182989 18334 raft_consensus.cc:515] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.202096 18334 leader_election.cc:290] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515)
I20260709 13:06:13.206029 18336 tablet_bootstrap.cc:654] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.233362 18144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5214f540f4d64609b515bb6b540791ec" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "500e80cda2474f66b0ad65f4bfa00258" is_pre_election: true
I20260709 13:06:13.236654 18335 tablet_bootstrap.cc:492] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258: No bootstrap required, opened a new log
I20260709 13:06:13.237270 18335 ts_tablet_manager.cc:1403] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258: Time spent bootstrapping tablet: real 0.089s	user 0.023s	sys 0.004s
W20260709 13:06:13.240763 18257 leader_election.cc:343] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.241092 18335 raft_consensus.cc:359] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } }
I20260709 13:06:13.242386 18335 raft_consensus.cc:385] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.243134 18335 raft_consensus.cc:740] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.244885 18332 tablet_bootstrap.cc:492] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a: No bootstrap required, opened a new log
I20260709 13:06:13.244910 18335 consensus_queue.cc:260] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } }
I20260709 13:06:13.247802 18336 tablet_bootstrap.cc:492] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8: No bootstrap required, opened a new log
I20260709 13:06:13.248447 18219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5214f540f4d64609b515bb6b540791ec" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8" is_pre_election: true
I20260709 13:06:13.248636 18169 heartbeater.cc:499] Master 127.6.255.62:37899 was elected leader, sending a full tablet report...
I20260709 13:06:13.245688 18332 ts_tablet_manager.cc:1403] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a: Time spent bootstrapping tablet: real 0.123s	user 0.008s	sys 0.012s
I20260709 13:06:13.250101 18335 ts_tablet_manager.cc:1434] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.004s
W20260709 13:06:13.250634 18257 leader_election.cc:343] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.251137 18257 leader_election.cc:304] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac455e76c74d4abd95c4d73003ca9d3a; no voters: 3272b165a59e4467925ea233bbb5f6f8, 500e80cda2474f66b0ad65f4bfa00258
I20260709 13:06:13.251467 18335 tablet_bootstrap.cc:492] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap starting.
I20260709 13:06:13.252348 18334 raft_consensus.cc:2749] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:13.248435 18336 ts_tablet_manager.cc:1403] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8: Time spent bootstrapping tablet: real 0.089s	user 0.018s	sys 0.012s
I20260709 13:06:13.254187 18332 raft_consensus.cc:359] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.255023 18332 raft_consensus.cc:385] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.255321 18332 raft_consensus.cc:740] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac455e76c74d4abd95c4d73003ca9d3a, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.256095 18332 consensus_queue.cc:260] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.258488 18332 ts_tablet_manager.cc:1434] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.005s
I20260709 13:06:13.256870 18336 raft_consensus.cc:359] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } }
I20260709 13:06:13.259389 18336 raft_consensus.cc:385] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.259625 18332 tablet_bootstrap.cc:492] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a: Bootstrap starting.
I20260709 13:06:13.260113 18336 raft_consensus.cc:740] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3272b165a59e4467925ea233bbb5f6f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.260375 18335 tablet_bootstrap.cc:654] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.261488 18336 consensus_queue.cc:260] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } }
I20260709 13:06:13.268366 18332 tablet_bootstrap.cc:654] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:13.272979 18319 tablet.cc:2406] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:06:13.281638 18332 tablet_bootstrap.cc:492] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a: No bootstrap required, opened a new log
I20260709 13:06:13.282254 18332 ts_tablet_manager.cc:1403] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.003s
I20260709 13:06:13.286222 18332 raft_consensus.cc:359] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.287050 18332 raft_consensus.cc:385] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.287582 18332 raft_consensus.cc:740] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac455e76c74d4abd95c4d73003ca9d3a, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.288844 18332 consensus_queue.cc:260] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.292716 18332 ts_tablet_manager.cc:1434] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 13:06:13.294018 18332 tablet_bootstrap.cc:492] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a: Bootstrap starting.
I20260709 13:06:13.299099 18335 tablet_bootstrap.cc:492] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258: No bootstrap required, opened a new log
I20260709 13:06:13.299764 18335 ts_tablet_manager.cc:1403] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258: Time spent bootstrapping tablet: real 0.049s	user 0.016s	sys 0.029s
I20260709 13:06:13.300242 18336 ts_tablet_manager.cc:1434] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8: Time spent starting tablet: real 0.047s	user 0.018s	sys 0.022s
I20260709 13:06:13.301040 18332 tablet_bootstrap.cc:654] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.303085 18335 raft_consensus.cc:359] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.304034 18335 raft_consensus.cc:385] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.304700 18244 heartbeater.cc:499] Master 127.6.255.62:37899 was elected leader, sending a full tablet report...
I20260709 13:06:13.304744 18335 raft_consensus.cc:740] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.306034 18335 consensus_queue.cc:260] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.311379 18335 ts_tablet_manager.cc:1434] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:06:13.313145 18332 tablet_bootstrap.cc:492] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a: No bootstrap required, opened a new log
I20260709 13:06:13.313628 18336 tablet_bootstrap.cc:492] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8: Bootstrap starting.
I20260709 13:06:13.313803 18332 ts_tablet_manager.cc:1403] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.007s
I20260709 13:06:13.317117 18332 raft_consensus.cc:359] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } }
I20260709 13:06:13.318202 18332 raft_consensus.cc:385] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.318317 18335 tablet_bootstrap.cc:492] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap starting.
I20260709 13:06:13.318553 18332 raft_consensus.cc:740] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac455e76c74d4abd95c4d73003ca9d3a, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.320209 18332 consensus_queue.cc:260] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } }
I20260709 13:06:13.322244 18336 tablet_bootstrap.cc:654] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.323553 18332 ts_tablet_manager.cc:1434] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.001s
I20260709 13:06:13.328083 18335 tablet_bootstrap.cc:654] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.330184 18332 tablet_bootstrap.cc:492] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a: Bootstrap starting.
W20260709 13:06:13.334390 18090 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:13.338294 18336 tablet_bootstrap.cc:492] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8: No bootstrap required, opened a new log
I20260709 13:06:13.339012 18336 ts_tablet_manager.cc:1403] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.012s
I20260709 13:06:13.340559 18332 tablet_bootstrap.cc:654] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.342371 18335 tablet_bootstrap.cc:492] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258: No bootstrap required, opened a new log
I20260709 13:06:13.342944 18335 ts_tablet_manager.cc:1403] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.004s
I20260709 13:06:13.342602 18336 raft_consensus.cc:359] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.343441 18336 raft_consensus.cc:385] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.343766 18336 raft_consensus.cc:740] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3272b165a59e4467925ea233bbb5f6f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.344980 18336 consensus_queue.cc:260] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.350973 18335 raft_consensus.cc:359] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.352022 18335 raft_consensus.cc:385] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.352316 18335 raft_consensus.cc:740] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.353083 18342 raft_consensus.cc:493] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.353659 18342 raft_consensus.cc:515] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } }
I20260709 13:06:13.353760 18335 consensus_queue.cc:260] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.355636 18332 tablet_bootstrap.cc:492] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a: No bootstrap required, opened a new log
I20260709 13:06:13.355985 18335 ts_tablet_manager.cc:1434] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:06:13.356387 18332 ts_tablet_manager.cc:1403] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.003s
I20260709 13:06:13.357151 18335 tablet_bootstrap.cc:492] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap starting.
I20260709 13:06:13.360199 18332 raft_consensus.cc:359] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.362233 18332 raft_consensus.cc:385] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.362596 18332 raft_consensus.cc:740] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac455e76c74d4abd95c4d73003ca9d3a, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.363720 18332 consensus_queue.cc:260] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.366560 18335 tablet_bootstrap.cc:654] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.367341 18342 leader_election.cc:290] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323), 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331)
I20260709 13:06:13.371117 18332 ts_tablet_manager.cc:1434] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a: Time spent starting tablet: real 0.014s	user 0.011s	sys 0.001s
I20260709 13:06:13.372615 18332 tablet_bootstrap.cc:492] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a: Bootstrap starting.
I20260709 13:06:13.379226 18336 ts_tablet_manager.cc:1434] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.002s
I20260709 13:06:13.380720 18336 tablet_bootstrap.cc:492] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8: Bootstrap starting.
I20260709 13:06:13.381280 18332 tablet_bootstrap.cc:654] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.389894 18334 raft_consensus.cc:493] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.390552 18334 raft_consensus.cc:515] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } }
I20260709 13:06:13.393312 18334 leader_election.cc:290] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515), 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331)
I20260709 13:06:13.395258 18144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490ba2fed8234928a81ff40ca50e58c4" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "500e80cda2474f66b0ad65f4bfa00258" is_pre_election: true
I20260709 13:06:13.396070 18144 raft_consensus.cc:2468] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac455e76c74d4abd95c4d73003ca9d3a in term 0.
I20260709 13:06:13.397521 18257 leader_election.cc:304] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 500e80cda2474f66b0ad65f4bfa00258, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 
I20260709 13:06:13.398542 18334 raft_consensus.cc:2804] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:13.398962 18334 raft_consensus.cc:493] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.399320 18334 raft_consensus.cc:3060] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.400283 18219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490ba2fed8234928a81ff40ca50e58c4" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8" is_pre_election: true
I20260709 13:06:13.401053 18219 raft_consensus.cc:2468] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac455e76c74d4abd95c4d73003ca9d3a in term 0.
I20260709 13:06:13.408634 18334 raft_consensus.cc:515] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } }
I20260709 13:06:13.411994 18219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490ba2fed8234928a81ff40ca50e58c4" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
I20260709 13:06:13.412971 18219 raft_consensus.cc:3060] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.416695 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490ba2fed8234928a81ff40ca50e58c4" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" is_pre_election: true
I20260709 13:06:13.417616 18144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490ba2fed8234928a81ff40ca50e58c4" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "500e80cda2474f66b0ad65f4bfa00258"
I20260709 13:06:13.417910 18293 raft_consensus.cc:2393] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3272b165a59e4467925ea233bbb5f6f8 in current term 1: Already voted for candidate ac455e76c74d4abd95c4d73003ca9d3a in this term.
I20260709 13:06:13.418274 18144 raft_consensus.cc:3060] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.423079 18219 raft_consensus.cc:2468] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac455e76c74d4abd95c4d73003ca9d3a in term 1.
I20260709 13:06:13.424556 18257 leader_election.cc:304] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 
I20260709 13:06:13.426856 18144 raft_consensus.cc:2468] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac455e76c74d4abd95c4d73003ca9d3a in term 1.
I20260709 13:06:13.428148 18336 tablet_bootstrap.cc:654] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.428819 18334 leader_election.cc:290] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 1 election: Requested vote from peers 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515), 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331)
I20260709 13:06:13.429553 18334 raft_consensus.cc:2804] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:13.430076 18334 raft_consensus.cc:697] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 LEADER]: Becoming Leader. State: Replica: ac455e76c74d4abd95c4d73003ca9d3a, State: Running, Role: LEADER
I20260709 13:06:13.431030 18334 consensus_queue.cc:237] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } }
I20260709 13:06:13.446678 18335 tablet_bootstrap.cc:492] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258: No bootstrap required, opened a new log
I20260709 13:06:13.447674 18332 tablet_bootstrap.cc:492] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a: No bootstrap required, opened a new log
I20260709 13:06:13.447726 18335 ts_tablet_manager.cc:1403] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258: Time spent bootstrapping tablet: real 0.091s	user 0.056s	sys 0.031s
I20260709 13:06:13.449560 18353 raft_consensus.cc:493] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.448249 18332 ts_tablet_manager.cc:1403] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a: Time spent bootstrapping tablet: real 0.076s	user 0.009s	sys 0.017s
I20260709 13:06:13.450096 18353 raft_consensus.cc:515] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.452407 18335 raft_consensus.cc:359] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.452724 18144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490ba2fed8234928a81ff40ca50e58c4" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "500e80cda2474f66b0ad65f4bfa00258" is_pre_election: true
I20260709 13:06:13.453764 18335 raft_consensus.cc:385] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.453781 18336 tablet_bootstrap.cc:492] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8: No bootstrap required, opened a new log
I20260709 13:06:13.454043 18144 raft_consensus.cc:2393] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3272b165a59e4467925ea233bbb5f6f8 in current term 1: Already voted for candidate ac455e76c74d4abd95c4d73003ca9d3a in this term.
I20260709 13:06:13.454295 18335 raft_consensus.cc:740] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.455545 18353 leader_election.cc:290] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515)
I20260709 13:06:13.455336 18335 consensus_queue.cc:260] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.455638 18183 leader_election.cc:304] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8; no voters: 500e80cda2474f66b0ad65f4bfa00258, ac455e76c74d4abd95c4d73003ca9d3a
I20260709 13:06:13.456501 18143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24794fe9a88c4959aa0101a5e612121f" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "500e80cda2474f66b0ad65f4bfa00258" is_pre_election: true
I20260709 13:06:13.457129 18342 raft_consensus.cc:2749] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:13.458081 18335 ts_tablet_manager.cc:1434] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.001s
I20260709 13:06:13.458283 18336 ts_tablet_manager.cc:1403] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8: Time spent bootstrapping tablet: real 0.078s	user 0.008s	sys 0.005s
I20260709 13:06:13.459437 18335 tablet_bootstrap.cc:492] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap starting.
I20260709 13:06:13.459784 18219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24794fe9a88c4959aa0101a5e612121f" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8" is_pre_election: true
I20260709 13:06:13.462525 18332 raft_consensus.cc:359] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.463392 18332 raft_consensus.cc:385] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.463675 18332 raft_consensus.cc:740] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac455e76c74d4abd95c4d73003ca9d3a, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.464244 18336 raft_consensus.cc:359] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.464417 18332 consensus_queue.cc:260] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.465036 18336 raft_consensus.cc:385] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.465351 18336 raft_consensus.cc:740] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3272b165a59e4467925ea233bbb5f6f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.466881 18332 ts_tablet_manager.cc:1434] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:06:13.466979 18335 tablet_bootstrap.cc:654] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.468063 18332 tablet_bootstrap.cc:492] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a: Bootstrap starting.
I20260709 13:06:13.468894 18336 consensus_queue.cc:260] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
W20260709 13:06:13.469790 18257 leader_election.cc:343] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:13.470811 18257 leader_election.cc:343] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.471284 18257 leader_election.cc:304] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac455e76c74d4abd95c4d73003ca9d3a; no voters: 3272b165a59e4467925ea233bbb5f6f8, 500e80cda2474f66b0ad65f4bfa00258
I20260709 13:06:13.474318 18336 ts_tablet_manager.cc:1434] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:06:13.475605 18353 raft_consensus.cc:2749] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:13.475770 18336 tablet_bootstrap.cc:492] T 5214f540f4d64609b515bb6b540791ec P 3272b165a59e4467925ea233bbb5f6f8: Bootstrap starting.
I20260709 13:06:13.478415 18332 tablet_bootstrap.cc:654] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.486415 18336 tablet_bootstrap.cc:654] T 5214f540f4d64609b515bb6b540791ec P 3272b165a59e4467925ea233bbb5f6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.487627 18341 raft_consensus.cc:493] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.488163 18341 raft_consensus.cc:515] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.503574 18332 tablet_bootstrap.cc:492] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a: No bootstrap required, opened a new log
I20260709 13:06:13.504447 18332 ts_tablet_manager.cc:1403] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a: Time spent bootstrapping tablet: real 0.037s	user 0.016s	sys 0.015s
I20260709 13:06:13.493669 18341 leader_election.cc:290] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:13.509119 18332 raft_consensus.cc:359] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.504256 18039 catalog_manager.cc:5697] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a reported cstate change: term changed from 0 to 1, leader changed from <none> to ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3). New cstate: current_term: 1 leader_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:13.510239 18332 raft_consensus.cc:385] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.510946 18332 raft_consensus.cc:740] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac455e76c74d4abd95c4d73003ca9d3a, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.513540 18332 consensus_queue.cc:260] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.515630 18332 ts_tablet_manager.cc:1434] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 13:06:13.517692 18332 tablet_bootstrap.cc:492] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a: Bootstrap starting.
I20260709 13:06:13.520722 18335 tablet_bootstrap.cc:492] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258: No bootstrap required, opened a new log
I20260709 13:06:13.521394 18335 ts_tablet_manager.cc:1403] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258: Time spent bootstrapping tablet: real 0.062s	user 0.027s	sys 0.015s
I20260709 13:06:13.525954 18332 tablet_bootstrap.cc:654] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.524794 18335 raft_consensus.cc:359] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.528959 18335 raft_consensus.cc:385] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.529374 18335 raft_consensus.cc:740] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.530398 18335 consensus_queue.cc:260] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.535765 18332 tablet_bootstrap.cc:492] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a: No bootstrap required, opened a new log
I20260709 13:06:13.536339 18332 ts_tablet_manager.cc:1403] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.003s
I20260709 13:06:13.536989 18335 ts_tablet_manager.cc:1434] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.004s
I20260709 13:06:13.538352 18335 tablet_bootstrap.cc:492] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap starting.
I20260709 13:06:13.539439 18332 raft_consensus.cc:359] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.540190 18332 raft_consensus.cc:385] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.540472 18332 raft_consensus.cc:740] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac455e76c74d4abd95c4d73003ca9d3a, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.541464 18332 consensus_queue.cc:260] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.544009 18332 ts_tablet_manager.cc:1434] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.001s
I20260709 13:06:13.545331 18332 tablet_bootstrap.cc:492] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a: Bootstrap starting.
I20260709 13:06:13.546583 18335 tablet_bootstrap.cc:654] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.546458 18219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308eb63baa0f4f51b9116e3b240b0932" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8" is_pre_election: true
W20260709 13:06:13.548209 18108 leader_election.cc:343] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.549155 18336 tablet_bootstrap.cc:492] T 5214f540f4d64609b515bb6b540791ec P 3272b165a59e4467925ea233bbb5f6f8: No bootstrap required, opened a new log
I20260709 13:06:13.548991 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308eb63baa0f4f51b9116e3b240b0932" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" is_pre_election: true
I20260709 13:06:13.549763 18293 raft_consensus.cc:2468] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 500e80cda2474f66b0ad65f4bfa00258 in term 0.
I20260709 13:06:13.551134 18107 leader_election.cc:304] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 500e80cda2474f66b0ad65f4bfa00258, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 3272b165a59e4467925ea233bbb5f6f8
I20260709 13:06:13.551990 18341 raft_consensus.cc:2804] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:13.552332 18341 raft_consensus.cc:493] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.552672 18341 raft_consensus.cc:3060] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.554544 18335 tablet_bootstrap.cc:492] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258: No bootstrap required, opened a new log
I20260709 13:06:13.549775 18336 ts_tablet_manager.cc:1403] T 5214f540f4d64609b515bb6b540791ec P 3272b165a59e4467925ea233bbb5f6f8: Time spent bootstrapping tablet: real 0.074s	user 0.016s	sys 0.009s
I20260709 13:06:13.555425 18335 ts_tablet_manager.cc:1403] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.000s
I20260709 13:06:13.558908 18332 tablet_bootstrap.cc:654] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.559367 18335 raft_consensus.cc:359] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.560261 18335 raft_consensus.cc:385] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.560508 18341 raft_consensus.cc:515] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.561151 18335 raft_consensus.cc:740] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.562435 18336 raft_consensus.cc:359] T 5214f540f4d64609b515bb6b540791ec P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.563185 18336 raft_consensus.cc:385] T 5214f540f4d64609b515bb6b540791ec P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.563547 18336 raft_consensus.cc:740] T 5214f540f4d64609b515bb6b540791ec P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3272b165a59e4467925ea233bbb5f6f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.562819 18335 consensus_queue.cc:260] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.564149 18219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308eb63baa0f4f51b9116e3b240b0932" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
I20260709 13:06:13.565583 18342 raft_consensus.cc:493] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:13.565682 18108 leader_election.cc:343] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.566196 18342 raft_consensus.cc:515] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.567871 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308eb63baa0f4f51b9116e3b240b0932" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
I20260709 13:06:13.568364 18342 leader_election.cc:290] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:13.568477 18293 raft_consensus.cc:3060] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.564514 18336 consensus_queue.cc:260] T 5214f540f4d64609b515bb6b540791ec P 3272b165a59e4467925ea233bbb5f6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.570822 18143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1a834c742b84d70859baac3a6ff751a" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "500e80cda2474f66b0ad65f4bfa00258" is_pre_election: true
I20260709 13:06:13.572943 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1a834c742b84d70859baac3a6ff751a" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" is_pre_election: true
I20260709 13:06:13.573912 18292 raft_consensus.cc:2468] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3272b165a59e4467925ea233bbb5f6f8 in term 0.
I20260709 13:06:13.574596 18336 ts_tablet_manager.cc:1434] T 5214f540f4d64609b515bb6b540791ec P 3272b165a59e4467925ea233bbb5f6f8: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:06:13.575734 18336 tablet_bootstrap.cc:492] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8: Bootstrap starting.
W20260709 13:06:13.575651 18183 leader_election.cc:343] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.577673 18182 leader_election.cc:304] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 500e80cda2474f66b0ad65f4bfa00258
I20260709 13:06:13.578401 18293 raft_consensus.cc:2468] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 500e80cda2474f66b0ad65f4bfa00258 in term 1.
I20260709 13:06:13.579581 18342 raft_consensus.cc:2804] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:13.580170 18342 raft_consensus.cc:493] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.579952 18107 leader_election.cc:304] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 500e80cda2474f66b0ad65f4bfa00258, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 3272b165a59e4467925ea233bbb5f6f8
I20260709 13:06:13.580816 18342 raft_consensus.cc:3060] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.581784 18341 leader_election.cc:290] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 election: Requested vote from peers 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:13.582513 18341 raft_consensus.cc:2804] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:13.583094 18341 raft_consensus.cc:697] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 LEADER]: Becoming Leader. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Running, Role: LEADER
I20260709 13:06:13.584820 18341 consensus_queue.cc:237] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.595388 18332 tablet_bootstrap.cc:492] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a: No bootstrap required, opened a new log
I20260709 13:06:13.596401 18332 ts_tablet_manager.cc:1403] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a: Time spent bootstrapping tablet: real 0.052s	user 0.016s	sys 0.000s
I20260709 13:06:13.596578 18335 ts_tablet_manager.cc:1434] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258: Time spent starting tablet: real 0.041s	user 0.015s	sys 0.023s
I20260709 13:06:13.598119 18335 tablet_bootstrap.cc:492] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap starting.
I20260709 13:06:13.599628 18332 raft_consensus.cc:359] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.600410 18332 raft_consensus.cc:385] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.600687 18332 raft_consensus.cc:740] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac455e76c74d4abd95c4d73003ca9d3a, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.603647 18336 tablet_bootstrap.cc:654] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.606137 18335 tablet_bootstrap.cc:654] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.601459 18332 consensus_queue.cc:260] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.608942 18332 ts_tablet_manager.cc:1434] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:06:13.609931 18332 tablet_bootstrap.cc:492] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a: Bootstrap starting.
I20260709 13:06:13.610977 18373 raft_consensus.cc:493] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.611476 18373 raft_consensus.cc:515] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.614262 18373 leader_election.cc:290] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:13.616693 18341 raft_consensus.cc:493] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.617274 18341 raft_consensus.cc:515] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.618818 18335 tablet_bootstrap.cc:492] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258: No bootstrap required, opened a new log
I20260709 13:06:13.619004 18332 tablet_bootstrap.cc:654] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.619545 18335 ts_tablet_manager.cc:1403] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.007s
I20260709 13:06:13.619537 18341 leader_election.cc:290] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:13.620397 18143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4a89e1ce0e14a47848477495dc4bb7b" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "500e80cda2474f66b0ad65f4bfa00258" is_pre_election: true
W20260709 13:06:13.624228 18183 leader_election.cc:343] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.624421 18219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8befc400255448babc5d5b6432d74740" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8" is_pre_election: true
I20260709 13:06:13.613322 18040 catalog_manager.cc:5697] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 reported cstate change: term changed from 0 to 1, leader changed from <none> to 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "500e80cda2474f66b0ad65f4bfa00258" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:13.626255 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8befc400255448babc5d5b6432d74740" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" is_pre_election: true
I20260709 13:06:13.626905 18293 raft_consensus.cc:2468] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 500e80cda2474f66b0ad65f4bfa00258 in term 0.
I20260709 13:06:13.628679 18107 leader_election.cc:304] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 500e80cda2474f66b0ad65f4bfa00258, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 
I20260709 13:06:13.624863 18335 raft_consensus.cc:359] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.629570 18335 raft_consensus.cc:385] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.629791 18341 raft_consensus.cc:2804] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:13.629921 18335 raft_consensus.cc:740] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.630272 18341 raft_consensus.cc:493] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.630924 18341 raft_consensus.cc:3060] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.631006 18335 consensus_queue.cc:260] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
W20260709 13:06:13.632511 18108 leader_election.cc:343] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.633472 18335 ts_tablet_manager.cc:1434] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258: Time spent starting tablet: real 0.013s	user 0.002s	sys 0.002s
I20260709 13:06:13.634582 18335 tablet_bootstrap.cc:492] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap starting.
I20260709 13:06:13.634804 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4a89e1ce0e14a47848477495dc4bb7b" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" is_pre_election: true
I20260709 13:06:13.635466 18293 raft_consensus.cc:2468] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3272b165a59e4467925ea233bbb5f6f8 in term 0.
I20260709 13:06:13.636850 18182 leader_election.cc:304] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 500e80cda2474f66b0ad65f4bfa00258
I20260709 13:06:13.638387 18373 raft_consensus.cc:2804] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:13.638769 18373 raft_consensus.cc:493] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.639117 18373 raft_consensus.cc:3060] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.639859 18336 tablet_bootstrap.cc:492] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8: No bootstrap required, opened a new log
I20260709 13:06:13.640515 18336 ts_tablet_manager.cc:1403] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8: Time spent bootstrapping tablet: real 0.065s	user 0.017s	sys 0.005s
I20260709 13:06:13.641678 18335 tablet_bootstrap.cc:654] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.644174 18368 raft_consensus.cc:493] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.643980 18342 raft_consensus.cc:515] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.645100 18368 raft_consensus.cc:515] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.647871 18368 leader_election.cc:290] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:13.648689 18342 leader_election.cc:290] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 1 election: Requested vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:13.649345 18219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5214f540f4d64609b515bb6b540791ec" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8" is_pre_election: true
I20260709 13:06:13.649945 18219 raft_consensus.cc:2468] T 5214f540f4d64609b515bb6b540791ec P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 500e80cda2474f66b0ad65f4bfa00258 in term 0.
I20260709 13:06:13.650797 18143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1a834c742b84d70859baac3a6ff751a" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "500e80cda2474f66b0ad65f4bfa00258"
I20260709 13:06:13.651010 18108 leader_election.cc:304] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, 500e80cda2474f66b0ad65f4bfa00258; no voters: 
I20260709 13:06:13.651551 18373 raft_consensus.cc:515] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.651895 18368 raft_consensus.cc:2804] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:06:13.652273 18183 leader_election.cc:343] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.652437 18368 raft_consensus.cc:493] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.652909 18368 raft_consensus.cc:3060] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.652365 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1a834c742b84d70859baac3a6ff751a" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
I20260709 13:06:13.654246 18292 raft_consensus.cc:3060] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.652963 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5214f540f4d64609b515bb6b540791ec" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" is_pre_election: true
I20260709 13:06:13.655745 18293 raft_consensus.cc:2468] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 500e80cda2474f66b0ad65f4bfa00258 in term 0.
I20260709 13:06:13.665764 18143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4a89e1ce0e14a47848477495dc4bb7b" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "500e80cda2474f66b0ad65f4bfa00258"
I20260709 13:06:13.666358 18368 raft_consensus.cc:515] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.668357 18373 leader_election.cc:290] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 1 election: Requested vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
W20260709 13:06:13.669078 18183 leader_election.cc:343] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.669373 18368 leader_election.cc:290] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 election: Requested vote from peers 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:13.670159 18219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5214f540f4d64609b515bb6b540791ec" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
I20260709 13:06:13.670686 18219 raft_consensus.cc:3060] T 5214f540f4d64609b515bb6b540791ec P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.671330 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4a89e1ce0e14a47848477495dc4bb7b" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
I20260709 13:06:13.671545 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5214f540f4d64609b515bb6b540791ec" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
I20260709 13:06:13.671986 18293 raft_consensus.cc:3060] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.672113 18291 raft_consensus.cc:3060] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.661379 18341 raft_consensus.cc:515] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.676497 18341 leader_election.cc:290] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 election: Requested vote from peers 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:13.676599 18218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8befc400255448babc5d5b6432d74740" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
I20260709 13:06:13.664808 18336 raft_consensus.cc:359] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.680281 18336 raft_consensus.cc:385] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.678309 18219 raft_consensus.cc:2468] T 5214f540f4d64609b515bb6b540791ec P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 500e80cda2474f66b0ad65f4bfa00258 in term 1.
I20260709 13:06:13.680626 18336 raft_consensus.cc:740] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3272b165a59e4467925ea233bbb5f6f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.681653 18290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8befc400255448babc5d5b6432d74740" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
I20260709 13:06:13.681764 18108 leader_election.cc:304] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, 500e80cda2474f66b0ad65f4bfa00258; no voters: 
I20260709 13:06:13.682379 18290 raft_consensus.cc:3060] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.682989 18341 raft_consensus.cc:2804] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:13.666338 18292 raft_consensus.cc:2468] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3272b165a59e4467925ea233bbb5f6f8 in term 1.
I20260709 13:06:13.683655 18291 raft_consensus.cc:2468] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 500e80cda2474f66b0ad65f4bfa00258 in term 1.
I20260709 13:06:13.684077 18341 raft_consensus.cc:697] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [term 1 LEADER]: Becoming Leader. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Running, Role: LEADER
I20260709 13:06:13.684989 18182 leader_election.cc:304] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 500e80cda2474f66b0ad65f4bfa00258
I20260709 13:06:13.685156 18336 consensus_queue.cc:260] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.685756 18341 consensus_queue.cc:237] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
W20260709 13:06:13.686719 18108 leader_election.cc:343] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.688347 18336 ts_tablet_manager.cc:1434] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8: Time spent starting tablet: real 0.047s	user 0.009s	sys 0.001s
I20260709 13:06:13.692855 18332 tablet_bootstrap.cc:492] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a: No bootstrap required, opened a new log
I20260709 13:06:13.683773 18293 raft_consensus.cc:2468] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3272b165a59e4467925ea233bbb5f6f8 in term 1.
I20260709 13:06:13.695535 18332 ts_tablet_manager.cc:1403] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a: Time spent bootstrapping tablet: real 0.086s	user 0.020s	sys 0.005s
I20260709 13:06:13.696388 18182 leader_election.cc:304] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 500e80cda2474f66b0ad65f4bfa00258
I20260709 13:06:13.697269 18342 raft_consensus.cc:2804] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:13.698824 18332 raft_consensus.cc:359] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.699427 18332 raft_consensus.cc:385] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.699681 18332 raft_consensus.cc:740] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac455e76c74d4abd95c4d73003ca9d3a, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.700541 18332 consensus_queue.cc:260] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.690838 18335 tablet_bootstrap.cc:492] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258: No bootstrap required, opened a new log
I20260709 13:06:13.702020 18335 ts_tablet_manager.cc:1403] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258: Time spent bootstrapping tablet: real 0.068s	user 0.014s	sys 0.004s
I20260709 13:06:13.704020 18335 raft_consensus.cc:359] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.704530 18335 raft_consensus.cc:385] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.704751 18335 raft_consensus.cc:740] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.705353 18335 consensus_queue.cc:260] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.706786 18332 ts_tablet_manager.cc:1434] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:06:13.707427 18335 ts_tablet_manager.cc:1434] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:06:13.694694 18290 raft_consensus.cc:2468] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 500e80cda2474f66b0ad65f4bfa00258 in term 1.
I20260709 13:06:13.708670 18335 tablet_bootstrap.cc:492] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap starting.
I20260709 13:06:13.709020 18107 leader_election.cc:304] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 500e80cda2474f66b0ad65f4bfa00258, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 3272b165a59e4467925ea233bbb5f6f8
I20260709 13:06:13.709923 18341 raft_consensus.cc:2804] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:13.692597 18336 tablet_bootstrap.cc:492] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8: Bootstrap starting.
I20260709 13:06:13.710630 18341 raft_consensus.cc:697] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 LEADER]: Becoming Leader. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Running, Role: LEADER
I20260709 13:06:13.711582 18341 consensus_queue.cc:237] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.716636 18335 tablet_bootstrap.cc:654] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.717521 18336 tablet_bootstrap.cc:654] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.734612 18335 tablet_bootstrap.cc:492] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258: No bootstrap required, opened a new log
I20260709 13:06:13.736295 18336 tablet_bootstrap.cc:492] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8: No bootstrap required, opened a new log
I20260709 13:06:13.738238 18335 ts_tablet_manager.cc:1403] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.015s
I20260709 13:06:13.729352 18040 catalog_manager.cc:5697] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 reported cstate change: term changed from 0 to 1, leader changed from <none> to 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "500e80cda2474f66b0ad65f4bfa00258" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:13.737181 18373 raft_consensus.cc:2804] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:13.739482 18336 ts_tablet_manager.cc:1403] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8: Time spent bootstrapping tablet: real 0.047s	user 0.021s	sys 0.005s
I20260709 13:06:13.741936 18335 raft_consensus.cc:359] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.742693 18335 raft_consensus.cc:385] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.742988 18335 raft_consensus.cc:740] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.743453 18336 raft_consensus.cc:359] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.744693 18336 raft_consensus.cc:385] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.745051 18336 raft_consensus.cc:740] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3272b165a59e4467925ea233bbb5f6f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.743826 18335 consensus_queue.cc:260] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.744242 18373 raft_consensus.cc:697] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [term 1 LEADER]: Becoming Leader. State: Replica: 3272b165a59e4467925ea233bbb5f6f8, State: Running, Role: LEADER
I20260709 13:06:13.746217 18336 consensus_queue.cc:260] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.746999 18373 consensus_queue.cc:237] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.749161 18342 raft_consensus.cc:697] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [term 1 LEADER]: Becoming Leader. State: Replica: 3272b165a59e4467925ea233bbb5f6f8, State: Running, Role: LEADER
I20260709 13:06:13.749341 18336 ts_tablet_manager.cc:1434] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:06:13.750170 18342 consensus_queue.cc:237] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.750643 18336 tablet_bootstrap.cc:492] T 8befc400255448babc5d5b6432d74740 P 3272b165a59e4467925ea233bbb5f6f8: Bootstrap starting.
I20260709 13:06:13.752033 18334 raft_consensus.cc:493] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.752697 18335 ts_tablet_manager.cc:1434] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:06:13.752542 18334 raft_consensus.cc:515] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.753705 18335 tablet_bootstrap.cc:492] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap starting.
I20260709 13:06:13.754637 18334 leader_election.cc:290] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515)
I20260709 13:06:13.756036 18143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44efaf24c849de9baeeb9a97271dbd" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "500e80cda2474f66b0ad65f4bfa00258" is_pre_election: true
I20260709 13:06:13.757009 18143 raft_consensus.cc:2468] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac455e76c74d4abd95c4d73003ca9d3a in term 0.
I20260709 13:06:13.756038 18218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44efaf24c849de9baeeb9a97271dbd" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8" is_pre_election: true
W20260709 13:06:13.759815 18257 leader_election.cc:343] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.760624 18257 leader_election.cc:304] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 500e80cda2474f66b0ad65f4bfa00258, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 3272b165a59e4467925ea233bbb5f6f8
I20260709 13:06:13.761610 18334 raft_consensus.cc:2804] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:13.762023 18334 raft_consensus.cc:493] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.762383 18334 raft_consensus.cc:3060] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.762435 18336 tablet_bootstrap.cc:654] T 8befc400255448babc5d5b6432d74740 P 3272b165a59e4467925ea233bbb5f6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.763278 18335 tablet_bootstrap.cc:654] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.763875 18039 catalog_manager.cc:5697] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 reported cstate change: term changed from 0 to 1, leader changed from <none> to 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "500e80cda2474f66b0ad65f4bfa00258" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:13.771163 18334 raft_consensus.cc:515] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.776986 18336 tablet_bootstrap.cc:492] T 8befc400255448babc5d5b6432d74740 P 3272b165a59e4467925ea233bbb5f6f8: No bootstrap required, opened a new log
I20260709 13:06:13.777606 18335 tablet_bootstrap.cc:492] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258: No bootstrap required, opened a new log
I20260709 13:06:13.777776 18336 ts_tablet_manager.cc:1403] T 8befc400255448babc5d5b6432d74740 P 3272b165a59e4467925ea233bbb5f6f8: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.005s
I20260709 13:06:13.778170 18335 ts_tablet_manager.cc:1403] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.000s
I20260709 13:06:13.779265 18334 leader_election.cc:290] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 1 election: Requested vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515)
I20260709 13:06:13.781227 18335 raft_consensus.cc:359] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.781980 18335 raft_consensus.cc:385] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.782531 18143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44efaf24c849de9baeeb9a97271dbd" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "500e80cda2474f66b0ad65f4bfa00258"
I20260709 13:06:13.782539 18335 raft_consensus.cc:740] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.783617 18143 raft_consensus.cc:3060] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.784253 18335 consensus_queue.cc:260] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.786917 18335 ts_tablet_manager.cc:1434] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 13:06:13.786861 18336 raft_consensus.cc:359] T 8befc400255448babc5d5b6432d74740 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.788002 18336 raft_consensus.cc:385] T 8befc400255448babc5d5b6432d74740 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.787822 18218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44efaf24c849de9baeeb9a97271dbd" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
I20260709 13:06:13.788364 18336 raft_consensus.cc:740] T 8befc400255448babc5d5b6432d74740 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3272b165a59e4467925ea233bbb5f6f8, State: Initialized, Role: FOLLOWER
W20260709 13:06:13.789402 18257 leader_election.cc:343] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.789808 18336 consensus_queue.cc:260] T 8befc400255448babc5d5b6432d74740 P 3272b165a59e4467925ea233bbb5f6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.791685 18336 ts_tablet_manager.cc:1434] T 8befc400255448babc5d5b6432d74740 P 3272b165a59e4467925ea233bbb5f6f8: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.001s
I20260709 13:06:13.792848 18336 tablet_bootstrap.cc:492] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8: Bootstrap starting.
I20260709 13:06:13.794061 18039 catalog_manager.cc:5697] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "3272b165a59e4467925ea233bbb5f6f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:13.795957 18039 catalog_manager.cc:5697] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "3272b165a59e4467925ea233bbb5f6f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:13.799177 18336 tablet_bootstrap.cc:654] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.800829 18341 raft_consensus.cc:493] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.801319 18143 raft_consensus.cc:2468] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac455e76c74d4abd95c4d73003ca9d3a in term 1.
I20260709 13:06:13.802147 18368 raft_consensus.cc:493] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.801409 18341 raft_consensus.cc:515] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.803025 18368 raft_consensus.cc:515] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.803994 18257 leader_election.cc:304] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 500e80cda2474f66b0ad65f4bfa00258, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 3272b165a59e4467925ea233bbb5f6f8
I20260709 13:06:13.804891 18341 leader_election.cc:290] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:13.805563 18368 leader_election.cc:290] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:13.805724 18334 raft_consensus.cc:2804] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:13.806221 18334 raft_consensus.cc:697] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [term 1 LEADER]: Becoming Leader. State: Replica: ac455e76c74d4abd95c4d73003ca9d3a, State: Running, Role: LEADER
I20260709 13:06:13.807189 18334 consensus_queue.cc:237] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.807461 18218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3970d05a344a0bb83eb2970cd735e6" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8" is_pre_election: true
I20260709 13:06:13.807539 18219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b45d6ca8b05141749dbacd2eaaf9eb49" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8" is_pre_election: true
I20260709 13:06:13.808185 18218 raft_consensus.cc:2468] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 500e80cda2474f66b0ad65f4bfa00258 in term 0.
I20260709 13:06:13.808284 18219 raft_consensus.cc:2468] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 500e80cda2474f66b0ad65f4bfa00258 in term 0.
I20260709 13:06:13.809341 18108 leader_election.cc:304] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, 500e80cda2474f66b0ad65f4bfa00258; no voters: 
I20260709 13:06:13.810392 18368 raft_consensus.cc:2804] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:13.810751 18368 raft_consensus.cc:493] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.810840 18108 leader_election.cc:304] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, 500e80cda2474f66b0ad65f4bfa00258; no voters: 
I20260709 13:06:13.811139 18368 raft_consensus.cc:3060] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.812014 18341 raft_consensus.cc:2804] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:13.812122 18290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3970d05a344a0bb83eb2970cd735e6" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" is_pre_election: true
I20260709 13:06:13.812482 18341 raft_consensus.cc:493] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.812866 18290 raft_consensus.cc:2468] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 500e80cda2474f66b0ad65f4bfa00258 in term 0.
I20260709 13:06:13.813234 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b45d6ca8b05141749dbacd2eaaf9eb49" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" is_pre_election: true
I20260709 13:06:13.812866 18341 raft_consensus.cc:3060] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.813902 18293 raft_consensus.cc:2468] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 500e80cda2474f66b0ad65f4bfa00258 in term 0.
I20260709 13:06:13.819664 18369 raft_consensus.cc:493] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.820478 18369 raft_consensus.cc:515] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.821712 18341 raft_consensus.cc:515] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.825440 18219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24794fe9a88c4959aa0101a5e612121f" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8" is_pre_election: true
I20260709 13:06:13.825807 18218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b45d6ca8b05141749dbacd2eaaf9eb49" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
I20260709 13:06:13.826440 18218 raft_consensus.cc:3060] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:13.827134 18108 leader_election.cc:343] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.827582 18341 leader_election.cc:290] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 election: Requested vote from peers 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:13.830197 18369 leader_election.cc:290] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:13.831694 18290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24794fe9a88c4959aa0101a5e612121f" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" is_pre_election: true
I20260709 13:06:13.832695 18290 raft_consensus.cc:2468] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 500e80cda2474f66b0ad65f4bfa00258 in term 0.
I20260709 13:06:13.832870 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b45d6ca8b05141749dbacd2eaaf9eb49" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
I20260709 13:06:13.833539 18293 raft_consensus.cc:3060] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.834395 18107 leader_election.cc:304] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 500e80cda2474f66b0ad65f4bfa00258, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 3272b165a59e4467925ea233bbb5f6f8
I20260709 13:06:13.835464 18218 raft_consensus.cc:2468] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 500e80cda2474f66b0ad65f4bfa00258 in term 1.
I20260709 13:06:13.835834 18369 raft_consensus.cc:2804] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:13.836517 18369 raft_consensus.cc:493] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.837639 18108 leader_election.cc:304] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, 500e80cda2474f66b0ad65f4bfa00258; no voters: 
I20260709 13:06:13.838115 18336 tablet_bootstrap.cc:492] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8: No bootstrap required, opened a new log
I20260709 13:06:13.838620 18369 raft_consensus.cc:3060] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.838238 18042 catalog_manager.cc:5697] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a reported cstate change: term changed from 0 to 1, leader changed from <none> to ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3). New cstate: current_term: 1 leader_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:13.838631 18336 ts_tablet_manager.cc:1403] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8: Time spent bootstrapping tablet: real 0.046s	user 0.007s	sys 0.003s
I20260709 13:06:13.841293 18293 raft_consensus.cc:2468] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 500e80cda2474f66b0ad65f4bfa00258 in term 1.
I20260709 13:06:13.842073 18368 raft_consensus.cc:515] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.844664 18368 leader_election.cc:290] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 election: Requested vote from peers 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:13.845592 18368 raft_consensus.cc:2804] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:13.845794 18218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3970d05a344a0bb83eb2970cd735e6" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
I20260709 13:06:13.846248 18368 raft_consensus.cc:697] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 LEADER]: Becoming Leader. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Running, Role: LEADER
I20260709 13:06:13.846623 18218 raft_consensus.cc:3060] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.847173 18290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3970d05a344a0bb83eb2970cd735e6" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
I20260709 13:06:13.848169 18290 raft_consensus.cc:3060] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.847131 18368 consensus_queue.cc:237] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.850404 18336 raft_consensus.cc:359] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.851243 18336 raft_consensus.cc:385] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.851498 18336 raft_consensus.cc:740] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3272b165a59e4467925ea233bbb5f6f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.852166 18336 consensus_queue.cc:260] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.854028 18336 ts_tablet_manager.cc:1434] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:06:13.857833 18369 raft_consensus.cc:515] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.859616 18336 tablet_bootstrap.cc:492] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8: Bootstrap starting.
I20260709 13:06:13.861339 18369 leader_election.cc:290] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 election: Requested vote from peers 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:13.861699 18219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24794fe9a88c4959aa0101a5e612121f" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
I20260709 13:06:13.862334 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24794fe9a88c4959aa0101a5e612121f" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
I20260709 13:06:13.862797 18219 raft_consensus.cc:3060] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.862711 18218 raft_consensus.cc:2468] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 500e80cda2474f66b0ad65f4bfa00258 in term 1.
I20260709 13:06:13.866671 18108 leader_election.cc:304] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, 500e80cda2474f66b0ad65f4bfa00258; no voters: 
I20260709 13:06:13.867479 18293 raft_consensus.cc:3060] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.870579 18369 raft_consensus.cc:2804] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:13.871599 18369 raft_consensus.cc:697] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 LEADER]: Becoming Leader. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Running, Role: LEADER
I20260709 13:06:13.872643 18369 consensus_queue.cc:237] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.876631 18293 raft_consensus.cc:2468] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 500e80cda2474f66b0ad65f4bfa00258 in term 1.
I20260709 13:06:13.878306 18336 tablet_bootstrap.cc:654] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.878268 18107 leader_election.cc:304] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 500e80cda2474f66b0ad65f4bfa00258, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 
I20260709 13:06:13.879305 18341 raft_consensus.cc:2804] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:13.879949 18341 raft_consensus.cc:697] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [term 1 LEADER]: Becoming Leader. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Running, Role: LEADER
I20260709 13:06:13.880896 18341 consensus_queue.cc:237] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.884657 18290 raft_consensus.cc:2468] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 500e80cda2474f66b0ad65f4bfa00258 in term 1.
I20260709 13:06:13.886040 18219 raft_consensus.cc:2468] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 500e80cda2474f66b0ad65f4bfa00258 in term 1.
I20260709 13:06:13.883175 18042 catalog_manager.cc:5697] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 reported cstate change: term changed from 0 to 1, leader changed from <none> to 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "500e80cda2474f66b0ad65f4bfa00258" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:13.895948 18336 tablet_bootstrap.cc:492] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8: No bootstrap required, opened a new log
I20260709 13:06:13.896713 18336 ts_tablet_manager.cc:1403] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8: Time spent bootstrapping tablet: real 0.037s	user 0.019s	sys 0.003s
I20260709 13:06:13.899176 18336 raft_consensus.cc:359] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.899801 18336 raft_consensus.cc:385] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.900007 18336 raft_consensus.cc:740] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3272b165a59e4467925ea233bbb5f6f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.900616 18336 consensus_queue.cc:260] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.902542 18336 ts_tablet_manager.cc:1434] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:06:13.903628 18336 tablet_bootstrap.cc:492] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8: Bootstrap starting.
I20260709 13:06:13.906008 18042 catalog_manager.cc:5697] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 reported cstate change: term changed from 0 to 1, leader changed from <none> to 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "500e80cda2474f66b0ad65f4bfa00258" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:13.907985 18336 tablet_bootstrap.cc:654] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.907953 18042 catalog_manager.cc:5697] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 reported cstate change: term changed from 0 to 1, leader changed from <none> to 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "500e80cda2474f66b0ad65f4bfa00258" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:13.914906 18342 raft_consensus.cc:493] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.915482 18342 raft_consensus.cc:515] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.918263 18143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44efaf24c849de9baeeb9a97271dbd" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "500e80cda2474f66b0ad65f4bfa00258" is_pre_election: true
I20260709 13:06:13.918690 18334 consensus_queue.cc:1048] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:13.919055 18143 raft_consensus.cc:2393] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3272b165a59e4467925ea233bbb5f6f8 in current term 1: Already voted for candidate ac455e76c74d4abd95c4d73003ca9d3a in this term.
I20260709 13:06:13.920207 18342 leader_election.cc:290] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:13.921659 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44efaf24c849de9baeeb9a97271dbd" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" is_pre_election: true
I20260709 13:06:13.923305 18336 tablet_bootstrap.cc:492] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8: No bootstrap required, opened a new log
I20260709 13:06:13.923384 18182 leader_election.cc:304] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8; no voters: 500e80cda2474f66b0ad65f4bfa00258, ac455e76c74d4abd95c4d73003ca9d3a
I20260709 13:06:13.924206 18336 ts_tablet_manager.cc:1403] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.005s
I20260709 13:06:13.924466 18342 raft_consensus.cc:3060] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.927155 18336 raft_consensus.cc:359] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.927883 18336 raft_consensus.cc:385] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.928122 18336 raft_consensus.cc:740] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3272b165a59e4467925ea233bbb5f6f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.928793 18336 consensus_queue.cc:260] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:13.930931 18336 ts_tablet_manager.cc:1434] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:06:13.938706 18342 raft_consensus.cc:2749] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:13.943997 18355 consensus_queue.cc:1048] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:13.988180 18341 consensus_queue.cc:1048] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:14.001732 18219 raft_consensus.cc:3060] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:14.009225 18368 consensus_queue.cc:1048] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:14.057476 18368 raft_consensus.cc:493] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.058003 18368 raft_consensus.cc:515] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:14.059720 18368 leader_election.cc:290] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:14.060643 18219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4a89e1ce0e14a47848477495dc4bb7b" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8" is_pre_election: true
I20260709 13:06:14.060732 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4a89e1ce0e14a47848477495dc4bb7b" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" is_pre_election: true
I20260709 13:06:14.061512 18293 raft_consensus.cc:2393] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 500e80cda2474f66b0ad65f4bfa00258 in current term 1: Already voted for candidate 3272b165a59e4467925ea233bbb5f6f8 in this term.
I20260709 13:06:14.062355 18107 leader_election.cc:304] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 500e80cda2474f66b0ad65f4bfa00258; no voters: 3272b165a59e4467925ea233bbb5f6f8, ac455e76c74d4abd95c4d73003ca9d3a
I20260709 13:06:14.063015 18368 raft_consensus.cc:3060] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:14.068645 18368 raft_consensus.cc:2749] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:14.075886 18368 consensus_queue.cc:1048] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:14.088690 18368 consensus_queue.cc:1048] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:14.090672  7164 ts_manager.cc:295] Set tserver state for 500e80cda2474f66b0ad65f4bfa00258 to MAINTENANCE_MODE
I20260709 13:06:14.092460  7164 tablet_server.cc:179] TabletServer@127.6.255.1:0 shutting down...
I20260709 13:06:14.093019 18368 raft_consensus.cc:493] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.093645 18368 raft_consensus.cc:515] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:14.096880 18218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1a834c742b84d70859baac3a6ff751a" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8" is_pre_election: true
I20260709 13:06:14.097553 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1a834c742b84d70859baac3a6ff751a" candidate_uuid: "500e80cda2474f66b0ad65f4bfa00258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" is_pre_election: true
I20260709 13:06:14.098645 18293 raft_consensus.cc:2393] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 500e80cda2474f66b0ad65f4bfa00258 in current term 1: Already voted for candidate 3272b165a59e4467925ea233bbb5f6f8 in this term.
I20260709 13:06:14.100296 18107 leader_election.cc:304] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 500e80cda2474f66b0ad65f4bfa00258; no voters: 3272b165a59e4467925ea233bbb5f6f8, ac455e76c74d4abd95c4d73003ca9d3a
I20260709 13:06:14.101364 18341 raft_consensus.cc:3060] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:14.101387 18368 leader_election.cc:290] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:14.112347 18341 raft_consensus.cc:2749] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:14.147317  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:14.148530  7164 tablet_replica.cc:333] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258: stopping tablet replica
I20260709 13:06:14.149294  7164 raft_consensus.cc:2243] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.149868  7164 raft_consensus.cc:2272] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.152400  7164 tablet_replica.cc:333] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258: stopping tablet replica
I20260709 13:06:14.153780  7164 raft_consensus.cc:2243] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.154217  7164 raft_consensus.cc:2272] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.156530  7164 tablet_replica.cc:333] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258: stopping tablet replica
I20260709 13:06:14.157230  7164 raft_consensus.cc:2243] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.157959  7164 pending_rounds.cc:70] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258: Trying to abort 1 pending ops.
I20260709 13:06:14.158118  7164 pending_rounds.cc:77] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305635323397533696 op_type: NO_OP noop_request { }
I20260709 13:06:14.158444  7164 raft_consensus.cc:2889] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:06:14.158793  7164 raft_consensus.cc:2272] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.161190  7164 tablet_replica.cc:333] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258: stopping tablet replica
I20260709 13:06:14.161818  7164 raft_consensus.cc:2243] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.162387  7164 pending_rounds.cc:70] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258: Trying to abort 1 pending ops.
I20260709 13:06:14.162591  7164 pending_rounds.cc:77] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305635323430797312 op_type: NO_OP noop_request { }
I20260709 13:06:14.162865  7164 raft_consensus.cc:2889] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:06:14.163137  7164 raft_consensus.cc:2272] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.165555  7164 tablet_replica.cc:333] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258: stopping tablet replica
I20260709 13:06:14.166100  7164 raft_consensus.cc:2243] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.166496  7164 raft_consensus.cc:2272] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.168092  7164 tablet_replica.cc:333] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258: stopping tablet replica
I20260709 13:06:14.168586  7164 raft_consensus.cc:2243] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.169375  7164 raft_consensus.cc:2272] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.171638  7164 tablet_replica.cc:333] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258: stopping tablet replica
I20260709 13:06:14.172176  7164 raft_consensus.cc:2243] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.172747  7164 pending_rounds.cc:70] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258: Trying to abort 1 pending ops.
I20260709 13:06:14.172945  7164 pending_rounds.cc:77] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305635323319296000 op_type: NO_OP noop_request { }
I20260709 13:06:14.173209  7164 raft_consensus.cc:2889] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:06:14.173503  7164 raft_consensus.cc:2272] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.175580  7164 tablet_replica.cc:333] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258: stopping tablet replica
I20260709 13:06:14.176115  7164 raft_consensus.cc:2243] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.176550  7164 raft_consensus.cc:2272] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.178846  7164 tablet_replica.cc:333] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258: stopping tablet replica
I20260709 13:06:14.179343  7164 raft_consensus.cc:2243] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.180132  7164 raft_consensus.cc:2272] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.183197  7164 tablet_replica.cc:333] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258: stopping tablet replica
I20260709 13:06:14.183785  7164 raft_consensus.cc:2243] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.184468  7164 pending_rounds.cc:70] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258: Trying to abort 1 pending ops.
I20260709 13:06:14.184695  7164 pending_rounds.cc:77] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305635322733375488 op_type: NO_OP noop_request { }
I20260709 13:06:14.185011  7164 raft_consensus.cc:2889] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:06:14.185309  7164 raft_consensus.cc:2272] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.194523 18342 raft_consensus.cc:493] T 8befc400255448babc5d5b6432d74740 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.195129 18342 raft_consensus.cc:515] T 8befc400255448babc5d5b6432d74740 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:14.197208 18342 leader_election.cc:290] T 8befc400255448babc5d5b6432d74740 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:14.198537 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8befc400255448babc5d5b6432d74740" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" is_pre_election: true
I20260709 13:06:14.199587 18293 raft_consensus.cc:2393] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3272b165a59e4467925ea233bbb5f6f8 in current term 1: Already voted for candidate 500e80cda2474f66b0ad65f4bfa00258 in this term.
W20260709 13:06:14.201442 18183 leader_election.cc:336] T 8befc400255448babc5d5b6432d74740 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:06:14.202005 18183 leader_election.cc:304] T 8befc400255448babc5d5b6432d74740 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8; no voters: 500e80cda2474f66b0ad65f4bfa00258, ac455e76c74d4abd95c4d73003ca9d3a
I20260709 13:06:14.202737 18342 raft_consensus.cc:3060] T 8befc400255448babc5d5b6432d74740 P 3272b165a59e4467925ea233bbb5f6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:14.210532  7164 tablet_server.cc:196] TabletServer@127.6.255.1:0 shutdown complete.
I20260709 13:06:14.212106 18342 raft_consensus.cc:2749] T 8befc400255448babc5d5b6432d74740 P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:06:14.214000 18183 consensus_peers.cc:597] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:14.218691 18355 consensus_queue.cc:1048] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:06:14.237794 18257 consensus_peers.cc:597] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:14.281374 18342 consensus_queue.cc:1048] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:14.284957 18342 consensus_queue.cc:1048] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:06:14.301837 18183 consensus_peers.cc:597] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:14.335563 18090 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:06:14.452876 18257 consensus_peers.cc:597] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:15.244935 18412 raft_consensus.cc:493] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.245499 18412 raft_consensus.cc:515] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:15.247329 18412 leader_election.cc:290] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515)
W20260709 13:06:15.248960 18257 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111) [suppressed 138 similar messages]
I20260709 13:06:15.249064 18219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8befc400255448babc5d5b6432d74740" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8" is_pre_election: true
I20260709 13:06:15.249874 18219 raft_consensus.cc:2468] T 8befc400255448babc5d5b6432d74740 P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac455e76c74d4abd95c4d73003ca9d3a in term 1.
I20260709 13:06:15.251356 18257 leader_election.cc:304] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 
I20260709 13:06:15.252039 18412 raft_consensus.cc:2804] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:06:15.252101 18257 leader_election.cc:336] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111)
I20260709 13:06:15.252480 18412 raft_consensus.cc:493] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.252827 18412 raft_consensus.cc:3060] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:15.258104 18412 raft_consensus.cc:515] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:15.259855 18412 leader_election.cc:290] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 election: Requested vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515)
I20260709 13:06:15.261574 18219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8befc400255448babc5d5b6432d74740" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
I20260709 13:06:15.262244 18219 raft_consensus.cc:3060] T 8befc400255448babc5d5b6432d74740 P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:15.263935 18257 leader_election.cc:336] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111)
I20260709 13:06:15.268373 18219 raft_consensus.cc:2468] T 8befc400255448babc5d5b6432d74740 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac455e76c74d4abd95c4d73003ca9d3a in term 2.
I20260709 13:06:15.269690 18257 leader_election.cc:304] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 500e80cda2474f66b0ad65f4bfa00258
I20260709 13:06:15.270491 18412 raft_consensus.cc:2804] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:15.270979 18412 raft_consensus.cc:697] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 LEADER]: Becoming Leader. State: Replica: ac455e76c74d4abd95c4d73003ca9d3a, State: Running, Role: LEADER
I20260709 13:06:15.271809 18412 consensus_queue.cc:237] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:15.279227 18039 catalog_manager.cc:5697] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a reported cstate change: term changed from 1 to 2, leader changed from 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1) to ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3). New cstate: current_term: 2 leader_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:15.336748 18090 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:15.502245 18412 raft_consensus.cc:493] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 500e80cda2474f66b0ad65f4bfa00258)
I20260709 13:06:15.503010 18412 raft_consensus.cc:515] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:15.504772 18412 leader_election.cc:290] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515)
I20260709 13:06:15.506651 18219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308eb63baa0f4f51b9116e3b240b0932" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8" is_pre_election: true
W20260709 13:06:15.508910 18257 leader_election.cc:336] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111)
I20260709 13:06:15.509300 18257 leader_election.cc:304] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac455e76c74d4abd95c4d73003ca9d3a; no voters: 3272b165a59e4467925ea233bbb5f6f8, 500e80cda2474f66b0ad65f4bfa00258
I20260709 13:06:15.510116 18412 raft_consensus.cc:2749] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:15.513576 18342 raft_consensus.cc:493] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 500e80cda2474f66b0ad65f4bfa00258)
I20260709 13:06:15.514128 18342 raft_consensus.cc:515] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:15.515815 18342 leader_election.cc:290] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:15.516871 18290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308eb63baa0f4f51b9116e3b240b0932" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" is_pre_election: true
I20260709 13:06:15.517590 18290 raft_consensus.cc:2468] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3272b165a59e4467925ea233bbb5f6f8 in term 1.
I20260709 13:06:15.518808 18182 leader_election.cc:304] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 
I20260709 13:06:15.519724 18342 raft_consensus.cc:2804] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:06:15.519943 18183 leader_election.cc:336] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111)
I20260709 13:06:15.520205 18342 raft_consensus.cc:493] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 500e80cda2474f66b0ad65f4bfa00258)
I20260709 13:06:15.520578 18342 raft_consensus.cc:3060] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:15.525987 18342 raft_consensus.cc:515] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:15.527721 18342 leader_election.cc:290] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 2 election: Requested vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:15.528970 18290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308eb63baa0f4f51b9116e3b240b0932" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
I20260709 13:06:15.529767 18290 raft_consensus.cc:3060] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:15.532092 18183 leader_election.cc:336] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111)
I20260709 13:06:15.536084 18290 raft_consensus.cc:2468] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3272b165a59e4467925ea233bbb5f6f8 in term 2.
I20260709 13:06:15.537317 18182 leader_election.cc:304] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 500e80cda2474f66b0ad65f4bfa00258
I20260709 13:06:15.538098 18342 raft_consensus.cc:2804] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:15.538970 18342 raft_consensus.cc:697] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 LEADER]: Becoming Leader. State: Replica: 3272b165a59e4467925ea233bbb5f6f8, State: Running, Role: LEADER
I20260709 13:06:15.539918 18342 consensus_queue.cc:237] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:15.549929 18039 catalog_manager.cc:5697] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 reported cstate change: term changed from 1 to 2, leader changed from 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1) to 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "3272b165a59e4467925ea233bbb5f6f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:15.588708 18412 raft_consensus.cc:493] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 500e80cda2474f66b0ad65f4bfa00258)
I20260709 13:06:15.589389 18412 raft_consensus.cc:515] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:15.591102 18412 leader_election.cc:290] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515)
I20260709 13:06:15.593053 18219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5214f540f4d64609b515bb6b540791ec" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8" is_pre_election: true
I20260709 13:06:15.593818 18219 raft_consensus.cc:2468] T 5214f540f4d64609b515bb6b540791ec P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac455e76c74d4abd95c4d73003ca9d3a in term 1.
I20260709 13:06:15.594238 18342 raft_consensus.cc:493] T 5214f540f4d64609b515bb6b540791ec P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 500e80cda2474f66b0ad65f4bfa00258)
I20260709 13:06:15.594728 18342 raft_consensus.cc:515] T 5214f540f4d64609b515bb6b540791ec P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:15.595116 18257 leader_election.cc:304] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 
I20260709 13:06:15.596046 18412 raft_consensus.cc:2804] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:06:15.596060 18257 leader_election.cc:336] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111)
I20260709 13:06:15.596549 18412 raft_consensus.cc:493] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 500e80cda2474f66b0ad65f4bfa00258)
I20260709 13:06:15.596684 18342 leader_election.cc:290] T 5214f540f4d64609b515bb6b540791ec P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:15.596997 18412 raft_consensus.cc:3060] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:15.598312 18290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5214f540f4d64609b515bb6b540791ec" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" is_pre_election: true
W20260709 13:06:15.600721 18183 leader_election.cc:336] T 5214f540f4d64609b515bb6b540791ec P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111)
I20260709 13:06:15.602649 18412 raft_consensus.cc:515] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:15.603765 18290 raft_consensus.cc:2393] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3272b165a59e4467925ea233bbb5f6f8 in current term 2: Already voted for candidate ac455e76c74d4abd95c4d73003ca9d3a in this term.
I20260709 13:06:15.604673 18412 leader_election.cc:290] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 election: Requested vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515)
I20260709 13:06:15.607021 18219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5214f540f4d64609b515bb6b540791ec" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
I20260709 13:06:15.607477 18219 raft_consensus.cc:3060] T 5214f540f4d64609b515bb6b540791ec P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:15.609882 18182 leader_election.cc:304] T 5214f540f4d64609b515bb6b540791ec P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8; no voters: 500e80cda2474f66b0ad65f4bfa00258, ac455e76c74d4abd95c4d73003ca9d3a
W20260709 13:06:15.612340 18257 leader_election.cc:336] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111)
I20260709 13:06:15.613065 18219 raft_consensus.cc:2468] T 5214f540f4d64609b515bb6b540791ec P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac455e76c74d4abd95c4d73003ca9d3a in term 2.
I20260709 13:06:15.613559 18342 raft_consensus.cc:2749] T 5214f540f4d64609b515bb6b540791ec P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:15.614454 18257 leader_election.cc:304] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 500e80cda2474f66b0ad65f4bfa00258
I20260709 13:06:15.615257 18412 raft_consensus.cc:2804] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:15.615736 18412 raft_consensus.cc:697] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [term 2 LEADER]: Becoming Leader. State: Replica: ac455e76c74d4abd95c4d73003ca9d3a, State: Running, Role: LEADER
I20260709 13:06:15.616477 18412 consensus_queue.cc:237] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:15.624011 18039 catalog_manager.cc:5697] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a reported cstate change: term changed from 1 to 2, leader changed from 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1) to ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3). New cstate: current_term: 2 leader_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:15.650007 18342 raft_consensus.cc:493] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.650045 18412 raft_consensus.cc:493] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.650612 18342 raft_consensus.cc:515] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:15.650933 18412 raft_consensus.cc:515] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:15.652844 18412 leader_election.cc:290] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515)
I20260709 13:06:15.653276 18342 leader_election.cc:290] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:15.654585 18290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3970d05a344a0bb83eb2970cd735e6" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" is_pre_election: true
I20260709 13:06:15.655301 18290 raft_consensus.cc:2468] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3272b165a59e4467925ea233bbb5f6f8 in term 1.
I20260709 13:06:15.655777 18219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3970d05a344a0bb83eb2970cd735e6" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8" is_pre_election: true
I20260709 13:06:15.656432 18219 raft_consensus.cc:2468] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac455e76c74d4abd95c4d73003ca9d3a in term 1.
I20260709 13:06:15.656569 18182 leader_election.cc:304] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 
I20260709 13:06:15.657405 18342 raft_consensus.cc:2804] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:15.657804 18342 raft_consensus.cc:493] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.657860 18257 leader_election.cc:304] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 
I20260709 13:06:15.658149 18342 raft_consensus.cc:3060] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:15.658236 18183 leader_election.cc:336] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111)
I20260709 13:06:15.658833 18412 raft_consensus.cc:2804] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:15.659204 18412 raft_consensus.cc:493] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.659582 18412 raft_consensus.cc:3060] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:15.659546 18257 leader_election.cc:336] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111)
I20260709 13:06:15.664815 18342 raft_consensus.cc:515] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:15.666018 18412 raft_consensus.cc:515] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:15.667281 18411 raft_consensus.cc:493] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.667371 18342 leader_election.cc:290] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 2 election: Requested vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:15.668013 18411 raft_consensus.cc:515] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:15.669289 18290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3970d05a344a0bb83eb2970cd735e6" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
I20260709 13:06:15.670141 18219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3970d05a344a0bb83eb2970cd735e6" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
I20260709 13:06:15.670405 18411 leader_election.cc:290] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515)
I20260709 13:06:15.670189 18290 raft_consensus.cc:2393] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3272b165a59e4467925ea233bbb5f6f8 in current term 2: Already voted for candidate ac455e76c74d4abd95c4d73003ca9d3a in this term.
I20260709 13:06:15.671088 18219 raft_consensus.cc:2393] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ac455e76c74d4abd95c4d73003ca9d3a in current term 2: Already voted for candidate 3272b165a59e4467925ea233bbb5f6f8 in this term.
I20260709 13:06:15.668042 18412 leader_election.cc:290] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 election: Requested vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515)
W20260709 13:06:15.672533 18183 leader_election.cc:336] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111)
I20260709 13:06:15.673072 18183 leader_election.cc:304] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8; no voters: 500e80cda2474f66b0ad65f4bfa00258, ac455e76c74d4abd95c4d73003ca9d3a
I20260709 13:06:15.674154 18219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b45d6ca8b05141749dbacd2eaaf9eb49" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8" is_pre_election: true
I20260709 13:06:15.674431 18342 raft_consensus.cc:2749] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:15.674896 18219 raft_consensus.cc:2468] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac455e76c74d4abd95c4d73003ca9d3a in term 1.
W20260709 13:06:15.675376 18257 leader_election.cc:336] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111)
I20260709 13:06:15.675875 18257 leader_election.cc:304] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac455e76c74d4abd95c4d73003ca9d3a; no voters: 3272b165a59e4467925ea233bbb5f6f8, 500e80cda2474f66b0ad65f4bfa00258
I20260709 13:06:15.676548 18412 raft_consensus.cc:2749] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:06:15.676652 18257 leader_election.cc:336] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111)
I20260709 13:06:15.677685 18257 leader_election.cc:304] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 500e80cda2474f66b0ad65f4bfa00258
I20260709 13:06:15.678448 18412 raft_consensus.cc:2804] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:15.678753 18412 raft_consensus.cc:493] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.679003 18412 raft_consensus.cc:3060] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:15.684033 18412 raft_consensus.cc:515] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:15.685722 18412 leader_election.cc:290] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 election: Requested vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515)
I20260709 13:06:15.687793 18219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b45d6ca8b05141749dbacd2eaaf9eb49" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
I20260709 13:06:15.688508 18219 raft_consensus.cc:3060] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:15.690238 18257 leader_election.cc:336] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111)
I20260709 13:06:15.694890 18219 raft_consensus.cc:2468] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac455e76c74d4abd95c4d73003ca9d3a in term 2.
I20260709 13:06:15.696007 18257 leader_election.cc:304] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 500e80cda2474f66b0ad65f4bfa00258
I20260709 13:06:15.696677 18412 raft_consensus.cc:2804] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:15.697074 18412 raft_consensus.cc:697] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 LEADER]: Becoming Leader. State: Replica: ac455e76c74d4abd95c4d73003ca9d3a, State: Running, Role: LEADER
I20260709 13:06:15.697945 18412 consensus_queue.cc:237] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:15.705014 18039 catalog_manager.cc:5697] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a reported cstate change: term changed from 1 to 2, leader changed from 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1) to ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3). New cstate: current_term: 2 leader_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:15.746143 18257 consensus_peers.cc:597] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:15.858912 18219 raft_consensus.cc:1275] T 8befc400255448babc5d5b6432d74740 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Refusing update from remote peer ac455e76c74d4abd95c4d73003ca9d3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:06:15.860388 18412 consensus_queue.cc:1048] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:15.864395 18342 raft_consensus.cc:493] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.864902 18342 raft_consensus.cc:515] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:15.867208 18342 leader_election.cc:290] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:15.868271 18290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24794fe9a88c4959aa0101a5e612121f" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" is_pre_election: true
I20260709 13:06:15.868974 18290 raft_consensus.cc:2468] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3272b165a59e4467925ea233bbb5f6f8 in term 1.
I20260709 13:06:15.870929 18182 leader_election.cc:304] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 
I20260709 13:06:15.872253 18342 raft_consensus.cc:2804] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:15.872658 18342 raft_consensus.cc:493] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.873008 18342 raft_consensus.cc:3060] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:15.873090 18183 leader_election.cc:336] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111)
I20260709 13:06:15.880951 18342 raft_consensus.cc:515] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:15.883006 18342 leader_election.cc:290] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 2 election: Requested vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:15.884174 18290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24794fe9a88c4959aa0101a5e612121f" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
I20260709 13:06:15.885123 18290 raft_consensus.cc:3060] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:15.889086 18183 leader_election.cc:336] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111)
I20260709 13:06:15.891556 18290 raft_consensus.cc:2468] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3272b165a59e4467925ea233bbb5f6f8 in term 2.
I20260709 13:06:15.892993 18182 leader_election.cc:304] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 500e80cda2474f66b0ad65f4bfa00258
I20260709 13:06:15.893859 18342 raft_consensus.cc:2804] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:15.894297 18342 raft_consensus.cc:697] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 [term 2 LEADER]: Becoming Leader. State: Replica: 3272b165a59e4467925ea233bbb5f6f8, State: Running, Role: LEADER
I20260709 13:06:15.895066 18342 consensus_queue.cc:237] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:15.903753 18040 catalog_manager.cc:5697] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 reported cstate change: term changed from 1 to 2, leader changed from 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1) to 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "3272b165a59e4467925ea233bbb5f6f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:16.043780 18219 raft_consensus.cc:1275] T 5214f540f4d64609b515bb6b540791ec P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Refusing update from remote peer ac455e76c74d4abd95c4d73003ca9d3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:16.045254 18411 consensus_queue.cc:1048] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:06:16.063371 18257 consensus_peers.cc:597] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:16.105055 18219 raft_consensus.cc:1275] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Refusing update from remote peer ac455e76c74d4abd95c4d73003ca9d3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:06:16.106421 18428 consensus_queue.cc:1048] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:06:16.121078 18257 consensus_peers.cc:597] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:16.128813 18290 raft_consensus.cc:1275] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Refusing update from remote peer 3272b165a59e4467925ea233bbb5f6f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:16.130317 18342 consensus_queue.cc:1048] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:06:16.140838 18183 consensus_peers.cc:597] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:16.336500 18183 consensus_peers.cc:597] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:16.337949 18090 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:16.358768 18290 raft_consensus.cc:1275] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Refusing update from remote peer 3272b165a59e4467925ea233bbb5f6f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:06:16.360430 18431 consensus_queue.cc:1048] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:06:16.462864 18183 consensus_peers.cc:597] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.567981 18257 consensus_peers.cc:597] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.667786 18183 consensus_peers.cc:597] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.874922 18257 consensus_peers.cc:597] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:17.338963 18090 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:17.514144 18431 raft_consensus.cc:493] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.514761 18431 raft_consensus.cc:515] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:17.516568 18431 leader_election.cc:290] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:17.517650 18290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3970d05a344a0bb83eb2970cd735e6" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" is_pre_election: true
I20260709 13:06:17.518381 18290 raft_consensus.cc:2468] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3272b165a59e4467925ea233bbb5f6f8 in term 2.
I20260709 13:06:17.519495 18182 leader_election.cc:304] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 
I20260709 13:06:17.520231 18431 raft_consensus.cc:2804] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:06:17.520426 18183 leader_election.cc:336] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111)
I20260709 13:06:17.520574 18431 raft_consensus.cc:493] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.521067 18431 raft_consensus.cc:3060] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:17.526877 18431 raft_consensus.cc:515] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:17.528710 18431 leader_election.cc:290] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 3 election: Requested vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:17.529815 18290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3970d05a344a0bb83eb2970cd735e6" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
I20260709 13:06:17.530398 18290 raft_consensus.cc:3060] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:17.532128 18183 leader_election.cc:336] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111)
I20260709 13:06:17.535704 18290 raft_consensus.cc:2468] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3272b165a59e4467925ea233bbb5f6f8 in term 3.
I20260709 13:06:17.536881 18182 leader_election.cc:304] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 500e80cda2474f66b0ad65f4bfa00258
I20260709 13:06:17.537608 18431 raft_consensus.cc:2804] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:17.538491 18431 raft_consensus.cc:697] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 3 LEADER]: Becoming Leader. State: Replica: 3272b165a59e4467925ea233bbb5f6f8, State: Running, Role: LEADER
I20260709 13:06:17.539429 18431 consensus_queue.cc:237] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:17.548779 18039 catalog_manager.cc:5697] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 reported cstate change: term changed from 1 to 3, leader changed from 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1) to 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2). New cstate: current_term: 3 leader_uuid: "3272b165a59e4467925ea233bbb5f6f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:18.066425 18183 consensus_peers.cc:597] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:18.106411 18257 consensus_peers.cc:597] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:18.148615 18290 raft_consensus.cc:1275] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 3 FOLLOWER]: Refusing update from remote peer 3272b165a59e4467925ea233bbb5f6f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 1. (index mismatch)
I20260709 13:06:18.150125 18434 consensus_queue.cc:1048] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:06:18.340282 18090 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:06:18.375451 18257 consensus_peers.cc:597] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:19.315200 18183 consensus_peers.cc:597] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:19.316337 18183 consensus_peers.cc:597] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:19.335318 18257 consensus_peers.cc:597] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:19.341745 18090 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:06:19.396473 18073 debug-util.cc:398] Leaking SignalData structure 0x7b08003ddf00 after lost signal to thread 7167
W20260709 13:06:19.398041 18073 debug-util.cc:398] Leaking SignalData structure 0x7b08001bda40 after lost signal to thread 18076
W20260709 13:06:19.399334 18073 debug-util.cc:398] Leaking SignalData structure 0x7b080031cc20 after lost signal to thread 18243
W20260709 13:06:19.400430 18073 debug-util.cc:398] Leaking SignalData structure 0x7b0800157080 after lost signal to thread 18317
W20260709 13:06:19.682989 18257 consensus_peers.cc:597] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:19.792174 18183 consensus_peers.cc:597] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:19.915426 18183 consensus_peers.cc:597] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:19.927884  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:19.937813 18453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:19.940057 18454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:19.948079 18456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:19.948096  7164 server_base.cc:1061] running on GCE node
I20260709 13:06:19.954906  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:19.955255  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:19.955482  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602379955465 us; error 0 us; skew 500 ppm
I20260709 13:06:19.956239  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:19.960407  7164 webserver.cc:533] Webserver started at http://127.6.255.1:35929/ using document root <none> and password file <none>
I20260709 13:06:19.961078  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:19.961321  7164 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:06:19.966279  7164 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:06:19.971199 18461 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:19.972246  7164 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.001s
I20260709 13:06:19.972585  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602046064212-7164-0/minicluster-data/ts-0-root
uuid: "500e80cda2474f66b0ad65f4bfa00258"
format_stamp: "Formatted at 2026-07-09 13:06:12 on dist-test-slave-rnv4"
I20260709 13:06:19.973100  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602046064212-7164-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602046064212-7164-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602046064212-7164-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:20.004230  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:20.005700  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:20.007575  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:20.012070 18468 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
I20260709 13:06:20.072938  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:06:20.073376  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.063s	user 0.000s	sys 0.003s
I20260709 13:06:20.073777  7164 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:06:20.083106 18468 tablet_bootstrap.cc:492] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap starting.
W20260709 13:06:20.103235 18257 consensus_peers.cc:597] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:20.103775 18468 tablet_bootstrap.cc:492] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:20.104732 18468 tablet_bootstrap.cc:492] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap complete.
I20260709 13:06:20.105595 18468 ts_tablet_manager.cc:1403] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.004s
I20260709 13:06:20.109315 18468 raft_consensus.cc:359] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:20.110674 18468 raft_consensus.cc:740] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.111629 18468 consensus_queue.cc:260] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:20.115164 18468 ts_tablet_manager.cc:1434] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:06:20.116107 18468 tablet_bootstrap.cc:492] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap starting.
I20260709 13:06:20.141245 18468 tablet_bootstrap.cc:492] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:20.142287 18468 tablet_bootstrap.cc:492] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap complete.
I20260709 13:06:20.143097 18468 ts_tablet_manager.cc:1403] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.008s
I20260709 13:06:20.146503 18468 raft_consensus.cc:359] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:20.147320 18468 raft_consensus.cc:740] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.148432 18468 consensus_queue.cc:260] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:20.150862 18468 ts_tablet_manager.cc:1434] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:06:20.151028  7164 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:06:20.151463  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.078s	user 0.065s	sys 0.004s
I20260709 13:06:20.151772 18468 tablet_bootstrap.cc:492] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap starting.
I20260709 13:06:20.172694 18468 tablet_bootstrap.cc:492] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:06:20.174000 18468 tablet_bootstrap.cc:492] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap complete.
I20260709 13:06:20.175123 18468 ts_tablet_manager.cc:1403] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.012s
I20260709 13:06:20.178841 18468 raft_consensus.cc:359] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:20.180087 18468 raft_consensus.cc:740] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.181084 18468 consensus_queue.cc:260] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:20.186772 18468 ts_tablet_manager.cc:1434] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:06:20.188014 18468 tablet_bootstrap.cc:492] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap starting.
I20260709 13:06:20.208925 18468 tablet_bootstrap.cc:492] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:06:20.210124 18468 tablet_bootstrap.cc:492] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap complete.
I20260709 13:06:20.210763 18468 ts_tablet_manager.cc:1403] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.003s
I20260709 13:06:20.213874 18468 raft_consensus.cc:359] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:20.214895 18468 raft_consensus.cc:740] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.215806 18468 consensus_queue.cc:260] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:20.218562 18468 ts_tablet_manager.cc:1434] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:06:20.219602 18468 tablet_bootstrap.cc:492] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap starting.
I20260709 13:06:20.226262  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.1:44331
I20260709 13:06:20.226390 18529 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.1:44331 every 8 connection(s)
I20260709 13:06:20.247311  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:20.260150 18468 tablet_bootstrap.cc:492] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:20.261274 18468 tablet_bootstrap.cc:492] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap complete.
I20260709 13:06:20.262023 18468 ts_tablet_manager.cc:1403] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258: Time spent bootstrapping tablet: real 0.043s	user 0.025s	sys 0.012s
I20260709 13:06:20.264853 18468 raft_consensus.cc:359] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:20.265656 18468 raft_consensus.cc:740] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.266562 18468 consensus_queue.cc:260] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:20.268994 18468 ts_tablet_manager.cc:1434] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:06:20.269986 18468 tablet_bootstrap.cc:492] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap starting.
I20260709 13:06:20.310173 18216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a4a89e1ce0e14a47848477495dc4bb7b"
dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
mode: GRACEFUL
new_leader_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
 from {username='slave'} at 127.0.0.1:46906
I20260709 13:06:20.311241 18216 raft_consensus.cc:606] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [term 1 LEADER]: Received request to transfer leadership to TS ac455e76c74d4abd95c4d73003ca9d3a
W20260709 13:06:20.342870 18090 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:20.345393 18468 tablet_bootstrap.cc:492] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:20.346851 18468 tablet_bootstrap.cc:492] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap complete.
I20260709 13:06:20.347610 18468 ts_tablet_manager.cc:1403] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258: Time spent bootstrapping tablet: real 0.078s	user 0.027s	sys 0.048s
I20260709 13:06:20.350698 18293 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b45d6ca8b05141749dbacd2eaaf9eb49"
dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
mode: GRACEFUL
new_leader_uuid: "3272b165a59e4467925ea233bbb5f6f8"
 from {username='slave'} at 127.0.0.1:58398
I20260709 13:06:20.350857 18468 raft_consensus.cc:359] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } }
I20260709 13:06:20.351557 18293 raft_consensus.cc:606] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 LEADER]: Received request to transfer leadership to TS 3272b165a59e4467925ea233bbb5f6f8
I20260709 13:06:20.351696 18468 raft_consensus.cc:740] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.352795 18468 consensus_queue.cc:260] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } }
I20260709 13:06:20.353657  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:06:20.354441  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:20.355470  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:20.355819 18468 ts_tablet_manager.cc:1434] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:06:20.356860 18468 tablet_bootstrap.cc:492] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap starting.
I20260709 13:06:20.360587 18530 heartbeater.cc:344] Connected to a master server at 127.6.255.62:37899
I20260709 13:06:20.361294 18530 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:20.362756 18530 heartbeater.cc:507] Master 127.6.255.62:37899 requested a full tablet report, sending...
I20260709 13:06:20.370743 18042 ts_manager.cc:194] Re-registered known tserver with Master: 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331)
I20260709 13:06:20.388145 18503 raft_consensus.cc:3060] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.394470 18502 raft_consensus.cc:3060] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.401413 18468 tablet_bootstrap.cc:492] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:06:20.402779 18468 tablet_bootstrap.cc:492] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap complete.
I20260709 13:06:20.403589 18468 ts_tablet_manager.cc:1403] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258: Time spent bootstrapping tablet: real 0.047s	user 0.022s	sys 0.015s
I20260709 13:06:20.406989 18503 pending_rounds.cc:85] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258: Aborting all ops after (but not including) 0
I20260709 13:06:20.406904 18468 raft_consensus.cc:359] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:20.407569 18503 pending_rounds.cc:107] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:06:20.408041 18503 raft_consensus.cc:2889] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:20.408814 18502 pending_rounds.cc:85] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258: Aborting all ops after (but not including) 0
I20260709 13:06:20.409263 18502 pending_rounds.cc:107] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:06:20.409643 18502 raft_consensus.cc:2889] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:20.408047 18468 raft_consensus.cc:740] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.411366 18468 consensus_queue.cc:260] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:20.412662 18042 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44972
I20260709 13:06:20.413780 18468 ts_tablet_manager.cc:1434] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:06:20.414834 18468 tablet_bootstrap.cc:492] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap starting.
I20260709 13:06:20.419356 18530 heartbeater.cc:499] Master 127.6.255.62:37899 was elected leader, sending a full tablet report...
I20260709 13:06:20.428815 18502 raft_consensus.cc:3060] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.437443 18468 tablet_bootstrap.cc:492] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:20.438555 18468 tablet_bootstrap.cc:492] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap complete.
I20260709 13:06:20.439330 18468 ts_tablet_manager.cc:1403] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258: Time spent bootstrapping tablet: real 0.025s	user 0.022s	sys 0.000s
I20260709 13:06:20.443001 18468 raft_consensus.cc:359] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:20.444283 18468 raft_consensus.cc:740] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.445551 18468 consensus_queue.cc:260] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:20.448132 18468 ts_tablet_manager.cc:1434] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:06:20.449211 18468 tablet_bootstrap.cc:492] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap starting.
I20260709 13:06:20.475373 18468 tablet_bootstrap.cc:492] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:06:20.476432 18468 tablet_bootstrap.cc:492] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap complete.
I20260709 13:06:20.477183 18468 ts_tablet_manager.cc:1403] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258: Time spent bootstrapping tablet: real 0.028s	user 0.018s	sys 0.005s
I20260709 13:06:20.479136 18468 raft_consensus.cc:359] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:20.479822 18468 raft_consensus.cc:740] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.480475 18468 consensus_queue.cc:260] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:20.482345 18468 ts_tablet_manager.cc:1434] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:06:20.483392 18468 tablet_bootstrap.cc:492] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap starting.
I20260709 13:06:20.497663 18468 tablet_bootstrap.cc:492] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:20.498413 18468 tablet_bootstrap.cc:492] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258: Bootstrap complete.
I20260709 13:06:20.498972 18468 ts_tablet_manager.cc:1403] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258: Time spent bootstrapping tablet: real 0.016s	user 0.007s	sys 0.007s
I20260709 13:06:20.500794 18468 raft_consensus.cc:359] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:20.501308 18468 raft_consensus.cc:740] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500e80cda2474f66b0ad65f4bfa00258, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.501906 18468 consensus_queue.cc:260] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:20.503762 18468 ts_tablet_manager.cc:1434] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:06:20.541702 18445 raft_consensus.cc:993] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8: : Instructing follower ac455e76c74d4abd95c4d73003ca9d3a to start an election
I20260709 13:06:20.542366 18536 raft_consensus.cc:1081] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [term 1 LEADER]: Signalling peer ac455e76c74d4abd95c4d73003ca9d3a to start an election
I20260709 13:06:20.544296 18293 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a4a89e1ce0e14a47848477495dc4bb7b"
dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
 from {username='slave'} at 127.0.0.1:58370
I20260709 13:06:20.544876 18293 raft_consensus.cc:493] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:20.545265 18293 raft_consensus.cc:3060] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.552009 18293 raft_consensus.cc:515] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:20.553874 18293 leader_election.cc:290] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 election: Requested vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515)
I20260709 13:06:20.554996 18503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4a89e1ce0e14a47848477495dc4bb7b" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "500e80cda2474f66b0ad65f4bfa00258"
I20260709 13:06:20.555390 18216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4a89e1ce0e14a47848477495dc4bb7b" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
I20260709 13:06:20.555837 18503 raft_consensus.cc:3060] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.556075 18216 raft_consensus.cc:3055] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:20.556447 18216 raft_consensus.cc:740] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3272b165a59e4467925ea233bbb5f6f8, State: Running, Role: LEADER
I20260709 13:06:20.557627 18216 consensus_queue.cc:260] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:20.559217 18216 raft_consensus.cc:3060] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.564817 18503 raft_consensus.cc:2468] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac455e76c74d4abd95c4d73003ca9d3a in term 2.
I20260709 13:06:20.566375 18257 leader_election.cc:304] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 500e80cda2474f66b0ad65f4bfa00258, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 
I20260709 13:06:20.567459 18542 raft_consensus.cc:2804] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:20.568037 18542 raft_consensus.cc:697] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a [term 2 LEADER]: Becoming Leader. State: Replica: ac455e76c74d4abd95c4d73003ca9d3a, State: Running, Role: LEADER
I20260709 13:06:20.568588 18216 raft_consensus.cc:2468] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac455e76c74d4abd95c4d73003ca9d3a in term 2.
I20260709 13:06:20.569305 18542 consensus_queue.cc:237] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:20.581179 18042 catalog_manager.cc:5697] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a reported cstate change: term changed from 1 to 2, leader changed from 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2) to ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3). New cstate: current_term: 2 leader_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:20.663132 18559 raft_consensus.cc:993] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a: : Instructing follower 3272b165a59e4467925ea233bbb5f6f8 to start an election
I20260709 13:06:20.663774 18447 raft_consensus.cc:1081] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 LEADER]: Signalling peer 3272b165a59e4467925ea233bbb5f6f8 to start an election
I20260709 13:06:20.665335 18216 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b45d6ca8b05141749dbacd2eaaf9eb49"
dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
 from {username='slave'} at 127.0.0.1:46888
I20260709 13:06:20.666000 18216 raft_consensus.cc:493] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:20.666410 18216 raft_consensus.cc:3060] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:20.675243 18502 raft_consensus.cc:3060] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:06:20.675774 18216 raft_consensus.cc:515] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:20.679217 18504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b45d6ca8b05141749dbacd2eaaf9eb49" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "500e80cda2474f66b0ad65f4bfa00258"
I20260709 13:06:20.679896 18504 raft_consensus.cc:3060] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:20.684090 18502 pending_rounds.cc:85] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258: Aborting all ops after (but not including) 0
I20260709 13:06:20.684568 18502 pending_rounds.cc:107] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:06:20.684944 18502 raft_consensus.cc:2889] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:20.691785 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b45d6ca8b05141749dbacd2eaaf9eb49" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
I20260709 13:06:20.692538 18293 raft_consensus.cc:3055] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:20.693017 18293 raft_consensus.cc:740] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ac455e76c74d4abd95c4d73003ca9d3a, State: Running, Role: LEADER
I20260709 13:06:20.694340 18293 consensus_queue.cc:260] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:20.696137 18293 raft_consensus.cc:3060] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:20.702137 18501 raft_consensus.cc:3060] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.706248 18216 leader_election.cc:290] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 3 election: Requested vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:20.707464 18504 raft_consensus.cc:2468] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3272b165a59e4467925ea233bbb5f6f8 in term 3.
I20260709 13:06:20.709062 18183 leader_election.cc:304] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, 500e80cda2474f66b0ad65f4bfa00258; no voters: 
I20260709 13:06:20.709982 18536 raft_consensus.cc:2804] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:20.710697 18536 raft_consensus.cc:697] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 3 LEADER]: Becoming Leader. State: Replica: 3272b165a59e4467925ea233bbb5f6f8, State: Running, Role: LEADER
I20260709 13:06:20.711766 18536 consensus_queue.cc:237] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:20.714809 18501 pending_rounds.cc:85] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258: Aborting all ops after (but not including) 0
I20260709 13:06:20.715287 18501 pending_rounds.cc:107] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:06:20.715642 18501 raft_consensus.cc:2889] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:20.709911 18293 raft_consensus.cc:2468] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3272b165a59e4467925ea233bbb5f6f8 in term 3.
I20260709 13:06:20.728981 18042 catalog_manager.cc:5697] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 reported cstate change: term changed from 2 to 3, leader changed from ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3) to 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2). New cstate: current_term: 3 leader_uuid: "3272b165a59e4467925ea233bbb5f6f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:20.883872 18501 raft_consensus.cc:3060] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:21.084185 18503 raft_consensus.cc:1275] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: Refusing update from remote peer ac455e76c74d4abd95c4d73003ca9d3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:21.085911 18559 consensus_queue.cc:1048] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:21.102931 18216 raft_consensus.cc:1275] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Refusing update from remote peer ac455e76c74d4abd95c4d73003ca9d3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:21.109526 18447 consensus_queue.cc:1048] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:21.137712 18293 raft_consensus.cc:1275] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 3 FOLLOWER]: Refusing update from remote peer 3272b165a59e4467925ea233bbb5f6f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:21.139276 18565 consensus_queue.cc:1048] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:21.151320 18501 raft_consensus.cc:1275] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 3 FOLLOWER]: Refusing update from remote peer 3272b165a59e4467925ea233bbb5f6f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:21.153694 18536 consensus_queue.cc:1048] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:06:21.344133 18090 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:21.376312  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:21.386197 18216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a1a834c742b84d70859baac3a6ff751a"
dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
mode: GRACEFUL
new_leader_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
 from {username='slave'} at 127.0.0.1:46906
I20260709 13:06:21.386904 18216 raft_consensus.cc:606] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [term 1 LEADER]: Received request to transfer leadership to TS ac455e76c74d4abd95c4d73003ca9d3a
I20260709 13:06:21.390077 18293 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fd44efaf24c849de9baeeb9a97271dbd"
dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
mode: GRACEFUL
new_leader_uuid: "3272b165a59e4467925ea233bbb5f6f8"
 from {username='slave'} at 127.0.0.1:58398
I20260709 13:06:21.390813 18293 raft_consensus.cc:606] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [term 1 LEADER]: Received request to transfer leadership to TS 3272b165a59e4467925ea233bbb5f6f8
I20260709 13:06:21.392604  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:06:21.393383  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:21.394268  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:21.690721 18570 raft_consensus.cc:993] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8: : Instructing follower ac455e76c74d4abd95c4d73003ca9d3a to start an election
I20260709 13:06:21.691887 18536 raft_consensus.cc:1081] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [term 1 LEADER]: Signalling peer ac455e76c74d4abd95c4d73003ca9d3a to start an election
I20260709 13:06:21.693699 18293 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a1a834c742b84d70859baac3a6ff751a"
dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
 from {username='slave'} at 127.0.0.1:58370
I20260709 13:06:21.694238 18293 raft_consensus.cc:493] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:21.694581 18293 raft_consensus.cc:3060] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:21.703739 18293 raft_consensus.cc:515] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:21.707445 18501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1a834c742b84d70859baac3a6ff751a" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "500e80cda2474f66b0ad65f4bfa00258"
I20260709 13:06:21.708040 18501 raft_consensus.cc:3060] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:21.719339 18293 leader_election.cc:290] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 election: Requested vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515)
I20260709 13:06:21.719625 18216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1a834c742b84d70859baac3a6ff751a" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
I20260709 13:06:21.720441 18216 raft_consensus.cc:3055] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:21.720839 18216 raft_consensus.cc:740] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3272b165a59e4467925ea233bbb5f6f8, State: Running, Role: LEADER
I20260709 13:06:21.722146 18501 raft_consensus.cc:2468] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac455e76c74d4abd95c4d73003ca9d3a in term 2.
I20260709 13:06:21.721905 18216 consensus_queue.cc:260] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:21.724123 18257 leader_election.cc:304] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 500e80cda2474f66b0ad65f4bfa00258, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 
I20260709 13:06:21.724236 18216 raft_consensus.cc:3060] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:21.724951 18447 raft_consensus.cc:2804] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:21.725550 18447 raft_consensus.cc:697] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [term 2 LEADER]: Becoming Leader. State: Replica: ac455e76c74d4abd95c4d73003ca9d3a, State: Running, Role: LEADER
I20260709 13:06:21.726289 18447 consensus_queue.cc:237] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:21.737030 18559 raft_consensus.cc:993] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a: : Instructing follower 3272b165a59e4467925ea233bbb5f6f8 to start an election
I20260709 13:06:21.737882 18559 raft_consensus.cc:1081] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [term 1 LEADER]: Signalling peer 3272b165a59e4467925ea233bbb5f6f8 to start an election
I20260709 13:06:21.743405 18217 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fd44efaf24c849de9baeeb9a97271dbd"
dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
 from {username='slave'} at 127.0.0.1:46888
I20260709 13:06:21.744215 18217 raft_consensus.cc:493] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:21.744681 18217 raft_consensus.cc:3060] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:21.737247 18216 raft_consensus.cc:2468] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac455e76c74d4abd95c4d73003ca9d3a in term 2.
I20260709 13:06:21.744870 18040 catalog_manager.cc:5697] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a reported cstate change: term changed from 1 to 2, leader changed from 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2) to ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3). New cstate: current_term: 2 leader_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:21.755136 18217 raft_consensus.cc:515] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:21.758064 18217 leader_election.cc:290] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 2 election: Requested vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:21.760139 18501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44efaf24c849de9baeeb9a97271dbd" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "500e80cda2474f66b0ad65f4bfa00258"
I20260709 13:06:21.760793 18501 raft_consensus.cc:3060] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:21.761926 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44efaf24c849de9baeeb9a97271dbd" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
I20260709 13:06:21.762567 18293 raft_consensus.cc:3055] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:21.762914 18293 raft_consensus.cc:740] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ac455e76c74d4abd95c4d73003ca9d3a, State: Running, Role: LEADER
I20260709 13:06:21.763983 18293 consensus_queue.cc:260] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:21.765270 18293 raft_consensus.cc:3060] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:21.770191 18501 raft_consensus.cc:2468] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3272b165a59e4467925ea233bbb5f6f8 in term 2.
I20260709 13:06:21.770881 18293 raft_consensus.cc:2468] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3272b165a59e4467925ea233bbb5f6f8 in term 2.
I20260709 13:06:21.771853 18183 leader_election.cc:304] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, 500e80cda2474f66b0ad65f4bfa00258; no voters: 
I20260709 13:06:21.772809 18536 raft_consensus.cc:2804] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:21.773300 18536 raft_consensus.cc:697] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [term 2 LEADER]: Becoming Leader. State: Replica: 3272b165a59e4467925ea233bbb5f6f8, State: Running, Role: LEADER
I20260709 13:06:21.774425 18536 consensus_queue.cc:237] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:21.784137 18040 catalog_manager.cc:5697] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 reported cstate change: term changed from 1 to 2, leader changed from ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3) to 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "3272b165a59e4467925ea233bbb5f6f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:22.122225 18217 raft_consensus.cc:1275] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Refusing update from remote peer ac455e76c74d4abd95c4d73003ca9d3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:22.124054 18559 consensus_queue.cc:1048] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:22.137151 18501 raft_consensus.cc:1275] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: Refusing update from remote peer ac455e76c74d4abd95c4d73003ca9d3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:22.139117 18542 consensus_queue.cc:1048] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:22.188078 18501 raft_consensus.cc:1275] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: Refusing update from remote peer 3272b165a59e4467925ea233bbb5f6f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:22.189590 18536 consensus_queue.cc:1048] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:22.206204 18293 raft_consensus.cc:1275] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Refusing update from remote peer 3272b165a59e4467925ea233bbb5f6f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:22.209136 18571 consensus_queue.cc:1048] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:06:22.345455 18090 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:22.406330  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:22.419235 18293 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a1a834c742b84d70859baac3a6ff751a"
dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
mode: GRACEFUL
new_leader_uuid: "3272b165a59e4467925ea233bbb5f6f8"
 from {username='slave'} at 127.0.0.1:58398
I20260709 13:06:22.419965 18293 raft_consensus.cc:606] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [term 2 LEADER]: Received request to transfer leadership to TS 3272b165a59e4467925ea233bbb5f6f8
I20260709 13:06:22.423440 18217 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b45d6ca8b05141749dbacd2eaaf9eb49"
dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
mode: GRACEFUL
new_leader_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
 from {username='slave'} at 127.0.0.1:46906
I20260709 13:06:22.424150 18217 raft_consensus.cc:606] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 3 LEADER]: Received request to transfer leadership to TS ac455e76c74d4abd95c4d73003ca9d3a
I20260709 13:06:22.425715  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:06:22.426453  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:22.427258  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:22.579622 18447 raft_consensus.cc:993] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a: : Instructing follower 3272b165a59e4467925ea233bbb5f6f8 to start an election
I20260709 13:06:22.580228 18542 raft_consensus.cc:1081] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [term 2 LEADER]: Signalling peer 3272b165a59e4467925ea233bbb5f6f8 to start an election
I20260709 13:06:22.581907 18217 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a1a834c742b84d70859baac3a6ff751a"
dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
 from {username='slave'} at 127.0.0.1:46888
I20260709 13:06:22.582437 18217 raft_consensus.cc:493] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:22.582767 18217 raft_consensus.cc:3060] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:22.588425 18217 raft_consensus.cc:515] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:22.591224 18217 leader_election.cc:290] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 3 election: Requested vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:22.592129 18501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1a834c742b84d70859baac3a6ff751a" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "500e80cda2474f66b0ad65f4bfa00258"
I20260709 13:06:22.592337 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1a834c742b84d70859baac3a6ff751a" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
I20260709 13:06:22.593050 18501 raft_consensus.cc:3060] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:22.593068 18293 raft_consensus.cc:3055] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:22.593740 18293 raft_consensus.cc:740] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ac455e76c74d4abd95c4d73003ca9d3a, State: Running, Role: LEADER
I20260709 13:06:22.594941 18293 consensus_queue.cc:260] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:22.596727 18293 raft_consensus.cc:3060] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:22.602253 18501 raft_consensus.cc:2468] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3272b165a59e4467925ea233bbb5f6f8 in term 3.
I20260709 13:06:22.603755 18183 leader_election.cc:304] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, 500e80cda2474f66b0ad65f4bfa00258; no voters: 
I20260709 13:06:22.604920 18572 raft_consensus.cc:2804] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:22.605412 18572 raft_consensus.cc:697] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [term 3 LEADER]: Becoming Leader. State: Replica: 3272b165a59e4467925ea233bbb5f6f8, State: Running, Role: LEADER
I20260709 13:06:22.606218 18572 consensus_queue.cc:237] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:22.609023 18293 raft_consensus.cc:2468] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3272b165a59e4467925ea233bbb5f6f8 in term 3.
I20260709 13:06:22.613210 18039 catalog_manager.cc:5697] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 reported cstate change: term changed from 2 to 3, leader changed from ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3) to 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2). New cstate: current_term: 3 leader_uuid: "3272b165a59e4467925ea233bbb5f6f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:22.667203 18572 raft_consensus.cc:993] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8: : Instructing follower ac455e76c74d4abd95c4d73003ca9d3a to start an election
I20260709 13:06:22.667783 18572 raft_consensus.cc:1081] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 3 LEADER]: Signalling peer ac455e76c74d4abd95c4d73003ca9d3a to start an election
I20260709 13:06:22.669551 18293 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b45d6ca8b05141749dbacd2eaaf9eb49"
dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
 from {username='slave'} at 127.0.0.1:58370
I20260709 13:06:22.670023 18293 raft_consensus.cc:493] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:22.670258 18293 raft_consensus.cc:3060] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:22.675571 18293 raft_consensus.cc:515] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:22.677320 18293 leader_election.cc:290] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 4 election: Requested vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515)
I20260709 13:06:22.678166 18503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b45d6ca8b05141749dbacd2eaaf9eb49" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "500e80cda2474f66b0ad65f4bfa00258"
I20260709 13:06:22.678689 18217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b45d6ca8b05141749dbacd2eaaf9eb49" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
I20260709 13:06:22.679001 18503 raft_consensus.cc:3060] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:22.679397 18217 raft_consensus.cc:3055] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:22.679740 18217 raft_consensus.cc:740] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3272b165a59e4467925ea233bbb5f6f8, State: Running, Role: LEADER
I20260709 13:06:22.680567 18217 consensus_queue.cc:260] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:22.681886 18217 raft_consensus.cc:3060] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:22.687276 18503 raft_consensus.cc:2468] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac455e76c74d4abd95c4d73003ca9d3a in term 4.
I20260709 13:06:22.688686 18257 leader_election.cc:304] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 500e80cda2474f66b0ad65f4bfa00258, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 
I20260709 13:06:22.689623 18447 raft_consensus.cc:2804] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:22.689843 18217 raft_consensus.cc:2468] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac455e76c74d4abd95c4d73003ca9d3a in term 4.
I20260709 13:06:22.690073 18447 raft_consensus.cc:697] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 4 LEADER]: Becoming Leader. State: Replica: ac455e76c74d4abd95c4d73003ca9d3a, State: Running, Role: LEADER
I20260709 13:06:22.691109 18447 consensus_queue.cc:237] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:22.698064 18039 catalog_manager.cc:5697] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a reported cstate change: term changed from 3 to 4, leader changed from 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2) to ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3). New cstate: current_term: 4 leader_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:23.019218 18503 raft_consensus.cc:1275] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258 [term 3 FOLLOWER]: Refusing update from remote peer 3272b165a59e4467925ea233bbb5f6f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:23.020722 18570 consensus_queue.cc:1048] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:23.032826 18293 raft_consensus.cc:1275] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [term 3 FOLLOWER]: Refusing update from remote peer 3272b165a59e4467925ea233bbb5f6f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:23.035934 18572 consensus_queue.cc:1048] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:23.130344 18501 raft_consensus.cc:1275] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 4 FOLLOWER]: Refusing update from remote peer ac455e76c74d4abd95c4d73003ca9d3a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:23.131843 18447 consensus_queue.cc:1048] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:23.141677 18217 raft_consensus.cc:1275] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 4 FOLLOWER]: Refusing update from remote peer ac455e76c74d4abd95c4d73003ca9d3a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:23.143106 18542 consensus_queue.cc:1048] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:06:23.346626 18090 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:23.438921  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:23.447484 18293 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "490ba2fed8234928a81ff40ca50e58c4"
dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
mode: GRACEFUL
new_leader_uuid: "3272b165a59e4467925ea233bbb5f6f8"
 from {username='slave'} at 127.0.0.1:58398
I20260709 13:06:23.448016 18293 raft_consensus.cc:606] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 LEADER]: Received request to transfer leadership to TS 3272b165a59e4467925ea233bbb5f6f8
I20260709 13:06:23.450771 18217 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fd44efaf24c849de9baeeb9a97271dbd"
dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
mode: GRACEFUL
new_leader_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
 from {username='slave'} at 127.0.0.1:46906
I20260709 13:06:23.451283 18217 raft_consensus.cc:606] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [term 2 LEADER]: Received request to transfer leadership to TS ac455e76c74d4abd95c4d73003ca9d3a
I20260709 13:06:23.452450  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:06:23.452981  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:23.453606  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:23.513347 18596 raft_consensus.cc:993] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a: : Instructing follower 3272b165a59e4467925ea233bbb5f6f8 to start an election
I20260709 13:06:23.513978 18542 raft_consensus.cc:1081] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 LEADER]: Signalling peer 3272b165a59e4467925ea233bbb5f6f8 to start an election
I20260709 13:06:23.515825 18217 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "490ba2fed8234928a81ff40ca50e58c4"
dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
 from {username='slave'} at 127.0.0.1:46888
I20260709 13:06:23.516366 18217 raft_consensus.cc:493] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:23.516676 18217 raft_consensus.cc:3060] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:23.522230 18217 raft_consensus.cc:515] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } }
I20260709 13:06:23.524269 18217 leader_election.cc:290] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 2 election: Requested vote from peers ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323), 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331)
I20260709 13:06:23.525252 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490ba2fed8234928a81ff40ca50e58c4" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
I20260709 13:06:23.525491 18503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490ba2fed8234928a81ff40ca50e58c4" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "500e80cda2474f66b0ad65f4bfa00258"
I20260709 13:06:23.526036 18293 raft_consensus.cc:3055] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:23.526106 18503 raft_consensus.cc:3060] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:23.526461 18293 raft_consensus.cc:740] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ac455e76c74d4abd95c4d73003ca9d3a, State: Running, Role: LEADER
I20260709 13:06:23.527304 18293 consensus_queue.cc:260] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } }
I20260709 13:06:23.528791 18293 raft_consensus.cc:3060] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:23.534356 18503 raft_consensus.cc:2468] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3272b165a59e4467925ea233bbb5f6f8 in term 2.
I20260709 13:06:23.535820 18183 leader_election.cc:304] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, 500e80cda2474f66b0ad65f4bfa00258; no voters: 
I20260709 13:06:23.535908 18293 raft_consensus.cc:2468] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3272b165a59e4467925ea233bbb5f6f8 in term 2.
I20260709 13:06:23.536692 18571 raft_consensus.cc:2804] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:23.537179 18571 raft_consensus.cc:697] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 LEADER]: Becoming Leader. State: Replica: 3272b165a59e4467925ea233bbb5f6f8, State: Running, Role: LEADER
I20260709 13:06:23.538080 18571 consensus_queue.cc:237] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } }
I20260709 13:06:23.548146 18039 catalog_manager.cc:5697] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 reported cstate change: term changed from 1 to 2, leader changed from ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3) to 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "3272b165a59e4467925ea233bbb5f6f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:23.601564 18572 raft_consensus.cc:993] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8: : Instructing follower ac455e76c74d4abd95c4d73003ca9d3a to start an election
I20260709 13:06:23.602169 18571 raft_consensus.cc:1081] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [term 2 LEADER]: Signalling peer ac455e76c74d4abd95c4d73003ca9d3a to start an election
I20260709 13:06:23.603654 18293 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fd44efaf24c849de9baeeb9a97271dbd"
dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
 from {username='slave'} at 127.0.0.1:58370
I20260709 13:06:23.604115 18293 raft_consensus.cc:493] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:23.604400 18293 raft_consensus.cc:3060] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:23.609016 18293 raft_consensus.cc:515] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:23.610570 18293 leader_election.cc:290] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 3 election: Requested vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515)
I20260709 13:06:23.611646 18503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44efaf24c849de9baeeb9a97271dbd" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "500e80cda2474f66b0ad65f4bfa00258"
I20260709 13:06:23.612171 18217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44efaf24c849de9baeeb9a97271dbd" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
I20260709 13:06:23.612342 18503 raft_consensus.cc:3060] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:23.612864 18217 raft_consensus.cc:3055] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:23.613236 18217 raft_consensus.cc:740] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3272b165a59e4467925ea233bbb5f6f8, State: Running, Role: LEADER
I20260709 13:06:23.614114 18217 consensus_queue.cc:260] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:23.615409 18217 raft_consensus.cc:3060] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:23.620308 18503 raft_consensus.cc:2468] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac455e76c74d4abd95c4d73003ca9d3a in term 3.
I20260709 13:06:23.621814 18257 leader_election.cc:304] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 500e80cda2474f66b0ad65f4bfa00258, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 
I20260709 13:06:23.622625 18542 raft_consensus.cc:2804] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:23.623081 18542 raft_consensus.cc:697] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [term 3 LEADER]: Becoming Leader. State: Replica: ac455e76c74d4abd95c4d73003ca9d3a, State: Running, Role: LEADER
I20260709 13:06:23.623824 18542 consensus_queue.cc:237] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:23.623998 18217 raft_consensus.cc:2468] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac455e76c74d4abd95c4d73003ca9d3a in term 3.
I20260709 13:06:23.632077 18039 catalog_manager.cc:5697] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a reported cstate change: term changed from 2 to 3, leader changed from 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2) to ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3). New cstate: current_term: 3 leader_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:24.062247 18503 raft_consensus.cc:1275] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: Refusing update from remote peer 3272b165a59e4467925ea233bbb5f6f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:24.063722 18603 consensus_queue.cc:1048] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:24.070966 18501 raft_consensus.cc:1275] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258 [term 3 FOLLOWER]: Refusing update from remote peer ac455e76c74d4abd95c4d73003ca9d3a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:24.072670 18604 consensus_queue.cc:1048] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:24.077533 18293 raft_consensus.cc:1275] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Refusing update from remote peer 3272b165a59e4467925ea233bbb5f6f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:24.079391 18603 consensus_queue.cc:1048] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:24.097179 18217 raft_consensus.cc:1275] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [term 3 FOLLOWER]: Refusing update from remote peer ac455e76c74d4abd95c4d73003ca9d3a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:24.098733 18604 consensus_queue.cc:1048] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:06:24.347802 18090 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:24.460317  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:24.469065 18293 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b45d6ca8b05141749dbacd2eaaf9eb49"
dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
mode: GRACEFUL
new_leader_uuid: "3272b165a59e4467925ea233bbb5f6f8"
 from {username='slave'} at 127.0.0.1:58398
I20260709 13:06:24.469919 18293 raft_consensus.cc:606] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 4 LEADER]: Received request to transfer leadership to TS 3272b165a59e4467925ea233bbb5f6f8
I20260709 13:06:24.474023 18217 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fd3970d05a344a0bb83eb2970cd735e6"
dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
mode: GRACEFUL
new_leader_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
 from {username='slave'} at 127.0.0.1:46906
I20260709 13:06:24.474731 18217 raft_consensus.cc:606] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 3 LEADER]: Received request to transfer leadership to TS ac455e76c74d4abd95c4d73003ca9d3a
I20260709 13:06:24.476174  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:06:24.476672  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:24.477236  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:24.642756 18616 raft_consensus.cc:993] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a: : Instructing follower 3272b165a59e4467925ea233bbb5f6f8 to start an election
I20260709 13:06:24.643245 18604 raft_consensus.cc:1081] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 4 LEADER]: Signalling peer 3272b165a59e4467925ea233bbb5f6f8 to start an election
I20260709 13:06:24.644675 18217 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b45d6ca8b05141749dbacd2eaaf9eb49"
dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
 from {username='slave'} at 127.0.0.1:46888
I20260709 13:06:24.645248 18217 raft_consensus.cc:493] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:24.645563 18217 raft_consensus.cc:3060] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:24.650624 18217 raft_consensus.cc:515] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:24.652491 18217 leader_election.cc:290] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 5 election: Requested vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3:42323)
I20260709 13:06:24.653518 18501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b45d6ca8b05141749dbacd2eaaf9eb49" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "500e80cda2474f66b0ad65f4bfa00258"
I20260709 13:06:24.653673 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b45d6ca8b05141749dbacd2eaaf9eb49" candidate_uuid: "3272b165a59e4467925ea233bbb5f6f8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
I20260709 13:06:24.654351 18293 raft_consensus.cc:3055] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:24.654573 18293 raft_consensus.cc:740] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ac455e76c74d4abd95c4d73003ca9d3a, State: Running, Role: LEADER
I20260709 13:06:24.654886 18501 raft_consensus.cc:3060] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:24.655324 18293 consensus_queue.cc:260] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:24.656334 18293 raft_consensus.cc:3060] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:24.657514 18614 raft_consensus.cc:993] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8: : Instructing follower ac455e76c74d4abd95c4d73003ca9d3a to start an election
I20260709 13:06:24.658083 18615 raft_consensus.cc:1081] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 3 LEADER]: Signalling peer ac455e76c74d4abd95c4d73003ca9d3a to start an election
I20260709 13:06:24.659952 18292 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fd3970d05a344a0bb83eb2970cd735e6"
dest_uuid: "ac455e76c74d4abd95c4d73003ca9d3a"
 from {username='slave'} at 127.0.0.1:58370
I20260709 13:06:24.660511 18292 raft_consensus.cc:493] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:24.660859 18292 raft_consensus.cc:3060] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:24.664422 18293 raft_consensus.cc:2468] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3272b165a59e4467925ea233bbb5f6f8 in term 5.
I20260709 13:06:24.664484 18501 raft_consensus.cc:2468] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3272b165a59e4467925ea233bbb5f6f8 in term 5.
I20260709 13:06:24.665899 18182 leader_election.cc:304] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3272b165a59e4467925ea233bbb5f6f8, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 
I20260709 13:06:24.666792 18615 raft_consensus.cc:2804] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:24.667301 18615 raft_consensus.cc:697] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 5 LEADER]: Becoming Leader. State: Replica: 3272b165a59e4467925ea233bbb5f6f8, State: Running, Role: LEADER
I20260709 13:06:24.668098 18615 consensus_queue.cc:237] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:24.669349 18292 raft_consensus.cc:515] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:24.671607 18292 leader_election.cc:290] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 4 election: Requested vote from peers 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331), 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515)
I20260709 13:06:24.672792 18501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3970d05a344a0bb83eb2970cd735e6" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 4 candidate_status { last_received { term: 3 index: 1 } } ignore_live_leader: true dest_uuid: "500e80cda2474f66b0ad65f4bfa00258"
I20260709 13:06:24.673621 18501 raft_consensus.cc:3060] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:24.673751 18217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3970d05a344a0bb83eb2970cd735e6" candidate_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" candidate_term: 4 candidate_status { last_received { term: 3 index: 1 } } ignore_live_leader: true dest_uuid: "3272b165a59e4467925ea233bbb5f6f8"
I20260709 13:06:24.674467 18217 raft_consensus.cc:3055] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:24.674872 18217 raft_consensus.cc:740] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3272b165a59e4467925ea233bbb5f6f8, State: Running, Role: LEADER
I20260709 13:06:24.675891 18217 consensus_queue.cc:260] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.1, Last appended by leader: 1, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:24.677093 18217 raft_consensus.cc:3060] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:24.677155 18039 catalog_manager.cc:5697] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 reported cstate change: term changed from 4 to 5, leader changed from ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3) to 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2). New cstate: current_term: 5 leader_uuid: "3272b165a59e4467925ea233bbb5f6f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:24.681517 18501 raft_consensus.cc:2468] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac455e76c74d4abd95c4d73003ca9d3a in term 4.
I20260709 13:06:24.682931 18257 leader_election.cc:304] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 500e80cda2474f66b0ad65f4bfa00258, ac455e76c74d4abd95c4d73003ca9d3a; no voters: 
I20260709 13:06:24.683785 18604 raft_consensus.cc:2804] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:24.684219 18604 raft_consensus.cc:697] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 4 LEADER]: Becoming Leader. State: Replica: ac455e76c74d4abd95c4d73003ca9d3a, State: Running, Role: LEADER
I20260709 13:06:24.684767 18217 raft_consensus.cc:2468] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac455e76c74d4abd95c4d73003ca9d3a in term 4.
I20260709 13:06:24.685015 18604 consensus_queue.cc:237] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } }
I20260709 13:06:24.695971 18040 catalog_manager.cc:5697] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a reported cstate change: term changed from 3 to 4, leader changed from 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2) to ac455e76c74d4abd95c4d73003ca9d3a (127.6.255.3). New cstate: current_term: 4 leader_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:25.082314 18501 raft_consensus.cc:1275] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [term 4 FOLLOWER]: Refusing update from remote peer ac455e76c74d4abd95c4d73003ca9d3a: Log matching property violated. Preceding OpId in replica: term: 3 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:06:25.087096 18623 consensus_queue.cc:1048] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:25.099556 18217 raft_consensus.cc:1275] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 4 FOLLOWER]: Refusing update from remote peer ac455e76c74d4abd95c4d73003ca9d3a: Log matching property violated. Preceding OpId in replica: term: 3 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:06:25.101327 18623 consensus_queue.cc:1048] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3272b165a59e4467925ea233bbb5f6f8" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 46515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:25.126073 18292 raft_consensus.cc:1275] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 5 FOLLOWER]: Refusing update from remote peer 3272b165a59e4467925ea233bbb5f6f8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:06:25.127496 18615 consensus_queue.cc:1048] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac455e76c74d4abd95c4d73003ca9d3a" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 42323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:25.136308 18501 raft_consensus.cc:1275] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 5 FOLLOWER]: Refusing update from remote peer 3272b165a59e4467925ea233bbb5f6f8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:06:25.138725 18615 consensus_queue.cc:1048] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "500e80cda2474f66b0ad65f4bfa00258" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 44331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:06:25.348991 18090 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:25.484787  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:25.519090  7164 tablet_server.cc:179] TabletServer@127.6.255.1:44331 shutting down...
W20260709 13:06:25.529004 18183 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 93 similar messages]
W20260709 13:06:25.532558 18183 consensus_peers.cc:597] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:25.537720 18183 consensus_peers.cc:597] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:25.556584  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:25.557516  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:25.558175  7164 raft_consensus.cc:2243] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:25.558691  7164 raft_consensus.cc:2272] T a1a834c742b84d70859baac3a6ff751a P 500e80cda2474f66b0ad65f4bfa00258 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.560840  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:25.561561  7164 raft_consensus.cc:2243] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:25.562014  7164 raft_consensus.cc:2272] T 24794fe9a88c4959aa0101a5e612121f P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.563967  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:25.564551  7164 raft_consensus.cc:2243] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:25.565003  7164 raft_consensus.cc:2272] T 490ba2fed8234928a81ff40ca50e58c4 P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.567040  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:25.567610  7164 raft_consensus.cc:2243] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:25.568061  7164 raft_consensus.cc:2272] T 308eb63baa0f4f51b9116e3b240b0932 P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.570562  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:25.571310  7164 raft_consensus.cc:2243] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:25.571919  7164 raft_consensus.cc:2272] T b45d6ca8b05141749dbacd2eaaf9eb49 P 500e80cda2474f66b0ad65f4bfa00258 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.575156  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:25.575845  7164 raft_consensus.cc:2243] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:25.576325  7164 raft_consensus.cc:2272] T 8befc400255448babc5d5b6432d74740 P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.578356  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:25.578971  7164 raft_consensus.cc:2243] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:25.579465  7164 raft_consensus.cc:2272] T fd3970d05a344a0bb83eb2970cd735e6 P 500e80cda2474f66b0ad65f4bfa00258 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.582165  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:25.582967  7164 raft_consensus.cc:2243] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:25.583541  7164 raft_consensus.cc:2272] T a4a89e1ce0e14a47848477495dc4bb7b P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.586024  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:25.586584  7164 raft_consensus.cc:2243] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:25.587028  7164 raft_consensus.cc:2272] T 5214f540f4d64609b515bb6b540791ec P 500e80cda2474f66b0ad65f4bfa00258 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.589057  7164 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:25.589659  7164 raft_consensus.cc:2243] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:25.590117  7164 raft_consensus.cc:2272] T fd44efaf24c849de9baeeb9a97271dbd P 500e80cda2474f66b0ad65f4bfa00258 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:25.603746 18257 consensus_peers.cc:597] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:25.611943  7164 tablet_server.cc:196] TabletServer@127.6.255.1:44331 shutdown complete.
W20260709 13:06:25.621821 18257 consensus_peers.cc:597] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:25.633662  7164 tablet_server.cc:179] TabletServer@127.6.255.2:0 shutting down...
W20260709 13:06:25.648674 18183 consensus_peers.cc:597] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:25.661592  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:25.664295  7164 tablet_replica.cc:333] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8: stopping tablet replica
I20260709 13:06:25.665256  7164 raft_consensus.cc:2243] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:25.666256  7164 raft_consensus.cc:2272] T 24794fe9a88c4959aa0101a5e612121f P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.668692  7164 tablet_replica.cc:333] T 8befc400255448babc5d5b6432d74740 P 3272b165a59e4467925ea233bbb5f6f8: stopping tablet replica
I20260709 13:06:25.669281  7164 raft_consensus.cc:2243] T 8befc400255448babc5d5b6432d74740 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:25.669842  7164 raft_consensus.cc:2272] T 8befc400255448babc5d5b6432d74740 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.672868  7164 tablet_replica.cc:333] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8: stopping tablet replica
I20260709 13:06:25.673764  7164 raft_consensus.cc:2243] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 5 LEADER]: Raft consensus shutting down.
W20260709 13:06:25.674005 18257 consensus_peers.cc:597] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:25.674768  7164 raft_consensus.cc:2272] T b45d6ca8b05141749dbacd2eaaf9eb49 P 3272b165a59e4467925ea233bbb5f6f8 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.677417  7164 tablet_replica.cc:333] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8: stopping tablet replica
I20260709 13:06:25.678026  7164 raft_consensus.cc:2243] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:25.678416  7164 raft_consensus.cc:2272] T a4a89e1ce0e14a47848477495dc4bb7b P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.680330  7164 tablet_replica.cc:333] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8: stopping tablet replica
I20260709 13:06:25.680800  7164 raft_consensus.cc:2243] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:25.681164  7164 raft_consensus.cc:2272] T fd3970d05a344a0bb83eb2970cd735e6 P 3272b165a59e4467925ea233bbb5f6f8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.683941  7164 tablet_replica.cc:333] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8: stopping tablet replica
I20260709 13:06:25.684490  7164 raft_consensus.cc:2243] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:25.684871  7164 raft_consensus.cc:2272] T fd44efaf24c849de9baeeb9a97271dbd P 3272b165a59e4467925ea233bbb5f6f8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.687074  7164 tablet_replica.cc:333] T 5214f540f4d64609b515bb6b540791ec P 3272b165a59e4467925ea233bbb5f6f8: stopping tablet replica
I20260709 13:06:25.687572  7164 raft_consensus.cc:2243] T 5214f540f4d64609b515bb6b540791ec P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:25.687954  7164 raft_consensus.cc:2272] T 5214f540f4d64609b515bb6b540791ec P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.690239  7164 tablet_replica.cc:333] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8: stopping tablet replica
I20260709 13:06:25.690699  7164 raft_consensus.cc:2243] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:25.691390  7164 raft_consensus.cc:2272] T a1a834c742b84d70859baac3a6ff751a P 3272b165a59e4467925ea233bbb5f6f8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.693409  7164 tablet_replica.cc:333] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8: stopping tablet replica
I20260709 13:06:25.693950  7164 raft_consensus.cc:2243] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:25.694710  7164 raft_consensus.cc:2272] T 308eb63baa0f4f51b9116e3b240b0932 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.696872  7164 tablet_replica.cc:333] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8: stopping tablet replica
I20260709 13:06:25.697283  7164 raft_consensus.cc:2243] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:25.698091  7164 raft_consensus.cc:2272] T 490ba2fed8234928a81ff40ca50e58c4 P 3272b165a59e4467925ea233bbb5f6f8 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:25.712735 18257 consensus_peers.cc:597] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a -> Peer 3272b165a59e4467925ea233bbb5f6f8 (127.6.255.2:46515): Couldn't send request to peer 3272b165a59e4467925ea233bbb5f6f8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:25.734184  7164 tablet_server.cc:196] TabletServer@127.6.255.2:0 shutdown complete.
W20260709 13:06:25.735152 18257 consensus_peers.cc:597] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a -> Peer 500e80cda2474f66b0ad65f4bfa00258 (127.6.255.1:44331): Couldn't send request to peer 500e80cda2474f66b0ad65f4bfa00258. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:44331: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:25.759997  7164 tablet_server.cc:179] TabletServer@127.6.255.3:0 shutting down...
I20260709 13:06:25.791080  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:25.791826  7164 tablet_replica.cc:333] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a: stopping tablet replica
I20260709 13:06:25.792649  7164 raft_consensus.cc:2243] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:25.793591  7164 raft_consensus.cc:2272] T a4a89e1ce0e14a47848477495dc4bb7b P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.795743  7164 tablet_replica.cc:333] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a: stopping tablet replica
I20260709 13:06:25.796692  7164 raft_consensus.cc:2243] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:06:25.797649  7164 raft_consensus.cc:2272] T fd3970d05a344a0bb83eb2970cd735e6 P ac455e76c74d4abd95c4d73003ca9d3a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.800716  7164 tablet_replica.cc:333] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a: stopping tablet replica
I20260709 13:06:25.801831  7164 raft_consensus.cc:2243] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:25.802803  7164 raft_consensus.cc:2272] T 8befc400255448babc5d5b6432d74740 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.805349  7164 tablet_replica.cc:333] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a: stopping tablet replica
I20260709 13:06:25.806108  7164 raft_consensus.cc:2243] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:25.806617  7164 raft_consensus.cc:2272] T b45d6ca8b05141749dbacd2eaaf9eb49 P ac455e76c74d4abd95c4d73003ca9d3a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.810009  7164 tablet_replica.cc:333] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a: stopping tablet replica
I20260709 13:06:25.810647  7164 raft_consensus.cc:2243] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:25.811187  7164 raft_consensus.cc:2272] T 308eb63baa0f4f51b9116e3b240b0932 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.813458  7164 tablet_replica.cc:333] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a: stopping tablet replica
I20260709 13:06:25.814052  7164 raft_consensus.cc:2243] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:25.814484  7164 raft_consensus.cc:2272] T 490ba2fed8234928a81ff40ca50e58c4 P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.816699  7164 tablet_replica.cc:333] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a: stopping tablet replica
I20260709 13:06:25.817368  7164 raft_consensus.cc:2243] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:25.817901  7164 raft_consensus.cc:2272] T 24794fe9a88c4959aa0101a5e612121f P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.820077  7164 tablet_replica.cc:333] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a: stopping tablet replica
I20260709 13:06:25.820618  7164 raft_consensus.cc:2243] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:25.821419  7164 raft_consensus.cc:2272] T a1a834c742b84d70859baac3a6ff751a P ac455e76c74d4abd95c4d73003ca9d3a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.824041  7164 tablet_replica.cc:333] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a: stopping tablet replica
I20260709 13:06:25.824618  7164 raft_consensus.cc:2243] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:25.825559  7164 raft_consensus.cc:2272] T fd44efaf24c849de9baeeb9a97271dbd P ac455e76c74d4abd95c4d73003ca9d3a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.828154  7164 tablet_replica.cc:333] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a: stopping tablet replica
I20260709 13:06:25.828687  7164 raft_consensus.cc:2243] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:25.829815  7164 raft_consensus.cc:2272] T 5214f540f4d64609b515bb6b540791ec P ac455e76c74d4abd95c4d73003ca9d3a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.864594  7164 tablet_server.cc:196] TabletServer@127.6.255.3:0 shutdown complete.
I20260709 13:06:25.892258  7164 master.cc:562] Master@127.6.255.62:37899 shutting down...
I20260709 13:06:25.921283  7164 raft_consensus.cc:2243] T 00000000000000000000000000000000 P abdef627a1e54172bbbc993071d7844f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:25.922049  7164 raft_consensus.cc:2272] T 00000000000000000000000000000000 P abdef627a1e54172bbbc993071d7844f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:25.922415  7164 tablet_replica.cc:333] T 00000000000000000000000000000000 P abdef627a1e54172bbbc993071d7844f: stopping tablet replica
I20260709 13:06:25.962244  7164 master.cc:584] Master@127.6.255.62:37899 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13833 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:06:26.016309  7164 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.255.62:38177
I20260709 13:06:26.017665  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:26.024315 18643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:26.024513 18642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:26.025561 18645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:26.026122  7164 server_base.cc:1061] running on GCE node
I20260709 13:06:26.027187  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:26.027426  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:26.027637  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602386027598 us; error 0 us; skew 500 ppm
I20260709 13:06:26.028225  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:26.031323  7164 webserver.cc:533] Webserver started at http://127.6.255.62:45851/ using document root <none> and password file <none>
I20260709 13:06:26.031867  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:26.032049  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:26.032410  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:26.033722  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602046064212-7164-0/minicluster-data/master-0-root/instance:
uuid: "5714b850b3de46dbb7d5ec4120bc5bbf"
format_stamp: "Formatted at 2026-07-09 13:06:26 on dist-test-slave-rnv4"
I20260709 13:06:26.038925  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:06:26.042999 18650 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:26.043931  7164 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:06:26.044341  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602046064212-7164-0/minicluster-data/master-0-root
uuid: "5714b850b3de46dbb7d5ec4120bc5bbf"
format_stamp: "Formatted at 2026-07-09 13:06:26 on dist-test-slave-rnv4"
I20260709 13:06:26.044706  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602046064212-7164-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602046064212-7164-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602046064212-7164-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:26.076162  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:26.077658  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:26.123519  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.62:38177
I20260709 13:06:26.123780 18701 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.62:38177 every 8 connection(s)
I20260709 13:06:26.127779 18702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:26.140273 18702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5714b850b3de46dbb7d5ec4120bc5bbf: Bootstrap starting.
I20260709 13:06:26.145722 18702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5714b850b3de46dbb7d5ec4120bc5bbf: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.151060 18702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5714b850b3de46dbb7d5ec4120bc5bbf: No bootstrap required, opened a new log
I20260709 13:06:26.153366 18702 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5714b850b3de46dbb7d5ec4120bc5bbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5714b850b3de46dbb7d5ec4120bc5bbf" member_type: VOTER }
I20260709 13:06:26.153800 18702 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5714b850b3de46dbb7d5ec4120bc5bbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.154065 18702 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5714b850b3de46dbb7d5ec4120bc5bbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5714b850b3de46dbb7d5ec4120bc5bbf, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.154678 18702 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5714b850b3de46dbb7d5ec4120bc5bbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5714b850b3de46dbb7d5ec4120bc5bbf" member_type: VOTER }
I20260709 13:06:26.155202 18702 raft_consensus.cc:399] T 00000000000000000000000000000000 P 5714b850b3de46dbb7d5ec4120bc5bbf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:06:26.155432 18702 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5714b850b3de46dbb7d5ec4120bc5bbf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:06:26.155735 18702 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5714b850b3de46dbb7d5ec4120bc5bbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.162075 18702 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5714b850b3de46dbb7d5ec4120bc5bbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5714b850b3de46dbb7d5ec4120bc5bbf" member_type: VOTER }
I20260709 13:06:26.162833 18702 leader_election.cc:304] T 00000000000000000000000000000000 P 5714b850b3de46dbb7d5ec4120bc5bbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5714b850b3de46dbb7d5ec4120bc5bbf; no voters: 
I20260709 13:06:26.164168 18702 leader_election.cc:290] T 00000000000000000000000000000000 P 5714b850b3de46dbb7d5ec4120bc5bbf [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:06:26.164549 18705 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5714b850b3de46dbb7d5ec4120bc5bbf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:26.166159 18705 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5714b850b3de46dbb7d5ec4120bc5bbf [term 1 LEADER]: Becoming Leader. State: Replica: 5714b850b3de46dbb7d5ec4120bc5bbf, State: Running, Role: LEADER
I20260709 13:06:26.166918 18705 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5714b850b3de46dbb7d5ec4120bc5bbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5714b850b3de46dbb7d5ec4120bc5bbf" member_type: VOTER }
I20260709 13:06:26.167189 18702 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5714b850b3de46dbb7d5ec4120bc5bbf [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:06:26.171838 18706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5714b850b3de46dbb7d5ec4120bc5bbf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5714b850b3de46dbb7d5ec4120bc5bbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5714b850b3de46dbb7d5ec4120bc5bbf" member_type: VOTER } }
I20260709 13:06:26.172294 18707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5714b850b3de46dbb7d5ec4120bc5bbf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5714b850b3de46dbb7d5ec4120bc5bbf. Latest consensus state: current_term: 1 leader_uuid: "5714b850b3de46dbb7d5ec4120bc5bbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5714b850b3de46dbb7d5ec4120bc5bbf" member_type: VOTER } }
I20260709 13:06:26.173127 18706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5714b850b3de46dbb7d5ec4120bc5bbf [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:26.173202 18707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5714b850b3de46dbb7d5ec4120bc5bbf [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:26.178683 18714 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:06:26.183790 18714 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:06:26.184943  7164 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:06:26.194275 18714 catalog_manager.cc:1383] Generated new cluster ID: efe3c0907fb44acfa004700e33776eb6
I20260709 13:06:26.194666 18714 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:06:26.222400 18714 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:06:26.224133 18714 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:06:26.254057 18714 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 5714b850b3de46dbb7d5ec4120bc5bbf: Generated new TSK 0
I20260709 13:06:26.254787 18714 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:06:26.319307  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:26.326280 18723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:26.327566 18724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:26.329252 18726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:26.329921  7164 server_base.cc:1061] running on GCE node
I20260709 13:06:26.331061  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:26.331349  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:26.331540  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602386331519 us; error 0 us; skew 500 ppm
I20260709 13:06:26.332265  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:26.334790  7164 webserver.cc:533] Webserver started at http://127.6.255.1:46847/ using document root <none> and password file <none>
I20260709 13:06:26.335258  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:26.335417  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:26.335611  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:26.336844  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602046064212-7164-0/minicluster-data/ts-0-root/instance:
uuid: "2e4bd2ed8fe9405793d0bab5682997c5"
format_stamp: "Formatted at 2026-07-09 13:06:26 on dist-test-slave-rnv4"
I20260709 13:06:26.341796  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:06:26.345384 18731 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:26.346581  7164 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:06:26.347043  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602046064212-7164-0/minicluster-data/ts-0-root
uuid: "2e4bd2ed8fe9405793d0bab5682997c5"
format_stamp: "Formatted at 2026-07-09 13:06:26 on dist-test-slave-rnv4"
I20260709 13:06:26.347390  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602046064212-7164-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602046064212-7164-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602046064212-7164-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:26.360822  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:26.362357  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:26.364176  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:26.367102  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:26.367358  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:26.367697  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:26.367892  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:26.416821  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.1:39709
I20260709 13:06:26.416917 18793 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.1:39709 every 8 connection(s)
I20260709 13:06:26.421953  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:26.428920 18798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:26.433105 18799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:26.436002  7164 server_base.cc:1061] running on GCE node
I20260709 13:06:26.436124 18794 heartbeater.cc:344] Connected to a master server at 127.6.255.62:38177
W20260709 13:06:26.436559 18801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:26.437038 18794 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:26.437911  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:26.438220  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:26.438129 18794 heartbeater.cc:507] Master 127.6.255.62:38177 requested a full tablet report, sending...
I20260709 13:06:26.438586  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602386438546 us; error 0 us; skew 500 ppm
I20260709 13:06:26.439260  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:26.440915 18667 ts_manager.cc:194] Registered new tserver with Master: 2e4bd2ed8fe9405793d0bab5682997c5 (127.6.255.1:39709)
I20260709 13:06:26.442039  7164 webserver.cc:533] Webserver started at http://127.6.255.2:38767/ using document root <none> and password file <none>
I20260709 13:06:26.442605  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:26.442822  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:26.443096  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:26.443501 18667 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35078
I20260709 13:06:26.444339  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602046064212-7164-0/minicluster-data/ts-1-root/instance:
uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6"
format_stamp: "Formatted at 2026-07-09 13:06:26 on dist-test-slave-rnv4"
I20260709 13:06:26.449415  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.003s
I20260709 13:06:26.453066 18806 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:26.453979  7164 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:06:26.454296  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602046064212-7164-0/minicluster-data/ts-1-root
uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6"
format_stamp: "Formatted at 2026-07-09 13:06:26 on dist-test-slave-rnv4"
I20260709 13:06:26.454615  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602046064212-7164-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602046064212-7164-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602046064212-7164-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:26.513829  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:26.515300  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:26.517148  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:26.519759  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:26.520020  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:26.520341  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:26.520761  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:26.569813  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.2:39573
I20260709 13:06:26.569931 18868 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.2:39573 every 8 connection(s)
I20260709 13:06:26.575343  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:26.584651 18872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:26.585973 18873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:26.589480 18869 heartbeater.cc:344] Connected to a master server at 127.6.255.62:38177
W20260709 13:06:26.589996 18875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:26.590114 18869 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:26.590641  7164 server_base.cc:1061] running on GCE node
I20260709 13:06:26.591140 18869 heartbeater.cc:507] Master 127.6.255.62:38177 requested a full tablet report, sending...
I20260709 13:06:26.591524  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:26.591787  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:26.591979  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602386591953 us; error 0 us; skew 500 ppm
I20260709 13:06:26.592578  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:26.593263 18667 ts_manager.cc:194] Registered new tserver with Master: 1e6b1a8ec1f74a2ebfab7f19e85d60d6 (127.6.255.2:39573)
I20260709 13:06:26.595304 18667 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35088
I20260709 13:06:26.595496  7164 webserver.cc:533] Webserver started at http://127.6.255.3:40659/ using document root <none> and password file <none>
I20260709 13:06:26.596163  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:26.596412  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:26.596693  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:26.597913  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602046064212-7164-0/minicluster-data/ts-2-root/instance:
uuid: "ab011c3efbec40e98169d325dc0fcc2d"
format_stamp: "Formatted at 2026-07-09 13:06:26 on dist-test-slave-rnv4"
I20260709 13:06:26.602639  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 13:06:26.606238 18880 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:26.607075  7164 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:06:26.607406  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602046064212-7164-0/minicluster-data/ts-2-root
uuid: "ab011c3efbec40e98169d325dc0fcc2d"
format_stamp: "Formatted at 2026-07-09 13:06:26 on dist-test-slave-rnv4"
I20260709 13:06:26.607708  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602046064212-7164-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602046064212-7164-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602046064212-7164-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:26.629772  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:26.630990  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:26.632509  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:26.634809  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:26.635066  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:26.635306  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:26.635480  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:26.678926  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.3:44099
I20260709 13:06:26.679014 18942 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.3:44099 every 8 connection(s)
I20260709 13:06:26.696210 18943 heartbeater.cc:344] Connected to a master server at 127.6.255.62:38177
I20260709 13:06:26.696727 18943 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:26.697633 18943 heartbeater.cc:507] Master 127.6.255.62:38177 requested a full tablet report, sending...
I20260709 13:06:26.699757 18667 ts_manager.cc:194] Registered new tserver with Master: ab011c3efbec40e98169d325dc0fcc2d (127.6.255.3:44099)
I20260709 13:06:26.699888  7164 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01777716s
I20260709 13:06:26.701623 18667 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35090
I20260709 13:06:26.728721 18667 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35092:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:26.731281 18667 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:26.785180 18908 tablet_service.cc:1511] Processing CreateTablet for tablet 48da62dff3be449cb227262ba5b0e2cf (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=dd81f7140e2347cca24566ecebb3664f]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:06:26.785194 18907 tablet_service.cc:1511] Processing CreateTablet for tablet 6d563bf7fb5e4132948e208bf86673c6 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=dd81f7140e2347cca24566ecebb3664f]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:06:26.786934 18908 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48da62dff3be449cb227262ba5b0e2cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:26.787612 18907 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d563bf7fb5e4132948e208bf86673c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:26.795128 18759 tablet_service.cc:1511] Processing CreateTablet for tablet 48da62dff3be449cb227262ba5b0e2cf (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=dd81f7140e2347cca24566ecebb3664f]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:06:26.796648 18759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48da62dff3be449cb227262ba5b0e2cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:26.797673 18834 tablet_service.cc:1511] Processing CreateTablet for tablet 48da62dff3be449cb227262ba5b0e2cf (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=dd81f7140e2347cca24566ecebb3664f]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:06:26.797992 18833 tablet_service.cc:1511] Processing CreateTablet for tablet 6d563bf7fb5e4132948e208bf86673c6 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=dd81f7140e2347cca24566ecebb3664f]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:06:26.799120 18834 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48da62dff3be449cb227262ba5b0e2cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:26.802208 18833 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d563bf7fb5e4132948e208bf86673c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:26.804246 18758 tablet_service.cc:1511] Processing CreateTablet for tablet 6d563bf7fb5e4132948e208bf86673c6 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=dd81f7140e2347cca24566ecebb3664f]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:06:26.805826 18758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d563bf7fb5e4132948e208bf86673c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:26.821800 18958 tablet_bootstrap.cc:492] T 48da62dff3be449cb227262ba5b0e2cf P ab011c3efbec40e98169d325dc0fcc2d: Bootstrap starting.
I20260709 13:06:26.828979 18958 tablet_bootstrap.cc:654] T 48da62dff3be449cb227262ba5b0e2cf P ab011c3efbec40e98169d325dc0fcc2d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.838099 18960 tablet_bootstrap.cc:492] T 6d563bf7fb5e4132948e208bf86673c6 P 2e4bd2ed8fe9405793d0bab5682997c5: Bootstrap starting.
I20260709 13:06:26.838227 18959 tablet_bootstrap.cc:492] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6: Bootstrap starting.
I20260709 13:06:26.841135 18958 tablet_bootstrap.cc:492] T 48da62dff3be449cb227262ba5b0e2cf P ab011c3efbec40e98169d325dc0fcc2d: No bootstrap required, opened a new log
I20260709 13:06:26.841825 18958 ts_tablet_manager.cc:1403] T 48da62dff3be449cb227262ba5b0e2cf P ab011c3efbec40e98169d325dc0fcc2d: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.004s
I20260709 13:06:26.845467 18958 raft_consensus.cc:359] T 48da62dff3be449cb227262ba5b0e2cf P ab011c3efbec40e98169d325dc0fcc2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } } peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } }
I20260709 13:06:26.846487 18958 raft_consensus.cc:385] T 48da62dff3be449cb227262ba5b0e2cf P ab011c3efbec40e98169d325dc0fcc2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.846865 18958 raft_consensus.cc:740] T 48da62dff3be449cb227262ba5b0e2cf P ab011c3efbec40e98169d325dc0fcc2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab011c3efbec40e98169d325dc0fcc2d, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.847702 18960 tablet_bootstrap.cc:654] T 6d563bf7fb5e4132948e208bf86673c6 P 2e4bd2ed8fe9405793d0bab5682997c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.847738 18958 consensus_queue.cc:260] T 48da62dff3be449cb227262ba5b0e2cf P ab011c3efbec40e98169d325dc0fcc2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } } peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } }
I20260709 13:06:26.848757 18959 tablet_bootstrap.cc:654] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.850687 18943 heartbeater.cc:499] Master 127.6.255.62:38177 was elected leader, sending a full tablet report...
I20260709 13:06:26.851421 18958 ts_tablet_manager.cc:1434] T 48da62dff3be449cb227262ba5b0e2cf P ab011c3efbec40e98169d325dc0fcc2d: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:06:26.852666 18958 tablet_bootstrap.cc:492] T 6d563bf7fb5e4132948e208bf86673c6 P ab011c3efbec40e98169d325dc0fcc2d: Bootstrap starting.
I20260709 13:06:26.860347 18960 tablet_bootstrap.cc:492] T 6d563bf7fb5e4132948e208bf86673c6 P 2e4bd2ed8fe9405793d0bab5682997c5: No bootstrap required, opened a new log
I20260709 13:06:26.860983 18960 ts_tablet_manager.cc:1403] T 6d563bf7fb5e4132948e208bf86673c6 P 2e4bd2ed8fe9405793d0bab5682997c5: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.005s
I20260709 13:06:26.863471 18958 tablet_bootstrap.cc:654] T 6d563bf7fb5e4132948e208bf86673c6 P ab011c3efbec40e98169d325dc0fcc2d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.864274 18959 tablet_bootstrap.cc:492] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6: No bootstrap required, opened a new log
I20260709 13:06:26.864836 18959 ts_tablet_manager.cc:1403] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6: Time spent bootstrapping tablet: real 0.027s	user 0.018s	sys 0.005s
I20260709 13:06:26.866364 18960 raft_consensus.cc:359] T 6d563bf7fb5e4132948e208bf86673c6 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:26.867606 18960 raft_consensus.cc:385] T 6d563bf7fb5e4132948e208bf86673c6 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.868160 18960 raft_consensus.cc:740] T 6d563bf7fb5e4132948e208bf86673c6 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e4bd2ed8fe9405793d0bab5682997c5, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.868602 18959 raft_consensus.cc:359] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:26.869463 18959 raft_consensus.cc:385] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.870401 18959 raft_consensus.cc:740] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e6b1a8ec1f74a2ebfab7f19e85d60d6, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.869418 18960 consensus_queue.cc:260] T 6d563bf7fb5e4132948e208bf86673c6 P 2e4bd2ed8fe9405793d0bab5682997c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:26.871604 18959 consensus_queue.cc:260] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:26.874747 18869 heartbeater.cc:499] Master 127.6.255.62:38177 was elected leader, sending a full tablet report...
I20260709 13:06:26.875221 18959 ts_tablet_manager.cc:1434] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 13:06:26.876405 18959 tablet_bootstrap.cc:492] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6: Bootstrap starting.
I20260709 13:06:26.877265 18958 tablet_bootstrap.cc:492] T 6d563bf7fb5e4132948e208bf86673c6 P ab011c3efbec40e98169d325dc0fcc2d: No bootstrap required, opened a new log
I20260709 13:06:26.878140 18958 ts_tablet_manager.cc:1403] T 6d563bf7fb5e4132948e208bf86673c6 P ab011c3efbec40e98169d325dc0fcc2d: Time spent bootstrapping tablet: real 0.026s	user 0.022s	sys 0.000s
I20260709 13:06:26.884128 18959 tablet_bootstrap.cc:654] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.883227 18958 raft_consensus.cc:359] T 6d563bf7fb5e4132948e208bf86673c6 P ab011c3efbec40e98169d325dc0fcc2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:26.884907 18958 raft_consensus.cc:385] T 6d563bf7fb5e4132948e208bf86673c6 P ab011c3efbec40e98169d325dc0fcc2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.885377 18958 raft_consensus.cc:740] T 6d563bf7fb5e4132948e208bf86673c6 P ab011c3efbec40e98169d325dc0fcc2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab011c3efbec40e98169d325dc0fcc2d, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.887799 18958 consensus_queue.cc:260] T 6d563bf7fb5e4132948e208bf86673c6 P ab011c3efbec40e98169d325dc0fcc2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:26.889374 18794 heartbeater.cc:499] Master 127.6.255.62:38177 was elected leader, sending a full tablet report...
I20260709 13:06:26.890100 18960 ts_tablet_manager.cc:1434] T 6d563bf7fb5e4132948e208bf86673c6 P 2e4bd2ed8fe9405793d0bab5682997c5: Time spent starting tablet: real 0.029s	user 0.015s	sys 0.014s
I20260709 13:06:26.890717 18958 ts_tablet_manager.cc:1434] T 6d563bf7fb5e4132948e208bf86673c6 P ab011c3efbec40e98169d325dc0fcc2d: Time spent starting tablet: real 0.012s	user 0.011s	sys 0.000s
I20260709 13:06:26.892059 18960 tablet_bootstrap.cc:492] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5: Bootstrap starting.
I20260709 13:06:26.897598 18959 tablet_bootstrap.cc:492] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6: No bootstrap required, opened a new log
I20260709 13:06:26.898376 18959 ts_tablet_manager.cc:1403] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.000s
I20260709 13:06:26.899825 18960 tablet_bootstrap.cc:654] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.902266 18959 raft_consensus.cc:359] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } } peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } }
I20260709 13:06:26.903461 18959 raft_consensus.cc:385] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.903789 18959 raft_consensus.cc:740] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e6b1a8ec1f74a2ebfab7f19e85d60d6, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.904702 18959 consensus_queue.cc:260] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } } peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } }
I20260709 13:06:26.907135 18959 ts_tablet_manager.cc:1434] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:06:26.911291 18960 tablet_bootstrap.cc:492] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5: No bootstrap required, opened a new log
I20260709 13:06:26.911948 18960 ts_tablet_manager.cc:1403] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.000s
I20260709 13:06:26.915153 18960 raft_consensus.cc:359] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } } peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } }
I20260709 13:06:26.915920 18960 raft_consensus.cc:385] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.916258 18960 raft_consensus.cc:740] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e4bd2ed8fe9405793d0bab5682997c5, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.917023 18960 consensus_queue.cc:260] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } } peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } }
I20260709 13:06:26.919198 18960 ts_tablet_manager.cc:1434] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:06:26.954551 18967 raft_consensus.cc:493] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.955293 18967 raft_consensus.cc:515] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } } peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } }
I20260709 13:06:26.958509 18967 leader_election.cc:290] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab011c3efbec40e98169d325dc0fcc2d (127.6.255.3:44099), 2e4bd2ed8fe9405793d0bab5682997c5 (127.6.255.1:39709)
I20260709 13:06:26.971145 18769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48da62dff3be449cb227262ba5b0e2cf" candidate_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" is_pre_election: true
I20260709 13:06:26.971902 18967 raft_consensus.cc:493] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.971774 18918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48da62dff3be449cb227262ba5b0e2cf" candidate_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab011c3efbec40e98169d325dc0fcc2d" is_pre_election: true
I20260709 13:06:26.972368 18769 raft_consensus.cc:2468] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e6b1a8ec1f74a2ebfab7f19e85d60d6 in term 0.
I20260709 13:06:26.972616 18967 raft_consensus.cc:515] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:26.972995 18918 raft_consensus.cc:2468] T 48da62dff3be449cb227262ba5b0e2cf P ab011c3efbec40e98169d325dc0fcc2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e6b1a8ec1f74a2ebfab7f19e85d60d6 in term 0.
I20260709 13:06:26.974311 18807 leader_election.cc:304] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e6b1a8ec1f74a2ebfab7f19e85d60d6, 2e4bd2ed8fe9405793d0bab5682997c5; no voters: 
I20260709 13:06:26.975106 18967 leader_election.cc:290] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e4bd2ed8fe9405793d0bab5682997c5 (127.6.255.1:39709), ab011c3efbec40e98169d325dc0fcc2d (127.6.255.3:44099)
I20260709 13:06:26.975888 18967 raft_consensus.cc:2804] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:26.976310 18967 raft_consensus.cc:493] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:26.976689 18967 raft_consensus.cc:3060] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.977481 18769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d563bf7fb5e4132948e208bf86673c6" candidate_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" is_pre_election: true
I20260709 13:06:26.977725 18918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d563bf7fb5e4132948e208bf86673c6" candidate_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab011c3efbec40e98169d325dc0fcc2d" is_pre_election: true
I20260709 13:06:26.978219 18769 raft_consensus.cc:2468] T 6d563bf7fb5e4132948e208bf86673c6 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e6b1a8ec1f74a2ebfab7f19e85d60d6 in term 0.
I20260709 13:06:26.978355 18918 raft_consensus.cc:2468] T 6d563bf7fb5e4132948e208bf86673c6 P ab011c3efbec40e98169d325dc0fcc2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e6b1a8ec1f74a2ebfab7f19e85d60d6 in term 0.
I20260709 13:06:26.979986 18807 leader_election.cc:304] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e6b1a8ec1f74a2ebfab7f19e85d60d6, 2e4bd2ed8fe9405793d0bab5682997c5; no voters: 
I20260709 13:06:26.981217 18971 raft_consensus.cc:2804] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:26.981762 18971 raft_consensus.cc:493] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:26.982054 18971 raft_consensus.cc:3060] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.984362 18967 raft_consensus.cc:515] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } } peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } }
I20260709 13:06:26.986299 18967 leader_election.cc:290] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [CANDIDATE]: Term 1 election: Requested vote from peers ab011c3efbec40e98169d325dc0fcc2d (127.6.255.3:44099), 2e4bd2ed8fe9405793d0bab5682997c5 (127.6.255.1:39709)
I20260709 13:06:26.987182 18918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48da62dff3be449cb227262ba5b0e2cf" candidate_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab011c3efbec40e98169d325dc0fcc2d"
I20260709 13:06:26.987360 18769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48da62dff3be449cb227262ba5b0e2cf" candidate_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e4bd2ed8fe9405793d0bab5682997c5"
I20260709 13:06:26.987888 18918 raft_consensus.cc:3060] T 48da62dff3be449cb227262ba5b0e2cf P ab011c3efbec40e98169d325dc0fcc2d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.988023 18769 raft_consensus.cc:3060] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.988713 18971 raft_consensus.cc:515] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:26.991185 18971 leader_election.cc:290] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [CANDIDATE]: Term 1 election: Requested vote from peers 2e4bd2ed8fe9405793d0bab5682997c5 (127.6.255.1:39709), ab011c3efbec40e98169d325dc0fcc2d (127.6.255.3:44099)
I20260709 13:06:26.992079 18768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d563bf7fb5e4132948e208bf86673c6" candidate_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e4bd2ed8fe9405793d0bab5682997c5"
I20260709 13:06:26.992933 18768 raft_consensus.cc:3060] T 6d563bf7fb5e4132948e208bf86673c6 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.993208 18917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d563bf7fb5e4132948e208bf86673c6" candidate_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab011c3efbec40e98169d325dc0fcc2d"
I20260709 13:06:26.994109 18917 raft_consensus.cc:3060] T 6d563bf7fb5e4132948e208bf86673c6 P ab011c3efbec40e98169d325dc0fcc2d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.997026 18769 raft_consensus.cc:2468] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e6b1a8ec1f74a2ebfab7f19e85d60d6 in term 1.
I20260709 13:06:26.997040 18918 raft_consensus.cc:2468] T 48da62dff3be449cb227262ba5b0e2cf P ab011c3efbec40e98169d325dc0fcc2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e6b1a8ec1f74a2ebfab7f19e85d60d6 in term 1.
I20260709 13:06:26.998462 18807 leader_election.cc:304] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e6b1a8ec1f74a2ebfab7f19e85d60d6, 2e4bd2ed8fe9405793d0bab5682997c5; no voters: 
I20260709 13:06:26.999436 18971 raft_consensus.cc:2804] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:26.999982 18971 raft_consensus.cc:697] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 1 LEADER]: Becoming Leader. State: Replica: 1e6b1a8ec1f74a2ebfab7f19e85d60d6, State: Running, Role: LEADER
I20260709 13:06:27.001240 18971 consensus_queue.cc:237] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } } peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } }
I20260709 13:06:27.002589 18768 raft_consensus.cc:2468] T 6d563bf7fb5e4132948e208bf86673c6 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e6b1a8ec1f74a2ebfab7f19e85d60d6 in term 1.
I20260709 13:06:27.003767 18917 raft_consensus.cc:2468] T 6d563bf7fb5e4132948e208bf86673c6 P ab011c3efbec40e98169d325dc0fcc2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e6b1a8ec1f74a2ebfab7f19e85d60d6 in term 1.
I20260709 13:06:27.004179 18807 leader_election.cc:304] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e6b1a8ec1f74a2ebfab7f19e85d60d6, 2e4bd2ed8fe9405793d0bab5682997c5; no voters: 
I20260709 13:06:27.005367 18967 raft_consensus.cc:2804] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:27.006109 18967 raft_consensus.cc:697] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 1 LEADER]: Becoming Leader. State: Replica: 1e6b1a8ec1f74a2ebfab7f19e85d60d6, State: Running, Role: LEADER
I20260709 13:06:27.007156 18967 consensus_queue.cc:237] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:27.012360 18664 catalog_manager.cc:5697] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e6b1a8ec1f74a2ebfab7f19e85d60d6 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:27.025995 18664 catalog_manager.cc:5697] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e6b1a8ec1f74a2ebfab7f19e85d60d6 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:27.083184 18664 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35106:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:27.085647 18664 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:27.113722 18759 tablet_service.cc:1511] Processing CreateTablet for tablet 209391c083b948038b2384ebefc13103 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=91fae317ca5241c8a5ec50b08691ae5a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:06:27.115288 18759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 209391c083b948038b2384ebefc13103. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:27.115311 18834 tablet_service.cc:1511] Processing CreateTablet for tablet 209391c083b948038b2384ebefc13103 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=91fae317ca5241c8a5ec50b08691ae5a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:06:27.116768 18834 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 209391c083b948038b2384ebefc13103. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:27.116914 18907 tablet_service.cc:1511] Processing CreateTablet for tablet 209391c083b948038b2384ebefc13103 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=91fae317ca5241c8a5ec50b08691ae5a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:06:27.118825 18907 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 209391c083b948038b2384ebefc13103. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:27.126299 18833 tablet_service.cc:1511] Processing CreateTablet for tablet 5ea48cf10fbf4fd0a93e826eae8356b3 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=91fae317ca5241c8a5ec50b08691ae5a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:06:27.127876 18833 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ea48cf10fbf4fd0a93e826eae8356b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:27.133416 18758 tablet_service.cc:1511] Processing CreateTablet for tablet 5ea48cf10fbf4fd0a93e826eae8356b3 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=91fae317ca5241c8a5ec50b08691ae5a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:06:27.134897 18960 tablet_bootstrap.cc:492] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5: Bootstrap starting.
I20260709 13:06:27.135306 18758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ea48cf10fbf4fd0a93e826eae8356b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:27.140069 18908 tablet_service.cc:1511] Processing CreateTablet for tablet 5ea48cf10fbf4fd0a93e826eae8356b3 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=91fae317ca5241c8a5ec50b08691ae5a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:06:27.141582 18908 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ea48cf10fbf4fd0a93e826eae8356b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:27.142735 18960 tablet_bootstrap.cc:654] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.152835 18959 tablet_bootstrap.cc:492] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6: Bootstrap starting.
I20260709 13:06:27.156183 18960 tablet_bootstrap.cc:492] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5: No bootstrap required, opened a new log
I20260709 13:06:27.156834 18960 ts_tablet_manager.cc:1403] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.000s
I20260709 13:06:27.159953 18958 tablet_bootstrap.cc:492] T 209391c083b948038b2384ebefc13103 P ab011c3efbec40e98169d325dc0fcc2d: Bootstrap starting.
I20260709 13:06:27.160435 18960 raft_consensus.cc:359] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:27.161335 18960 raft_consensus.cc:385] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.161708 18960 raft_consensus.cc:740] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e4bd2ed8fe9405793d0bab5682997c5, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.162726 18959 tablet_bootstrap.cc:654] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.162818 18960 consensus_queue.cc:260] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:27.165546 18960 ts_tablet_manager.cc:1434] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5: Time spent starting tablet: real 0.008s	user 0.011s	sys 0.000s
I20260709 13:06:27.170178 18958 tablet_bootstrap.cc:654] T 209391c083b948038b2384ebefc13103 P ab011c3efbec40e98169d325dc0fcc2d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.178833 18960 tablet_bootstrap.cc:492] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5: Bootstrap starting.
W20260709 13:06:27.179534 18715 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:27.185179 18958 tablet_bootstrap.cc:492] T 209391c083b948038b2384ebefc13103 P ab011c3efbec40e98169d325dc0fcc2d: No bootstrap required, opened a new log
I20260709 13:06:27.185395 18959 tablet_bootstrap.cc:492] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6: No bootstrap required, opened a new log
I20260709 13:06:27.185947 18958 ts_tablet_manager.cc:1403] T 209391c083b948038b2384ebefc13103 P ab011c3efbec40e98169d325dc0fcc2d: Time spent bootstrapping tablet: real 0.026s	user 0.007s	sys 0.008s
I20260709 13:06:27.186020 18959 ts_tablet_manager.cc:1403] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6: Time spent bootstrapping tablet: real 0.033s	user 0.015s	sys 0.004s
I20260709 13:06:27.187879 18960 tablet_bootstrap.cc:654] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.189635 18958 raft_consensus.cc:359] T 209391c083b948038b2384ebefc13103 P ab011c3efbec40e98169d325dc0fcc2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:27.190048 18959 raft_consensus.cc:359] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:27.190524 18958 raft_consensus.cc:385] T 209391c083b948038b2384ebefc13103 P ab011c3efbec40e98169d325dc0fcc2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.190913 18959 raft_consensus.cc:385] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.191015 18958 raft_consensus.cc:740] T 209391c083b948038b2384ebefc13103 P ab011c3efbec40e98169d325dc0fcc2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab011c3efbec40e98169d325dc0fcc2d, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.191601 18959 raft_consensus.cc:740] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e6b1a8ec1f74a2ebfab7f19e85d60d6, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.192181 18958 consensus_queue.cc:260] T 209391c083b948038b2384ebefc13103 P ab011c3efbec40e98169d325dc0fcc2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:27.192567 18959 consensus_queue.cc:260] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:27.194325 18960 tablet_bootstrap.cc:492] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5: No bootstrap required, opened a new log
I20260709 13:06:27.194660 18958 ts_tablet_manager.cc:1434] T 209391c083b948038b2384ebefc13103 P ab011c3efbec40e98169d325dc0fcc2d: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:06:27.194880 18960 ts_tablet_manager.cc:1403] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.003s
I20260709 13:06:27.195819 18958 tablet_bootstrap.cc:492] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d: Bootstrap starting.
I20260709 13:06:27.198484 18959 ts_tablet_manager.cc:1434] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 13:06:27.199770 18959 tablet_bootstrap.cc:492] T 209391c083b948038b2384ebefc13103 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6: Bootstrap starting.
I20260709 13:06:27.203470 18958 tablet_bootstrap.cc:654] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.207927 18960 raft_consensus.cc:359] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:27.209251 18960 raft_consensus.cc:385] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.209673 18960 raft_consensus.cc:740] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e4bd2ed8fe9405793d0bab5682997c5, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.210934 18960 consensus_queue.cc:260] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:27.214404 18960 ts_tablet_manager.cc:1434] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:06:27.214773 18959 tablet_bootstrap.cc:654] T 209391c083b948038b2384ebefc13103 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.215890 18958 tablet_bootstrap.cc:492] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d: No bootstrap required, opened a new log
I20260709 13:06:27.216516 18958 ts_tablet_manager.cc:1403] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.009s
I20260709 13:06:27.220703 18959 tablet_bootstrap.cc:492] T 209391c083b948038b2384ebefc13103 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6: No bootstrap required, opened a new log
I20260709 13:06:27.220238 18958 raft_consensus.cc:359] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:27.221140 18958 raft_consensus.cc:385] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.221493 18958 raft_consensus.cc:740] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab011c3efbec40e98169d325dc0fcc2d, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.221781 18959 ts_tablet_manager.cc:1403] T 209391c083b948038b2384ebefc13103 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.001s
I20260709 13:06:27.222337 18958 consensus_queue.cc:260] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:27.224905 18958 ts_tablet_manager.cc:1434] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:06:27.225281 18959 raft_consensus.cc:359] T 209391c083b948038b2384ebefc13103 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:27.225739 18962 raft_consensus.cc:493] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.226322 18962 raft_consensus.cc:515] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:27.226125 18959 raft_consensus.cc:385] T 209391c083b948038b2384ebefc13103 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.227438 18959 raft_consensus.cc:740] T 209391c083b948038b2384ebefc13103 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e6b1a8ec1f74a2ebfab7f19e85d60d6, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.228235 18959 consensus_queue.cc:260] T 209391c083b948038b2384ebefc13103 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:27.230625 18959 ts_tablet_manager.cc:1434] T 209391c083b948038b2384ebefc13103 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 13:06:27.231463 18962 leader_election.cc:290] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e4bd2ed8fe9405793d0bab5682997c5 (127.6.255.1:39709), 1e6b1a8ec1f74a2ebfab7f19e85d60d6 (127.6.255.2:39573)
I20260709 13:06:27.244338 18768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ea48cf10fbf4fd0a93e826eae8356b3" candidate_uuid: "ab011c3efbec40e98169d325dc0fcc2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" is_pre_election: true
I20260709 13:06:27.244771 18844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ea48cf10fbf4fd0a93e826eae8356b3" candidate_uuid: "ab011c3efbec40e98169d325dc0fcc2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" is_pre_election: true
I20260709 13:06:27.245234 18768 raft_consensus.cc:2468] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab011c3efbec40e98169d325dc0fcc2d in term 0.
I20260709 13:06:27.245544 18844 raft_consensus.cc:2468] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab011c3efbec40e98169d325dc0fcc2d in term 0.
I20260709 13:06:27.246513 18881 leader_election.cc:304] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e4bd2ed8fe9405793d0bab5682997c5, ab011c3efbec40e98169d325dc0fcc2d; no voters: 
I20260709 13:06:27.247483 18962 raft_consensus.cc:2804] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:27.247804 18962 raft_consensus.cc:493] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:27.248116 18962 raft_consensus.cc:3060] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.253897 18962 raft_consensus.cc:515] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:27.255471 18962 leader_election.cc:290] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [CANDIDATE]: Term 1 election: Requested vote from peers 2e4bd2ed8fe9405793d0bab5682997c5 (127.6.255.1:39709), 1e6b1a8ec1f74a2ebfab7f19e85d60d6 (127.6.255.2:39573)
I20260709 13:06:27.256645 18768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ea48cf10fbf4fd0a93e826eae8356b3" candidate_uuid: "ab011c3efbec40e98169d325dc0fcc2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e4bd2ed8fe9405793d0bab5682997c5"
I20260709 13:06:27.256807 18844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ea48cf10fbf4fd0a93e826eae8356b3" candidate_uuid: "ab011c3efbec40e98169d325dc0fcc2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6"
I20260709 13:06:27.257520 18768 raft_consensus.cc:3060] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.257570 18844 raft_consensus.cc:3060] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.265045 18768 raft_consensus.cc:2468] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab011c3efbec40e98169d325dc0fcc2d in term 1.
I20260709 13:06:27.265045 18844 raft_consensus.cc:2468] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab011c3efbec40e98169d325dc0fcc2d in term 1.
I20260709 13:06:27.266649 18884 leader_election.cc:304] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e6b1a8ec1f74a2ebfab7f19e85d60d6, ab011c3efbec40e98169d325dc0fcc2d; no voters: 
I20260709 13:06:27.267545 18962 raft_consensus.cc:2804] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:27.268898 18962 raft_consensus.cc:697] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 1 LEADER]: Becoming Leader. State: Replica: ab011c3efbec40e98169d325dc0fcc2d, State: Running, Role: LEADER
I20260709 13:06:27.269734 18962 consensus_queue.cc:237] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:27.277794 18667 catalog_manager.cc:5697] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d reported cstate change: term changed from 0 to 1, leader changed from <none> to ab011c3efbec40e98169d325dc0fcc2d (127.6.255.3). New cstate: current_term: 1 leader_uuid: "ab011c3efbec40e98169d325dc0fcc2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:27.379683 18966 raft_consensus.cc:493] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.380342 18966 raft_consensus.cc:515] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:27.382743 18966 leader_election.cc:290] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e6b1a8ec1f74a2ebfab7f19e85d60d6 (127.6.255.2:39573), ab011c3efbec40e98169d325dc0fcc2d (127.6.255.3:44099)
I20260709 13:06:27.391965 18967 consensus_queue.cc:1048] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:27.396458 18917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "209391c083b948038b2384ebefc13103" candidate_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab011c3efbec40e98169d325dc0fcc2d" is_pre_election: true
I20260709 13:06:27.397006 18844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "209391c083b948038b2384ebefc13103" candidate_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" is_pre_election: true
I20260709 13:06:27.397229 18917 raft_consensus.cc:2468] T 209391c083b948038b2384ebefc13103 P ab011c3efbec40e98169d325dc0fcc2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e4bd2ed8fe9405793d0bab5682997c5 in term 0.
I20260709 13:06:27.397789 18844 raft_consensus.cc:2468] T 209391c083b948038b2384ebefc13103 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e4bd2ed8fe9405793d0bab5682997c5 in term 0.
I20260709 13:06:27.399294 18732 leader_election.cc:304] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e4bd2ed8fe9405793d0bab5682997c5, ab011c3efbec40e98169d325dc0fcc2d; no voters: 
I20260709 13:06:27.400224 18966 raft_consensus.cc:2804] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:27.400601 18966 raft_consensus.cc:493] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:27.400970 18966 raft_consensus.cc:3060] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.411958 18966 raft_consensus.cc:515] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:27.414800 18995 consensus_queue.cc:1048] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:27.416860 18966 leader_election.cc:290] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5 [CANDIDATE]: Term 1 election: Requested vote from peers 1e6b1a8ec1f74a2ebfab7f19e85d60d6 (127.6.255.2:39573), ab011c3efbec40e98169d325dc0fcc2d (127.6.255.3:44099)
I20260709 13:06:27.418406 18844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "209391c083b948038b2384ebefc13103" candidate_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6"
I20260709 13:06:27.418838 18918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "209391c083b948038b2384ebefc13103" candidate_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab011c3efbec40e98169d325dc0fcc2d"
I20260709 13:06:27.419296 18844 raft_consensus.cc:3060] T 209391c083b948038b2384ebefc13103 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.419678 18918 raft_consensus.cc:3060] T 209391c083b948038b2384ebefc13103 P ab011c3efbec40e98169d325dc0fcc2d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.427486 18844 raft_consensus.cc:2468] T 209391c083b948038b2384ebefc13103 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e4bd2ed8fe9405793d0bab5682997c5 in term 1.
I20260709 13:06:27.429320 18735 leader_election.cc:304] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e6b1a8ec1f74a2ebfab7f19e85d60d6, 2e4bd2ed8fe9405793d0bab5682997c5; no voters: 
I20260709 13:06:27.430591 18966 raft_consensus.cc:2804] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:27.432083 18918 raft_consensus.cc:2468] T 209391c083b948038b2384ebefc13103 P ab011c3efbec40e98169d325dc0fcc2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e4bd2ed8fe9405793d0bab5682997c5 in term 1.
I20260709 13:06:27.432099 18966 raft_consensus.cc:697] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 1 LEADER]: Becoming Leader. State: Replica: 2e4bd2ed8fe9405793d0bab5682997c5, State: Running, Role: LEADER
I20260709 13:06:27.433821 18966 consensus_queue.cc:237] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:27.445475 18664 catalog_manager.cc:5697] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e4bd2ed8fe9405793d0bab5682997c5 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:27.485110 18995 consensus_queue.cc:1048] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:27.497115 18967 consensus_queue.cc:1048] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:27.538250 18844 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48da62dff3be449cb227262ba5b0e2cf"
dest_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6"
mode: GRACEFUL
new_leader_uuid: "2e4bd2ed8fe9405793d0bab5682997c5"
 from {username='slave'} at 127.0.0.1:49620
I20260709 13:06:27.538836 18844 raft_consensus.cc:606] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 1 LEADER]: Received request to transfer leadership to TS 2e4bd2ed8fe9405793d0bab5682997c5
I20260709 13:06:27.560822 18918 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ea48cf10fbf4fd0a93e826eae8356b3"
dest_uuid: "ab011c3efbec40e98169d325dc0fcc2d"
mode: GRACEFUL
new_leader_uuid: "2e4bd2ed8fe9405793d0bab5682997c5"
 from {username='slave'} at 127.0.0.1:49964
I20260709 13:06:27.561529 18918 raft_consensus.cc:606] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 1 LEADER]: Received request to transfer leadership to TS 2e4bd2ed8fe9405793d0bab5682997c5
I20260709 13:06:27.780145 18962 consensus_queue.cc:1048] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:27.788370 19003 raft_consensus.cc:993] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d: : Instructing follower 2e4bd2ed8fe9405793d0bab5682997c5 to start an election
I20260709 13:06:27.789108 18962 raft_consensus.cc:1081] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 1 LEADER]: Signalling peer 2e4bd2ed8fe9405793d0bab5682997c5 to start an election
I20260709 13:06:27.791234 18769 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ea48cf10fbf4fd0a93e826eae8356b3"
dest_uuid: "2e4bd2ed8fe9405793d0bab5682997c5"
 from {username='slave'} at 127.0.0.1:53008
I20260709 13:06:27.791810 18769 raft_consensus.cc:493] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:27.792215 18769 raft_consensus.cc:3060] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:27.795400 18962 consensus_queue.cc:1048] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:27.800940 18769 raft_consensus.cc:515] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:27.809325 18769 leader_election.cc:290] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [CANDIDATE]: Term 2 election: Requested vote from peers 1e6b1a8ec1f74a2ebfab7f19e85d60d6 (127.6.255.2:39573), ab011c3efbec40e98169d325dc0fcc2d (127.6.255.3:44099)
I20260709 13:06:27.810194 18844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ea48cf10fbf4fd0a93e826eae8356b3" candidate_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6"
I20260709 13:06:27.810917 18844 raft_consensus.cc:3060] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:27.811590 18918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ea48cf10fbf4fd0a93e826eae8356b3" candidate_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ab011c3efbec40e98169d325dc0fcc2d"
I20260709 13:06:27.812252 18918 raft_consensus.cc:3055] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:27.812590 18918 raft_consensus.cc:740] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ab011c3efbec40e98169d325dc0fcc2d, State: Running, Role: LEADER
I20260709 13:06:27.813422 18918 consensus_queue.cc:260] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:27.814819 18918 raft_consensus.cc:3060] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:27.820478 18844 raft_consensus.cc:2468] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e4bd2ed8fe9405793d0bab5682997c5 in term 2.
I20260709 13:06:27.822189 18735 leader_election.cc:304] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e6b1a8ec1f74a2ebfab7f19e85d60d6, 2e4bd2ed8fe9405793d0bab5682997c5; no voters: 
I20260709 13:06:27.822274 18918 raft_consensus.cc:2468] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e4bd2ed8fe9405793d0bab5682997c5 in term 2.
I20260709 13:06:27.823544 18994 raft_consensus.cc:2804] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:27.824903 18994 raft_consensus.cc:697] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 2 LEADER]: Becoming Leader. State: Replica: 2e4bd2ed8fe9405793d0bab5682997c5, State: Running, Role: LEADER
I20260709 13:06:27.825858 18994 consensus_queue.cc:237] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:27.823570 18966 consensus_queue.cc:1048] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:06:27.833081 18966 consensus_queue.cc:1048] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:27.837076 18667 catalog_manager.cc:5697] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 reported cstate change: term changed from 1 to 2, leader changed from ab011c3efbec40e98169d325dc0fcc2d (127.6.255.3) to 2e4bd2ed8fe9405793d0bab5682997c5 (127.6.255.1). New cstate: current_term: 2 leader_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:28.043320 18995 raft_consensus.cc:993] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6: : Instructing follower 2e4bd2ed8fe9405793d0bab5682997c5 to start an election
I20260709 13:06:28.043939 18967 raft_consensus.cc:1081] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 1 LEADER]: Signalling peer 2e4bd2ed8fe9405793d0bab5682997c5 to start an election
I20260709 13:06:28.045521 18769 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "48da62dff3be449cb227262ba5b0e2cf"
dest_uuid: "2e4bd2ed8fe9405793d0bab5682997c5"
 from {username='slave'} at 127.0.0.1:52994
I20260709 13:06:28.046025 18769 raft_consensus.cc:493] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:28.046339 18769 raft_consensus.cc:3060] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:28.052237 18769 raft_consensus.cc:515] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } } peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } }
I20260709 13:06:28.054325 18769 leader_election.cc:290] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5 [CANDIDATE]: Term 2 election: Requested vote from peers ab011c3efbec40e98169d325dc0fcc2d (127.6.255.3:44099), 1e6b1a8ec1f74a2ebfab7f19e85d60d6 (127.6.255.2:39573)
I20260709 13:06:28.055327 18918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48da62dff3be449cb227262ba5b0e2cf" candidate_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ab011c3efbec40e98169d325dc0fcc2d"
I20260709 13:06:28.055437 18844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48da62dff3be449cb227262ba5b0e2cf" candidate_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6"
I20260709 13:06:28.056363 18918 raft_consensus.cc:3060] T 48da62dff3be449cb227262ba5b0e2cf P ab011c3efbec40e98169d325dc0fcc2d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:28.056612 18844 raft_consensus.cc:3055] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:28.057067 18844 raft_consensus.cc:740] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1e6b1a8ec1f74a2ebfab7f19e85d60d6, State: Running, Role: LEADER
I20260709 13:06:28.057937 18844 consensus_queue.cc:260] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } } peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } }
I20260709 13:06:28.059386 18844 raft_consensus.cc:3060] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:28.063489 18918 raft_consensus.cc:2468] T 48da62dff3be449cb227262ba5b0e2cf P ab011c3efbec40e98169d325dc0fcc2d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e4bd2ed8fe9405793d0bab5682997c5 in term 2.
I20260709 13:06:28.064888 18732 leader_election.cc:304] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e4bd2ed8fe9405793d0bab5682997c5, ab011c3efbec40e98169d325dc0fcc2d; no voters: 
I20260709 13:06:28.065826 19007 raft_consensus.cc:2804] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:28.066318 19007 raft_consensus.cc:697] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5 [term 2 LEADER]: Becoming Leader. State: Replica: 2e4bd2ed8fe9405793d0bab5682997c5, State: Running, Role: LEADER
I20260709 13:06:28.066511 18844 raft_consensus.cc:2468] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e4bd2ed8fe9405793d0bab5682997c5 in term 2.
I20260709 13:06:28.067219 19007 consensus_queue.cc:237] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } } peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } }
I20260709 13:06:28.074807 18665 catalog_manager.cc:5697] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5 reported cstate change: term changed from 1 to 2, leader changed from 1e6b1a8ec1f74a2ebfab7f19e85d60d6 (127.6.255.2) to 2e4bd2ed8fe9405793d0bab5682997c5 (127.6.255.1). New cstate: current_term: 2 leader_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:28.259366 18918 raft_consensus.cc:1275] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 2 FOLLOWER]: Refusing update from remote peer 2e4bd2ed8fe9405793d0bab5682997c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:28.260664 19007 consensus_queue.cc:1048] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:28.269105 18844 raft_consensus.cc:1275] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 2 FOLLOWER]: Refusing update from remote peer 2e4bd2ed8fe9405793d0bab5682997c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:28.270838 19007 consensus_queue.cc:1048] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:28.303185  7164 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:06:28.304872  7164 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:06:28.305289  7164 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:06:28.320243 18769 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ea48cf10fbf4fd0a93e826eae8356b3"
dest_uuid: "2e4bd2ed8fe9405793d0bab5682997c5"
mode: GRACEFUL
new_leader_uuid: "ab011c3efbec40e98169d325dc0fcc2d"
 from {username='slave'} at 127.0.0.1:53020
I20260709 13:06:28.320824 18769 raft_consensus.cc:606] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 2 LEADER]: Received request to transfer leadership to TS ab011c3efbec40e98169d325dc0fcc2d
I20260709 13:06:28.322208  7164 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:06:28.515110 18844 raft_consensus.cc:1275] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 2 FOLLOWER]: Refusing update from remote peer 2e4bd2ed8fe9405793d0bab5682997c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:28.516537 19007 consensus_queue.cc:1048] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:28.525990 18918 raft_consensus.cc:1275] T 48da62dff3be449cb227262ba5b0e2cf P ab011c3efbec40e98169d325dc0fcc2d [term 2 FOLLOWER]: Refusing update from remote peer 2e4bd2ed8fe9405793d0bab5682997c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:28.528136 18994 consensus_queue.cc:1048] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:28.718405 18966 raft_consensus.cc:993] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5: : Instructing follower ab011c3efbec40e98169d325dc0fcc2d to start an election
I20260709 13:06:28.718847 18966 raft_consensus.cc:1081] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 2 LEADER]: Signalling peer ab011c3efbec40e98169d325dc0fcc2d to start an election
I20260709 13:06:28.720574 18918 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ea48cf10fbf4fd0a93e826eae8356b3"
dest_uuid: "ab011c3efbec40e98169d325dc0fcc2d"
 from {username='slave'} at 127.0.0.1:49950
I20260709 13:06:28.721143 18918 raft_consensus.cc:493] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:28.721536 18918 raft_consensus.cc:3060] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:28.727710 18918 raft_consensus.cc:515] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:28.729833 18918 leader_election.cc:290] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [CANDIDATE]: Term 3 election: Requested vote from peers 2e4bd2ed8fe9405793d0bab5682997c5 (127.6.255.1:39709), 1e6b1a8ec1f74a2ebfab7f19e85d60d6 (127.6.255.2:39573)
I20260709 13:06:28.730870 18769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ea48cf10fbf4fd0a93e826eae8356b3" candidate_uuid: "ab011c3efbec40e98169d325dc0fcc2d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2e4bd2ed8fe9405793d0bab5682997c5"
I20260709 13:06:28.730953 18844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ea48cf10fbf4fd0a93e826eae8356b3" candidate_uuid: "ab011c3efbec40e98169d325dc0fcc2d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6"
I20260709 13:06:28.731554 18769 raft_consensus.cc:3055] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:28.731658 18844 raft_consensus.cc:3060] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:28.731871 18769 raft_consensus.cc:740] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2e4bd2ed8fe9405793d0bab5682997c5, State: Running, Role: LEADER
I20260709 13:06:28.732784 18769 consensus_queue.cc:260] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:28.734009 18769 raft_consensus.cc:3060] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:28.737803 18844 raft_consensus.cc:2468] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab011c3efbec40e98169d325dc0fcc2d in term 3.
I20260709 13:06:28.739502 18884 leader_election.cc:304] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e6b1a8ec1f74a2ebfab7f19e85d60d6, ab011c3efbec40e98169d325dc0fcc2d; no voters: 
I20260709 13:06:28.740446 19005 raft_consensus.cc:2804] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:28.740878 18769 raft_consensus.cc:2468] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab011c3efbec40e98169d325dc0fcc2d in term 3.
I20260709 13:06:28.741554 19005 raft_consensus.cc:697] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 3 LEADER]: Becoming Leader. State: Replica: ab011c3efbec40e98169d325dc0fcc2d, State: Running, Role: LEADER
I20260709 13:06:28.742527 19005 consensus_queue.cc:237] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } }
I20260709 13:06:28.750047 18667 catalog_manager.cc:5697] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d reported cstate change: term changed from 2 to 3, leader changed from 2e4bd2ed8fe9405793d0bab5682997c5 (127.6.255.1) to ab011c3efbec40e98169d325dc0fcc2d (127.6.255.3). New cstate: current_term: 3 leader_uuid: "ab011c3efbec40e98169d325dc0fcc2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bd2ed8fe9405793d0bab5682997c5" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 39709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e6b1a8ec1f74a2ebfab7f19e85d60d6" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 39573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab011c3efbec40e98169d325dc0fcc2d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 44099 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:28.954119  7164 tablet_server.cc:179] TabletServer@127.6.255.1:0 shutting down...
I20260709 13:06:28.981793  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:28.982688  7164 tablet_replica.cc:333] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5: stopping tablet replica
I20260709 13:06:28.983403  7164 raft_consensus.cc:2243] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.983948  7164 raft_consensus.cc:2272] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.986965  7164 tablet_replica.cc:333] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5: stopping tablet replica
I20260709 13:06:28.987618  7164 raft_consensus.cc:2243] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:28.988555  7164 raft_consensus.cc:2272] T 48da62dff3be449cb227262ba5b0e2cf P 2e4bd2ed8fe9405793d0bab5682997c5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.991303  7164 tablet_replica.cc:333] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5: stopping tablet replica
I20260709 13:06:28.992100  7164 raft_consensus.cc:2243] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:28.993297  7164 raft_consensus.cc:2272] T 209391c083b948038b2384ebefc13103 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.995745  7164 tablet_replica.cc:333] T 6d563bf7fb5e4132948e208bf86673c6 P 2e4bd2ed8fe9405793d0bab5682997c5: stopping tablet replica
I20260709 13:06:28.996256  7164 raft_consensus.cc:2243] T 6d563bf7fb5e4132948e208bf86673c6 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.996642  7164 raft_consensus.cc:2272] T 6d563bf7fb5e4132948e208bf86673c6 P 2e4bd2ed8fe9405793d0bab5682997c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:29.020702  7164 tablet_server.cc:196] TabletServer@127.6.255.1:0 shutdown complete.
I20260709 13:06:29.034030  7164 tablet_server.cc:179] TabletServer@127.6.255.2:0 shutting down...
I20260709 13:06:29.054636  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:29.055263  7164 tablet_replica.cc:333] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6: stopping tablet replica
I20260709 13:06:29.055902  7164 raft_consensus.cc:2243] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:29.056542  7164 raft_consensus.cc:2272] T 5ea48cf10fbf4fd0a93e826eae8356b3 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:29.058992  7164 tablet_replica.cc:333] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6: stopping tablet replica
I20260709 13:06:29.059609  7164 raft_consensus.cc:2243] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:29.060076  7164 raft_consensus.cc:2272] T 48da62dff3be449cb227262ba5b0e2cf P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:29.062867  7164 tablet_replica.cc:333] T 209391c083b948038b2384ebefc13103 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6: stopping tablet replica
I20260709 13:06:29.063585  7164 raft_consensus.cc:2243] T 209391c083b948038b2384ebefc13103 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:29.064107  7164 raft_consensus.cc:2272] T 209391c083b948038b2384ebefc13103 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:29.066500  7164 tablet_replica.cc:333] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6: stopping tablet replica
I20260709 13:06:29.067031  7164 raft_consensus.cc:2243] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:29.067879  7164 raft_consensus.cc:2272] T 6d563bf7fb5e4132948e208bf86673c6 P 1e6b1a8ec1f74a2ebfab7f19e85d60d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:29.129833  7164 tablet_server.cc:196] TabletServer@127.6.255.2:0 shutdown complete.
W20260709 13:06:29.146375 18881 consensus_peers.cc:597] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d -> Peer 2e4bd2ed8fe9405793d0bab5682997c5 (127.6.255.1:39709): Couldn't send request to peer 2e4bd2ed8fe9405793d0bab5682997c5. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:39709: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:29.165102  7164 tablet_server.cc:179] TabletServer@127.6.255.3:0 shutting down...
I20260709 13:06:29.202311  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:29.203265  7164 tablet_replica.cc:333] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d: stopping tablet replica
I20260709 13:06:29.204422  7164 raft_consensus.cc:2243] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:29.205824  7164 pending_rounds.cc:70] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d: Trying to abort 1 pending ops.
I20260709 13:06:29.206074  7164 pending_rounds.cc:77] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305635384297435136 op_type: NO_OP noop_request { }
I20260709 13:06:29.206537  7164 raft_consensus.cc:2889] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:06:29.206890  7164 raft_consensus.cc:2272] T 5ea48cf10fbf4fd0a93e826eae8356b3 P ab011c3efbec40e98169d325dc0fcc2d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:29.211470  7164 tablet_replica.cc:333] T 209391c083b948038b2384ebefc13103 P ab011c3efbec40e98169d325dc0fcc2d: stopping tablet replica
I20260709 13:06:29.212252  7164 raft_consensus.cc:2243] T 209391c083b948038b2384ebefc13103 P ab011c3efbec40e98169d325dc0fcc2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:29.212949  7164 raft_consensus.cc:2272] T 209391c083b948038b2384ebefc13103 P ab011c3efbec40e98169d325dc0fcc2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:29.216157  7164 tablet_replica.cc:333] T 6d563bf7fb5e4132948e208bf86673c6 P ab011c3efbec40e98169d325dc0fcc2d: stopping tablet replica
I20260709 13:06:29.216747  7164 raft_consensus.cc:2243] T 6d563bf7fb5e4132948e208bf86673c6 P ab011c3efbec40e98169d325dc0fcc2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:29.217398  7164 raft_consensus.cc:2272] T 6d563bf7fb5e4132948e208bf86673c6 P ab011c3efbec40e98169d325dc0fcc2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:29.220485  7164 tablet_replica.cc:333] T 48da62dff3be449cb227262ba5b0e2cf P ab011c3efbec40e98169d325dc0fcc2d: stopping tablet replica
I20260709 13:06:29.221066  7164 raft_consensus.cc:2243] T 48da62dff3be449cb227262ba5b0e2cf P ab011c3efbec40e98169d325dc0fcc2d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:29.221808  7164 raft_consensus.cc:2272] T 48da62dff3be449cb227262ba5b0e2cf P ab011c3efbec40e98169d325dc0fcc2d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:29.252125  7164 tablet_server.cc:196] TabletServer@127.6.255.3:0 shutdown complete.
I20260709 13:06:29.275295  7164 master.cc:562] Master@127.6.255.62:38177 shutting down...
I20260709 13:06:29.301772  7164 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 5714b850b3de46dbb7d5ec4120bc5bbf [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:29.302601  7164 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 5714b850b3de46dbb7d5ec4120bc5bbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:29.303017  7164 tablet_replica.cc:333] T 00000000000000000000000000000000 P 5714b850b3de46dbb7d5ec4120bc5bbf: stopping tablet replica
I20260709 13:06:29.325364  7164 master.cc:584] Master@127.6.255.62:38177 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (3358 ms)
[----------] 6 tests from LeaderRebalancerTest (343147 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:06:29.375103  7164 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.255.62:41289
I20260709 13:06:29.376327  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:29.383292 19022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:29.383416 19021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:29.384872 19024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:29.386785  7164 server_base.cc:1061] running on GCE node
I20260709 13:06:29.387701  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:29.387894  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:29.388069  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602389388056 us; error 0 us; skew 500 ppm
I20260709 13:06:29.388541  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:29.391688  7164 webserver.cc:533] Webserver started at http://127.6.255.62:35137/ using document root <none> and password file <none>
I20260709 13:06:29.392163  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:29.392316  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:29.392554  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:29.394003  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602046064212-7164-0/minicluster-data/master-0-root/instance:
uuid: "6e3e7c5ed6f04c6f9df5fb20e212c2ee"
format_stamp: "Formatted at 2026-07-09 13:06:29 on dist-test-slave-rnv4"
I20260709 13:06:29.399443  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:06:29.403429 19029 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:29.404719  7164 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:06:29.405234  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602046064212-7164-0/minicluster-data/master-0-root
uuid: "6e3e7c5ed6f04c6f9df5fb20e212c2ee"
format_stamp: "Formatted at 2026-07-09 13:06:29 on dist-test-slave-rnv4"
I20260709 13:06:29.405675  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602046064212-7164-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602046064212-7164-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602046064212-7164-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:29.452154  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:29.453634  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:29.495883  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.62:41289
I20260709 13:06:29.496037 19080 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.62:41289 every 8 connection(s)
I20260709 13:06:29.500277 19081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:29.513898 19081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee: Bootstrap starting.
I20260709 13:06:29.519744 19081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:29.524756 19081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee: No bootstrap required, opened a new log
I20260709 13:06:29.527175 19081 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3e7c5ed6f04c6f9df5fb20e212c2ee" member_type: VOTER }
I20260709 13:06:29.527609 19081 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:29.527890 19081 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e3e7c5ed6f04c6f9df5fb20e212c2ee, State: Initialized, Role: FOLLOWER
I20260709 13:06:29.528613 19081 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3e7c5ed6f04c6f9df5fb20e212c2ee" member_type: VOTER }
I20260709 13:06:29.529093 19081 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:06:29.529321 19081 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:06:29.529633 19081 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:29.534672 19081 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3e7c5ed6f04c6f9df5fb20e212c2ee" member_type: VOTER }
I20260709 13:06:29.535271 19081 leader_election.cc:304] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e3e7c5ed6f04c6f9df5fb20e212c2ee; no voters: 
I20260709 13:06:29.536525 19081 leader_election.cc:290] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:06:29.537070 19084 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:29.538770 19084 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee [term 1 LEADER]: Becoming Leader. State: Replica: 6e3e7c5ed6f04c6f9df5fb20e212c2ee, State: Running, Role: LEADER
I20260709 13:06:29.539768 19081 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:06:29.539662 19084 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3e7c5ed6f04c6f9df5fb20e212c2ee" member_type: VOTER }
I20260709 13:06:29.543139 19085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e3e7c5ed6f04c6f9df5fb20e212c2ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3e7c5ed6f04c6f9df5fb20e212c2ee" member_type: VOTER } }
I20260709 13:06:29.543139 19086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e3e7c5ed6f04c6f9df5fb20e212c2ee. Latest consensus state: current_term: 1 leader_uuid: "6e3e7c5ed6f04c6f9df5fb20e212c2ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3e7c5ed6f04c6f9df5fb20e212c2ee" member_type: VOTER } }
I20260709 13:06:29.543965 19085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:29.543989 19086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:29.548596 19089 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:06:29.558060 19089 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:06:29.559131  7164 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:06:29.568240 19089 catalog_manager.cc:1383] Generated new cluster ID: 92646e5151f048e9b3edcd54addbfee3
I20260709 13:06:29.568661 19089 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:06:29.584913 19089 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:06:29.586539 19089 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:06:29.600306 19089 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee: Generated new TSK 0
I20260709 13:06:29.600970 19089 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:06:29.626245  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:29.633000 19102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:29.634591 19103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:29.636359 19105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:29.636472  7164 server_base.cc:1061] running on GCE node
I20260709 13:06:29.637758  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:29.638077  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:29.638278  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602389638235 us; error 0 us; skew 500 ppm
I20260709 13:06:29.638921  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:29.641788  7164 webserver.cc:533] Webserver started at http://127.6.255.1:36879/ using document root <none> and password file <none>
I20260709 13:06:29.642383  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:29.642598  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:29.642925  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:29.644233  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602046064212-7164-0/minicluster-data/ts-0-root/instance:
uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
format_stamp: "Formatted at 2026-07-09 13:06:29 on dist-test-slave-rnv4"
I20260709 13:06:29.649678  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.005s
I20260709 13:06:29.653867 19110 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:29.654728  7164 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.004s
I20260709 13:06:29.655100  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602046064212-7164-0/minicluster-data/ts-0-root
uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
format_stamp: "Formatted at 2026-07-09 13:06:29 on dist-test-slave-rnv4"
I20260709 13:06:29.655377  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602046064212-7164-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602046064212-7164-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602046064212-7164-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:29.672780  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:29.674158  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:29.675787  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:29.678297  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:29.678488  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:29.678809  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:29.678983  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:29.725863  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.1:35991
I20260709 13:06:29.725992 19172 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.1:35991 every 8 connection(s)
I20260709 13:06:29.731652  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:29.740351 19178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:29.740913 19177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:29.742915 19180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:29.745412  7164 server_base.cc:1061] running on GCE node
I20260709 13:06:29.746273 19173 heartbeater.cc:344] Connected to a master server at 127.6.255.62:41289
I20260709 13:06:29.746734  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:06:29.746771 19173 heartbeater.cc:461] Registering TS with master...
W20260709 13:06:29.747193  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:29.747444  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602389747425 us; error 0 us; skew 500 ppm
I20260709 13:06:29.748006 19173 heartbeater.cc:507] Master 127.6.255.62:41289 requested a full tablet report, sending...
I20260709 13:06:29.748421  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:29.751003 19046 ts_manager.cc:194] Registered new tserver with Master: b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991)
I20260709 13:06:29.752336  7164 webserver.cc:533] Webserver started at http://127.6.255.2:39729/ using document root <none> and password file <none>
I20260709 13:06:29.752930  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:29.753150  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:29.753324 19046 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54504
I20260709 13:06:29.753489  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:29.755204  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602046064212-7164-0/minicluster-data/ts-1-root/instance:
uuid: "0e552e70a13c4e4c817d78cdde2861cb"
format_stamp: "Formatted at 2026-07-09 13:06:29 on dist-test-slave-rnv4"
I20260709 13:06:29.761759  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:06:29.765854 19185 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:29.766884  7164 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:06:29.767292  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602046064212-7164-0/minicluster-data/ts-1-root
uuid: "0e552e70a13c4e4c817d78cdde2861cb"
format_stamp: "Formatted at 2026-07-09 13:06:29 on dist-test-slave-rnv4"
I20260709 13:06:29.767689  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602046064212-7164-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602046064212-7164-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602046064212-7164-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:29.796725  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:29.798130  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:29.799721  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:29.802371  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:29.802579  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:29.802882  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:29.803047  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:29.852888  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.2:42093
I20260709 13:06:29.853044 19247 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.2:42093 every 8 connection(s)
I20260709 13:06:29.858640  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:29.868245 19251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:29.870579 19252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:29.873834 19254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:29.874043  7164 server_base.cc:1061] running on GCE node
I20260709 13:06:29.875460  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:29.875779  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:29.876055  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602389876035 us; error 0 us; skew 500 ppm
I20260709 13:06:29.876209 19248 heartbeater.cc:344] Connected to a master server at 127.6.255.62:41289
I20260709 13:06:29.876740 19248 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:29.876850  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:29.877815 19248 heartbeater.cc:507] Master 127.6.255.62:41289 requested a full tablet report, sending...
I20260709 13:06:29.880328  7164 webserver.cc:533] Webserver started at http://127.6.255.3:33389/ using document root <none> and password file <none>
I20260709 13:06:29.880693 19046 ts_manager.cc:194] Registered new tserver with Master: 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093)
I20260709 13:06:29.881285  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:29.881881  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:29.882277  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:29.883234 19046 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54514
I20260709 13:06:29.884009  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602046064212-7164-0/minicluster-data/ts-2-root/instance:
uuid: "3e8380a11eb9479cae35d2f49f8591dc"
format_stamp: "Formatted at 2026-07-09 13:06:29 on dist-test-slave-rnv4"
I20260709 13:06:29.890719  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:06:29.895009 19259 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:29.896304  7164 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:06:29.896731  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602046064212-7164-0/minicluster-data/ts-2-root
uuid: "3e8380a11eb9479cae35d2f49f8591dc"
format_stamp: "Formatted at 2026-07-09 13:06:29 on dist-test-slave-rnv4"
I20260709 13:06:29.897336  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602046064212-7164-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602046064212-7164-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602046064212-7164-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:29.909623  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:29.911031  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:29.912479  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:29.915299  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:29.915596  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:29.915899  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:29.916118  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:29.965042  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.3:43973
I20260709 13:06:29.965167 19321 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.3:43973 every 8 connection(s)
I20260709 13:06:29.983407 19322 heartbeater.cc:344] Connected to a master server at 127.6.255.62:41289
I20260709 13:06:29.984035 19322 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:29.984869 19322 heartbeater.cc:507] Master 127.6.255.62:41289 requested a full tablet report, sending...
I20260709 13:06:29.987454 19046 ts_manager.cc:194] Registered new tserver with Master: 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:29.988307  7164 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.019827975s
I20260709 13:06:29.989650 19046 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54530
I20260709 13:06:30.020386 19046 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54534:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:30.023880 19046 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:30.103327 19213 tablet_service.cc:1511] Processing CreateTablet for tablet a67334720ba743938c7b5af66099ddcb (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:30.103987 19212 tablet_service.cc:1511] Processing CreateTablet for tablet 7d271ad4ab1a46c988238f4154400c61 (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:30.105017 19213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a67334720ba743938c7b5af66099ddcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.105744 19212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d271ad4ab1a46c988238f4154400c61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.116026 19211 tablet_service.cc:1511] Processing CreateTablet for tablet 8d50426588914ed2b6d59c26481beb6c (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:30.118108 19211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d50426588914ed2b6d59c26481beb6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.118674 19138 tablet_service.cc:1511] Processing CreateTablet for tablet a67334720ba743938c7b5af66099ddcb (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:30.120383 19138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a67334720ba743938c7b5af66099ddcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.120244 19210 tablet_service.cc:1511] Processing CreateTablet for tablet a54f724cc37d4a86b39037771aaa58a7 (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:30.121907 19210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a54f724cc37d4a86b39037771aaa58a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.122570 19137 tablet_service.cc:1511] Processing CreateTablet for tablet 7d271ad4ab1a46c988238f4154400c61 (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:30.123790 19209 tablet_service.cc:1511] Processing CreateTablet for tablet ffc2b16ab3f54c858df9b008cecdf70c (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:30.125396 19209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffc2b16ab3f54c858df9b008cecdf70c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.127068 19135 tablet_service.cc:1511] Processing CreateTablet for tablet a54f724cc37d4a86b39037771aaa58a7 (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:30.129277 19136 tablet_service.cc:1511] Processing CreateTablet for tablet 8d50426588914ed2b6d59c26481beb6c (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:30.132153 19134 tablet_service.cc:1511] Processing CreateTablet for tablet ffc2b16ab3f54c858df9b008cecdf70c (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:30.124260 19137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d271ad4ab1a46c988238f4154400c61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.135911 19136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d50426588914ed2b6d59c26481beb6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.136746 19135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a54f724cc37d4a86b39037771aaa58a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.139236 19134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffc2b16ab3f54c858df9b008cecdf70c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.157174 19287 tablet_service.cc:1511] Processing CreateTablet for tablet a67334720ba743938c7b5af66099ddcb (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:30.158958 19287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a67334720ba743938c7b5af66099ddcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.162895 19286 tablet_service.cc:1511] Processing CreateTablet for tablet 7d271ad4ab1a46c988238f4154400c61 (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:30.164578 19286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d271ad4ab1a46c988238f4154400c61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.166869 19285 tablet_service.cc:1511] Processing CreateTablet for tablet 8d50426588914ed2b6d59c26481beb6c (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:30.169132 19285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d50426588914ed2b6d59c26481beb6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.171499 19284 tablet_service.cc:1511] Processing CreateTablet for tablet a54f724cc37d4a86b39037771aaa58a7 (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:30.174629 19284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a54f724cc37d4a86b39037771aaa58a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.205961 19283 tablet_service.cc:1511] Processing CreateTablet for tablet ffc2b16ab3f54c858df9b008cecdf70c (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:30.207093 19137 tablet_service.cc:1511] Processing CreateTablet for tablet 47e715af44cf464aa61c1f683f5c6b96 (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:30.208730 19137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47e715af44cf464aa61c1f683f5c6b96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.209112 19283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffc2b16ab3f54c858df9b008cecdf70c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.212368 19212 tablet_service.cc:1511] Processing CreateTablet for tablet 47e715af44cf464aa61c1f683f5c6b96 (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:30.213990 19212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47e715af44cf464aa61c1f683f5c6b96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.219219 19136 tablet_service.cc:1511] Processing CreateTablet for tablet e13f9d6c506846e1bb4f69bf7341622d (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:30.219812 19282 tablet_service.cc:1511] Processing CreateTablet for tablet 47e715af44cf464aa61c1f683f5c6b96 (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:30.220912 19136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e13f9d6c506846e1bb4f69bf7341622d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.221300 19282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47e715af44cf464aa61c1f683f5c6b96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.220961 19208 tablet_service.cc:1511] Processing CreateTablet for tablet e13f9d6c506846e1bb4f69bf7341622d (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:30.222674 19208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e13f9d6c506846e1bb4f69bf7341622d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.251981 19281 tablet_service.cc:1511] Processing CreateTablet for tablet e13f9d6c506846e1bb4f69bf7341622d (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:30.253647 19281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e13f9d6c506846e1bb4f69bf7341622d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.255286 19337 tablet_bootstrap.cc:492] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2: Bootstrap starting.
I20260709 13:06:30.274905 19207 tablet_service.cc:1511] Processing CreateTablet for tablet 2d02069d652c4ef99b2876fb113575f2 (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:30.281283 19207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d02069d652c4ef99b2876fb113575f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.284166 19280 tablet_service.cc:1511] Processing CreateTablet for tablet 2d02069d652c4ef99b2876fb113575f2 (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:30.285569 19338 tablet_bootstrap.cc:492] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc: Bootstrap starting.
I20260709 13:06:30.285710 19280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d02069d652c4ef99b2876fb113575f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.300585 19336 tablet_bootstrap.cc:492] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb: Bootstrap starting.
I20260709 13:06:30.302381 19137 tablet_service.cc:1511] Processing CreateTablet for tablet 2d02069d652c4ef99b2876fb113575f2 (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:30.303828 19137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d02069d652c4ef99b2876fb113575f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.311256 19138 tablet_service.cc:1511] Processing CreateTablet for tablet a6cf4b8385a54efd9d4a624eeecb5c80 (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:30.312781 19338 tablet_bootstrap.cc:654] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.312644 19210 tablet_service.cc:1511] Processing CreateTablet for tablet a6cf4b8385a54efd9d4a624eeecb5c80 (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:30.312971 19138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6cf4b8385a54efd9d4a624eeecb5c80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.314150 19210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6cf4b8385a54efd9d4a624eeecb5c80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.320936 19337 tablet_bootstrap.cc:654] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.329977 19338 tablet_bootstrap.cc:492] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc: No bootstrap required, opened a new log
I20260709 13:06:30.330647 19338 ts_tablet_manager.cc:1403] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc: Time spent bootstrapping tablet: real 0.050s	user 0.014s	sys 0.000s
I20260709 13:06:30.339082 19337 tablet_bootstrap.cc:492] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2: No bootstrap required, opened a new log
I20260709 13:06:30.339694 19337 ts_tablet_manager.cc:1403] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent bootstrapping tablet: real 0.085s	user 0.014s	sys 0.000s
I20260709 13:06:30.342420 19338 raft_consensus.cc:359] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.342792 19337 raft_consensus.cc:359] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.343410 19338 raft_consensus.cc:385] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.334743 19281 tablet_service.cc:1511] Processing CreateTablet for tablet a6cf4b8385a54efd9d4a624eeecb5c80 (DEFAULT_TABLE table=test-workload [id=8b851c18e4aa46be8761d56ed9cfaa80]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:30.344063 19338 raft_consensus.cc:740] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.344794 19281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6cf4b8385a54efd9d4a624eeecb5c80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:30.343618 19337 raft_consensus.cc:385] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.331065 19336 tablet_bootstrap.cc:654] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.345769 19337 raft_consensus.cc:740] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.345669 19338 consensus_queue.cc:260] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.347090 19337 consensus_queue.cc:260] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.356802 19322 heartbeater.cc:499] Master 127.6.255.62:41289 was elected leader, sending a full tablet report...
I20260709 13:06:30.356751 19338 ts_tablet_manager.cc:1434] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc: Time spent starting tablet: real 0.018s	user 0.010s	sys 0.004s
I20260709 13:06:30.359390 19338 tablet_bootstrap.cc:492] T 7d271ad4ab1a46c988238f4154400c61 P 3e8380a11eb9479cae35d2f49f8591dc: Bootstrap starting.
I20260709 13:06:30.367657 19338 tablet_bootstrap.cc:654] T 7d271ad4ab1a46c988238f4154400c61 P 3e8380a11eb9479cae35d2f49f8591dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.380999 19336 tablet_bootstrap.cc:492] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb: No bootstrap required, opened a new log
I20260709 13:06:30.381843 19336 ts_tablet_manager.cc:1403] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb: Time spent bootstrapping tablet: real 0.082s	user 0.011s	sys 0.010s
I20260709 13:06:30.386714 19173 heartbeater.cc:499] Master 127.6.255.62:41289 was elected leader, sending a full tablet report...
I20260709 13:06:30.387238 19337 ts_tablet_manager.cc:1434] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent starting tablet: real 0.047s	user 0.012s	sys 0.032s
I20260709 13:06:30.388590 19337 tablet_bootstrap.cc:492] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2: Bootstrap starting.
I20260709 13:06:30.394234 19336 raft_consensus.cc:359] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.395440 19336 raft_consensus.cc:385] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.395872 19336 raft_consensus.cc:740] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.397806 19336 consensus_queue.cc:260] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.400688 19248 heartbeater.cc:499] Master 127.6.255.62:41289 was elected leader, sending a full tablet report...
I20260709 13:06:30.401366 19336 ts_tablet_manager.cc:1434] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 13:06:30.402118 19337 tablet_bootstrap.cc:654] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.402592 19336 tablet_bootstrap.cc:492] T 7d271ad4ab1a46c988238f4154400c61 P 0e552e70a13c4e4c817d78cdde2861cb: Bootstrap starting.
I20260709 13:06:30.408561 19337 tablet_bootstrap.cc:492] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2: No bootstrap required, opened a new log
I20260709 13:06:30.409160 19337 ts_tablet_manager.cc:1403] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.003s
I20260709 13:06:30.412357 19337 raft_consensus.cc:359] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.413157 19337 raft_consensus.cc:385] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.413630 19337 raft_consensus.cc:740] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.415462 19337 consensus_queue.cc:260] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.416568 19338 tablet_bootstrap.cc:492] T 7d271ad4ab1a46c988238f4154400c61 P 3e8380a11eb9479cae35d2f49f8591dc: No bootstrap required, opened a new log
I20260709 13:06:30.417236 19338 ts_tablet_manager.cc:1403] T 7d271ad4ab1a46c988238f4154400c61 P 3e8380a11eb9479cae35d2f49f8591dc: Time spent bootstrapping tablet: real 0.059s	user 0.013s	sys 0.025s
I20260709 13:06:30.417858 19336 tablet_bootstrap.cc:654] T 7d271ad4ab1a46c988238f4154400c61 P 0e552e70a13c4e4c817d78cdde2861cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.418371 19337 ts_tablet_manager.cc:1434] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.006s
I20260709 13:06:30.419620 19337 tablet_bootstrap.cc:492] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2: Bootstrap starting.
I20260709 13:06:30.420511 19338 raft_consensus.cc:359] T 7d271ad4ab1a46c988238f4154400c61 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.421391 19338 raft_consensus.cc:385] T 7d271ad4ab1a46c988238f4154400c61 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.421762 19338 raft_consensus.cc:740] T 7d271ad4ab1a46c988238f4154400c61 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.422783 19336 tablet_bootstrap.cc:492] T 7d271ad4ab1a46c988238f4154400c61 P 0e552e70a13c4e4c817d78cdde2861cb: No bootstrap required, opened a new log
I20260709 13:06:30.423527 19336 ts_tablet_manager.cc:1403] T 7d271ad4ab1a46c988238f4154400c61 P 0e552e70a13c4e4c817d78cdde2861cb: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.002s
I20260709 13:06:30.422793 19338 consensus_queue.cc:260] T 7d271ad4ab1a46c988238f4154400c61 P 3e8380a11eb9479cae35d2f49f8591dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.425796 19338 ts_tablet_manager.cc:1434] T 7d271ad4ab1a46c988238f4154400c61 P 3e8380a11eb9479cae35d2f49f8591dc: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 13:06:30.427000 19338 tablet_bootstrap.cc:492] T a54f724cc37d4a86b39037771aaa58a7 P 3e8380a11eb9479cae35d2f49f8591dc: Bootstrap starting.
I20260709 13:06:30.426914 19336 raft_consensus.cc:359] T 7d271ad4ab1a46c988238f4154400c61 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.427834 19336 raft_consensus.cc:385] T 7d271ad4ab1a46c988238f4154400c61 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.428150 19336 raft_consensus.cc:740] T 7d271ad4ab1a46c988238f4154400c61 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.428622 19337 tablet_bootstrap.cc:654] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.429050 19336 consensus_queue.cc:260] T 7d271ad4ab1a46c988238f4154400c61 P 0e552e70a13c4e4c817d78cdde2861cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.431730 19336 ts_tablet_manager.cc:1434] T 7d271ad4ab1a46c988238f4154400c61 P 0e552e70a13c4e4c817d78cdde2861cb: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:06:30.432714 19336 tablet_bootstrap.cc:492] T 8d50426588914ed2b6d59c26481beb6c P 0e552e70a13c4e4c817d78cdde2861cb: Bootstrap starting.
I20260709 13:06:30.435645 19338 tablet_bootstrap.cc:654] T a54f724cc37d4a86b39037771aaa58a7 P 3e8380a11eb9479cae35d2f49f8591dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.439862 19336 tablet_bootstrap.cc:654] T 8d50426588914ed2b6d59c26481beb6c P 0e552e70a13c4e4c817d78cdde2861cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.451941 19336 tablet_bootstrap.cc:492] T 8d50426588914ed2b6d59c26481beb6c P 0e552e70a13c4e4c817d78cdde2861cb: No bootstrap required, opened a new log
I20260709 13:06:30.452525 19336 ts_tablet_manager.cc:1403] T 8d50426588914ed2b6d59c26481beb6c P 0e552e70a13c4e4c817d78cdde2861cb: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.009s
I20260709 13:06:30.455470 19336 raft_consensus.cc:359] T 8d50426588914ed2b6d59c26481beb6c P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.456246 19336 raft_consensus.cc:385] T 8d50426588914ed2b6d59c26481beb6c P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.456544 19336 raft_consensus.cc:740] T 8d50426588914ed2b6d59c26481beb6c P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.457518 19336 consensus_queue.cc:260] T 8d50426588914ed2b6d59c26481beb6c P 0e552e70a13c4e4c817d78cdde2861cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.460155 19336 ts_tablet_manager.cc:1434] T 8d50426588914ed2b6d59c26481beb6c P 0e552e70a13c4e4c817d78cdde2861cb: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:06:30.461206 19336 tablet_bootstrap.cc:492] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb: Bootstrap starting.
I20260709 13:06:30.468039 19336 tablet_bootstrap.cc:654] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.481686 19336 tablet_bootstrap.cc:492] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb: No bootstrap required, opened a new log
I20260709 13:06:30.482226 19336 ts_tablet_manager.cc:1403] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.011s
I20260709 13:06:30.484745 19342 raft_consensus.cc:493] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.485562 19337 tablet_bootstrap.cc:492] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2: No bootstrap required, opened a new log
I20260709 13:06:30.485285 19342 raft_consensus.cc:515] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.486189 19337 ts_tablet_manager.cc:1403] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent bootstrapping tablet: real 0.067s	user 0.014s	sys 0.046s
I20260709 13:06:30.486299 19338 tablet_bootstrap.cc:492] T a54f724cc37d4a86b39037771aaa58a7 P 3e8380a11eb9479cae35d2f49f8591dc: No bootstrap required, opened a new log
I20260709 13:06:30.486931 19338 ts_tablet_manager.cc:1403] T a54f724cc37d4a86b39037771aaa58a7 P 3e8380a11eb9479cae35d2f49f8591dc: Time spent bootstrapping tablet: real 0.060s	user 0.026s	sys 0.025s
I20260709 13:06:30.486721 19336 raft_consensus.cc:359] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.487680 19336 raft_consensus.cc:385] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.488071 19336 raft_consensus.cc:740] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.488907 19336 consensus_queue.cc:260] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.489825 19342 leader_election.cc:290] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:30.490703 19338 raft_consensus.cc:359] T a54f724cc37d4a86b39037771aaa58a7 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.493475 19338 raft_consensus.cc:385] T a54f724cc37d4a86b39037771aaa58a7 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.493832 19338 raft_consensus.cc:740] T a54f724cc37d4a86b39037771aaa58a7 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.494728 19338 consensus_queue.cc:260] T a54f724cc37d4a86b39037771aaa58a7 P 3e8380a11eb9479cae35d2f49f8591dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.497354 19338 ts_tablet_manager.cc:1434] T a54f724cc37d4a86b39037771aaa58a7 P 3e8380a11eb9479cae35d2f49f8591dc: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:06:30.491253 19336 ts_tablet_manager.cc:1434] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:06:30.498705 19338 tablet_bootstrap.cc:492] T ffc2b16ab3f54c858df9b008cecdf70c P 3e8380a11eb9479cae35d2f49f8591dc: Bootstrap starting.
I20260709 13:06:30.499614 19336 tablet_bootstrap.cc:492] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb: Bootstrap starting.
I20260709 13:06:30.490754 19337 raft_consensus.cc:359] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.501636 19337 raft_consensus.cc:385] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.502028 19337 raft_consensus.cc:740] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.502882 19337 consensus_queue.cc:260] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.505456 19337 ts_tablet_manager.cc:1434] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.002s
I20260709 13:06:30.506531 19337 tablet_bootstrap.cc:492] T e13f9d6c506846e1bb4f69bf7341622d P b38ff4cb1ee546abba5eb00fe4cf50f2: Bootstrap starting.
I20260709 13:06:30.513202 19338 tablet_bootstrap.cc:654] T ffc2b16ab3f54c858df9b008cecdf70c P 3e8380a11eb9479cae35d2f49f8591dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.520251 19336 tablet_bootstrap.cc:654] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.522056 19337 tablet_bootstrap.cc:654] T e13f9d6c506846e1bb4f69bf7341622d P b38ff4cb1ee546abba5eb00fe4cf50f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.537725 19337 tablet_bootstrap.cc:492] T e13f9d6c506846e1bb4f69bf7341622d P b38ff4cb1ee546abba5eb00fe4cf50f2: No bootstrap required, opened a new log
I20260709 13:06:30.538486 19337 ts_tablet_manager.cc:1403] T e13f9d6c506846e1bb4f69bf7341622d P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent bootstrapping tablet: real 0.032s	user 0.006s	sys 0.016s
I20260709 13:06:30.542801 19337 raft_consensus.cc:359] T e13f9d6c506846e1bb4f69bf7341622d P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.543851 19337 raft_consensus.cc:385] T e13f9d6c506846e1bb4f69bf7341622d P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.544239 19337 raft_consensus.cc:740] T e13f9d6c506846e1bb4f69bf7341622d P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.545320 19337 consensus_queue.cc:260] T e13f9d6c506846e1bb4f69bf7341622d P b38ff4cb1ee546abba5eb00fe4cf50f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.547108 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a54f724cc37d4a86b39037771aaa58a7" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc" is_pre_election: true
I20260709 13:06:30.547986 19297 raft_consensus.cc:2468] T a54f724cc37d4a86b39037771aaa58a7 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 0.
I20260709 13:06:30.547168 19223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a54f724cc37d4a86b39037771aaa58a7" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb" is_pre_election: true
I20260709 13:06:30.549518 19337 ts_tablet_manager.cc:1434] T e13f9d6c506846e1bb4f69bf7341622d P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent starting tablet: real 0.011s	user 0.011s	sys 0.000s
I20260709 13:06:30.549783 19336 tablet_bootstrap.cc:492] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb: No bootstrap required, opened a new log
I20260709 13:06:30.550423 19336 ts_tablet_manager.cc:1403] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb: Time spent bootstrapping tablet: real 0.051s	user 0.013s	sys 0.008s
W20260709 13:06:30.551455 19114 leader_election.cc:343] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:30.552588 19094 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:30.554065 19114 leader_election.cc:304] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e8380a11eb9479cae35d2f49f8591dc, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 0e552e70a13c4e4c817d78cdde2861cb
I20260709 13:06:30.555666 19342 raft_consensus.cc:2804] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:30.556037 19342 raft_consensus.cc:493] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.556402 19342 raft_consensus.cc:3060] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.554222 19337 tablet_bootstrap.cc:492] T ffc2b16ab3f54c858df9b008cecdf70c P b38ff4cb1ee546abba5eb00fe4cf50f2: Bootstrap starting.
I20260709 13:06:30.558092 19338 tablet_bootstrap.cc:492] T ffc2b16ab3f54c858df9b008cecdf70c P 3e8380a11eb9479cae35d2f49f8591dc: No bootstrap required, opened a new log
I20260709 13:06:30.554209 19336 raft_consensus.cc:359] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.558598 19336 raft_consensus.cc:385] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.558771 19338 ts_tablet_manager.cc:1403] T ffc2b16ab3f54c858df9b008cecdf70c P 3e8380a11eb9479cae35d2f49f8591dc: Time spent bootstrapping tablet: real 0.060s	user 0.025s	sys 0.024s
I20260709 13:06:30.559007 19336 raft_consensus.cc:740] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.560021 19336 consensus_queue.cc:260] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.562463 19338 raft_consensus.cc:359] T ffc2b16ab3f54c858df9b008cecdf70c P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.563344 19338 raft_consensus.cc:385] T ffc2b16ab3f54c858df9b008cecdf70c P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.563700 19338 raft_consensus.cc:740] T ffc2b16ab3f54c858df9b008cecdf70c P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.564585 19338 consensus_queue.cc:260] T ffc2b16ab3f54c858df9b008cecdf70c P 3e8380a11eb9479cae35d2f49f8591dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.565229 19342 raft_consensus.cc:515] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.567705 19337 tablet_bootstrap.cc:654] T ffc2b16ab3f54c858df9b008cecdf70c P b38ff4cb1ee546abba5eb00fe4cf50f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.568276 19223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a54f724cc37d4a86b39037771aaa58a7" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
W20260709 13:06:30.569876 19114 leader_election.cc:343] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:30.571604 19342 leader_election.cc:290] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 election: Requested vote from peers 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:30.573681 19336 ts_tablet_manager.cc:1434] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 13:06:30.574117 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a54f724cc37d4a86b39037771aaa58a7" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
I20260709 13:06:30.574813 19297 raft_consensus.cc:3060] T a54f724cc37d4a86b39037771aaa58a7 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.577648 19336 tablet_bootstrap.cc:492] T a54f724cc37d4a86b39037771aaa58a7 P 0e552e70a13c4e4c817d78cdde2861cb: Bootstrap starting.
I20260709 13:06:30.577893 19338 ts_tablet_manager.cc:1434] T ffc2b16ab3f54c858df9b008cecdf70c P 3e8380a11eb9479cae35d2f49f8591dc: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:06:30.582934 19338 tablet_bootstrap.cc:492] T a67334720ba743938c7b5af66099ddcb P 3e8380a11eb9479cae35d2f49f8591dc: Bootstrap starting.
I20260709 13:06:30.590444 19336 tablet_bootstrap.cc:654] T a54f724cc37d4a86b39037771aaa58a7 P 0e552e70a13c4e4c817d78cdde2861cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.585206 19297 raft_consensus.cc:2468] T a54f724cc37d4a86b39037771aaa58a7 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 1.
I20260709 13:06:30.591326 19338 tablet_bootstrap.cc:654] T a67334720ba743938c7b5af66099ddcb P 3e8380a11eb9479cae35d2f49f8591dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.591997 19114 leader_election.cc:304] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e8380a11eb9479cae35d2f49f8591dc, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 0e552e70a13c4e4c817d78cdde2861cb
I20260709 13:06:30.592653 19337 tablet_bootstrap.cc:492] T ffc2b16ab3f54c858df9b008cecdf70c P b38ff4cb1ee546abba5eb00fe4cf50f2: No bootstrap required, opened a new log
I20260709 13:06:30.593395 19337 ts_tablet_manager.cc:1403] T ffc2b16ab3f54c858df9b008cecdf70c P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent bootstrapping tablet: real 0.039s	user 0.016s	sys 0.015s
I20260709 13:06:30.593262 19342 raft_consensus.cc:2804] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:30.594940 19342 raft_consensus.cc:697] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 LEADER]: Becoming Leader. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Running, Role: LEADER
I20260709 13:06:30.596117 19342 consensus_queue.cc:237] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.603790 19338 tablet_bootstrap.cc:492] T a67334720ba743938c7b5af66099ddcb P 3e8380a11eb9479cae35d2f49f8591dc: No bootstrap required, opened a new log
I20260709 13:06:30.604637 19338 ts_tablet_manager.cc:1403] T a67334720ba743938c7b5af66099ddcb P 3e8380a11eb9479cae35d2f49f8591dc: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.004s
I20260709 13:06:30.603859 19337 raft_consensus.cc:359] T ffc2b16ab3f54c858df9b008cecdf70c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.606000 19337 raft_consensus.cc:385] T ffc2b16ab3f54c858df9b008cecdf70c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.606314 19337 raft_consensus.cc:740] T ffc2b16ab3f54c858df9b008cecdf70c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.606886 19336 tablet_bootstrap.cc:492] T a54f724cc37d4a86b39037771aaa58a7 P 0e552e70a13c4e4c817d78cdde2861cb: No bootstrap required, opened a new log
I20260709 13:06:30.607652 19336 ts_tablet_manager.cc:1403] T a54f724cc37d4a86b39037771aaa58a7 P 0e552e70a13c4e4c817d78cdde2861cb: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.007s
I20260709 13:06:30.607116 19337 consensus_queue.cc:260] T ffc2b16ab3f54c858df9b008cecdf70c P b38ff4cb1ee546abba5eb00fe4cf50f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.608537 19338 raft_consensus.cc:359] T a67334720ba743938c7b5af66099ddcb P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.609273 19338 raft_consensus.cc:385] T a67334720ba743938c7b5af66099ddcb P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.609932 19338 raft_consensus.cc:740] T a67334720ba743938c7b5af66099ddcb P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.611503 19338 consensus_queue.cc:260] T a67334720ba743938c7b5af66099ddcb P 3e8380a11eb9479cae35d2f49f8591dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.613117 19337 ts_tablet_manager.cc:1434] T ffc2b16ab3f54c858df9b008cecdf70c P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:06:30.614611 19337 tablet_bootstrap.cc:492] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2: Bootstrap starting.
I20260709 13:06:30.615862 19338 ts_tablet_manager.cc:1434] T a67334720ba743938c7b5af66099ddcb P 3e8380a11eb9479cae35d2f49f8591dc: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.004s
I20260709 13:06:30.617007 19338 tablet_bootstrap.cc:492] T e13f9d6c506846e1bb4f69bf7341622d P 3e8380a11eb9479cae35d2f49f8591dc: Bootstrap starting.
I20260709 13:06:30.616384 19044 catalog_manager.cc:5697] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:30.621786 19337 tablet_bootstrap.cc:654] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.625152 19338 tablet_bootstrap.cc:654] T e13f9d6c506846e1bb4f69bf7341622d P 3e8380a11eb9479cae35d2f49f8591dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.627759 19337 tablet_bootstrap.cc:492] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2: No bootstrap required, opened a new log
I20260709 13:06:30.628353 19337 ts_tablet_manager.cc:1403] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.002s
I20260709 13:06:30.631481 19337 raft_consensus.cc:359] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.632227 19337 raft_consensus.cc:385] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.632422 19338 tablet_bootstrap.cc:492] T e13f9d6c506846e1bb4f69bf7341622d P 3e8380a11eb9479cae35d2f49f8591dc: No bootstrap required, opened a new log
I20260709 13:06:30.632530 19337 raft_consensus.cc:740] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.633072 19338 ts_tablet_manager.cc:1403] T e13f9d6c506846e1bb4f69bf7341622d P 3e8380a11eb9479cae35d2f49f8591dc: Time spent bootstrapping tablet: real 0.016s	user 0.008s	sys 0.004s
I20260709 13:06:30.633489 19337 consensus_queue.cc:260] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.635161 19336 raft_consensus.cc:359] T a54f724cc37d4a86b39037771aaa58a7 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.635932 19336 raft_consensus.cc:385] T a54f724cc37d4a86b39037771aaa58a7 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.636159 19345 raft_consensus.cc:493] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.636376 19336 raft_consensus.cc:740] T a54f724cc37d4a86b39037771aaa58a7 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.636574 19337 ts_tablet_manager.cc:1434] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:06:30.636749 19345 raft_consensus.cc:515] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.637794 19337 tablet_bootstrap.cc:492] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2: Bootstrap starting.
I20260709 13:06:30.637398 19336 consensus_queue.cc:260] T a54f724cc37d4a86b39037771aaa58a7 P 0e552e70a13c4e4c817d78cdde2861cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.639098 19338 raft_consensus.cc:359] T e13f9d6c506846e1bb4f69bf7341622d P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.639894 19338 raft_consensus.cc:385] T e13f9d6c506846e1bb4f69bf7341622d P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.640180 19338 raft_consensus.cc:740] T e13f9d6c506846e1bb4f69bf7341622d P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.641319 19338 consensus_queue.cc:260] T e13f9d6c506846e1bb4f69bf7341622d P 3e8380a11eb9479cae35d2f49f8591dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.644145 19338 ts_tablet_manager.cc:1434] T e13f9d6c506846e1bb4f69bf7341622d P 3e8380a11eb9479cae35d2f49f8591dc: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:06:30.645232 19338 tablet_bootstrap.cc:492] T 47e715af44cf464aa61c1f683f5c6b96 P 3e8380a11eb9479cae35d2f49f8591dc: Bootstrap starting.
I20260709 13:06:30.645512 19337 tablet_bootstrap.cc:654] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.650904 19345 leader_election.cc:290] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:30.653856 19338 tablet_bootstrap.cc:654] T 47e715af44cf464aa61c1f683f5c6b96 P 3e8380a11eb9479cae35d2f49f8591dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.657092 19337 tablet_bootstrap.cc:492] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2: No bootstrap required, opened a new log
I20260709 13:06:30.657719 19337 ts_tablet_manager.cc:1403] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.004s
I20260709 13:06:30.661830 19337 raft_consensus.cc:359] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.662887 19337 raft_consensus.cc:385] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.663281 19337 raft_consensus.cc:740] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.664472 19337 consensus_queue.cc:260] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.668107 19337 ts_tablet_manager.cc:1434] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.003s
I20260709 13:06:30.669673 19337 tablet_bootstrap.cc:492] T 2d02069d652c4ef99b2876fb113575f2 P b38ff4cb1ee546abba5eb00fe4cf50f2: Bootstrap starting.
I20260709 13:06:30.675314 19336 ts_tablet_manager.cc:1434] T a54f724cc37d4a86b39037771aaa58a7 P 0e552e70a13c4e4c817d78cdde2861cb: Time spent starting tablet: real 0.065s	user 0.007s	sys 0.004s
I20260709 13:06:30.677018 19336 tablet_bootstrap.cc:492] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb: Bootstrap starting.
I20260709 13:06:30.679114 19337 tablet_bootstrap.cc:654] T 2d02069d652c4ef99b2876fb113575f2 P b38ff4cb1ee546abba5eb00fe4cf50f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.679525 19338 tablet_bootstrap.cc:492] T 47e715af44cf464aa61c1f683f5c6b96 P 3e8380a11eb9479cae35d2f49f8591dc: No bootstrap required, opened a new log
I20260709 13:06:30.680133 19338 ts_tablet_manager.cc:1403] T 47e715af44cf464aa61c1f683f5c6b96 P 3e8380a11eb9479cae35d2f49f8591dc: Time spent bootstrapping tablet: real 0.035s	user 0.017s	sys 0.015s
I20260709 13:06:30.683792 19338 raft_consensus.cc:359] T 47e715af44cf464aa61c1f683f5c6b96 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.685117 19338 raft_consensus.cc:385] T 47e715af44cf464aa61c1f683f5c6b96 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.685658 19338 raft_consensus.cc:740] T 47e715af44cf464aa61c1f683f5c6b96 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.686825 19338 consensus_queue.cc:260] T 47e715af44cf464aa61c1f683f5c6b96 P 3e8380a11eb9479cae35d2f49f8591dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.689558 19342 raft_consensus.cc:493] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.690299 19342 raft_consensus.cc:515] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.694536 19223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d50426588914ed2b6d59c26481beb6c" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb" is_pre_election: true
I20260709 13:06:30.695225 19223 raft_consensus.cc:2468] T 8d50426588914ed2b6d59c26481beb6c P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 0.
I20260709 13:06:30.695891 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d50426588914ed2b6d59c26481beb6c" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc" is_pre_election: true
I20260709 13:06:30.696738 19297 raft_consensus.cc:2468] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 0.
I20260709 13:06:30.698140 19114 leader_election.cc:304] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8380a11eb9479cae35d2f49f8591dc, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:30.699955 19360 raft_consensus.cc:2804] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:30.700341 19360 raft_consensus.cc:493] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.700680 19360 raft_consensus.cc:3060] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.700861 19148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a67334720ba743938c7b5af66099ddcb" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" is_pre_election: true
I20260709 13:06:30.701655 19148 raft_consensus.cc:2468] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 0.
I20260709 13:06:30.705876 19342 leader_election.cc:290] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:30.706365 19189 leader_election.cc:304] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:30.708364 19365 raft_consensus.cc:2804] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:30.709020 19365 raft_consensus.cc:493] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.709484 19365 raft_consensus.cc:3060] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.711020 19360 raft_consensus.cc:515] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.690289 19338 ts_tablet_manager.cc:1434] T 47e715af44cf464aa61c1f683f5c6b96 P 3e8380a11eb9479cae35d2f49f8591dc: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:06:30.715076 19223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d50426588914ed2b6d59c26481beb6c" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
I20260709 13:06:30.715685 19223 raft_consensus.cc:3060] T 8d50426588914ed2b6d59c26481beb6c P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.716131 19338 tablet_bootstrap.cc:492] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc: Bootstrap starting.
I20260709 13:06:30.721629 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d50426588914ed2b6d59c26481beb6c" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
I20260709 13:06:30.722250 19297 raft_consensus.cc:3060] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.724171 19223 raft_consensus.cc:2468] T 8d50426588914ed2b6d59c26481beb6c P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 1.
I20260709 13:06:30.725466 19296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a67334720ba743938c7b5af66099ddcb" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc" is_pre_election: true
I20260709 13:06:30.726245 19296 raft_consensus.cc:2468] T a67334720ba743938c7b5af66099ddcb P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 0.
I20260709 13:06:30.725530 19114 leader_election.cc:304] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:30.727705 19337 tablet_bootstrap.cc:492] T 2d02069d652c4ef99b2876fb113575f2 P b38ff4cb1ee546abba5eb00fe4cf50f2: No bootstrap required, opened a new log
I20260709 13:06:30.728273 19337 ts_tablet_manager.cc:1403] T 2d02069d652c4ef99b2876fb113575f2 P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent bootstrapping tablet: real 0.059s	user 0.015s	sys 0.015s
I20260709 13:06:30.728446 19342 raft_consensus.cc:2804] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:30.729266 19360 leader_election.cc:290] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 election: Requested vote from peers 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:30.721657 19336 tablet_bootstrap.cc:654] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.737025 19365 raft_consensus.cc:515] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.740437 19148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a67334720ba743938c7b5af66099ddcb" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
I20260709 13:06:30.731177 19337 raft_consensus.cc:359] T 2d02069d652c4ef99b2876fb113575f2 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.741289 19148 raft_consensus.cc:3060] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.741578 19338 tablet_bootstrap.cc:654] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.741611 19337 raft_consensus.cc:385] T 2d02069d652c4ef99b2876fb113575f2 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.742304 19337 raft_consensus.cc:740] T 2d02069d652c4ef99b2876fb113575f2 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.742297 19296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a67334720ba743938c7b5af66099ddcb" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
I20260709 13:06:30.743232 19296 raft_consensus.cc:3060] T a67334720ba743938c7b5af66099ddcb P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.743659 19337 consensus_queue.cc:260] T 2d02069d652c4ef99b2876fb113575f2 P b38ff4cb1ee546abba5eb00fe4cf50f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.730340 19297 raft_consensus.cc:2468] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 1.
I20260709 13:06:30.746033 19337 ts_tablet_manager.cc:1434] T 2d02069d652c4ef99b2876fb113575f2 P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.002s
I20260709 13:06:30.747138 19337 tablet_bootstrap.cc:492] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2: Bootstrap starting.
I20260709 13:06:30.749547 19342 raft_consensus.cc:697] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 LEADER]: Becoming Leader. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Running, Role: LEADER
I20260709 13:06:30.750803 19342 consensus_queue.cc:237] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.751505 19296 raft_consensus.cc:2468] T a67334720ba743938c7b5af66099ddcb P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 1.
I20260709 13:06:30.752875 19189 leader_election.cc:304] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, 3e8380a11eb9479cae35d2f49f8591dc; no voters: 
I20260709 13:06:30.753685 19337 tablet_bootstrap.cc:654] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.753772 19345 raft_consensus.cc:2804] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:30.754946 19148 raft_consensus.cc:2468] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 1.
I20260709 13:06:30.762904 19374 raft_consensus.cc:493] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.763744 19374 raft_consensus.cc:515] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.768878 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d271ad4ab1a46c988238f4154400c61" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc" is_pre_election: true
I20260709 13:06:30.768982 19374 leader_election.cc:290] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:30.770028 19297 raft_consensus.cc:2468] T 7d271ad4ab1a46c988238f4154400c61 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 0.
I20260709 13:06:30.771771 19114 leader_election.cc:304] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8380a11eb9479cae35d2f49f8591dc, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:30.772920 19374 raft_consensus.cc:2804] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:30.773375 19374 raft_consensus.cc:493] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.773809 19374 raft_consensus.cc:3060] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.768888 19365 leader_election.cc:290] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 election: Requested vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:30.780642 19365 raft_consensus.cc:493] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.785368 19378 raft_consensus.cc:493] T 7d271ad4ab1a46c988238f4154400c61 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.786134 19336 tablet_bootstrap.cc:492] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb: No bootstrap required, opened a new log
I20260709 13:06:30.785975 19378 raft_consensus.cc:515] T 7d271ad4ab1a46c988238f4154400c61 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.786727 19336 ts_tablet_manager.cc:1403] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb: Time spent bootstrapping tablet: real 0.110s	user 0.019s	sys 0.013s
I20260709 13:06:30.781271 19365 raft_consensus.cc:515] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.790344 19336 raft_consensus.cc:359] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.790558 19147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffc2b16ab3f54c858df9b008cecdf70c" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" is_pre_election: true
I20260709 13:06:30.791241 19336 raft_consensus.cc:385] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.791373 19296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffc2b16ab3f54c858df9b008cecdf70c" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc" is_pre_election: true
I20260709 13:06:30.791508 19147 raft_consensus.cc:2468] T ffc2b16ab3f54c858df9b008cecdf70c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 0.
I20260709 13:06:30.791693 19336 raft_consensus.cc:740] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.792109 19296 raft_consensus.cc:2468] T ffc2b16ab3f54c858df9b008cecdf70c P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 0.
I20260709 13:06:30.792740 19336 consensus_queue.cc:260] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.792133 19223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d271ad4ab1a46c988238f4154400c61" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb" is_pre_election: true
I20260709 13:06:30.793627 19189 leader_election.cc:304] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:30.793910 19223 raft_consensus.cc:2468] T 7d271ad4ab1a46c988238f4154400c61 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 0.
I20260709 13:06:30.793819 19365 leader_election.cc:290] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:30.790518 19148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d271ad4ab1a46c988238f4154400c61" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" is_pre_election: true
I20260709 13:06:30.794973 19365 raft_consensus.cc:2804] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:30.796288 19345 raft_consensus.cc:697] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Becoming Leader. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Running, Role: LEADER
I20260709 13:06:30.797487 19336 ts_tablet_manager.cc:1434] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.006s
I20260709 13:06:30.797555 19345 consensus_queue.cc:237] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.798956 19336 tablet_bootstrap.cc:492] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb: Bootstrap starting.
I20260709 13:06:30.795471 19365 raft_consensus.cc:493] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.802909 19365 raft_consensus.cc:3060] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.807777 19377 raft_consensus.cc:493] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.807943 19336 tablet_bootstrap.cc:654] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.808530 19377 raft_consensus.cc:515] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.811282 19338 tablet_bootstrap.cc:492] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc: No bootstrap required, opened a new log
I20260709 13:06:30.810139 19046 catalog_manager.cc:5697] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:30.811870 19338 ts_tablet_manager.cc:1403] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc: Time spent bootstrapping tablet: real 0.096s	user 0.018s	sys 0.016s
I20260709 13:06:30.811972 19342 raft_consensus.cc:493] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.812525 19342 raft_consensus.cc:515] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.815277 19338 raft_consensus.cc:359] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.815837 19223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e715af44cf464aa61c1f683f5c6b96" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb" is_pre_election: true
I20260709 13:06:30.816601 19223 raft_consensus.cc:2468] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 0.
I20260709 13:06:30.816102 19338 raft_consensus.cc:385] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.817950 19338 raft_consensus.cc:740] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.818058 19114 leader_election.cc:304] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:30.819049 19360 raft_consensus.cc:2804] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:30.818951 19338 consensus_queue.cc:260] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.819514 19360 raft_consensus.cc:493] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.819936 19360 raft_consensus.cc:3060] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.824144 19338 ts_tablet_manager.cc:1434] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.002s
I20260709 13:06:30.825980 19338 tablet_bootstrap.cc:492] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc: Bootstrap starting.
I20260709 13:06:30.835148 19147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13f9d6c506846e1bb4f69bf7341622d" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" is_pre_election: true
I20260709 13:06:30.836061 19147 raft_consensus.cc:2468] T e13f9d6c506846e1bb4f69bf7341622d P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 0.
I20260709 13:06:30.837991 19189 leader_election.cc:304] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:30.836050 19338 tablet_bootstrap.cc:654] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.838927 19345 raft_consensus.cc:2804] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:30.839048 19337 tablet_bootstrap.cc:492] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2: No bootstrap required, opened a new log
I20260709 13:06:30.839937 19345 raft_consensus.cc:493] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.840335 19337 ts_tablet_manager.cc:1403] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent bootstrapping tablet: real 0.093s	user 0.042s	sys 0.029s
I20260709 13:06:30.843714 19337 raft_consensus.cc:359] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.844105 19365 raft_consensus.cc:515] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.840359 19345 raft_consensus.cc:3060] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.843856 19043 catalog_manager.cc:5697] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2). New cstate: current_term: 1 leader_uuid: "0e552e70a13c4e4c817d78cdde2861cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:30.844529 19337 raft_consensus.cc:385] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.846237 19337 raft_consensus.cc:740] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.847039 19337 consensus_queue.cc:260] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.850117 19336 tablet_bootstrap.cc:492] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb: No bootstrap required, opened a new log
I20260709 13:06:30.850756 19336 ts_tablet_manager.cc:1403] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb: Time spent bootstrapping tablet: real 0.052s	user 0.014s	sys 0.002s
I20260709 13:06:30.854231 19336 raft_consensus.cc:359] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.854986 19336 raft_consensus.cc:385] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.855274 19336 raft_consensus.cc:740] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.855969 19345 raft_consensus.cc:515] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.871412 19342 leader_election.cc:290] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:30.872777 19296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e715af44cf464aa61c1f683f5c6b96" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc" is_pre_election: true
I20260709 13:06:30.873869 19296 raft_consensus.cc:2468] T 47e715af44cf464aa61c1f683f5c6b96 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 0.
I20260709 13:06:30.877089 19377 leader_election.cc:290] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:30.829026 19360 raft_consensus.cc:515] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.886142 19296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13f9d6c506846e1bb4f69bf7341622d" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc" is_pre_election: true
I20260709 13:06:30.886395 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e715af44cf464aa61c1f683f5c6b96" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
I20260709 13:06:30.887250 19296 raft_consensus.cc:2468] T e13f9d6c506846e1bb4f69bf7341622d P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 0.
I20260709 13:06:30.887338 19297 raft_consensus.cc:3060] T 47e715af44cf464aa61c1f683f5c6b96 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.890659 19336 consensus_queue.cc:260] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.895059 19147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffc2b16ab3f54c858df9b008cecdf70c" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
I20260709 13:06:30.895941 19147 raft_consensus.cc:3060] T ffc2b16ab3f54c858df9b008cecdf70c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.896683 19295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffc2b16ab3f54c858df9b008cecdf70c" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
I20260709 13:06:30.897397 19295 raft_consensus.cc:3060] T ffc2b16ab3f54c858df9b008cecdf70c P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.897284 19223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e715af44cf464aa61c1f683f5c6b96" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
I20260709 13:06:30.900174 19223 raft_consensus.cc:3060] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.900852 19380 raft_consensus.cc:493] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.895020 19365 leader_election.cc:290] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 election: Requested vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:30.901557 19380 raft_consensus.cc:515] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.902881 19378 leader_election.cc:290] T 7d271ad4ab1a46c988238f4154400c61 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:30.903040 19296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d271ad4ab1a46c988238f4154400c61" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc" is_pre_election: true
I20260709 13:06:30.903784 19296 raft_consensus.cc:2468] T 7d271ad4ab1a46c988238f4154400c61 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 0.
I20260709 13:06:30.903905 19380 leader_election.cc:290] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:30.904340 19147 raft_consensus.cc:2468] T ffc2b16ab3f54c858df9b008cecdf70c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 1.
I20260709 13:06:30.905042 19189 leader_election.cc:304] T 7d271ad4ab1a46c988238f4154400c61 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, 3e8380a11eb9479cae35d2f49f8591dc; no voters: 
I20260709 13:06:30.906167 19377 raft_consensus.cc:2804] T 7d271ad4ab1a46c988238f4154400c61 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:30.906610 19377 raft_consensus.cc:493] T 7d271ad4ab1a46c988238f4154400c61 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.906334 19189 leader_election.cc:304] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:30.907569 19377 raft_consensus.cc:3060] T 7d271ad4ab1a46c988238f4154400c61 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.907775 19380 raft_consensus.cc:2804] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:30.908505 19380 raft_consensus.cc:697] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Becoming Leader. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Running, Role: LEADER
I20260709 13:06:30.909839 19380 consensus_queue.cc:237] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.913996 19374 raft_consensus.cc:515] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.902009 19336 ts_tablet_manager.cc:1434] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb: Time spent starting tablet: real 0.051s	user 0.011s	sys 0.001s
I20260709 13:06:30.914824 19360 leader_election.cc:290] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 election: Requested vote from peers 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:30.916344 19336 tablet_bootstrap.cc:492] T a6cf4b8385a54efd9d4a624eeecb5c80 P 0e552e70a13c4e4c817d78cdde2861cb: Bootstrap starting.
I20260709 13:06:30.915553 19297 raft_consensus.cc:2468] T 47e715af44cf464aa61c1f683f5c6b96 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 1.
I20260709 13:06:30.918869 19148 raft_consensus.cc:2393] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e552e70a13c4e4c817d78cdde2861cb in current term 1: Already voted for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in this term.
I20260709 13:06:30.917829 19147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13f9d6c506846e1bb4f69bf7341622d" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
I20260709 13:06:30.919638 19147 raft_consensus.cc:3060] T e13f9d6c506846e1bb4f69bf7341622d P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.920677 19114 leader_election.cc:304] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8380a11eb9479cae35d2f49f8591dc, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:30.920994 19222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d271ad4ab1a46c988238f4154400c61" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
I20260709 13:06:30.923883 19146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d02069d652c4ef99b2876fb113575f2" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" is_pre_election: true
I20260709 13:06:30.926290 19146 raft_consensus.cc:2468] T 2d02069d652c4ef99b2876fb113575f2 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 0.
I20260709 13:06:30.926586 19374 leader_election.cc:290] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 election: Requested vote from peers 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:30.927582 19385 raft_consensus.cc:2804] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:30.928174 19385 raft_consensus.cc:697] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 LEADER]: Becoming Leader. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Running, Role: LEADER
I20260709 13:06:30.927392 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d271ad4ab1a46c988238f4154400c61" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
I20260709 13:06:30.929322 19385 consensus_queue.cc:237] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.929858 19189 leader_election.cc:304] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:30.930631 19296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d02069d652c4ef99b2876fb113575f2" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc" is_pre_election: true
I20260709 13:06:30.931385 19296 raft_consensus.cc:2468] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 0.
I20260709 13:06:30.933996 19378 raft_consensus.cc:2804] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:30.934544 19378 raft_consensus.cc:493] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.934979 19378 raft_consensus.cc:3060] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.936699 19223 raft_consensus.cc:2468] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 1.
I20260709 13:06:30.938197 19337 ts_tablet_manager.cc:1434] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent starting tablet: real 0.097s	user 0.021s	sys 0.032s
I20260709 13:06:30.930152 19297 raft_consensus.cc:3060] T 7d271ad4ab1a46c988238f4154400c61 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.939548 19377 raft_consensus.cc:515] T 7d271ad4ab1a46c988238f4154400c61 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.940929 19222 raft_consensus.cc:2393] T 7d271ad4ab1a46c988238f4154400c61 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in current term 1: Already voted for candidate 0e552e70a13c4e4c817d78cdde2861cb in this term.
I20260709 13:06:30.938675 19345 leader_election.cc:290] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 election: Requested vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:30.941651 19296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13f9d6c506846e1bb4f69bf7341622d" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
I20260709 13:06:30.942421 19296 raft_consensus.cc:3060] T e13f9d6c506846e1bb4f69bf7341622d P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.947108 19377 leader_election.cc:290] T 7d271ad4ab1a46c988238f4154400c61 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 election: Requested vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:30.936872 19336 tablet_bootstrap.cc:654] T a6cf4b8385a54efd9d4a624eeecb5c80 P 0e552e70a13c4e4c817d78cdde2861cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.949587 19146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d271ad4ab1a46c988238f4154400c61" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
I20260709 13:06:30.951020 19146 raft_consensus.cc:2393] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e552e70a13c4e4c817d78cdde2861cb in current term 1: Already voted for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in this term.
I20260709 13:06:30.952183 19294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d271ad4ab1a46c988238f4154400c61" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
I20260709 13:06:30.949537 19295 raft_consensus.cc:2468] T ffc2b16ab3f54c858df9b008cecdf70c P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 1.
I20260709 13:06:30.936693 19147 raft_consensus.cc:2468] T e13f9d6c506846e1bb4f69bf7341622d P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 1.
I20260709 13:06:30.963943 19338 tablet_bootstrap.cc:492] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc: No bootstrap required, opened a new log
I20260709 13:06:30.964510 19338 ts_tablet_manager.cc:1403] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc: Time spent bootstrapping tablet: real 0.139s	user 0.012s	sys 0.060s
I20260709 13:06:30.966259 19297 raft_consensus.cc:2468] T 7d271ad4ab1a46c988238f4154400c61 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 1.
I20260709 13:06:30.968472 19338 raft_consensus.cc:359] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.966367 19189 leader_election.cc:304] T 7d271ad4ab1a46c988238f4154400c61 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb; no voters: 3e8380a11eb9479cae35d2f49f8591dc, b38ff4cb1ee546abba5eb00fe4cf50f2
I20260709 13:06:30.970021 19338 raft_consensus.cc:385] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.969164 19336 tablet_bootstrap.cc:492] T a6cf4b8385a54efd9d4a624eeecb5c80 P 0e552e70a13c4e4c817d78cdde2861cb: No bootstrap required, opened a new log
I20260709 13:06:30.970418 19380 raft_consensus.cc:2749] T 7d271ad4ab1a46c988238f4154400c61 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:30.970844 19336 ts_tablet_manager.cc:1403] T a6cf4b8385a54efd9d4a624eeecb5c80 P 0e552e70a13c4e4c817d78cdde2861cb: Time spent bootstrapping tablet: real 0.055s	user 0.008s	sys 0.012s
I20260709 13:06:30.970443 19338 raft_consensus.cc:740] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.972203 19338 consensus_queue.cc:260] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.974365 19189 leader_election.cc:304] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:30.974766 19336 raft_consensus.cc:359] T a6cf4b8385a54efd9d4a624eeecb5c80 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.975764 19336 raft_consensus.cc:385] T a6cf4b8385a54efd9d4a624eeecb5c80 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.976179 19336 raft_consensus.cc:740] T a6cf4b8385a54efd9d4a624eeecb5c80 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.977012 19338 ts_tablet_manager.cc:1434] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.006s
I20260709 13:06:30.977530 19114 leader_election.cc:304] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e8380a11eb9479cae35d2f49f8591dc, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 0e552e70a13c4e4c817d78cdde2861cb
I20260709 13:06:30.977052 19336 consensus_queue.cc:260] T a6cf4b8385a54efd9d4a624eeecb5c80 P 0e552e70a13c4e4c817d78cdde2861cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.975153 19378 raft_consensus.cc:515] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.980392 19380 raft_consensus.cc:2804] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:30.981220 19380 raft_consensus.cc:697] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Becoming Leader. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Running, Role: LEADER
I20260709 13:06:30.982197 19380 consensus_queue.cc:237] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:30.982038 19046 catalog_manager.cc:5697] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:30.983254 19378 leader_election.cc:290] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 election: Requested vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:30.984840 19336 ts_tablet_manager.cc:1434] T a6cf4b8385a54efd9d4a624eeecb5c80 P 0e552e70a13c4e4c817d78cdde2861cb: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:06:30.989380 19044 catalog_manager.cc:5697] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2). New cstate: current_term: 1 leader_uuid: "0e552e70a13c4e4c817d78cdde2861cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:30.994345 19296 raft_consensus.cc:2468] T e13f9d6c506846e1bb4f69bf7341622d P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 1.
I20260709 13:06:30.994568 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d02069d652c4ef99b2876fb113575f2" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
I20260709 13:06:30.995663 19297 raft_consensus.cc:3060] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.995748 19147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d02069d652c4ef99b2876fb113575f2" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
I20260709 13:06:30.996594 19147 raft_consensus.cc:3060] T 2d02069d652c4ef99b2876fb113575f2 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.004729 19385 raft_consensus.cc:2804] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:31.007954 19297 raft_consensus.cc:2468] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 1.
I20260709 13:06:31.009174 19385 raft_consensus.cc:697] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 LEADER]: Becoming Leader. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Running, Role: LEADER
I20260709 13:06:31.006935 19147 raft_consensus.cc:2468] T 2d02069d652c4ef99b2876fb113575f2 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 1.
I20260709 13:06:31.015338 19385 consensus_queue.cc:237] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:31.017721 19189 leader_election.cc:304] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, 3e8380a11eb9479cae35d2f49f8591dc; no voters: 
I20260709 13:06:31.018774 19380 raft_consensus.cc:2804] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:31.020011 19380 raft_consensus.cc:697] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Becoming Leader. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Running, Role: LEADER
I20260709 13:06:31.020996 19380 consensus_queue.cc:237] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:31.030443 19044 catalog_manager.cc:5697] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:31.039601 19046 catalog_manager.cc:5697] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2). New cstate: current_term: 1 leader_uuid: "0e552e70a13c4e4c817d78cdde2861cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:31.041218 19046 catalog_manager.cc:5697] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2). New cstate: current_term: 1 leader_uuid: "0e552e70a13c4e4c817d78cdde2861cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:31.055528 19385 consensus_queue.cc:1048] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:31.066989 19223 raft_consensus.cc:3060] T a54f724cc37d4a86b39037771aaa58a7 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.077558 19380 raft_consensus.cc:493] T a54f724cc37d4a86b39037771aaa58a7 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.078094 19380 raft_consensus.cc:515] T a54f724cc37d4a86b39037771aaa58a7 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:31.078598 19385 consensus_queue.cc:1048] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:31.082829 19147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a54f724cc37d4a86b39037771aaa58a7" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" is_pre_election: true
I20260709 13:06:31.083153 19380 leader_election.cc:290] T a54f724cc37d4a86b39037771aaa58a7 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:31.084620 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a54f724cc37d4a86b39037771aaa58a7" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc" is_pre_election: true
I20260709 13:06:31.086589 19189 leader_election.cc:304] T a54f724cc37d4a86b39037771aaa58a7 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb; no voters: 3e8380a11eb9479cae35d2f49f8591dc, b38ff4cb1ee546abba5eb00fe4cf50f2
I20260709 13:06:31.087661 19380 raft_consensus.cc:2749] T a54f724cc37d4a86b39037771aaa58a7 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:31.216355 19385 raft_consensus.cc:493] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.217003 19385 raft_consensus.cc:515] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:31.218981 19385 leader_election.cc:290] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:31.219622 19223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6cf4b8385a54efd9d4a624eeecb5c80" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb" is_pre_election: true
I20260709 13:06:31.220327 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6cf4b8385a54efd9d4a624eeecb5c80" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc" is_pre_election: true
I20260709 13:06:31.220388 19223 raft_consensus.cc:2468] T a6cf4b8385a54efd9d4a624eeecb5c80 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 0.
I20260709 13:06:31.220683 19385 consensus_queue.cc:1048] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:31.222157 19297 raft_consensus.cc:2468] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 0.
I20260709 13:06:31.222878 19114 leader_election.cc:304] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:31.223672 19385 raft_consensus.cc:2804] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:31.224185 19385 raft_consensus.cc:493] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:31.224622 19385 raft_consensus.cc:3060] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.233253 19385 raft_consensus.cc:515] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:31.235777 19385 leader_election.cc:290] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 election: Requested vote from peers 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:31.237568 19296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6cf4b8385a54efd9d4a624eeecb5c80" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
I20260709 13:06:31.238356 19296 raft_consensus.cc:3060] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.239585 19222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6cf4b8385a54efd9d4a624eeecb5c80" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
I20260709 13:06:31.240314 19222 raft_consensus.cc:3060] T a6cf4b8385a54efd9d4a624eeecb5c80 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.240110 19385 consensus_queue.cc:1048] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:31.245966 19296 raft_consensus.cc:2468] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 1.
I20260709 13:06:31.252693 19222 raft_consensus.cc:2468] T a6cf4b8385a54efd9d4a624eeecb5c80 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 1.
I20260709 13:06:31.253140 19114 leader_election.cc:304] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8380a11eb9479cae35d2f49f8591dc, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:31.254534 19342 raft_consensus.cc:2804] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:31.254914 19380 consensus_queue.cc:1048] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:31.255039 19342 raft_consensus.cc:697] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 LEADER]: Becoming Leader. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Running, Role: LEADER
I20260709 13:06:31.257454 19342 consensus_queue.cc:237] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:31.277907 19377 consensus_queue.cc:1048] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:31.281015 19045 catalog_manager.cc:5697] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:31.330395 19342 consensus_queue.cc:1048] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:31.342756 19342 consensus_queue.cc:1048] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:31.381963 19377 consensus_queue.cc:1048] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:31.395982 19377 consensus_queue.cc:1048] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:31.425848 19380 consensus_queue.cc:1048] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:31.429491 19380 consensus_queue.cc:1048] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:31.429924 19374 consensus_queue.cc:1048] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:31.448675 19378 consensus_queue.cc:1048] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:31.456228 19378 consensus_queue.cc:1048] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:31.495443 19342 consensus_queue.cc:1048] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:31.564857 19222 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a67334720ba743938c7b5af66099ddcb"
dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
mode: GRACEFUL
new_leader_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
 from {username='slave'} at 127.0.0.1:52694
I20260709 13:06:31.565644 19222 raft_consensus.cc:606] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Received request to transfer leadership to TS b38ff4cb1ee546abba5eb00fe4cf50f2
I20260709 13:06:31.596994 19147 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "47e715af44cf464aa61c1f683f5c6b96"
dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
mode: GRACEFUL
new_leader_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
 from {username='slave'} at 127.0.0.1:38072
I20260709 13:06:31.597581 19147 raft_consensus.cc:606] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 LEADER]: Received request to transfer leadership to TS 0e552e70a13c4e4c817d78cdde2861cb
I20260709 13:06:31.611351 19147 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a6cf4b8385a54efd9d4a624eeecb5c80"
dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
mode: GRACEFUL
new_leader_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
 from {username='slave'} at 127.0.0.1:38072
I20260709 13:06:31.612228 19147 raft_consensus.cc:606] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 LEADER]: Received request to transfer leadership to TS 3e8380a11eb9479cae35d2f49f8591dc
I20260709 13:06:31.641803 19342 consensus_queue.cc:1048] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:31.649187 19360 raft_consensus.cc:993] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2: : Instructing follower 3e8380a11eb9479cae35d2f49f8591dc to start an election
I20260709 13:06:31.649806 19385 raft_consensus.cc:1081] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 LEADER]: Signalling peer 3e8380a11eb9479cae35d2f49f8591dc to start an election
I20260709 13:06:31.653623 19297 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a6cf4b8385a54efd9d4a624eeecb5c80"
dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
 from {username='slave'} at 127.0.0.1:41202
I20260709 13:06:31.654280 19297 raft_consensus.cc:493] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:31.654660 19297 raft_consensus.cc:3060] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:31.657068 19385 consensus_queue.cc:1048] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:31.662781 19297 raft_consensus.cc:515] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:31.666836 19297 leader_election.cc:290] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 2 election: Requested vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093)
I20260709 13:06:31.676076 19380 raft_consensus.cc:993] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb: : Instructing follower b38ff4cb1ee546abba5eb00fe4cf50f2 to start an election
I20260709 13:06:31.676600 19380 raft_consensus.cc:1081] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Signalling peer b38ff4cb1ee546abba5eb00fe4cf50f2 to start an election
I20260709 13:06:31.678381 19147 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a67334720ba743938c7b5af66099ddcb"
dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
 from {username='slave'} at 127.0.0.1:38070
I20260709 13:06:31.678988 19147 raft_consensus.cc:493] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:31.679375 19147 raft_consensus.cc:3060] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:31.687058 19147 raft_consensus.cc:515] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:31.689178 19146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6cf4b8385a54efd9d4a624eeecb5c80" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
I20260709 13:06:31.689905 19146 raft_consensus.cc:3055] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:31.690210 19146 raft_consensus.cc:740] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Running, Role: LEADER
I20260709 13:06:31.690670 19222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a67334720ba743938c7b5af66099ddcb" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
I20260709 13:06:31.691167 19147 leader_election.cc:290] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 2 election: Requested vote from peers 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:31.691340 19222 raft_consensus.cc:3055] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:31.690953 19146 consensus_queue.cc:260] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:31.691706 19222 raft_consensus.cc:740] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Running, Role: LEADER
I20260709 13:06:31.691833 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a67334720ba743938c7b5af66099ddcb" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
I20260709 13:06:31.692436 19297 raft_consensus.cc:3060] T a67334720ba743938c7b5af66099ddcb P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:31.692458 19146 raft_consensus.cc:3060] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:31.692577 19222 consensus_queue.cc:260] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:31.694128 19222 raft_consensus.cc:3060] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:31.696811 19223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6cf4b8385a54efd9d4a624eeecb5c80" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
I20260709 13:06:31.697463 19223 raft_consensus.cc:3060] T a6cf4b8385a54efd9d4a624eeecb5c80 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:31.700239 19297 raft_consensus.cc:2468] T a67334720ba743938c7b5af66099ddcb P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 2.
I20260709 13:06:31.701704 19114 leader_election.cc:304] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8380a11eb9479cae35d2f49f8591dc, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:31.702620 19342 raft_consensus.cc:2804] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:31.701689 19146 raft_consensus.cc:2468] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e8380a11eb9479cae35d2f49f8591dc in term 2.
I20260709 13:06:31.703228 19342 raft_consensus.cc:697] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 LEADER]: Becoming Leader. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Running, Role: LEADER
I20260709 13:06:31.703646 19222 raft_consensus.cc:2468] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 2.
I20260709 13:06:31.704134 19342 consensus_queue.cc:237] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:31.704015 19263 leader_election.cc:304] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8380a11eb9479cae35d2f49f8591dc, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:31.706049 19407 raft_consensus.cc:2804] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:31.706449 19223 raft_consensus.cc:2468] T a6cf4b8385a54efd9d4a624eeecb5c80 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e8380a11eb9479cae35d2f49f8591dc in term 2.
I20260709 13:06:31.706496 19407 raft_consensus.cc:697] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 LEADER]: Becoming Leader. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Running, Role: LEADER
I20260709 13:06:31.707644 19407 consensus_queue.cc:237] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:31.712955 19045 catalog_manager.cc:5697] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2 reported cstate change: term changed from 1 to 2, leader changed from 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2) to b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1). New cstate: current_term: 2 leader_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:31.713958 19043 catalog_manager.cc:5697] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc reported cstate change: term changed from 1 to 2, leader changed from b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1) to 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3). New cstate: current_term: 2 leader_uuid: "3e8380a11eb9479cae35d2f49f8591dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:31.854228 19360 raft_consensus.cc:993] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2: : Instructing follower 0e552e70a13c4e4c817d78cdde2861cb to start an election
I20260709 13:06:31.854794 19342 raft_consensus.cc:1081] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 LEADER]: Signalling peer 0e552e70a13c4e4c817d78cdde2861cb to start an election
I20260709 13:06:31.856559 19223 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "47e715af44cf464aa61c1f683f5c6b96"
dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
 from {username='slave'} at 127.0.0.1:52684
I20260709 13:06:31.857307 19223 raft_consensus.cc:493] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:31.857656 19223 raft_consensus.cc:3060] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:31.862962 19223 raft_consensus.cc:515] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:31.864634 19223 leader_election.cc:290] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 2 election: Requested vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:31.865603 19146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e715af44cf464aa61c1f683f5c6b96" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
I20260709 13:06:31.866272 19146 raft_consensus.cc:3055] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:31.866559 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e715af44cf464aa61c1f683f5c6b96" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
I20260709 13:06:31.866582 19146 raft_consensus.cc:740] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Running, Role: LEADER
I20260709 13:06:31.867187 19297 raft_consensus.cc:3060] T 47e715af44cf464aa61c1f683f5c6b96 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:31.867623 19146 consensus_queue.cc:260] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:31.868669 19146 raft_consensus.cc:3060] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:31.873769 19297 raft_consensus.cc:2468] T 47e715af44cf464aa61c1f683f5c6b96 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 2.
I20260709 13:06:31.874404 19146 raft_consensus.cc:2468] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 2.
I20260709 13:06:31.874984 19189 leader_election.cc:304] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, 3e8380a11eb9479cae35d2f49f8591dc; no voters: 
I20260709 13:06:31.875871 19380 raft_consensus.cc:2804] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:31.876549 19380 raft_consensus.cc:697] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 LEADER]: Becoming Leader. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Running, Role: LEADER
I20260709 13:06:31.877403 19380 consensus_queue.cc:237] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:31.884002 19043 catalog_manager.cc:5697] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb reported cstate change: term changed from 1 to 2, leader changed from b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1) to 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2). New cstate: current_term: 2 leader_uuid: "0e552e70a13c4e4c817d78cdde2861cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:32.088476 19223 raft_consensus.cc:1275] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Refusing update from remote peer b38ff4cb1ee546abba5eb00fe4cf50f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:32.090158 19342 consensus_queue.cc:1048] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:32.099809 19296 raft_consensus.cc:1275] T a67334720ba743938c7b5af66099ddcb P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Refusing update from remote peer b38ff4cb1ee546abba5eb00fe4cf50f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:32.102296 19342 consensus_queue.cc:1048] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:32.189496 19223 raft_consensus.cc:1275] T a6cf4b8385a54efd9d4a624eeecb5c80 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Refusing update from remote peer 3e8380a11eb9479cae35d2f49f8591dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:32.191154 19407 consensus_queue.cc:1048] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:32.200796 19146 raft_consensus.cc:1275] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Refusing update from remote peer 3e8380a11eb9479cae35d2f49f8591dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:32.203418 19341 consensus_queue.cc:1048] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:32.310539 19296 raft_consensus.cc:1275] T 47e715af44cf464aa61c1f683f5c6b96 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Refusing update from remote peer 0e552e70a13c4e4c817d78cdde2861cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:32.312011 19380 consensus_queue.cc:1048] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:32.320960 19146 raft_consensus.cc:1275] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Refusing update from remote peer 0e552e70a13c4e4c817d78cdde2861cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:32.323729 19378 consensus_queue.cc:1048] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:34.644941 19043 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33592:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:34.647286 19043 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:34.699669 19137 tablet_service.cc:1511] Processing CreateTablet for tablet f75151a1c3d546d88baf9b1b511c2407 (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:34.701004 19137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f75151a1c3d546d88baf9b1b511c2407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.701033 19210 tablet_service.cc:1511] Processing CreateTablet for tablet f75151a1c3d546d88baf9b1b511c2407 (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:34.702656 19210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f75151a1c3d546d88baf9b1b511c2407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.702862 19281 tablet_service.cc:1511] Processing CreateTablet for tablet f75151a1c3d546d88baf9b1b511c2407 (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:34.704090 19281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f75151a1c3d546d88baf9b1b511c2407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.708117 19207 tablet_service.cc:1511] Processing CreateTablet for tablet 535ad37217c24424ac08ebe1cf5a5252 (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:34.708539 19280 tablet_service.cc:1511] Processing CreateTablet for tablet 535ad37217c24424ac08ebe1cf5a5252 (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:34.709059 19138 tablet_service.cc:1511] Processing CreateTablet for tablet 535ad37217c24424ac08ebe1cf5a5252 (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:34.709585 19207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 535ad37217c24424ac08ebe1cf5a5252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.709887 19280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 535ad37217c24424ac08ebe1cf5a5252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.710318 19138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 535ad37217c24424ac08ebe1cf5a5252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.710261 19212 tablet_service.cc:1511] Processing CreateTablet for tablet 50f3b5b62c7f4e89a139a79d44b47496 (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:34.711680 19212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50f3b5b62c7f4e89a139a79d44b47496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.712713 19135 tablet_service.cc:1511] Processing CreateTablet for tablet 50f3b5b62c7f4e89a139a79d44b47496 (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:34.713996 19135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50f3b5b62c7f4e89a139a79d44b47496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.729954 19211 tablet_service.cc:1511] Processing CreateTablet for tablet f37bdae5606e44e588830063efe2e301 (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:34.731458 19211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f37bdae5606e44e588830063efe2e301. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.734911 19282 tablet_service.cc:1511] Processing CreateTablet for tablet 50f3b5b62c7f4e89a139a79d44b47496 (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:34.736284 19282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50f3b5b62c7f4e89a139a79d44b47496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.738777 19208 tablet_service.cc:1511] Processing CreateTablet for tablet 04abaf75391342018eef037f9e2f92db (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:34.740118 19208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04abaf75391342018eef037f9e2f92db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.742941 19287 tablet_service.cc:1511] Processing CreateTablet for tablet f37bdae5606e44e588830063efe2e301 (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:34.744269 19287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f37bdae5606e44e588830063efe2e301. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.748564 19134 tablet_service.cc:1511] Processing CreateTablet for tablet f37bdae5606e44e588830063efe2e301 (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:34.749996 19134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f37bdae5606e44e588830063efe2e301. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.755336 19136 tablet_service.cc:1511] Processing CreateTablet for tablet 04abaf75391342018eef037f9e2f92db (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:34.757042 19136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04abaf75391342018eef037f9e2f92db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.757874 19209 tablet_service.cc:1511] Processing CreateTablet for tablet eb0e2bad45494ca38c6c9107a7656899 (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:34.759238 19209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb0e2bad45494ca38c6c9107a7656899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.762768 19285 tablet_service.cc:1511] Processing CreateTablet for tablet 04abaf75391342018eef037f9e2f92db (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:34.764149 19285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04abaf75391342018eef037f9e2f92db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.767308 19133 tablet_service.cc:1511] Processing CreateTablet for tablet eb0e2bad45494ca38c6c9107a7656899 (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:34.768664 19133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb0e2bad45494ca38c6c9107a7656899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.773128 19213 tablet_service.cc:1511] Processing CreateTablet for tablet 2fa0f701a2794dee89d3afba337c51fa (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:34.774573 19213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fa0f701a2794dee89d3afba337c51fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.778811 19282 tablet_service.cc:1511] Processing CreateTablet for tablet eb0e2bad45494ca38c6c9107a7656899 (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:34.781389 19132 tablet_service.cc:1511] Processing CreateTablet for tablet 2fa0f701a2794dee89d3afba337c51fa (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:34.781172 19131 tablet_service.cc:1511] Processing CreateTablet for tablet 5e600380d13a47f38e3369753d2ce794 (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:34.782799 19132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fa0f701a2794dee89d3afba337c51fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.786599 19438 tablet_bootstrap.cc:492] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc: Bootstrap starting.
I20260709 13:06:34.789191 19131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e600380d13a47f38e3369753d2ce794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.798584 19282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb0e2bad45494ca38c6c9107a7656899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.804283 19206 tablet_service.cc:1511] Processing CreateTablet for tablet 5e600380d13a47f38e3369753d2ce794 (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:34.806820 19206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e600380d13a47f38e3369753d2ce794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.806421 19205 tablet_service.cc:1511] Processing CreateTablet for tablet 38f843895e814f8a9ed5c4b4e8978ac4 (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:34.808454 19205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38f843895e814f8a9ed5c4b4e8978ac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.806419 19281 tablet_service.cc:1511] Processing CreateTablet for tablet 38f843895e814f8a9ed5c4b4e8978ac4 (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:34.814339 19281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38f843895e814f8a9ed5c4b4e8978ac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.818497 19280 tablet_service.cc:1511] Processing CreateTablet for tablet 2fa0f701a2794dee89d3afba337c51fa (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:34.819937 19280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fa0f701a2794dee89d3afba337c51fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.839404 19287 tablet_service.cc:1511] Processing CreateTablet for tablet 5e600380d13a47f38e3369753d2ce794 (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:34.841005 19287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e600380d13a47f38e3369753d2ce794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.866083 19438 tablet_bootstrap.cc:654] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:34.885633 19441 tablet_bootstrap.cc:492] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb: Bootstrap starting.
W20260709 13:06:34.883450 19114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.3:43973, user_credentials={real_user=slave}} blocked reactor thread for 83812us
I20260709 13:06:34.893996 19138 tablet_service.cc:1511] Processing CreateTablet for tablet 38f843895e814f8a9ed5c4b4e8978ac4 (DEFAULT_TABLE table=soft_deleted_table [id=46a235393ae445be9239216af30cc03a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:34.896982 19138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38f843895e814f8a9ed5c4b4e8978ac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.900694 19439 tablet_bootstrap.cc:492] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2: Bootstrap starting.
I20260709 13:06:34.892913 19441 tablet_bootstrap.cc:654] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:34.922061 19441 tablet_bootstrap.cc:492] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb: No bootstrap required, opened a new log
I20260709 13:06:34.922587 19441 ts_tablet_manager.cc:1403] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb: Time spent bootstrapping tablet: real 0.037s	user 0.007s	sys 0.007s
I20260709 13:06:34.923100 19438 tablet_bootstrap.cc:492] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc: No bootstrap required, opened a new log
I20260709 13:06:34.923729 19438 ts_tablet_manager.cc:1403] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc: Time spent bootstrapping tablet: real 0.137s	user 0.011s	sys 0.022s
I20260709 13:06:34.926699 19441 raft_consensus.cc:359] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:34.927560 19441 raft_consensus.cc:385] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:34.927927 19441 raft_consensus.cc:740] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Initialized, Role: FOLLOWER
I20260709 13:06:34.928715 19441 consensus_queue.cc:260] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:34.933133 19441 ts_tablet_manager.cc:1434] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:06:34.926829 19438 raft_consensus.cc:359] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:34.933877 19438 raft_consensus.cc:385] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:34.934303 19438 raft_consensus.cc:740] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Initialized, Role: FOLLOWER
I20260709 13:06:34.935225 19441 tablet_bootstrap.cc:492] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb: Bootstrap starting.
I20260709 13:06:34.935595 19438 consensus_queue.cc:260] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:34.940241 19438 ts_tablet_manager.cc:1434] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.003s
I20260709 13:06:34.942119 19438 tablet_bootstrap.cc:492] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc: Bootstrap starting.
I20260709 13:06:34.946862 19441 tablet_bootstrap.cc:654] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:34.935443 19439 tablet_bootstrap.cc:654] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:34.960707 19438 tablet_bootstrap.cc:654] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:34.964243 19441 tablet_bootstrap.cc:492] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb: No bootstrap required, opened a new log
I20260709 13:06:34.964996 19441 ts_tablet_manager.cc:1403] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.005s
I20260709 13:06:34.969508 19441 raft_consensus.cc:359] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:34.970571 19441 raft_consensus.cc:385] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:34.971004 19441 raft_consensus.cc:740] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Initialized, Role: FOLLOWER
I20260709 13:06:34.972077 19441 consensus_queue.cc:260] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:34.974028 19438 tablet_bootstrap.cc:492] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc: No bootstrap required, opened a new log
I20260709 13:06:34.974705 19438 ts_tablet_manager.cc:1403] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.009s
I20260709 13:06:34.975800 19441 ts_tablet_manager.cc:1434] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.003s
I20260709 13:06:34.976994 19441 tablet_bootstrap.cc:492] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb: Bootstrap starting.
I20260709 13:06:34.977387 19439 tablet_bootstrap.cc:492] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2: No bootstrap required, opened a new log
I20260709 13:06:34.978025 19439 ts_tablet_manager.cc:1403] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent bootstrapping tablet: real 0.078s	user 0.013s	sys 0.010s
I20260709 13:06:34.978156 19438 raft_consensus.cc:359] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:34.979069 19438 raft_consensus.cc:385] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:34.979441 19438 raft_consensus.cc:740] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Initialized, Role: FOLLOWER
I20260709 13:06:34.980263 19438 consensus_queue.cc:260] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:34.982668 19438 ts_tablet_manager.cc:1434] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.006s
I20260709 13:06:34.983664 19438 tablet_bootstrap.cc:492] T 50f3b5b62c7f4e89a139a79d44b47496 P 3e8380a11eb9479cae35d2f49f8591dc: Bootstrap starting.
I20260709 13:06:34.985340 19441 tablet_bootstrap.cc:654] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:34.991999 19438 tablet_bootstrap.cc:654] T 50f3b5b62c7f4e89a139a79d44b47496 P 3e8380a11eb9479cae35d2f49f8591dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:34.995985 19439 raft_consensus.cc:359] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:34.996963 19439 raft_consensus.cc:385] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:34.997300 19439 raft_consensus.cc:740] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Initialized, Role: FOLLOWER
I20260709 13:06:34.997596 19438 tablet_bootstrap.cc:492] T 50f3b5b62c7f4e89a139a79d44b47496 P 3e8380a11eb9479cae35d2f49f8591dc: No bootstrap required, opened a new log
I20260709 13:06:34.998160 19439 consensus_queue.cc:260] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:34.999507 19438 ts_tablet_manager.cc:1403] T 50f3b5b62c7f4e89a139a79d44b47496 P 3e8380a11eb9479cae35d2f49f8591dc: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.000s
I20260709 13:06:35.001300 19439 ts_tablet_manager.cc:1434] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.001s
I20260709 13:06:35.002569 19439 tablet_bootstrap.cc:492] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2: Bootstrap starting.
I20260709 13:06:35.004446 19438 raft_consensus.cc:359] T 50f3b5b62c7f4e89a139a79d44b47496 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.005204 19438 raft_consensus.cc:385] T 50f3b5b62c7f4e89a139a79d44b47496 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.005602 19438 raft_consensus.cc:740] T 50f3b5b62c7f4e89a139a79d44b47496 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Initialized, Role: FOLLOWER
I20260709 13:06:35.006479 19438 consensus_queue.cc:260] T 50f3b5b62c7f4e89a139a79d44b47496 P 3e8380a11eb9479cae35d2f49f8591dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.008455 19441 tablet_bootstrap.cc:492] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb: No bootstrap required, opened a new log
I20260709 13:06:35.009112 19438 ts_tablet_manager.cc:1434] T 50f3b5b62c7f4e89a139a79d44b47496 P 3e8380a11eb9479cae35d2f49f8591dc: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.002s
I20260709 13:06:35.009136 19441 ts_tablet_manager.cc:1403] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb: Time spent bootstrapping tablet: real 0.032s	user 0.006s	sys 0.015s
I20260709 13:06:35.009660 19439 tablet_bootstrap.cc:654] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:35.010668 19438 tablet_bootstrap.cc:492] T f37bdae5606e44e588830063efe2e301 P 3e8380a11eb9479cae35d2f49f8591dc: Bootstrap starting.
I20260709 13:06:35.015930 19441 raft_consensus.cc:359] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.016760 19441 raft_consensus.cc:385] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.017094 19441 raft_consensus.cc:740] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Initialized, Role: FOLLOWER
I20260709 13:06:35.017946 19441 consensus_queue.cc:260] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.020795 19441 ts_tablet_manager.cc:1434] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.003s
I20260709 13:06:35.021029 19438 tablet_bootstrap.cc:654] T f37bdae5606e44e588830063efe2e301 P 3e8380a11eb9479cae35d2f49f8591dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:35.022020 19441 tablet_bootstrap.cc:492] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb: Bootstrap starting.
I20260709 13:06:35.026553 19427 raft_consensus.cc:493] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.027298 19427 raft_consensus.cc:515] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.027900 19438 tablet_bootstrap.cc:492] T f37bdae5606e44e588830063efe2e301 P 3e8380a11eb9479cae35d2f49f8591dc: No bootstrap required, opened a new log
I20260709 13:06:35.028522 19438 ts_tablet_manager.cc:1403] T f37bdae5606e44e588830063efe2e301 P 3e8380a11eb9479cae35d2f49f8591dc: Time spent bootstrapping tablet: real 0.018s	user 0.005s	sys 0.007s
I20260709 13:06:35.029721 19441 tablet_bootstrap.cc:654] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:35.031962 19438 raft_consensus.cc:359] T f37bdae5606e44e588830063efe2e301 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.032820 19438 raft_consensus.cc:385] T f37bdae5606e44e588830063efe2e301 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.033208 19438 raft_consensus.cc:740] T f37bdae5606e44e588830063efe2e301 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Initialized, Role: FOLLOWER
I20260709 13:06:35.034584 19438 consensus_queue.cc:260] T f37bdae5606e44e588830063efe2e301 P 3e8380a11eb9479cae35d2f49f8591dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.037357 19296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb0e2bad45494ca38c6c9107a7656899" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc" is_pre_election: true
I20260709 13:06:35.037715 19438 ts_tablet_manager.cc:1434] T f37bdae5606e44e588830063efe2e301 P 3e8380a11eb9479cae35d2f49f8591dc: Time spent starting tablet: real 0.009s	user 0.011s	sys 0.000s
I20260709 13:06:35.039032 19438 tablet_bootstrap.cc:492] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc: Bootstrap starting.
W20260709 13:06:35.039541 19189 leader_election.cc:343] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:35.033103 19146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb0e2bad45494ca38c6c9107a7656899" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" is_pre_election: true
I20260709 13:06:35.035351 19427 leader_election.cc:290] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:35.066704 19439 tablet_bootstrap.cc:492] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2: No bootstrap required, opened a new log
I20260709 13:06:35.067344 19439 ts_tablet_manager.cc:1403] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent bootstrapping tablet: real 0.065s	user 0.022s	sys 0.027s
I20260709 13:06:35.067437 19441 tablet_bootstrap.cc:492] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb: No bootstrap required, opened a new log
I20260709 13:06:35.068064 19441 ts_tablet_manager.cc:1403] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb: Time spent bootstrapping tablet: real 0.046s	user 0.005s	sys 0.028s
I20260709 13:06:35.071904 19441 raft_consensus.cc:359] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.072738 19441 raft_consensus.cc:385] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.073078 19441 raft_consensus.cc:740] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Initialized, Role: FOLLOWER
W20260709 13:06:35.043215 19189 leader_election.cc:343] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:35.046538 19438 tablet_bootstrap.cc:654] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:35.073964 19441 consensus_queue.cc:260] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.074678 19439 raft_consensus.cc:359] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.075486 19439 raft_consensus.cc:385] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.075755 19439 raft_consensus.cc:740] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Initialized, Role: FOLLOWER
I20260709 13:06:35.076424 19439 consensus_queue.cc:260] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.076715 19441 ts_tablet_manager.cc:1434] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:06:35.077981 19441 tablet_bootstrap.cc:492] T 50f3b5b62c7f4e89a139a79d44b47496 P 0e552e70a13c4e4c817d78cdde2861cb: Bootstrap starting.
I20260709 13:06:35.077777 19189 leader_election.cc:304] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb; no voters: 3e8380a11eb9479cae35d2f49f8591dc, b38ff4cb1ee546abba5eb00fe4cf50f2
I20260709 13:06:35.078698 19427 raft_consensus.cc:2749] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:35.084939 19440 raft_consensus.cc:493] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.085587 19440 raft_consensus.cc:515] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.086201 19441 tablet_bootstrap.cc:654] T 50f3b5b62c7f4e89a139a79d44b47496 P 0e552e70a13c4e4c817d78cdde2861cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:35.086671 19439 ts_tablet_manager.cc:1434] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.003s
I20260709 13:06:35.088143 19440 leader_election.cc:290] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:35.088959 19427 raft_consensus.cc:493] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.089612 19427 raft_consensus.cc:515] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.091895 19427 leader_election.cc:290] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:35.094678 19223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50f3b5b62c7f4e89a139a79d44b47496" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb" is_pre_election: true
I20260709 13:06:35.096186 19439 tablet_bootstrap.cc:492] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2: Bootstrap starting.
I20260709 13:06:35.096966 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50f3b5b62c7f4e89a139a79d44b47496" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc" is_pre_election: true
I20260709 13:06:35.097843 19297 raft_consensus.cc:2468] T 50f3b5b62c7f4e89a139a79d44b47496 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 0.
I20260709 13:06:35.101999 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f37bdae5606e44e588830063efe2e301" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc" is_pre_election: true
I20260709 13:06:35.102767 19297 raft_consensus.cc:2468] T f37bdae5606e44e588830063efe2e301 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 0.
I20260709 13:06:35.104725 19439 tablet_bootstrap.cc:654] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:35.104940 19114 leader_election.cc:304] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8380a11eb9479cae35d2f49f8591dc, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:35.108023 19425 raft_consensus.cc:493] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.108945 19426 raft_consensus.cc:2804] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:35.109386 19426 raft_consensus.cc:493] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.109769 19426 raft_consensus.cc:3060] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:35.114073 19114 leader_election.cc:343] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:35.108616 19425 raft_consensus.cc:515] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.105971 19146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f37bdae5606e44e588830063efe2e301" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" is_pre_election: true
I20260709 13:06:35.107034 19189 leader_election.cc:304] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, 3e8380a11eb9479cae35d2f49f8591dc; no voters: 
I20260709 13:06:35.116660 19425 leader_election.cc:290] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:35.117290 19425 raft_consensus.cc:2804] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:35.117717 19425 raft_consensus.cc:493] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.118093 19425 raft_consensus.cc:3060] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:35.122057 19189 leader_election.cc:343] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:35.124075 19296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fa0f701a2794dee89d3afba337c51fa" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc" is_pre_election: true
I20260709 13:06:35.128330 19146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fa0f701a2794dee89d3afba337c51fa" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" is_pre_election: true
W20260709 13:06:35.130162 19189 leader_election.cc:343] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:35.130970 19189 leader_election.cc:343] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:35.135758 19189 leader_election.cc:304] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb; no voters: 3e8380a11eb9479cae35d2f49f8591dc, b38ff4cb1ee546abba5eb00fe4cf50f2
I20260709 13:06:35.127332 19425 raft_consensus.cc:515] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.138995 19425 leader_election.cc:290] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 election: Requested vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:35.139786 19427 raft_consensus.cc:2749] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:35.142393 19146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f37bdae5606e44e588830063efe2e301" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
I20260709 13:06:35.144774 19296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f37bdae5606e44e588830063efe2e301" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
I20260709 13:06:35.148173 19296 raft_consensus.cc:3060] T f37bdae5606e44e588830063efe2e301 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:35.160466 19189 leader_election.cc:343] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:35.165058 19427 raft_consensus.cc:493] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.166234 19427 raft_consensus.cc:515] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.173933 19423 raft_consensus.cc:493] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.134009 19426 raft_consensus.cc:515] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.174706 19423 raft_consensus.cc:515] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.177565 19426 leader_election.cc:290] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 election: Requested vote from peers 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:35.178802 19146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "535ad37217c24424ac08ebe1cf5a5252" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" is_pre_election: true
I20260709 13:06:35.186780 19147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "535ad37217c24424ac08ebe1cf5a5252" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" is_pre_election: true
I20260709 13:06:35.187178 19146 raft_consensus.cc:2468] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e8380a11eb9479cae35d2f49f8591dc in term 0.
I20260709 13:06:35.179585 19223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50f3b5b62c7f4e89a139a79d44b47496" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
I20260709 13:06:35.181258 19423 leader_election.cc:290] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093)
I20260709 13:06:35.181517 19222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "535ad37217c24424ac08ebe1cf5a5252" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb" is_pre_election: true
I20260709 13:06:35.169004 19427 leader_election.cc:290] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:35.181375 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50f3b5b62c7f4e89a139a79d44b47496" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
I20260709 13:06:35.188975 19222 raft_consensus.cc:2468] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e8380a11eb9479cae35d2f49f8591dc in term 0.
W20260709 13:06:35.189314 19114 leader_election.cc:343] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:35.185405 19438 tablet_bootstrap.cc:492] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc: No bootstrap required, opened a new log
I20260709 13:06:35.189405 19297 raft_consensus.cc:3060] T 50f3b5b62c7f4e89a139a79d44b47496 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.190558 19438 ts_tablet_manager.cc:1403] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc: Time spent bootstrapping tablet: real 0.152s	user 0.044s	sys 0.067s
I20260709 13:06:35.186874 19295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "535ad37217c24424ac08ebe1cf5a5252" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc" is_pre_election: true
I20260709 13:06:35.191365 19295 raft_consensus.cc:2468] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 0.
I20260709 13:06:35.190366 19263 leader_election.cc:304] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, 3e8380a11eb9479cae35d2f49f8591dc; no voters: 
I20260709 13:06:35.193992 19189 leader_election.cc:304] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, 3e8380a11eb9479cae35d2f49f8591dc; no voters: b38ff4cb1ee546abba5eb00fe4cf50f2
I20260709 13:06:35.194769 19439 tablet_bootstrap.cc:492] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2: No bootstrap required, opened a new log
I20260709 13:06:35.196611 19441 tablet_bootstrap.cc:492] T 50f3b5b62c7f4e89a139a79d44b47496 P 0e552e70a13c4e4c817d78cdde2861cb: No bootstrap required, opened a new log
I20260709 13:06:35.194958 19427 raft_consensus.cc:2804] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:35.194808 19423 raft_consensus.cc:2804] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:35.188930 19296 raft_consensus.cc:2468] T f37bdae5606e44e588830063efe2e301 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 1.
I20260709 13:06:35.197700 19423 raft_consensus.cc:493] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.197418 19427 raft_consensus.cc:493] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.198652 19427 raft_consensus.cc:3060] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.198836 19189 leader_election.cc:304] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, 3e8380a11eb9479cae35d2f49f8591dc; no voters: b38ff4cb1ee546abba5eb00fe4cf50f2
I20260709 13:06:35.198005 19441 ts_tablet_manager.cc:1403] T 50f3b5b62c7f4e89a139a79d44b47496 P 0e552e70a13c4e4c817d78cdde2861cb: Time spent bootstrapping tablet: real 0.120s	user 0.029s	sys 0.067s
I20260709 13:06:35.200124 19425 raft_consensus.cc:2804] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:35.197353 19438 raft_consensus.cc:359] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.200628 19425 raft_consensus.cc:697] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Becoming Leader. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Running, Role: LEADER
I20260709 13:06:35.198865 19423 raft_consensus.cc:3060] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.201534 19425 consensus_queue.cc:237] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.200987 19438 raft_consensus.cc:385] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.202632 19438 raft_consensus.cc:740] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Initialized, Role: FOLLOWER
I20260709 13:06:35.203553 19438 consensus_queue.cc:260] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.197979 19439 ts_tablet_manager.cc:1403] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent bootstrapping tablet: real 0.102s	user 0.032s	sys 0.028s
I20260709 13:06:35.207486 19427 raft_consensus.cc:515] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.205992 19441 raft_consensus.cc:359] T 50f3b5b62c7f4e89a139a79d44b47496 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.211787 19441 raft_consensus.cc:385] T 50f3b5b62c7f4e89a139a79d44b47496 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.212083 19427 leader_election.cc:290] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 election: Requested vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:35.212247 19441 raft_consensus.cc:740] T 50f3b5b62c7f4e89a139a79d44b47496 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Initialized, Role: FOLLOWER
I20260709 13:06:35.213217 19295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "535ad37217c24424ac08ebe1cf5a5252" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
I20260709 13:06:35.213296 19441 consensus_queue.cc:260] T 50f3b5b62c7f4e89a139a79d44b47496 P 0e552e70a13c4e4c817d78cdde2861cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.217263 19441 ts_tablet_manager.cc:1434] T 50f3b5b62c7f4e89a139a79d44b47496 P 0e552e70a13c4e4c817d78cdde2861cb: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 13:06:35.219213 19441 tablet_bootstrap.cc:492] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb: Bootstrap starting.
I20260709 13:06:35.220223 19439 raft_consensus.cc:359] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.219229 19146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "535ad37217c24424ac08ebe1cf5a5252" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
I20260709 13:06:35.221478 19146 raft_consensus.cc:3060] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.221036 19439 raft_consensus.cc:385] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.221980 19439 raft_consensus.cc:740] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Initialized, Role: FOLLOWER
I20260709 13:06:35.222770 19439 consensus_queue.cc:260] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.225135 19439 ts_tablet_manager.cc:1434] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.002s
I20260709 13:06:35.226629 19439 tablet_bootstrap.cc:492] T 04abaf75391342018eef037f9e2f92db P b38ff4cb1ee546abba5eb00fe4cf50f2: Bootstrap starting.
I20260709 13:06:35.212203 19423 raft_consensus.cc:515] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.229547 19295 raft_consensus.cc:2393] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e552e70a13c4e4c817d78cdde2861cb in current term 1: Already voted for candidate 3e8380a11eb9479cae35d2f49f8591dc in this term.
I20260709 13:06:35.217237 19297 raft_consensus.cc:2468] T 50f3b5b62c7f4e89a139a79d44b47496 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 1.
I20260709 13:06:35.234532 19114 leader_election.cc:304] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e8380a11eb9479cae35d2f49f8591dc, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 0e552e70a13c4e4c817d78cdde2861cb
I20260709 13:06:35.235666 19426 raft_consensus.cc:2804] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:35.237697 19146 raft_consensus.cc:2468] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 1.
I20260709 13:06:35.238907 19189 leader_election.cc:304] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:35.239820 19427 raft_consensus.cc:2804] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:35.240352 19427 raft_consensus.cc:697] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Becoming Leader. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Running, Role: LEADER
I20260709 13:06:35.241345 19427 consensus_queue.cc:237] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.243098 19441 tablet_bootstrap.cc:654] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:35.244482 19439 tablet_bootstrap.cc:654] T 04abaf75391342018eef037f9e2f92db P b38ff4cb1ee546abba5eb00fe4cf50f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:35.245738 19457 raft_consensus.cc:493] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.246388 19457 raft_consensus.cc:515] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.248049 19438 ts_tablet_manager.cc:1434] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc: Time spent starting tablet: real 0.057s	user 0.020s	sys 0.010s
I20260709 13:06:35.248904 19457 leader_election.cc:290] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093)
I20260709 13:06:35.235713 19423 leader_election.cc:290] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 1 election: Requested vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093)
I20260709 13:06:35.255275 19426 raft_consensus.cc:697] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 LEADER]: Becoming Leader. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Running, Role: LEADER
I20260709 13:06:35.262112 19438 tablet_bootstrap.cc:492] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc: Bootstrap starting.
I20260709 13:06:35.256942 19426 consensus_queue.cc:237] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.268301 19043 catalog_manager.cc:5697] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2). New cstate: current_term: 1 leader_uuid: "0e552e70a13c4e4c817d78cdde2861cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:35.274469 19146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "535ad37217c24424ac08ebe1cf5a5252" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
I20260709 13:06:35.275641 19146 raft_consensus.cc:2393] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e8380a11eb9479cae35d2f49f8591dc in current term 1: Already voted for candidate 0e552e70a13c4e4c817d78cdde2861cb in this term.
I20260709 13:06:35.279529 19222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "535ad37217c24424ac08ebe1cf5a5252" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
I20260709 13:06:35.283622 19146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f75151a1c3d546d88baf9b1b511c2407" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" is_pre_election: true
I20260709 13:06:35.284103 19438 tablet_bootstrap.cc:654] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:35.284597 19146 raft_consensus.cc:2468] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e8380a11eb9479cae35d2f49f8591dc in term 0.
I20260709 13:06:35.284405 19223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f75151a1c3d546d88baf9b1b511c2407" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb" is_pre_election: true
I20260709 13:06:35.286782 19263 leader_election.cc:304] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e8380a11eb9479cae35d2f49f8591dc; no voters: 0e552e70a13c4e4c817d78cdde2861cb, b38ff4cb1ee546abba5eb00fe4cf50f2
I20260709 13:06:35.287914 19458 raft_consensus.cc:2749] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:06:35.288637 19263 leader_election.cc:343] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:35.289260 19441 tablet_bootstrap.cc:492] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb: No bootstrap required, opened a new log
I20260709 13:06:35.290416 19441 ts_tablet_manager.cc:1403] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb: Time spent bootstrapping tablet: real 0.072s	user 0.024s	sys 0.017s
I20260709 13:06:35.293944 19441 raft_consensus.cc:359] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.294018 19263 leader_election.cc:304] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e8380a11eb9479cae35d2f49f8591dc, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 0e552e70a13c4e4c817d78cdde2861cb
I20260709 13:06:35.295895 19458 raft_consensus.cc:2804] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:35.296118 19441 raft_consensus.cc:385] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.296238 19458 raft_consensus.cc:493] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.296708 19441 raft_consensus.cc:740] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Initialized, Role: FOLLOWER
I20260709 13:06:35.296830 19458 raft_consensus.cc:3060] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.297024 19439 tablet_bootstrap.cc:492] T 04abaf75391342018eef037f9e2f92db P b38ff4cb1ee546abba5eb00fe4cf50f2: No bootstrap required, opened a new log
I20260709 13:06:35.297859 19439 ts_tablet_manager.cc:1403] T 04abaf75391342018eef037f9e2f92db P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent bootstrapping tablet: real 0.072s	user 0.030s	sys 0.016s
I20260709 13:06:35.297669 19441 consensus_queue.cc:260] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.301187 19441 ts_tablet_manager.cc:1434] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.002s
I20260709 13:06:35.301396 19046 catalog_manager.cc:5697] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:35.302333 19441 tablet_bootstrap.cc:492] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb: Bootstrap starting.
I20260709 13:06:35.301416 19439 raft_consensus.cc:359] T 04abaf75391342018eef037f9e2f92db P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.305373 19439 raft_consensus.cc:385] T 04abaf75391342018eef037f9e2f92db P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.305759 19439 raft_consensus.cc:740] T 04abaf75391342018eef037f9e2f92db P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Initialized, Role: FOLLOWER
I20260709 13:06:35.307834 19439 consensus_queue.cc:260] T 04abaf75391342018eef037f9e2f92db P b38ff4cb1ee546abba5eb00fe4cf50f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.309791 19439 ts_tablet_manager.cc:1434] T 04abaf75391342018eef037f9e2f92db P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:06:35.310886 19439 tablet_bootstrap.cc:492] T 5e600380d13a47f38e3369753d2ce794 P b38ff4cb1ee546abba5eb00fe4cf50f2: Bootstrap starting.
I20260709 13:06:35.312140 19440 raft_consensus.cc:493] T 04abaf75391342018eef037f9e2f92db P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.312918 19440 raft_consensus.cc:515] T 04abaf75391342018eef037f9e2f92db P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.316782 19440 leader_election.cc:290] T 04abaf75391342018eef037f9e2f92db P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:35.316726 19223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04abaf75391342018eef037f9e2f92db" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb" is_pre_election: true
I20260709 13:06:35.318436 19441 tablet_bootstrap.cc:654] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:35.318734 19114 leader_election.cc:343] T 04abaf75391342018eef037f9e2f92db P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:35.318498 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04abaf75391342018eef037f9e2f92db" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc" is_pre_election: true
W20260709 13:06:35.320297 19114 leader_election.cc:343] T 04abaf75391342018eef037f9e2f92db P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:35.320739 19114 leader_election.cc:304] T 04abaf75391342018eef037f9e2f92db P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 0e552e70a13c4e4c817d78cdde2861cb, 3e8380a11eb9479cae35d2f49f8591dc
I20260709 13:06:35.321986 19426 raft_consensus.cc:2749] T 04abaf75391342018eef037f9e2f92db P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:35.322136 19458 raft_consensus.cc:515] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.324717 19458 leader_election.cc:290] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 1 election: Requested vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093)
I20260709 13:06:35.325875 19146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f75151a1c3d546d88baf9b1b511c2407" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
I20260709 13:06:35.326115 19223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f75151a1c3d546d88baf9b1b511c2407" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
I20260709 13:06:35.326538 19146 raft_consensus.cc:3060] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.327656 19439 tablet_bootstrap.cc:654] T 5e600380d13a47f38e3369753d2ce794 P b38ff4cb1ee546abba5eb00fe4cf50f2: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:35.327791 19263 leader_election.cc:343] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:35.328732 19458 raft_consensus.cc:493] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.329238 19458 raft_consensus.cc:515] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.332022 19147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb0e2bad45494ca38c6c9107a7656899" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" is_pre_election: true
W20260709 13:06:35.333173 19263 leader_election.cc:343] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:35.335511 19438 tablet_bootstrap.cc:492] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc: No bootstrap required, opened a new log
I20260709 13:06:35.335213 19146 raft_consensus.cc:2468] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e8380a11eb9479cae35d2f49f8591dc in term 1.
I20260709 13:06:35.335160 19441 tablet_bootstrap.cc:492] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb: No bootstrap required, opened a new log
I20260709 13:06:35.336614 19263 leader_election.cc:304] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e8380a11eb9479cae35d2f49f8591dc, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 0e552e70a13c4e4c817d78cdde2861cb
I20260709 13:06:35.337385 19423 raft_consensus.cc:2804] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:35.336084 19438 ts_tablet_manager.cc:1403] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc: Time spent bootstrapping tablet: real 0.074s	user 0.006s	sys 0.008s
I20260709 13:06:35.337921 19423 raft_consensus.cc:697] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 LEADER]: Becoming Leader. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Running, Role: LEADER
I20260709 13:06:35.338829 19423 consensus_queue.cc:237] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.336742 19441 ts_tablet_manager.cc:1403] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.009s
I20260709 13:06:35.340961 19438 raft_consensus.cc:359] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.341733 19438 raft_consensus.cc:385] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.342249 19438 raft_consensus.cc:740] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Initialized, Role: FOLLOWER
I20260709 13:06:35.342976 19438 consensus_queue.cc:260] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.345417 19223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb0e2bad45494ca38c6c9107a7656899" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb" is_pre_election: true
I20260709 13:06:35.346168 19223 raft_consensus.cc:2468] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e8380a11eb9479cae35d2f49f8591dc in term 0.
I20260709 13:06:35.347433 19263 leader_election.cc:304] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, 3e8380a11eb9479cae35d2f49f8591dc; no voters: b38ff4cb1ee546abba5eb00fe4cf50f2
I20260709 13:06:35.348163 19458 leader_election.cc:290] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093)
I20260709 13:06:35.347499 19441 raft_consensus.cc:359] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.349215 19441 raft_consensus.cc:385] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.349861 19457 raft_consensus.cc:2804] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:35.352880 19457 raft_consensus.cc:493] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.353302 19457 raft_consensus.cc:3060] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.360602 19441 raft_consensus.cc:740] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Initialized, Role: FOLLOWER
I20260709 13:06:35.352455 19438 ts_tablet_manager.cc:1434] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.002s
I20260709 13:06:35.361544 19441 consensus_queue.cc:260] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.363109 19438 tablet_bootstrap.cc:492] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc: Bootstrap starting.
I20260709 13:06:35.363955 19441 ts_tablet_manager.cc:1434] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:06:35.365996 19441 tablet_bootstrap.cc:492] T f75151a1c3d546d88baf9b1b511c2407 P 0e552e70a13c4e4c817d78cdde2861cb: Bootstrap starting.
I20260709 13:06:35.368212 19043 catalog_manager.cc:5697] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2). New cstate: current_term: 1 leader_uuid: "0e552e70a13c4e4c817d78cdde2861cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:35.366209 19457 raft_consensus.cc:515] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.371528 19439 tablet_bootstrap.cc:492] T 5e600380d13a47f38e3369753d2ce794 P b38ff4cb1ee546abba5eb00fe4cf50f2: No bootstrap required, opened a new log
I20260709 13:06:35.371050 19046 catalog_manager.cc:5697] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3). New cstate: current_term: 1 leader_uuid: "3e8380a11eb9479cae35d2f49f8591dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:35.374224 19457 leader_election.cc:290] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 1 election: Requested vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093)
I20260709 13:06:35.374794 19439 ts_tablet_manager.cc:1403] T 5e600380d13a47f38e3369753d2ce794 P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent bootstrapping tablet: real 0.064s	user 0.021s	sys 0.005s
I20260709 13:06:35.372668 19146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb0e2bad45494ca38c6c9107a7656899" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
I20260709 13:06:35.373366 19438 tablet_bootstrap.cc:654] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:35.373720 19223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb0e2bad45494ca38c6c9107a7656899" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
I20260709 13:06:35.376351 19223 raft_consensus.cc:3060] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:35.377849 19263 leader_election.cc:343] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:35.385006 19441 tablet_bootstrap.cc:654] T f75151a1c3d546d88baf9b1b511c2407 P 0e552e70a13c4e4c817d78cdde2861cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:35.385440 19439 raft_consensus.cc:359] T 5e600380d13a47f38e3369753d2ce794 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.389355 19439 raft_consensus.cc:385] T 5e600380d13a47f38e3369753d2ce794 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.389806 19439 raft_consensus.cc:740] T 5e600380d13a47f38e3369753d2ce794 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Initialized, Role: FOLLOWER
I20260709 13:06:35.390507 19439 consensus_queue.cc:260] T 5e600380d13a47f38e3369753d2ce794 P b38ff4cb1ee546abba5eb00fe4cf50f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.392722 19439 ts_tablet_manager.cc:1434] T 5e600380d13a47f38e3369753d2ce794 P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.000s
I20260709 13:06:35.393884 19439 tablet_bootstrap.cc:492] T eb0e2bad45494ca38c6c9107a7656899 P b38ff4cb1ee546abba5eb00fe4cf50f2: Bootstrap starting.
I20260709 13:06:35.395193 19223 raft_consensus.cc:2468] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e8380a11eb9479cae35d2f49f8591dc in term 1.
I20260709 13:06:35.401278 19263 leader_election.cc:304] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, 3e8380a11eb9479cae35d2f49f8591dc; no voters: b38ff4cb1ee546abba5eb00fe4cf50f2
I20260709 13:06:35.402561 19457 raft_consensus.cc:2804] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:35.403057 19457 raft_consensus.cc:697] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 LEADER]: Becoming Leader. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Running, Role: LEADER
I20260709 13:06:35.403597 19439 tablet_bootstrap.cc:654] T eb0e2bad45494ca38c6c9107a7656899 P b38ff4cb1ee546abba5eb00fe4cf50f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:35.403836 19457 consensus_queue.cc:237] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.407888 19441 tablet_bootstrap.cc:492] T f75151a1c3d546d88baf9b1b511c2407 P 0e552e70a13c4e4c817d78cdde2861cb: No bootstrap required, opened a new log
I20260709 13:06:35.408545 19441 ts_tablet_manager.cc:1403] T f75151a1c3d546d88baf9b1b511c2407 P 0e552e70a13c4e4c817d78cdde2861cb: Time spent bootstrapping tablet: real 0.043s	user 0.010s	sys 0.004s
I20260709 13:06:35.411490 19441 raft_consensus.cc:359] T f75151a1c3d546d88baf9b1b511c2407 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.412232 19441 raft_consensus.cc:385] T f75151a1c3d546d88baf9b1b511c2407 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.412563 19441 raft_consensus.cc:740] T f75151a1c3d546d88baf9b1b511c2407 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Initialized, Role: FOLLOWER
I20260709 13:06:35.413388 19441 consensus_queue.cc:260] T f75151a1c3d546d88baf9b1b511c2407 P 0e552e70a13c4e4c817d78cdde2861cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.414398 19438 tablet_bootstrap.cc:492] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc: No bootstrap required, opened a new log
I20260709 13:06:35.415098 19438 ts_tablet_manager.cc:1403] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc: Time spent bootstrapping tablet: real 0.052s	user 0.007s	sys 0.009s
I20260709 13:06:35.415889 19441 ts_tablet_manager.cc:1434] T f75151a1c3d546d88baf9b1b511c2407 P 0e552e70a13c4e4c817d78cdde2861cb: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:06:35.414561 19439 tablet_bootstrap.cc:492] T eb0e2bad45494ca38c6c9107a7656899 P b38ff4cb1ee546abba5eb00fe4cf50f2: No bootstrap required, opened a new log
I20260709 13:06:35.417022 19441 tablet_bootstrap.cc:492] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb: Bootstrap starting.
I20260709 13:06:35.417531 19439 ts_tablet_manager.cc:1403] T eb0e2bad45494ca38c6c9107a7656899 P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent bootstrapping tablet: real 0.024s	user 0.004s	sys 0.008s
I20260709 13:06:35.418496 19438 raft_consensus.cc:359] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.419389 19438 raft_consensus.cc:385] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.419703 19438 raft_consensus.cc:740] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Initialized, Role: FOLLOWER
I20260709 13:06:35.420571 19438 consensus_queue.cc:260] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.423420 19438 ts_tablet_manager.cc:1434] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 13:06:35.424456 19441 tablet_bootstrap.cc:654] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:35.424496 19438 tablet_bootstrap.cc:492] T 5e600380d13a47f38e3369753d2ce794 P 3e8380a11eb9479cae35d2f49f8591dc: Bootstrap starting.
I20260709 13:06:35.425372 19043 catalog_manager.cc:5697] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3). New cstate: current_term: 1 leader_uuid: "3e8380a11eb9479cae35d2f49f8591dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:35.434255 19438 tablet_bootstrap.cc:654] T 5e600380d13a47f38e3369753d2ce794 P 3e8380a11eb9479cae35d2f49f8591dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:35.436311 19441 tablet_bootstrap.cc:492] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb: No bootstrap required, opened a new log
I20260709 13:06:35.436945 19441 ts_tablet_manager.cc:1403] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.002s
I20260709 13:06:35.440892 19441 raft_consensus.cc:359] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.442114 19441 raft_consensus.cc:385] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.442523 19441 raft_consensus.cc:740] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Initialized, Role: FOLLOWER
I20260709 13:06:35.420787 19439 raft_consensus.cc:359] T eb0e2bad45494ca38c6c9107a7656899 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.443502 19439 raft_consensus.cc:385] T eb0e2bad45494ca38c6c9107a7656899 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.443872 19439 raft_consensus.cc:740] T eb0e2bad45494ca38c6c9107a7656899 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Initialized, Role: FOLLOWER
I20260709 13:06:35.443521 19441 consensus_queue.cc:260] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.444762 19439 consensus_queue.cc:260] T eb0e2bad45494ca38c6c9107a7656899 P b38ff4cb1ee546abba5eb00fe4cf50f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.447270 19441 ts_tablet_manager.cc:1434] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 13:06:35.450757 19438 tablet_bootstrap.cc:492] T 5e600380d13a47f38e3369753d2ce794 P 3e8380a11eb9479cae35d2f49f8591dc: No bootstrap required, opened a new log
I20260709 13:06:35.451362 19438 ts_tablet_manager.cc:1403] T 5e600380d13a47f38e3369753d2ce794 P 3e8380a11eb9479cae35d2f49f8591dc: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.009s
I20260709 13:06:35.452956 19439 ts_tablet_manager.cc:1434] T eb0e2bad45494ca38c6c9107a7656899 P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.001s
I20260709 13:06:35.454109 19439 tablet_bootstrap.cc:492] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2: Bootstrap starting.
I20260709 13:06:35.453819 19438 raft_consensus.cc:359] T 5e600380d13a47f38e3369753d2ce794 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.454581 19438 raft_consensus.cc:385] T 5e600380d13a47f38e3369753d2ce794 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.454896 19438 raft_consensus.cc:740] T 5e600380d13a47f38e3369753d2ce794 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Initialized, Role: FOLLOWER
I20260709 13:06:35.455632 19438 consensus_queue.cc:260] T 5e600380d13a47f38e3369753d2ce794 P 3e8380a11eb9479cae35d2f49f8591dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.458138 19438 ts_tablet_manager.cc:1434] T 5e600380d13a47f38e3369753d2ce794 P 3e8380a11eb9479cae35d2f49f8591dc: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:06:35.459062 19438 tablet_bootstrap.cc:492] T 04abaf75391342018eef037f9e2f92db P 3e8380a11eb9479cae35d2f49f8591dc: Bootstrap starting.
I20260709 13:06:35.463745 19439 tablet_bootstrap.cc:654] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:35.465168 19438 tablet_bootstrap.cc:654] T 04abaf75391342018eef037f9e2f92db P 3e8380a11eb9479cae35d2f49f8591dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:35.474040 19438 tablet_bootstrap.cc:492] T 04abaf75391342018eef037f9e2f92db P 3e8380a11eb9479cae35d2f49f8591dc: No bootstrap required, opened a new log
I20260709 13:06:35.474298 19439 tablet_bootstrap.cc:492] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2: No bootstrap required, opened a new log
I20260709 13:06:35.474457 19438 ts_tablet_manager.cc:1403] T 04abaf75391342018eef037f9e2f92db P 3e8380a11eb9479cae35d2f49f8591dc: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.004s
I20260709 13:06:35.474892 19439 ts_tablet_manager.cc:1403] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.001s
I20260709 13:06:35.476706 19438 raft_consensus.cc:359] T 04abaf75391342018eef037f9e2f92db P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.477371 19438 raft_consensus.cc:385] T 04abaf75391342018eef037f9e2f92db P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.477669 19438 raft_consensus.cc:740] T 04abaf75391342018eef037f9e2f92db P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Initialized, Role: FOLLOWER
I20260709 13:06:35.477488 19439 raft_consensus.cc:359] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.478360 19438 consensus_queue.cc:260] T 04abaf75391342018eef037f9e2f92db P 3e8380a11eb9479cae35d2f49f8591dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.478935 19439 raft_consensus.cc:385] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.479312 19439 raft_consensus.cc:740] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Initialized, Role: FOLLOWER
I20260709 13:06:35.480019 19439 consensus_queue.cc:260] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.480633 19438 ts_tablet_manager.cc:1434] T 04abaf75391342018eef037f9e2f92db P 3e8380a11eb9479cae35d2f49f8591dc: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:06:35.482236 19439 ts_tablet_manager.cc:1434] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:06:35.483354 19439 tablet_bootstrap.cc:492] T 38f843895e814f8a9ed5c4b4e8978ac4 P b38ff4cb1ee546abba5eb00fe4cf50f2: Bootstrap starting.
I20260709 13:06:35.490629 19439 tablet_bootstrap.cc:654] T 38f843895e814f8a9ed5c4b4e8978ac4 P b38ff4cb1ee546abba5eb00fe4cf50f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:35.500560 19439 tablet_bootstrap.cc:492] T 38f843895e814f8a9ed5c4b4e8978ac4 P b38ff4cb1ee546abba5eb00fe4cf50f2: No bootstrap required, opened a new log
I20260709 13:06:35.500969 19439 ts_tablet_manager.cc:1403] T 38f843895e814f8a9ed5c4b4e8978ac4 P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.007s
I20260709 13:06:35.503207 19439 raft_consensus.cc:359] T 38f843895e814f8a9ed5c4b4e8978ac4 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.503746 19439 raft_consensus.cc:385] T 38f843895e814f8a9ed5c4b4e8978ac4 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.503945 19439 raft_consensus.cc:740] T 38f843895e814f8a9ed5c4b4e8978ac4 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Initialized, Role: FOLLOWER
I20260709 13:06:35.504421 19440 raft_consensus.cc:493] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.504711 19439 consensus_queue.cc:260] T 38f843895e814f8a9ed5c4b4e8978ac4 P b38ff4cb1ee546abba5eb00fe4cf50f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.504911 19440 raft_consensus.cc:515] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.506805 19439 ts_tablet_manager.cc:1434] T 38f843895e814f8a9ed5c4b4e8978ac4 P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 13:06:35.506947 19425 raft_consensus.cc:493] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.507121 19440 leader_election.cc:290] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:35.507566 19425 raft_consensus.cc:515] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.508327 19439 tablet_bootstrap.cc:492] T 2fa0f701a2794dee89d3afba337c51fa P b38ff4cb1ee546abba5eb00fe4cf50f2: Bootstrap starting.
I20260709 13:06:35.509016 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f37bdae5606e44e588830063efe2e301" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc" is_pre_election: true
I20260709 13:06:35.510499 19297 raft_consensus.cc:2393] T f37bdae5606e44e588830063efe2e301 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in current term 1: Already voted for candidate 0e552e70a13c4e4c817d78cdde2861cb in this term.
I20260709 13:06:35.509022 19223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f37bdae5606e44e588830063efe2e301" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb" is_pre_election: true
I20260709 13:06:35.511214 19427 raft_consensus.cc:493] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.511802 19427 raft_consensus.cc:515] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.512666 19114 leader_election.cc:304] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 0e552e70a13c4e4c817d78cdde2861cb, 3e8380a11eb9479cae35d2f49f8591dc
I20260709 13:06:35.512753 19146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38f843895e814f8a9ed5c4b4e8978ac4" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" is_pre_election: true
I20260709 13:06:35.513603 19146 raft_consensus.cc:2468] T 38f843895e814f8a9ed5c4b4e8978ac4 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 0.
I20260709 13:06:35.513958 19440 raft_consensus.cc:3060] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.514178 19427 leader_election.cc:290] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:35.514753 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38f843895e814f8a9ed5c4b4e8978ac4" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc" is_pre_election: true
I20260709 13:06:35.515496 19297 raft_consensus.cc:2468] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 0.
I20260709 13:06:35.516500 19295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e600380d13a47f38e3369753d2ce794" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc" is_pre_election: true
I20260709 13:06:35.510591 19425 leader_election.cc:290] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:35.517289 19295 raft_consensus.cc:2468] T 5e600380d13a47f38e3369753d2ce794 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 0.
I20260709 13:06:35.517494 19146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e600380d13a47f38e3369753d2ce794" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" is_pre_election: true
I20260709 13:06:35.518208 19146 raft_consensus.cc:2468] T 5e600380d13a47f38e3369753d2ce794 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 0.
I20260709 13:06:35.518241 19189 leader_election.cc:304] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:35.519318 19189 leader_election.cc:304] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:35.520140 19427 raft_consensus.cc:2804] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:35.520638 19427 raft_consensus.cc:493] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.521078 19427 raft_consensus.cc:3060] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.521159 19425 raft_consensus.cc:2804] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:35.521721 19425 raft_consensus.cc:493] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.517283 19439 tablet_bootstrap.cc:654] T 2fa0f701a2794dee89d3afba337c51fa P b38ff4cb1ee546abba5eb00fe4cf50f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:35.522396 19440 raft_consensus.cc:2749] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:35.522082 19425 raft_consensus.cc:3060] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.530263 19439 tablet_bootstrap.cc:492] T 2fa0f701a2794dee89d3afba337c51fa P b38ff4cb1ee546abba5eb00fe4cf50f2: No bootstrap required, opened a new log
I20260709 13:06:35.530840 19439 ts_tablet_manager.cc:1403] T 2fa0f701a2794dee89d3afba337c51fa P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent bootstrapping tablet: real 0.023s	user 0.008s	sys 0.006s
I20260709 13:06:35.533586 19439 raft_consensus.cc:359] T 2fa0f701a2794dee89d3afba337c51fa P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.534261 19439 raft_consensus.cc:385] T 2fa0f701a2794dee89d3afba337c51fa P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.534551 19439 raft_consensus.cc:740] T 2fa0f701a2794dee89d3afba337c51fa P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Initialized, Role: FOLLOWER
I20260709 13:06:35.534499 19425 raft_consensus.cc:515] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.535620 19439 consensus_queue.cc:260] T 2fa0f701a2794dee89d3afba337c51fa P b38ff4cb1ee546abba5eb00fe4cf50f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.535966 19427 raft_consensus.cc:515] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.537619 19425 leader_election.cc:290] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 election: Requested vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:35.538228 19439 ts_tablet_manager.cc:1434] T 2fa0f701a2794dee89d3afba337c51fa P b38ff4cb1ee546abba5eb00fe4cf50f2: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:06:35.538530 19146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e600380d13a47f38e3369753d2ce794" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
I20260709 13:06:35.539069 19146 raft_consensus.cc:3060] T 5e600380d13a47f38e3369753d2ce794 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.543998 19295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e600380d13a47f38e3369753d2ce794" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
I20260709 13:06:35.544659 19295 raft_consensus.cc:3060] T 5e600380d13a47f38e3369753d2ce794 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.545449 19146 raft_consensus.cc:2468] T 5e600380d13a47f38e3369753d2ce794 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 1.
I20260709 13:06:35.546947 19189 leader_election.cc:304] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:35.548364 19455 raft_consensus.cc:2804] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:35.548504 19427 leader_election.cc:290] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 election: Requested vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:35.551959 19295 raft_consensus.cc:2468] T 5e600380d13a47f38e3369753d2ce794 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 1.
I20260709 13:06:35.552229 19455 raft_consensus.cc:697] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Becoming Leader. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Running, Role: LEADER
I20260709 13:06:35.551695 19146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38f843895e814f8a9ed5c4b4e8978ac4" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
I20260709 13:06:35.553808 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38f843895e814f8a9ed5c4b4e8978ac4" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
I20260709 13:06:35.554117 19146 raft_consensus.cc:3060] T 38f843895e814f8a9ed5c4b4e8978ac4 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.554519 19297 raft_consensus.cc:3060] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.553866 19455 consensus_queue.cc:237] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.560261 19297 raft_consensus.cc:2468] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 1.
I20260709 13:06:35.563158 19146 raft_consensus.cc:2468] T 38f843895e814f8a9ed5c4b4e8978ac4 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 1.
I20260709 13:06:35.561587 19189 leader_election.cc:304] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, 3e8380a11eb9479cae35d2f49f8591dc; no voters: 
I20260709 13:06:35.564378 19427 raft_consensus.cc:2804] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:35.564419 19481 raft_consensus.cc:493] T 50f3b5b62c7f4e89a139a79d44b47496 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.565203 19427 raft_consensus.cc:697] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Becoming Leader. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Running, Role: LEADER
I20260709 13:06:35.565055 19481 raft_consensus.cc:515] T 50f3b5b62c7f4e89a139a79d44b47496 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.566131 19427 consensus_queue.cc:237] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.567400 19481 leader_election.cc:290] T 50f3b5b62c7f4e89a139a79d44b47496 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:35.568025 19044 catalog_manager.cc:5697] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2). New cstate: current_term: 1 leader_uuid: "0e552e70a13c4e4c817d78cdde2861cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:35.569506 19147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50f3b5b62c7f4e89a139a79d44b47496" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" is_pre_election: true
I20260709 13:06:35.569977 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50f3b5b62c7f4e89a139a79d44b47496" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc" is_pre_election: true
I20260709 13:06:35.571067 19297 raft_consensus.cc:2393] T 50f3b5b62c7f4e89a139a79d44b47496 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e552e70a13c4e4c817d78cdde2861cb in current term 1: Already voted for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in this term.
I20260709 13:06:35.573958 19189 leader_election.cc:304] T 50f3b5b62c7f4e89a139a79d44b47496 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb; no voters: 3e8380a11eb9479cae35d2f49f8591dc, b38ff4cb1ee546abba5eb00fe4cf50f2
I20260709 13:06:35.575002 19455 raft_consensus.cc:3060] T 50f3b5b62c7f4e89a139a79d44b47496 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.581949 19455 raft_consensus.cc:2749] T 50f3b5b62c7f4e89a139a79d44b47496 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:35.585078 19044 catalog_manager.cc:5697] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2). New cstate: current_term: 1 leader_uuid: "0e552e70a13c4e4c817d78cdde2861cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:35.587607 19455 raft_consensus.cc:493] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.588128 19455 raft_consensus.cc:515] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.590071 19455 leader_election.cc:290] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:35.590799 19147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04abaf75391342018eef037f9e2f92db" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" is_pre_election: true
I20260709 13:06:35.591064 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04abaf75391342018eef037f9e2f92db" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc" is_pre_election: true
I20260709 13:06:35.591554 19147 raft_consensus.cc:2468] T 04abaf75391342018eef037f9e2f92db P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 0.
I20260709 13:06:35.591805 19297 raft_consensus.cc:2468] T 04abaf75391342018eef037f9e2f92db P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 0.
I20260709 13:06:35.592905 19189 leader_election.cc:304] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, 3e8380a11eb9479cae35d2f49f8591dc; no voters: 
I20260709 13:06:35.593680 19455 raft_consensus.cc:2804] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:35.594079 19455 raft_consensus.cc:493] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.594496 19455 raft_consensus.cc:3060] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.599346 19455 raft_consensus.cc:515] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.601054 19455 leader_election.cc:290] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 election: Requested vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:35.601867 19147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04abaf75391342018eef037f9e2f92db" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
I20260709 13:06:35.602196 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04abaf75391342018eef037f9e2f92db" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
I20260709 13:06:35.602468 19147 raft_consensus.cc:3060] T 04abaf75391342018eef037f9e2f92db P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.602855 19297 raft_consensus.cc:3060] T 04abaf75391342018eef037f9e2f92db P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.607971 19147 raft_consensus.cc:2468] T 04abaf75391342018eef037f9e2f92db P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 1.
I20260709 13:06:35.608062 19297 raft_consensus.cc:2468] T 04abaf75391342018eef037f9e2f92db P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 1.
I20260709 13:06:35.609148 19189 leader_election.cc:304] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:35.610028 19455 raft_consensus.cc:2804] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:35.610473 19455 raft_consensus.cc:697] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Becoming Leader. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Running, Role: LEADER
I20260709 13:06:35.611254 19455 consensus_queue.cc:237] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
W20260709 13:06:35.614005 19094 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:35.620007 19044 catalog_manager.cc:5697] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2). New cstate: current_term: 1 leader_uuid: "0e552e70a13c4e4c817d78cdde2861cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:35.669744 19457 raft_consensus.cc:493] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.670073 19440 consensus_queue.cc:1048] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:35.670392 19457 raft_consensus.cc:515] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.672784 19457 leader_election.cc:290] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093)
I20260709 13:06:35.673448 19147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fa0f701a2794dee89d3afba337c51fa" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" is_pre_election: true
I20260709 13:06:35.674055 19223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fa0f701a2794dee89d3afba337c51fa" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb" is_pre_election: true
I20260709 13:06:35.674242 19147 raft_consensus.cc:2468] T 2fa0f701a2794dee89d3afba337c51fa P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e8380a11eb9479cae35d2f49f8591dc in term 0.
I20260709 13:06:35.674896 19223 raft_consensus.cc:2468] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e8380a11eb9479cae35d2f49f8591dc in term 0.
I20260709 13:06:35.675467 19263 leader_election.cc:304] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8380a11eb9479cae35d2f49f8591dc, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:35.676926 19423 raft_consensus.cc:2804] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:35.677290 19423 raft_consensus.cc:493] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.677680 19423 raft_consensus.cc:3060] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.680054 19455 consensus_queue.cc:1048] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:35.687647 19423 raft_consensus.cc:515] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.692096 19426 consensus_queue.cc:1048] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:35.692528 19146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fa0f701a2794dee89d3afba337c51fa" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
I20260709 13:06:35.693277 19146 raft_consensus.cc:3060] T 2fa0f701a2794dee89d3afba337c51fa P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.694067 19423 leader_election.cc:290] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 1 election: Requested vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093)
I20260709 13:06:35.698796 19223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fa0f701a2794dee89d3afba337c51fa" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
I20260709 13:06:35.699657 19223 raft_consensus.cc:3060] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.714231 19146 raft_consensus.cc:2468] T 2fa0f701a2794dee89d3afba337c51fa P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e8380a11eb9479cae35d2f49f8591dc in term 1.
I20260709 13:06:35.716300 19263 leader_election.cc:304] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8380a11eb9479cae35d2f49f8591dc, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:35.718101 19427 consensus_queue.cc:1048] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:35.718780 19423 raft_consensus.cc:2804] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:35.719640 19423 raft_consensus.cc:697] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [term 1 LEADER]: Becoming Leader. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Running, Role: LEADER
I20260709 13:06:35.721381 19223 raft_consensus.cc:2468] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e8380a11eb9479cae35d2f49f8591dc in term 1.
I20260709 13:06:35.720898 19423 consensus_queue.cc:237] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:35.744352 19425 consensus_queue.cc:1048] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:35.744485 19044 catalog_manager.cc:5697] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3). New cstate: current_term: 1 leader_uuid: "3e8380a11eb9479cae35d2f49f8591dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:35.760196 19495 consensus_queue.cc:1048] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:35.806252 19222 raft_consensus.cc:3060] T f75151a1c3d546d88baf9b1b511c2407 P 0e552e70a13c4e4c817d78cdde2861cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.812521 19423 consensus_queue.cc:1048] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:35.823282 19458 consensus_queue.cc:1048] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:35.861112 19458 consensus_queue.cc:1048] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:35.872254 19146 raft_consensus.cc:3060] T eb0e2bad45494ca38c6c9107a7656899 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.880856 19501 consensus_queue.cc:1048] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:35.933058 19297 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f75151a1c3d546d88baf9b1b511c2407"
dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
mode: GRACEFUL
new_leader_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
 from {username='slave'} at 127.0.0.1:58700
I20260709 13:06:35.933619 19297 raft_consensus.cc:606] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 LEADER]: Received request to transfer leadership to TS b38ff4cb1ee546abba5eb00fe4cf50f2
I20260709 13:06:35.938918 19222 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "535ad37217c24424ac08ebe1cf5a5252"
dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
mode: GRACEFUL
new_leader_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
 from {username='slave'} at 127.0.0.1:52694
I20260709 13:06:35.939504 19222 raft_consensus.cc:606] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Received request to transfer leadership to TS b38ff4cb1ee546abba5eb00fe4cf50f2
I20260709 13:06:35.948081 19222 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f37bdae5606e44e588830063efe2e301"
dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
mode: GRACEFUL
new_leader_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
 from {username='slave'} at 127.0.0.1:52694
I20260709 13:06:35.948823 19222 raft_consensus.cc:606] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Received request to transfer leadership to TS b38ff4cb1ee546abba5eb00fe4cf50f2
I20260709 13:06:35.957695 19295 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb0e2bad45494ca38c6c9107a7656899"
dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
mode: GRACEFUL
new_leader_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
 from {username='slave'} at 127.0.0.1:58700
I20260709 13:06:35.958249 19295 raft_consensus.cc:606] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 LEADER]: Received request to transfer leadership to TS 0e552e70a13c4e4c817d78cdde2861cb
I20260709 13:06:35.964067 19295 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fa0f701a2794dee89d3afba337c51fa"
dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
mode: GRACEFUL
new_leader_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
 from {username='slave'} at 127.0.0.1:58700
I20260709 13:06:35.964773 19295 raft_consensus.cc:606] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [term 1 LEADER]: Received request to transfer leadership to TS 0e552e70a13c4e4c817d78cdde2861cb
I20260709 13:06:35.974263 19222 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38f843895e814f8a9ed5c4b4e8978ac4"
dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
mode: GRACEFUL
new_leader_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
 from {username='slave'} at 127.0.0.1:52694
I20260709 13:06:35.975054 19222 raft_consensus.cc:606] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Received request to transfer leadership to TS 3e8380a11eb9479cae35d2f49f8591dc
I20260709 13:06:35.990921 19495 consensus_queue.cc:1048] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:35.999253 19427 raft_consensus.cc:993] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb: : Instructing follower 3e8380a11eb9479cae35d2f49f8591dc to start an election
I20260709 13:06:35.999960 19427 raft_consensus.cc:1081] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Signalling peer 3e8380a11eb9479cae35d2f49f8591dc to start an election
I20260709 13:06:36.003185 19297 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "38f843895e814f8a9ed5c4b4e8978ac4"
dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
 from {username='slave'} at 127.0.0.1:41218
I20260709 13:06:36.004343 19297 raft_consensus.cc:493] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:36.004765 19297 raft_consensus.cc:3060] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:36.005844 19425 consensus_queue.cc:1048] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:36.012588 19297 raft_consensus.cc:515] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:36.014941 19297 leader_election.cc:290] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 2 election: Requested vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093)
I20260709 13:06:36.015909 19147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38f843895e814f8a9ed5c4b4e8978ac4" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
I20260709 13:06:36.016559 19147 raft_consensus.cc:3060] T 38f843895e814f8a9ed5c4b4e8978ac4 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:36.020095 19222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38f843895e814f8a9ed5c4b4e8978ac4" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
I20260709 13:06:36.020838 19222 raft_consensus.cc:3055] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:36.021209 19222 raft_consensus.cc:740] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Running, Role: LEADER
I20260709 13:06:36.022194 19222 consensus_queue.cc:260] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:36.023435 19222 raft_consensus.cc:3060] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:36.023567 19147 raft_consensus.cc:2468] T 38f843895e814f8a9ed5c4b4e8978ac4 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e8380a11eb9479cae35d2f49f8591dc in term 2.
I20260709 13:06:36.025398 19263 leader_election.cc:304] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8380a11eb9479cae35d2f49f8591dc, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:36.026252 19501 raft_consensus.cc:2804] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:36.026713 19501 raft_consensus.cc:697] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 LEADER]: Becoming Leader. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Running, Role: LEADER
I20260709 13:06:36.027539 19501 consensus_queue.cc:237] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:36.031211 19222 raft_consensus.cc:2468] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e8380a11eb9479cae35d2f49f8591dc in term 2.
I20260709 13:06:36.038075 19043 catalog_manager.cc:5697] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc reported cstate change: term changed from 1 to 2, leader changed from 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2) to 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3). New cstate: current_term: 2 leader_uuid: "3e8380a11eb9479cae35d2f49f8591dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:36.043596 19427 consensus_queue.cc:1048] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:36.055469 19427 consensus_queue.cc:1048] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:36.063297 19427 consensus_queue.cc:1048] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:36.066843 19427 consensus_queue.cc:1048] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:36.245091 19425 raft_consensus.cc:993] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb: : Instructing follower b38ff4cb1ee546abba5eb00fe4cf50f2 to start an election
I20260709 13:06:36.245663 19427 raft_consensus.cc:1081] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Signalling peer b38ff4cb1ee546abba5eb00fe4cf50f2 to start an election
I20260709 13:06:36.247437 19147 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f37bdae5606e44e588830063efe2e301"
dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
 from {username='slave'} at 127.0.0.1:38070
I20260709 13:06:36.247871 19147 raft_consensus.cc:493] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:36.248116 19147 raft_consensus.cc:3060] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:36.253553 19147 raft_consensus.cc:515] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:36.255339 19147 leader_election.cc:290] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 2 election: Requested vote from peers 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:36.255936 19222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f37bdae5606e44e588830063efe2e301" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
I20260709 13:06:36.256354 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f37bdae5606e44e588830063efe2e301" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
I20260709 13:06:36.256691 19222 raft_consensus.cc:3055] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:36.256897 19297 raft_consensus.cc:3060] T f37bdae5606e44e588830063efe2e301 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:36.257079 19222 raft_consensus.cc:740] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Running, Role: LEADER
I20260709 13:06:36.257962 19222 consensus_queue.cc:260] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:36.258924 19222 raft_consensus.cc:3060] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:36.261868 19297 raft_consensus.cc:2468] T f37bdae5606e44e588830063efe2e301 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 2.
I20260709 13:06:36.262990 19114 leader_election.cc:304] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8380a11eb9479cae35d2f49f8591dc, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:36.263664 19487 raft_consensus.cc:2804] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:36.264202 19487 raft_consensus.cc:697] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 LEADER]: Becoming Leader. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Running, Role: LEADER
I20260709 13:06:36.264232 19222 raft_consensus.cc:2468] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 2.
I20260709 13:06:36.265137 19487 consensus_queue.cc:237] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:36.272261 19044 catalog_manager.cc:5697] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2 reported cstate change: term changed from 1 to 2, leader changed from 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2) to b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1). New cstate: current_term: 2 leader_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:36.299296 19501 consensus_queue.cc:1048] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:36.306681 19457 raft_consensus.cc:993] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc: : Instructing follower 0e552e70a13c4e4c817d78cdde2861cb to start an election
I20260709 13:06:36.307199 19457 raft_consensus.cc:1081] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [term 1 LEADER]: Signalling peer 0e552e70a13c4e4c817d78cdde2861cb to start an election
I20260709 13:06:36.309616 19223 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2fa0f701a2794dee89d3afba337c51fa"
dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
 from {username='slave'} at 127.0.0.1:52696
I20260709 13:06:36.312088 19223 raft_consensus.cc:493] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:36.312531 19223 raft_consensus.cc:3060] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:36.313509 19457 consensus_queue.cc:1048] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:36.320143 19223 raft_consensus.cc:515] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:36.325848 19223 leader_election.cc:290] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 2 election: Requested vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:36.326525 19147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fa0f701a2794dee89d3afba337c51fa" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
I20260709 13:06:36.327250 19147 raft_consensus.cc:3060] T 2fa0f701a2794dee89d3afba337c51fa P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:36.328735 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fa0f701a2794dee89d3afba337c51fa" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
I20260709 13:06:36.329339 19297 raft_consensus.cc:3055] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:36.329674 19297 raft_consensus.cc:740] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Running, Role: LEADER
I20260709 13:06:36.330368 19297 consensus_queue.cc:260] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:36.331404 19297 raft_consensus.cc:3060] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:36.333640 19147 raft_consensus.cc:2468] T 2fa0f701a2794dee89d3afba337c51fa P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 2.
I20260709 13:06:36.336593 19297 raft_consensus.cc:2468] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 2.
I20260709 13:06:36.337361 19189 leader_election.cc:304] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:36.338402 19425 raft_consensus.cc:2804] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:36.339008 19425 raft_consensus.cc:697] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [term 2 LEADER]: Becoming Leader. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Running, Role: LEADER
I20260709 13:06:36.339756 19425 consensus_queue.cc:237] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:36.343883 19495 raft_consensus.cc:993] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb: : Instructing follower b38ff4cb1ee546abba5eb00fe4cf50f2 to start an election
I20260709 13:06:36.344364 19425 raft_consensus.cc:1081] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Signalling peer b38ff4cb1ee546abba5eb00fe4cf50f2 to start an election
I20260709 13:06:36.345825 19147 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "535ad37217c24424ac08ebe1cf5a5252"
dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
 from {username='slave'} at 127.0.0.1:38070
I20260709 13:06:36.346441 19147 raft_consensus.cc:493] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:36.346833 19147 raft_consensus.cc:3060] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:36.349608 19043 catalog_manager.cc:5697] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb reported cstate change: term changed from 1 to 2, leader changed from 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3) to 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2). New cstate: current_term: 2 leader_uuid: "0e552e70a13c4e4c817d78cdde2861cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:36.354626 19147 raft_consensus.cc:515] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:36.356952 19147 leader_election.cc:290] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 2 election: Requested vote from peers 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:36.358273 19223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "535ad37217c24424ac08ebe1cf5a5252" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
I20260709 13:06:36.358939 19223 raft_consensus.cc:3055] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:36.359342 19223 raft_consensus.cc:740] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Running, Role: LEADER
I20260709 13:06:36.359468 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "535ad37217c24424ac08ebe1cf5a5252" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
I20260709 13:06:36.360044 19297 raft_consensus.cc:3060] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:36.360365 19223 consensus_queue.cc:260] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:36.361615 19423 raft_consensus.cc:993] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc: : Instructing follower 0e552e70a13c4e4c817d78cdde2861cb to start an election
I20260709 13:06:36.361770 19223 raft_consensus.cc:3060] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:36.362149 19457 raft_consensus.cc:1081] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 LEADER]: Signalling peer 0e552e70a13c4e4c817d78cdde2861cb to start an election
I20260709 13:06:36.363801 19222 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb0e2bad45494ca38c6c9107a7656899"
dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
 from {username='slave'} at 127.0.0.1:52696
I20260709 13:06:36.364346 19222 raft_consensus.cc:493] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:36.364703 19222 raft_consensus.cc:3060] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:36.366959 19297 raft_consensus.cc:2468] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 2.
I20260709 13:06:36.368101 19114 leader_election.cc:304] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8380a11eb9479cae35d2f49f8591dc, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:36.368876 19487 raft_consensus.cc:2804] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:36.368954 19223 raft_consensus.cc:2468] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 2.
I20260709 13:06:36.369508 19487 raft_consensus.cc:697] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 LEADER]: Becoming Leader. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Running, Role: LEADER
I20260709 13:06:36.370512 19487 consensus_queue.cc:237] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:36.371172 19222 raft_consensus.cc:515] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:36.374367 19147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb0e2bad45494ca38c6c9107a7656899" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
I20260709 13:06:36.375006 19147 raft_consensus.cc:3060] T eb0e2bad45494ca38c6c9107a7656899 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:36.375231 19222 leader_election.cc:290] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 2 election: Requested vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:36.376302 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb0e2bad45494ca38c6c9107a7656899" candidate_uuid: "0e552e70a13c4e4c817d78cdde2861cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
I20260709 13:06:36.377167 19297 raft_consensus.cc:3055] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:36.377501 19297 raft_consensus.cc:740] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Running, Role: LEADER
I20260709 13:06:36.378430 19297 consensus_queue.cc:260] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:36.379779 19297 raft_consensus.cc:3060] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:36.379542 19043 catalog_manager.cc:5697] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2 reported cstate change: term changed from 1 to 2, leader changed from 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2) to b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1). New cstate: current_term: 2 leader_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:36.382987 19147 raft_consensus.cc:2468] T eb0e2bad45494ca38c6c9107a7656899 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 2.
I20260709 13:06:36.384279 19189 leader_election.cc:304] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:36.385149 19425 raft_consensus.cc:2804] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:36.385699 19425 raft_consensus.cc:697] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 LEADER]: Becoming Leader. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Running, Role: LEADER
I20260709 13:06:36.386554 19425 consensus_queue.cc:237] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:36.387002 19297 raft_consensus.cc:2468] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e552e70a13c4e4c817d78cdde2861cb in term 2.
I20260709 13:06:36.394182 19043 catalog_manager.cc:5697] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb reported cstate change: term changed from 1 to 2, leader changed from 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3) to 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2). New cstate: current_term: 2 leader_uuid: "0e552e70a13c4e4c817d78cdde2861cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:36.399811 19423 raft_consensus.cc:993] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc: : Instructing follower b38ff4cb1ee546abba5eb00fe4cf50f2 to start an election
I20260709 13:06:36.400403 19457 raft_consensus.cc:1081] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 LEADER]: Signalling peer b38ff4cb1ee546abba5eb00fe4cf50f2 to start an election
I20260709 13:06:36.402773 19147 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f75151a1c3d546d88baf9b1b511c2407"
dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
 from {username='slave'} at 127.0.0.1:38074
I20260709 13:06:36.403407 19147 raft_consensus.cc:493] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:36.403807 19147 raft_consensus.cc:3060] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:36.409920 19147 raft_consensus.cc:515] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:36.411756 19147 leader_election.cc:290] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 2 election: Requested vote from peers 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093), 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3:43973)
I20260709 13:06:36.412516 19222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f75151a1c3d546d88baf9b1b511c2407" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
I20260709 13:06:36.412779 19297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f75151a1c3d546d88baf9b1b511c2407" candidate_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
I20260709 13:06:36.413093 19222 raft_consensus.cc:3060] T f75151a1c3d546d88baf9b1b511c2407 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:36.413405 19297 raft_consensus.cc:3055] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:36.413759 19297 raft_consensus.cc:740] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Running, Role: LEADER
I20260709 13:06:36.414397 19297 consensus_queue.cc:260] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:36.415223 19297 raft_consensus.cc:3060] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:36.418931 19222 raft_consensus.cc:2468] T f75151a1c3d546d88baf9b1b511c2407 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 2.
I20260709 13:06:36.420087 19114 leader_election.cc:304] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:36.420382 19223 raft_consensus.cc:1275] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Refusing update from remote peer 3e8380a11eb9479cae35d2f49f8591dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:36.420902 19487 raft_consensus.cc:2804] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:36.421201 19297 raft_consensus.cc:2468] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b38ff4cb1ee546abba5eb00fe4cf50f2 in term 2.
I20260709 13:06:36.421500 19487 raft_consensus.cc:697] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 LEADER]: Becoming Leader. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Running, Role: LEADER
I20260709 13:06:36.421716 19457 consensus_queue.cc:1048] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:36.422410 19487 consensus_queue.cc:237] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:36.432958 19147 raft_consensus.cc:1275] T 38f843895e814f8a9ed5c4b4e8978ac4 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Refusing update from remote peer 3e8380a11eb9479cae35d2f49f8591dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:36.430102 19043 catalog_manager.cc:5697] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 reported cstate change: term changed from 1 to 2, leader changed from 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3) to b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1). New cstate: current_term: 2 leader_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:36.434821 19457 consensus_queue.cc:1048] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:36.713271 19295 raft_consensus.cc:1275] T f37bdae5606e44e588830063efe2e301 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Refusing update from remote peer b38ff4cb1ee546abba5eb00fe4cf50f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:36.714798 19426 consensus_queue.cc:1048] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:36.726833 19223 raft_consensus.cc:1275] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Refusing update from remote peer b38ff4cb1ee546abba5eb00fe4cf50f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:36.730221 19487 consensus_queue.cc:1048] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:36.810699 19147 raft_consensus.cc:1275] T eb0e2bad45494ca38c6c9107a7656899 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Refusing update from remote peer 0e552e70a13c4e4c817d78cdde2861cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:36.811889 19425 consensus_queue.cc:1048] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:36.819531 19295 raft_consensus.cc:1275] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Refusing update from remote peer 0e552e70a13c4e4c817d78cdde2861cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:36.820780 19481 consensus_queue.cc:1048] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:36.828591 19295 raft_consensus.cc:1275] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Refusing update from remote peer b38ff4cb1ee546abba5eb00fe4cf50f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:36.830034 19487 consensus_queue.cc:1048] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:36.839610 19223 raft_consensus.cc:1275] T f75151a1c3d546d88baf9b1b511c2407 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Refusing update from remote peer b38ff4cb1ee546abba5eb00fe4cf50f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:36.841415 19513 consensus_queue.cc:1048] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:36.860590 19147 raft_consensus.cc:1275] T 2fa0f701a2794dee89d3afba337c51fa P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Refusing update from remote peer 0e552e70a13c4e4c817d78cdde2861cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:36.861922 19481 consensus_queue.cc:1048] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:36.864414 19295 raft_consensus.cc:1275] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Refusing update from remote peer 0e552e70a13c4e4c817d78cdde2861cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:36.866510 19481 consensus_queue.cc:1048] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:36.908259 19223 raft_consensus.cc:1275] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Refusing update from remote peer b38ff4cb1ee546abba5eb00fe4cf50f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:36.909696 19513 consensus_queue.cc:1048] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:36.919414 19297 raft_consensus.cc:1275] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Refusing update from remote peer b38ff4cb1ee546abba5eb00fe4cf50f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:36.921063 19487 consensus_queue.cc:1048] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:06:38.489907 19318 debug-util.cc:398] Leaking SignalData structure 0x7b0800099460 after lost signal to thread 7167
W20260709 13:06:38.490903 19318 debug-util.cc:398] Leaking SignalData structure 0x7b0800149a80 after lost signal to thread 19080
W20260709 13:06:38.492003 19318 debug-util.cc:398] Leaking SignalData structure 0x7b080035d740 after lost signal to thread 19172
W20260709 13:06:38.493016 19318 debug-util.cc:398] Leaking SignalData structure 0x7b08003769a0 after lost signal to thread 19247
W20260709 13:06:38.494197 19318 debug-util.cc:398] Leaking SignalData structure 0x7b08002bc520 after lost signal to thread 19321
I20260709 13:06:39.201850 19043 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33592:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:06:39.250064 19043 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33592:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:06:39.350628  7164 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 13:06:39.585341 19541 log.cc:927] Time spent T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2: Append to log took a long time: real 0.144s	user 0.001s	sys 0.004s
W20260709 13:06:39.703687 19577 log.cc:927] Time spent T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb: Append to log took a long time: real 0.064s	user 0.000s	sys 0.002s
I20260709 13:06:39.686064 19580 tablet.cc:1724] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:39.649186 19575 tablet.cc:1724] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:06:39.735482 19563 log.cc:927] Time spent T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc: Append to log took a long time: real 0.114s	user 0.002s	sys 0.000s
I20260709 13:06:39.777863 19588 tablet.cc:1724] T eb0e2bad45494ca38c6c9107a7656899 P b38ff4cb1ee546abba5eb00fe4cf50f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:39.705734 19582 tablet.cc:1724] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:06:39.818390 19174 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:50f3b5b62c7f4e89a139a79d44b47496)
W20260709 13:06:39.817173 19189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:35991, user_credentials={real_user=slave}} blocked reactor thread for 90174.1us
I20260709 13:06:39.826229 19589 tablet.cc:1724] T 50f3b5b62c7f4e89a139a79d44b47496 P 3e8380a11eb9479cae35d2f49f8591dc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:39.861166 19590 tablet.cc:1724] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:06:39.865690 19323 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:38f843895e814f8a9ed5c4b4e8978ac4)
I20260709 13:06:39.862572 19591 tablet.cc:1724] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:06:39.866487 19249 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:eb0e2bad45494ca38c6c9107a7656899)
W20260709 13:06:39.864910 19263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.1:35991, user_credentials={real_user=slave}} blocked reactor thread for 52103us
W20260709 13:06:39.879072 19576 log.cc:927] Time spent T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2: Append to log took a long time: real 0.177s	user 0.002s	sys 0.000s
I20260709 13:06:39.811889 19596 tablet.cc:1724] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:39.894696 19600 tablet.cc:1724] T f37bdae5606e44e588830063efe2e301 P 3e8380a11eb9479cae35d2f49f8591dc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:39.893000 19223 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb0e2bad45494ca38c6c9107a7656899"
dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
mode: GRACEFUL
new_leader_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
 from {username='slave'} at 127.0.0.1:51588
I20260709 13:06:39.896652 19223 raft_consensus.cc:606] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 LEADER]: Received request to transfer leadership to TS 3e8380a11eb9479cae35d2f49f8591dc
I20260709 13:06:39.897619 19598 tablet.cc:1724] T 38f843895e814f8a9ed5c4b4e8978ac4 P b38ff4cb1ee546abba5eb00fe4cf50f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:39.894556 19593 tablet.cc:1724] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:39.942421 19595 tablet.cc:1724] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:39.963146 19593 tablet.cc:1724] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:39.977392 19589 tablet.cc:1724] T 04abaf75391342018eef037f9e2f92db P 3e8380a11eb9479cae35d2f49f8591dc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:39.978731 19600 tablet.cc:1724] T 5e600380d13a47f38e3369753d2ce794 P 3e8380a11eb9479cae35d2f49f8591dc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:39.980202 19045 catalog_manager.cc:5789] TS 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093) does not have the latest schema for tablet 2fa0f701a2794dee89d3afba337c51fa (table soft_deleted_table [id=46a235393ae445be9239216af30cc03a]). Expected version 1 got 0
I20260709 13:06:39.980767 19595 tablet.cc:1724] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:39.999804 19427 raft_consensus.cc:993] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb: : Instructing follower 3e8380a11eb9479cae35d2f49f8591dc to start an election
I20260709 13:06:40.001313 19427 raft_consensus.cc:1081] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 LEADER]: Signalling peer 3e8380a11eb9479cae35d2f49f8591dc to start an election
I20260709 13:06:40.004721 19591 tablet.cc:1724] T 04abaf75391342018eef037f9e2f92db P b38ff4cb1ee546abba5eb00fe4cf50f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:40.005805 19593 tablet.cc:1724] T 50f3b5b62c7f4e89a139a79d44b47496 P 0e552e70a13c4e4c817d78cdde2861cb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:40.022974 19292 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb0e2bad45494ca38c6c9107a7656899"
dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
 from {username='slave'} at 127.0.0.1:41218
I20260709 13:06:40.023892 19292 raft_consensus.cc:493] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:40.019538 19598 tablet.cc:1724] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:40.024374 19292 raft_consensus.cc:3060] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:40.003242 19580 tablet.cc:1724] T 2fa0f701a2794dee89d3afba337c51fa P b38ff4cb1ee546abba5eb00fe4cf50f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:40.040422 19292 raft_consensus.cc:515] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:40.044193 19144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb0e2bad45494ca38c6c9107a7656899" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
I20260709 13:06:40.044450 19223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb0e2bad45494ca38c6c9107a7656899" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
I20260709 13:06:40.044957 19144 raft_consensus.cc:3060] T eb0e2bad45494ca38c6c9107a7656899 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:40.045042 19223 raft_consensus.cc:3055] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:40.045413 19223 raft_consensus.cc:740] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Running, Role: LEADER
I20260709 13:06:40.046321 19223 consensus_queue.cc:260] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:40.047498 19223 raft_consensus.cc:3060] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:40.055368 19223 raft_consensus.cc:2468] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e8380a11eb9479cae35d2f49f8591dc in term 3.
I20260709 13:06:40.057462 19263 leader_election.cc:304] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, 3e8380a11eb9479cae35d2f49f8591dc; no voters: 
I20260709 13:06:40.057773 19600 tablet.cc:1724] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:40.055913 19292 leader_election.cc:290] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 3 election: Requested vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093)
I20260709 13:06:40.058609 19601 raft_consensus.cc:2804] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:40.059403 19601 raft_consensus.cc:697] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [term 3 LEADER]: Becoming Leader. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Running, Role: LEADER
I20260709 13:06:40.060473 19601 consensus_queue.cc:237] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:40.061232 19598 tablet.cc:1724] T 5e600380d13a47f38e3369753d2ce794 P b38ff4cb1ee546abba5eb00fe4cf50f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:40.061558 19596 tablet.cc:1724] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:40.055069 19582 tablet.cc:1724] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:40.069532 19588 tablet.cc:1724] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:40.089056 19575 tablet.cc:1724] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:40.097535 19144 raft_consensus.cc:2468] T eb0e2bad45494ca38c6c9107a7656899 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e8380a11eb9479cae35d2f49f8591dc in term 3.
I20260709 13:06:40.089089 19043 catalog_manager.cc:5697] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc reported cstate change: term changed from 2 to 3, leader changed from 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2) to 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3). New cstate: current_term: 3 leader_uuid: "3e8380a11eb9479cae35d2f49f8591dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:40.106594 19045 catalog_manager.cc:5789] TS b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991) does not have the latest schema for tablet f75151a1c3d546d88baf9b1b511c2407 (table soft_deleted_table [id=46a235393ae445be9239216af30cc03a]). Expected version 1 got 0
I20260709 13:06:40.116703 19144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f75151a1c3d546d88baf9b1b511c2407"
dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
mode: GRACEFUL
new_leader_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
 from {username='slave'} at 127.0.0.1:51668
I20260709 13:06:40.117714 19144 raft_consensus.cc:606] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 LEADER]: Received request to transfer leadership to TS 3e8380a11eb9479cae35d2f49f8591dc
I20260709 13:06:40.120020  7164 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:06:40.120750  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:40.101410 19593 tablet.cc:1724] T f75151a1c3d546d88baf9b1b511c2407 P 0e552e70a13c4e4c817d78cdde2861cb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:40.162693 19223 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2d02069d652c4ef99b2876fb113575f2"
dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
mode: GRACEFUL
new_leader_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
 from {username='slave'} at 127.0.0.1:51588
I20260709 13:06:40.163352 19223 raft_consensus.cc:606] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Received request to transfer leadership to TS 3e8380a11eb9479cae35d2f49f8591dc
I20260709 13:06:40.167187 19144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d50426588914ed2b6d59c26481beb6c"
dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
mode: GRACEFUL
new_leader_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
 from {username='slave'} at 127.0.0.1:51668
I20260709 13:06:40.167888 19144 raft_consensus.cc:606] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 LEADER]: Received request to transfer leadership to TS 3e8380a11eb9479cae35d2f49f8591dc
I20260709 13:06:40.170750  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:06:40.171691  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:40.187642 19045 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee: soft_deleted_table [id=46a235393ae445be9239216af30cc03a] alter complete (version 1)
I20260709 13:06:40.242606 19599 raft_consensus.cc:993] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb: : Instructing follower 3e8380a11eb9479cae35d2f49f8591dc to start an election
I20260709 13:06:40.243314 19587 raft_consensus.cc:1081] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Signalling peer 3e8380a11eb9479cae35d2f49f8591dc to start an election
I20260709 13:06:40.245213 19292 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2d02069d652c4ef99b2876fb113575f2"
dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
 from {username='slave'} at 127.0.0.1:41218
I20260709 13:06:40.245900 19292 raft_consensus.cc:493] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:40.246286 19292 raft_consensus.cc:3060] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:40.252396 19292 raft_consensus.cc:515] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:40.254484 19292 leader_election.cc:290] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 2 election: Requested vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093)
I20260709 13:06:40.255362 19144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d02069d652c4ef99b2876fb113575f2" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
I20260709 13:06:40.255563 19223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d02069d652c4ef99b2876fb113575f2" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
I20260709 13:06:40.256073 19144 raft_consensus.cc:3060] T 2d02069d652c4ef99b2876fb113575f2 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:40.256321 19223 raft_consensus.cc:3055] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:40.256776 19223 raft_consensus.cc:740] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e552e70a13c4e4c817d78cdde2861cb, State: Running, Role: LEADER
I20260709 13:06:40.257799 19223 consensus_queue.cc:260] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:40.259164 19223 raft_consensus.cc:3060] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:40.263310 19144 raft_consensus.cc:2468] T 2d02069d652c4ef99b2876fb113575f2 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e8380a11eb9479cae35d2f49f8591dc in term 2.
I20260709 13:06:40.264866 19263 leader_election.cc:304] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8380a11eb9479cae35d2f49f8591dc, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:40.265815 19601 raft_consensus.cc:2804] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:40.266520 19601 raft_consensus.cc:697] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 LEADER]: Becoming Leader. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Running, Role: LEADER
I20260709 13:06:40.267742 19601 consensus_queue.cc:237] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:40.268239 19223 raft_consensus.cc:2468] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e8380a11eb9479cae35d2f49f8591dc in term 2.
I20260709 13:06:40.280376 19044 catalog_manager.cc:5697] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc reported cstate change: term changed from 1 to 2, leader changed from 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2) to 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3). New cstate: current_term: 2 leader_uuid: "3e8380a11eb9479cae35d2f49f8591dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:40.284840 19513 raft_consensus.cc:993] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2: : Instructing follower 3e8380a11eb9479cae35d2f49f8591dc to start an election
I20260709 13:06:40.285640 19581 raft_consensus.cc:1081] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 LEADER]: Signalling peer 3e8380a11eb9479cae35d2f49f8591dc to start an election
I20260709 13:06:40.288409 19292 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8d50426588914ed2b6d59c26481beb6c"
dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
 from {username='slave'} at 127.0.0.1:41202
I20260709 13:06:40.289016 19292 raft_consensus.cc:493] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:40.289387 19292 raft_consensus.cc:3060] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:40.295434 19292 raft_consensus.cc:515] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:40.297361 19292 leader_election.cc:290] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 2 election: Requested vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093)
I20260709 13:06:40.298738 19223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d50426588914ed2b6d59c26481beb6c" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
I20260709 13:06:40.298864 19144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d50426588914ed2b6d59c26481beb6c" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
I20260709 13:06:40.299468 19223 raft_consensus.cc:3060] T 8d50426588914ed2b6d59c26481beb6c P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:40.299695 19144 raft_consensus.cc:3055] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:40.300220 19144 raft_consensus.cc:740] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Running, Role: LEADER
I20260709 13:06:40.301319 19144 consensus_queue.cc:260] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:40.303102 19144 raft_consensus.cc:3060] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:40.305940 19223 raft_consensus.cc:2468] T 8d50426588914ed2b6d59c26481beb6c P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e8380a11eb9479cae35d2f49f8591dc in term 2.
I20260709 13:06:40.307509 19263 leader_election.cc:304] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e552e70a13c4e4c817d78cdde2861cb, 3e8380a11eb9479cae35d2f49f8591dc; no voters: 
I20260709 13:06:40.308502 19601 raft_consensus.cc:2804] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:40.309671 19601 raft_consensus.cc:697] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc [term 2 LEADER]: Becoming Leader. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Running, Role: LEADER
I20260709 13:06:40.311198 19601 consensus_queue.cc:237] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:40.313001 19144 raft_consensus.cc:2468] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e8380a11eb9479cae35d2f49f8591dc in term 2.
I20260709 13:06:40.322752 19044 catalog_manager.cc:5697] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc reported cstate change: term changed from 1 to 2, leader changed from b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1) to 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3). New cstate: current_term: 2 leader_uuid: "3e8380a11eb9479cae35d2f49f8591dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:40.473089 19144 raft_consensus.cc:1275] T eb0e2bad45494ca38c6c9107a7656899 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 3 FOLLOWER]: Refusing update from remote peer 3e8380a11eb9479cae35d2f49f8591dc: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:06:40.474869 19601 consensus_queue.cc:1048] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:40.498303 19223 raft_consensus.cc:1275] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [term 3 FOLLOWER]: Refusing update from remote peer 3e8380a11eb9479cae35d2f49f8591dc: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:06:40.500406 19601 consensus_queue.cc:1048] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:40.563333 19513 raft_consensus.cc:993] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2: : Instructing follower 3e8380a11eb9479cae35d2f49f8591dc to start an election
I20260709 13:06:40.563886 19581 raft_consensus.cc:1081] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 LEADER]: Signalling peer 3e8380a11eb9479cae35d2f49f8591dc to start an election
I20260709 13:06:40.565733 19292 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f75151a1c3d546d88baf9b1b511c2407"
dest_uuid: "3e8380a11eb9479cae35d2f49f8591dc"
 from {username='slave'} at 127.0.0.1:41202
I20260709 13:06:40.566373 19292 raft_consensus.cc:493] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:40.566946 19292 raft_consensus.cc:3060] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:40.576653 19292 raft_consensus.cc:515] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:40.579473 19292 leader_election.cc:290] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 3 election: Requested vote from peers b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991), 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093)
I20260709 13:06:40.580171 19144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f75151a1c3d546d88baf9b1b511c2407" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2"
I20260709 13:06:40.580793 19223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f75151a1c3d546d88baf9b1b511c2407" candidate_uuid: "3e8380a11eb9479cae35d2f49f8591dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "0e552e70a13c4e4c817d78cdde2861cb"
I20260709 13:06:40.580999 19144 raft_consensus.cc:3055] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:40.581537 19144 raft_consensus.cc:740] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b38ff4cb1ee546abba5eb00fe4cf50f2, State: Running, Role: LEADER
I20260709 13:06:40.581832 19223 raft_consensus.cc:3060] T f75151a1c3d546d88baf9b1b511c2407 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:40.582683 19144 consensus_queue.cc:260] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:40.584076 19144 raft_consensus.cc:3060] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:40.592331 19144 raft_consensus.cc:2468] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e8380a11eb9479cae35d2f49f8591dc in term 3.
I20260709 13:06:40.593312 19223 raft_consensus.cc:2468] T f75151a1c3d546d88baf9b1b511c2407 P 0e552e70a13c4e4c817d78cdde2861cb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e8380a11eb9479cae35d2f49f8591dc in term 3.
I20260709 13:06:40.593613 19263 leader_election.cc:304] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8380a11eb9479cae35d2f49f8591dc, b38ff4cb1ee546abba5eb00fe4cf50f2; no voters: 
I20260709 13:06:40.594584 19601 raft_consensus.cc:2804] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:40.595108 19601 raft_consensus.cc:697] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [term 3 LEADER]: Becoming Leader. State: Replica: 3e8380a11eb9479cae35d2f49f8591dc, State: Running, Role: LEADER
I20260709 13:06:40.596084 19601 consensus_queue.cc:237] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } }
I20260709 13:06:40.603250 19044 catalog_manager.cc:5697] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc reported cstate change: term changed from 2 to 3, leader changed from b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1) to 3e8380a11eb9479cae35d2f49f8591dc (127.6.255.3). New cstate: current_term: 3 leader_uuid: "3e8380a11eb9479cae35d2f49f8591dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e8380a11eb9479cae35d2f49f8591dc" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 43973 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:40.768601 19144 raft_consensus.cc:1275] T 2d02069d652c4ef99b2876fb113575f2 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Refusing update from remote peer 3e8380a11eb9479cae35d2f49f8591dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:40.770301 19601 consensus_queue.cc:1048] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:40.784631 19223 raft_consensus.cc:1275] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Refusing update from remote peer 3e8380a11eb9479cae35d2f49f8591dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:40.788208 19601 consensus_queue.cc:1048] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:40.884908 19223 raft_consensus.cc:1275] T 8d50426588914ed2b6d59c26481beb6c P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Refusing update from remote peer 3e8380a11eb9479cae35d2f49f8591dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:40.886780 19519 consensus_queue.cc:1048] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:40.907939 19144 raft_consensus.cc:1275] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Refusing update from remote peer 3e8380a11eb9479cae35d2f49f8591dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:40.911115 19578 consensus_queue.cc:1048] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:41.045382 19223 raft_consensus.cc:1275] T f75151a1c3d546d88baf9b1b511c2407 P 0e552e70a13c4e4c817d78cdde2861cb [term 3 FOLLOWER]: Refusing update from remote peer 3e8380a11eb9479cae35d2f49f8591dc: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:06:41.046782 19519 consensus_queue.cc:1048] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e552e70a13c4e4c817d78cdde2861cb" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:41.058966 19144 raft_consensus.cc:1275] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 3 FOLLOWER]: Refusing update from remote peer 3e8380a11eb9479cae35d2f49f8591dc: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:06:41.066324 19519 consensus_queue.cc:1048] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b38ff4cb1ee546abba5eb00fe4cf50f2" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 35991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:41.253461  7164 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:06:41.263604  7164 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:06:41.265372  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:41.275874  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:06:41.276515  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:42.313759  7164 tablet_server.cc:179] TabletServer@127.6.255.1:0 shutting down...
W20260709 13:06:42.345927 19263 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 8 similar messages]
W20260709 13:06:42.349313 19263 consensus_peers.cc:597] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc -> Peer b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991): Couldn't send request to peer b38ff4cb1ee546abba5eb00fe4cf50f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:42.363068  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:42.364130  7164 tablet_replica.cc:333] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2: stopping tablet replica
I20260709 13:06:42.365320  7164 raft_consensus.cc:2243] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.367516  7164 raft_consensus.cc:2272] T f37bdae5606e44e588830063efe2e301 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.371111  7164 tablet_replica.cc:333] T 5e600380d13a47f38e3369753d2ce794 P b38ff4cb1ee546abba5eb00fe4cf50f2: stopping tablet replica
I20260709 13:06:42.372175  7164 raft_consensus.cc:2243] T 5e600380d13a47f38e3369753d2ce794 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.372833  7164 raft_consensus.cc:2272] T 5e600380d13a47f38e3369753d2ce794 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.376374  7164 tablet_replica.cc:333] T 2fa0f701a2794dee89d3afba337c51fa P b38ff4cb1ee546abba5eb00fe4cf50f2: stopping tablet replica
W20260709 13:06:42.376509 19263 consensus_peers.cc:597] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc -> Peer b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991): Couldn't send request to peer b38ff4cb1ee546abba5eb00fe4cf50f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:42.377468  7164 raft_consensus.cc:2243] T 2fa0f701a2794dee89d3afba337c51fa P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.378163  7164 raft_consensus.cc:2272] T 2fa0f701a2794dee89d3afba337c51fa P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.381102  7164 tablet_replica.cc:333] T 04abaf75391342018eef037f9e2f92db P b38ff4cb1ee546abba5eb00fe4cf50f2: stopping tablet replica
I20260709 13:06:42.382087  7164 raft_consensus.cc:2243] T 04abaf75391342018eef037f9e2f92db P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.382664  7164 raft_consensus.cc:2272] T 04abaf75391342018eef037f9e2f92db P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.385785  7164 tablet_replica.cc:333] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2: stopping tablet replica
I20260709 13:06:42.386698  7164 raft_consensus.cc:2243] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.387868  7164 raft_consensus.cc:2272] T 535ad37217c24424ac08ebe1cf5a5252 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.391470  7164 tablet_replica.cc:333] T 2d02069d652c4ef99b2876fb113575f2 P b38ff4cb1ee546abba5eb00fe4cf50f2: stopping tablet replica
I20260709 13:06:42.392421  7164 raft_consensus.cc:2243] T 2d02069d652c4ef99b2876fb113575f2 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.393014  7164 raft_consensus.cc:2272] T 2d02069d652c4ef99b2876fb113575f2 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.396236  7164 tablet_replica.cc:333] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2: stopping tablet replica
I20260709 13:06:42.397133  7164 raft_consensus.cc:2243] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.398376  7164 raft_consensus.cc:2272] T a67334720ba743938c7b5af66099ddcb P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.401839  7164 tablet_replica.cc:333] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2: stopping tablet replica
I20260709 13:06:42.402750  7164 raft_consensus.cc:2243] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.403343  7164 raft_consensus.cc:2272] T 47e715af44cf464aa61c1f683f5c6b96 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.406669  7164 tablet_replica.cc:333] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2: stopping tablet replica
I20260709 13:06:42.407481  7164 raft_consensus.cc:2243] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.408607  7164 raft_consensus.cc:2272] T 7d271ad4ab1a46c988238f4154400c61 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.411818  7164 tablet_replica.cc:333] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2: stopping tablet replica
I20260709 13:06:42.412588  7164 raft_consensus.cc:2243] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.413251  7164 raft_consensus.cc:2272] T a6cf4b8385a54efd9d4a624eeecb5c80 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.416703  7164 tablet_replica.cc:333] T eb0e2bad45494ca38c6c9107a7656899 P b38ff4cb1ee546abba5eb00fe4cf50f2: stopping tablet replica
I20260709 13:06:42.417658  7164 raft_consensus.cc:2243] T eb0e2bad45494ca38c6c9107a7656899 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.418259  7164 raft_consensus.cc:2272] T eb0e2bad45494ca38c6c9107a7656899 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.421746  7164 tablet_replica.cc:333] T e13f9d6c506846e1bb4f69bf7341622d P b38ff4cb1ee546abba5eb00fe4cf50f2: stopping tablet replica
I20260709 13:06:42.422636  7164 raft_consensus.cc:2243] T e13f9d6c506846e1bb4f69bf7341622d P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.423250  7164 raft_consensus.cc:2272] T e13f9d6c506846e1bb4f69bf7341622d P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.426707  7164 tablet_replica.cc:333] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2: stopping tablet replica
I20260709 13:06:42.427657  7164 raft_consensus.cc:2243] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.428221  7164 raft_consensus.cc:2272] T 8d50426588914ed2b6d59c26481beb6c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.431334  7164 tablet_replica.cc:333] T ffc2b16ab3f54c858df9b008cecdf70c P b38ff4cb1ee546abba5eb00fe4cf50f2: stopping tablet replica
I20260709 13:06:42.431912  7164 raft_consensus.cc:2243] T ffc2b16ab3f54c858df9b008cecdf70c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.432428  7164 raft_consensus.cc:2272] T ffc2b16ab3f54c858df9b008cecdf70c P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.434814  7164 tablet_replica.cc:333] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2: stopping tablet replica
I20260709 13:06:42.435372  7164 raft_consensus.cc:2243] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.435806  7164 raft_consensus.cc:2272] T f75151a1c3d546d88baf9b1b511c2407 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.438717  7164 tablet_replica.cc:333] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2: stopping tablet replica
I20260709 13:06:42.439563  7164 raft_consensus.cc:2243] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.440610  7164 raft_consensus.cc:2272] T 50f3b5b62c7f4e89a139a79d44b47496 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:42.440685 19263 consensus_peers.cc:597] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc -> Peer b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991): Couldn't send request to peer b38ff4cb1ee546abba5eb00fe4cf50f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:42.443176  7164 tablet_replica.cc:333] T 38f843895e814f8a9ed5c4b4e8978ac4 P b38ff4cb1ee546abba5eb00fe4cf50f2: stopping tablet replica
I20260709 13:06:42.443789  7164 raft_consensus.cc:2243] T 38f843895e814f8a9ed5c4b4e8978ac4 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.444249  7164 raft_consensus.cc:2272] T 38f843895e814f8a9ed5c4b4e8978ac4 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.447216  7164 tablet_replica.cc:333] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2: stopping tablet replica
I20260709 13:06:42.447973  7164 raft_consensus.cc:2243] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.449745  7164 raft_consensus.cc:2272] T a54f724cc37d4a86b39037771aaa58a7 P b38ff4cb1ee546abba5eb00fe4cf50f2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:42.481765 19189 consensus_peers.cc:597] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb -> Peer b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991): Couldn't send request to peer b38ff4cb1ee546abba5eb00fe4cf50f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:42.490407 19189 consensus_peers.cc:597] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb -> Peer b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991): Couldn't send request to peer b38ff4cb1ee546abba5eb00fe4cf50f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:42.507508  7164 tablet_server.cc:196] TabletServer@127.6.255.1:0 shutdown complete.
W20260709 13:06:42.521658 19189 consensus_peers.cc:597] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb -> Peer b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991): Couldn't send request to peer b38ff4cb1ee546abba5eb00fe4cf50f2. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:35991: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:42.558475  7164 tablet_server.cc:179] TabletServer@127.6.255.2:0 shutting down...
W20260709 13:06:42.597015 19189 consensus_peers.cc:597] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb -> Peer b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991): Couldn't send request to peer b38ff4cb1ee546abba5eb00fe4cf50f2. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:35991: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:42.599690 19189 consensus_peers.cc:597] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb -> Peer b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991): Couldn't send request to peer b38ff4cb1ee546abba5eb00fe4cf50f2. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:35991: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:42.607951  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:42.608952  7164 tablet_replica.cc:333] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb: stopping tablet replica
I20260709 13:06:42.609943  7164 raft_consensus.cc:2243] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.610975  7164 raft_consensus.cc:2272] T 5e600380d13a47f38e3369753d2ce794 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.614111  7164 tablet_replica.cc:333] T f75151a1c3d546d88baf9b1b511c2407 P 0e552e70a13c4e4c817d78cdde2861cb: stopping tablet replica
I20260709 13:06:42.614965  7164 raft_consensus.cc:2243] T f75151a1c3d546d88baf9b1b511c2407 P 0e552e70a13c4e4c817d78cdde2861cb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.615468  7164 raft_consensus.cc:2272] T f75151a1c3d546d88baf9b1b511c2407 P 0e552e70a13c4e4c817d78cdde2861cb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.618016  7164 tablet_replica.cc:333] T 50f3b5b62c7f4e89a139a79d44b47496 P 0e552e70a13c4e4c817d78cdde2861cb: stopping tablet replica
I20260709 13:06:42.618834  7164 raft_consensus.cc:2243] T 50f3b5b62c7f4e89a139a79d44b47496 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.619323  7164 raft_consensus.cc:2272] T 50f3b5b62c7f4e89a139a79d44b47496 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.621599  7164 tablet_replica.cc:333] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb: stopping tablet replica
I20260709 13:06:42.622219  7164 raft_consensus.cc:2243] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.622701  7164 raft_consensus.cc:2272] T f37bdae5606e44e588830063efe2e301 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.624728  7164 tablet_replica.cc:333] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb: stopping tablet replica
I20260709 13:06:42.625538  7164 raft_consensus.cc:2243] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.626407  7164 raft_consensus.cc:2272] T e13f9d6c506846e1bb4f69bf7341622d P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.628871  7164 tablet_replica.cc:333] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb: stopping tablet replica
I20260709 13:06:42.629549  7164 raft_consensus.cc:2243] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.630296  7164 raft_consensus.cc:2272] T eb0e2bad45494ca38c6c9107a7656899 P 0e552e70a13c4e4c817d78cdde2861cb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.632320  7164 tablet_replica.cc:333] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb: stopping tablet replica
I20260709 13:06:42.632933  7164 raft_consensus.cc:2243] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.633766  7164 raft_consensus.cc:2272] T 04abaf75391342018eef037f9e2f92db P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.635790  7164 tablet_replica.cc:333] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb: stopping tablet replica
I20260709 13:06:42.636361  7164 raft_consensus.cc:2243] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.637117  7164 raft_consensus.cc:2272] T 2fa0f701a2794dee89d3afba337c51fa P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.639189  7164 tablet_replica.cc:333] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb: stopping tablet replica
I20260709 13:06:42.639714  7164 raft_consensus.cc:2243] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.640131  7164 raft_consensus.cc:2272] T 535ad37217c24424ac08ebe1cf5a5252 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.642617  7164 tablet_replica.cc:333] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb: stopping tablet replica
I20260709 13:06:42.643201  7164 raft_consensus.cc:2243] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.643618  7164 raft_consensus.cc:2272] T 2d02069d652c4ef99b2876fb113575f2 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.645745  7164 tablet_replica.cc:333] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb: stopping tablet replica
I20260709 13:06:42.646301  7164 raft_consensus.cc:2243] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.647511  7164 raft_consensus.cc:2272] T 47e715af44cf464aa61c1f683f5c6b96 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.649909  7164 tablet_replica.cc:333] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb: stopping tablet replica
I20260709 13:06:42.650532  7164 raft_consensus.cc:2243] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.650954  7164 raft_consensus.cc:2272] T 38f843895e814f8a9ed5c4b4e8978ac4 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.653048  7164 tablet_replica.cc:333] T a54f724cc37d4a86b39037771aaa58a7 P 0e552e70a13c4e4c817d78cdde2861cb: stopping tablet replica
I20260709 13:06:42.653649  7164 raft_consensus.cc:2243] T a54f724cc37d4a86b39037771aaa58a7 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.654047  7164 raft_consensus.cc:2272] T a54f724cc37d4a86b39037771aaa58a7 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.656121  7164 tablet_replica.cc:333] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb: stopping tablet replica
I20260709 13:06:42.656623  7164 raft_consensus.cc:2243] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.657403  7164 raft_consensus.cc:2272] T ffc2b16ab3f54c858df9b008cecdf70c P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.659749  7164 tablet_replica.cc:333] T a6cf4b8385a54efd9d4a624eeecb5c80 P 0e552e70a13c4e4c817d78cdde2861cb: stopping tablet replica
I20260709 13:06:42.660339  7164 raft_consensus.cc:2243] T a6cf4b8385a54efd9d4a624eeecb5c80 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.660758  7164 raft_consensus.cc:2272] T a6cf4b8385a54efd9d4a624eeecb5c80 P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.662878  7164 tablet_replica.cc:333] T 8d50426588914ed2b6d59c26481beb6c P 0e552e70a13c4e4c817d78cdde2861cb: stopping tablet replica
I20260709 13:06:42.663405  7164 raft_consensus.cc:2243] T 8d50426588914ed2b6d59c26481beb6c P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.663786  7164 raft_consensus.cc:2272] T 8d50426588914ed2b6d59c26481beb6c P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.665848  7164 tablet_replica.cc:333] T 7d271ad4ab1a46c988238f4154400c61 P 0e552e70a13c4e4c817d78cdde2861cb: stopping tablet replica
I20260709 13:06:42.666349  7164 raft_consensus.cc:2243] T 7d271ad4ab1a46c988238f4154400c61 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.666759  7164 raft_consensus.cc:2272] T 7d271ad4ab1a46c988238f4154400c61 P 0e552e70a13c4e4c817d78cdde2861cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.668839  7164 tablet_replica.cc:333] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb: stopping tablet replica
I20260709 13:06:42.669348  7164 raft_consensus.cc:2243] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.669946  7164 raft_consensus.cc:2272] T a67334720ba743938c7b5af66099ddcb P 0e552e70a13c4e4c817d78cdde2861cb [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:42.674672 19263 consensus_peers.cc:597] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc -> Peer b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991): Couldn't send request to peer b38ff4cb1ee546abba5eb00fe4cf50f2. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:35991: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:42.695369 19263 consensus_peers.cc:597] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc -> Peer b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991): Couldn't send request to peer b38ff4cb1ee546abba5eb00fe4cf50f2. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:35991: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:42.714989  7164 tablet_server.cc:196] TabletServer@127.6.255.2:0 shutdown complete.
W20260709 13:06:42.750249 19263 consensus_peers.cc:597] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc -> Peer b38ff4cb1ee546abba5eb00fe4cf50f2 (127.6.255.1:35991): Couldn't send request to peer b38ff4cb1ee546abba5eb00fe4cf50f2. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:35991: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:42.753857 19263 consensus_peers.cc:597] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc -> Peer 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093): Couldn't send request to peer 0e552e70a13c4e4c817d78cdde2861cb. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:42093: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:42.764428  7164 tablet_server.cc:179] TabletServer@127.6.255.3:0 shutting down...
W20260709 13:06:42.774523 19263 consensus_peers.cc:597] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc -> Peer 0e552e70a13c4e4c817d78cdde2861cb (127.6.255.2:42093): Couldn't send request to peer 0e552e70a13c4e4c817d78cdde2861cb. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:42093: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:42.799005  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:42.800029  7164 tablet_replica.cc:333] T 5e600380d13a47f38e3369753d2ce794 P 3e8380a11eb9479cae35d2f49f8591dc: stopping tablet replica
I20260709 13:06:42.801187  7164 raft_consensus.cc:2243] T 5e600380d13a47f38e3369753d2ce794 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.801991  7164 raft_consensus.cc:2272] T 5e600380d13a47f38e3369753d2ce794 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.804489  7164 tablet_replica.cc:333] T f37bdae5606e44e588830063efe2e301 P 3e8380a11eb9479cae35d2f49f8591dc: stopping tablet replica
I20260709 13:06:42.805083  7164 raft_consensus.cc:2243] T f37bdae5606e44e588830063efe2e301 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.805621  7164 raft_consensus.cc:2272] T f37bdae5606e44e588830063efe2e301 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.807811  7164 tablet_replica.cc:333] T 04abaf75391342018eef037f9e2f92db P 3e8380a11eb9479cae35d2f49f8591dc: stopping tablet replica
I20260709 13:06:42.808373  7164 raft_consensus.cc:2243] T 04abaf75391342018eef037f9e2f92db P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.808822  7164 raft_consensus.cc:2272] T 04abaf75391342018eef037f9e2f92db P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.810881  7164 tablet_replica.cc:333] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc: stopping tablet replica
I20260709 13:06:42.811404  7164 raft_consensus.cc:2243] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.811893  7164 raft_consensus.cc:2272] T 2fa0f701a2794dee89d3afba337c51fa P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.814036  7164 tablet_replica.cc:333] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc: stopping tablet replica
I20260709 13:06:42.814597  7164 raft_consensus.cc:2243] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.815172  7164 raft_consensus.cc:2272] T 535ad37217c24424ac08ebe1cf5a5252 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.817320  7164 tablet_replica.cc:333] T 50f3b5b62c7f4e89a139a79d44b47496 P 3e8380a11eb9479cae35d2f49f8591dc: stopping tablet replica
I20260709 13:06:42.818008  7164 raft_consensus.cc:2243] T 50f3b5b62c7f4e89a139a79d44b47496 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.818445  7164 raft_consensus.cc:2272] T 50f3b5b62c7f4e89a139a79d44b47496 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.820494  7164 tablet_replica.cc:333] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc: stopping tablet replica
I20260709 13:06:42.821053  7164 raft_consensus.cc:2243] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.821861  7164 raft_consensus.cc:2272] T f75151a1c3d546d88baf9b1b511c2407 P 3e8380a11eb9479cae35d2f49f8591dc [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.823941  7164 tablet_replica.cc:333] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc: stopping tablet replica
I20260709 13:06:42.824510  7164 raft_consensus.cc:2243] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.825215  7164 raft_consensus.cc:2272] T 2d02069d652c4ef99b2876fb113575f2 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.827538  7164 tablet_replica.cc:333] T 47e715af44cf464aa61c1f683f5c6b96 P 3e8380a11eb9479cae35d2f49f8591dc: stopping tablet replica
I20260709 13:06:42.828054  7164 raft_consensus.cc:2243] T 47e715af44cf464aa61c1f683f5c6b96 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.828504  7164 raft_consensus.cc:2272] T 47e715af44cf464aa61c1f683f5c6b96 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.830775  7164 tablet_replica.cc:333] T a67334720ba743938c7b5af66099ddcb P 3e8380a11eb9479cae35d2f49f8591dc: stopping tablet replica
I20260709 13:06:42.831415  7164 raft_consensus.cc:2243] T a67334720ba743938c7b5af66099ddcb P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.831914  7164 raft_consensus.cc:2272] T a67334720ba743938c7b5af66099ddcb P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.834085  7164 tablet_replica.cc:333] T ffc2b16ab3f54c858df9b008cecdf70c P 3e8380a11eb9479cae35d2f49f8591dc: stopping tablet replica
I20260709 13:06:42.834591  7164 raft_consensus.cc:2243] T ffc2b16ab3f54c858df9b008cecdf70c P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.834983  7164 raft_consensus.cc:2272] T ffc2b16ab3f54c858df9b008cecdf70c P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.837072  7164 tablet_replica.cc:333] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc: stopping tablet replica
I20260709 13:06:42.837604  7164 raft_consensus.cc:2243] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.838392  7164 raft_consensus.cc:2272] T 38f843895e814f8a9ed5c4b4e8978ac4 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.840422  7164 tablet_replica.cc:333] T a54f724cc37d4a86b39037771aaa58a7 P 3e8380a11eb9479cae35d2f49f8591dc: stopping tablet replica
I20260709 13:06:42.840971  7164 raft_consensus.cc:2243] T a54f724cc37d4a86b39037771aaa58a7 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.841387  7164 raft_consensus.cc:2272] T a54f724cc37d4a86b39037771aaa58a7 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.843633  7164 tablet_replica.cc:333] T 7d271ad4ab1a46c988238f4154400c61 P 3e8380a11eb9479cae35d2f49f8591dc: stopping tablet replica
I20260709 13:06:42.844102  7164 raft_consensus.cc:2243] T 7d271ad4ab1a46c988238f4154400c61 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.844600  7164 raft_consensus.cc:2272] T 7d271ad4ab1a46c988238f4154400c61 P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.846746  7164 tablet_replica.cc:333] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc: stopping tablet replica
I20260709 13:06:42.847234  7164 raft_consensus.cc:2243] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.848022  7164 raft_consensus.cc:2272] T a6cf4b8385a54efd9d4a624eeecb5c80 P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.850438  7164 tablet_replica.cc:333] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc: stopping tablet replica
I20260709 13:06:42.850922  7164 raft_consensus.cc:2243] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.851661  7164 raft_consensus.cc:2272] T eb0e2bad45494ca38c6c9107a7656899 P 3e8380a11eb9479cae35d2f49f8591dc [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.853767  7164 tablet_replica.cc:333] T e13f9d6c506846e1bb4f69bf7341622d P 3e8380a11eb9479cae35d2f49f8591dc: stopping tablet replica
I20260709 13:06:42.854200  7164 raft_consensus.cc:2243] T e13f9d6c506846e1bb4f69bf7341622d P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.854614  7164 raft_consensus.cc:2272] T e13f9d6c506846e1bb4f69bf7341622d P 3e8380a11eb9479cae35d2f49f8591dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.856716  7164 tablet_replica.cc:333] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc: stopping tablet replica
I20260709 13:06:42.857189  7164 raft_consensus.cc:2243] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.858042  7164 raft_consensus.cc:2272] T 8d50426588914ed2b6d59c26481beb6c P 3e8380a11eb9479cae35d2f49f8591dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.922036  7164 tablet_server.cc:196] TabletServer@127.6.255.3:0 shutdown complete.
I20260709 13:06:42.959367  7164 master.cc:562] Master@127.6.255.62:41289 shutting down...
I20260709 13:06:42.988391  7164 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.989040  7164 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.989367  7164 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6e3e7c5ed6f04c6f9df5fb20e212c2ee: stopping tablet replica
I20260709 13:06:43.011273  7164 master.cc:584] Master@127.6.255.62:41289 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (13692 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:06:43.067415  7164 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.255.62:37417
I20260709 13:06:43.068569  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:43.074908 19626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:43.074987 19627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:43.075217 19629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:43.076224  7164 server_base.cc:1061] running on GCE node
I20260709 13:06:43.077143  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:43.077327  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:43.077947  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602403077926 us; error 0 us; skew 500 ppm
I20260709 13:06:43.078522  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:43.081034  7164 webserver.cc:533] Webserver started at http://127.6.255.62:41277/ using document root <none> and password file <none>
I20260709 13:06:43.081775  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:43.082017  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:43.082341  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:43.083536  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602046064212-7164-0/minicluster-data/master-0-root/instance:
uuid: "1740df38a6644ce6aa8ad917ff3a7eef"
format_stamp: "Formatted at 2026-07-09 13:06:43 on dist-test-slave-rnv4"
I20260709 13:06:43.088538  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:06:43.092564 19634 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:43.093520  7164 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:06:43.093844  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602046064212-7164-0/minicluster-data/master-0-root
uuid: "1740df38a6644ce6aa8ad917ff3a7eef"
format_stamp: "Formatted at 2026-07-09 13:06:43 on dist-test-slave-rnv4"
I20260709 13:06:43.094085  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602046064212-7164-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602046064212-7164-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602046064212-7164-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:43.111616  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:43.112926  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:43.155586  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.62:37417
I20260709 13:06:43.155699 19685 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.62:37417 every 8 connection(s)
I20260709 13:06:43.160410 19686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.172888 19686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef: Bootstrap starting.
I20260709 13:06:43.178335 19686 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:43.183352 19686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef: No bootstrap required, opened a new log
I20260709 13:06:43.186293 19686 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1740df38a6644ce6aa8ad917ff3a7eef" member_type: VOTER }
I20260709 13:06:43.186934 19686 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:43.187188 19686 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1740df38a6644ce6aa8ad917ff3a7eef, State: Initialized, Role: FOLLOWER
I20260709 13:06:43.187953 19686 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1740df38a6644ce6aa8ad917ff3a7eef" member_type: VOTER }
I20260709 13:06:43.188488 19686 raft_consensus.cc:399] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:06:43.188705 19686 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:06:43.188936 19686 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:43.194398 19686 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1740df38a6644ce6aa8ad917ff3a7eef" member_type: VOTER }
I20260709 13:06:43.195168 19686 leader_election.cc:304] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1740df38a6644ce6aa8ad917ff3a7eef; no voters: 
I20260709 13:06:43.196691 19686 leader_election.cc:290] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:06:43.197073 19689 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:43.198763 19689 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef [term 1 LEADER]: Becoming Leader. State: Replica: 1740df38a6644ce6aa8ad917ff3a7eef, State: Running, Role: LEADER
I20260709 13:06:43.199601 19689 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1740df38a6644ce6aa8ad917ff3a7eef" member_type: VOTER }
I20260709 13:06:43.200280 19686 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:06:43.204490 19690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1740df38a6644ce6aa8ad917ff3a7eef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1740df38a6644ce6aa8ad917ff3a7eef" member_type: VOTER } }
I20260709 13:06:43.205332 19690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:43.206212 19691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1740df38a6644ce6aa8ad917ff3a7eef. Latest consensus state: current_term: 1 leader_uuid: "1740df38a6644ce6aa8ad917ff3a7eef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1740df38a6644ce6aa8ad917ff3a7eef" member_type: VOTER } }
I20260709 13:06:43.207005 19691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:43.210666 19695 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:06:43.216053 19695 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:06:43.219576  7164 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:06:43.225693 19695 catalog_manager.cc:1383] Generated new cluster ID: 344efe0033c9444fb008736ce1306615
I20260709 13:06:43.226045 19695 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:06:43.239943 19695 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:06:43.241290 19695 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:06:43.259634 19695 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef: Generated new TSK 0
I20260709 13:06:43.260272 19695 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:06:43.287817  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:43.294368 19707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:43.295390 19708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:43.296983 19710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:43.298066  7164 server_base.cc:1061] running on GCE node
I20260709 13:06:43.299027  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:43.299228  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:43.299353  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602403299342 us; error 0 us; skew 500 ppm
I20260709 13:06:43.299800  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:43.302600  7164 webserver.cc:533] Webserver started at http://127.6.255.1:33865/ using document root <none> and password file <none>
I20260709 13:06:43.303081  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:43.303258  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:43.303490  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:43.304730  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602046064212-7164-0/minicluster-data/ts-0-root/instance:
uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
format_stamp: "Formatted at 2026-07-09 13:06:43 on dist-test-slave-rnv4"
I20260709 13:06:43.309880  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:06:43.314061 19715 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:43.315006  7164 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 13:06:43.315390  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602046064212-7164-0/minicluster-data/ts-0-root
uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
format_stamp: "Formatted at 2026-07-09 13:06:43 on dist-test-slave-rnv4"
I20260709 13:06:43.315732  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602046064212-7164-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602046064212-7164-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602046064212-7164-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:43.358505  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:43.360350  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:43.362056  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:43.364583  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:43.364827  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:43.365111  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:43.365314  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:43.411983  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.1:46037
I20260709 13:06:43.412431 19777 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.1:46037 every 8 connection(s)
I20260709 13:06:43.417544  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:43.425379 19782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:43.427227 19783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:43.430380  7164 server_base.cc:1061] running on GCE node
W20260709 13:06:43.431221 19785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:43.432737  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:43.433005  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:43.433113 19778 heartbeater.cc:344] Connected to a master server at 127.6.255.62:37417
I20260709 13:06:43.433236  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602403433196 us; error 0 us; skew 500 ppm
I20260709 13:06:43.433701 19778 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:43.434082  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:43.434551 19778 heartbeater.cc:507] Master 127.6.255.62:37417 requested a full tablet report, sending...
I20260709 13:06:43.437037 19651 ts_manager.cc:194] Registered new tserver with Master: 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037)
I20260709 13:06:43.437875  7164 webserver.cc:533] Webserver started at http://127.6.255.2:35391/ using document root <none> and password file <none>
I20260709 13:06:43.438409  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:43.438596  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:43.438881  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:43.439075 19651 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50038
I20260709 13:06:43.440166  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602046064212-7164-0/minicluster-data/ts-1-root/instance:
uuid: "cf68973149544eae8929005c48bb84b0"
format_stamp: "Formatted at 2026-07-09 13:06:43 on dist-test-slave-rnv4"
I20260709 13:06:43.444821  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20260709 13:06:43.448475 19790 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:43.449375  7164 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:06:43.449811  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602046064212-7164-0/minicluster-data/ts-1-root
uuid: "cf68973149544eae8929005c48bb84b0"
format_stamp: "Formatted at 2026-07-09 13:06:43 on dist-test-slave-rnv4"
I20260709 13:06:43.450076  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602046064212-7164-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602046064212-7164-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602046064212-7164-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:43.471611  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:43.472939  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:43.474633  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:43.477056  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:43.477317  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:43.477614  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:43.477814  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:43.524822  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.2:43553
I20260709 13:06:43.524899 19852 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.2:43553 every 8 connection(s)
I20260709 13:06:43.530351  7164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:43.539685 19856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:43.541962 19857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:43.542511 19853 heartbeater.cc:344] Connected to a master server at 127.6.255.62:37417
I20260709 13:06:43.543072 19853 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:43.543826  7164 server_base.cc:1061] running on GCE node
I20260709 13:06:43.544195 19853 heartbeater.cc:507] Master 127.6.255.62:37417 requested a full tablet report, sending...
W20260709 13:06:43.545464 19859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:43.546516 19651 ts_manager.cc:194] Registered new tserver with Master: cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553)
I20260709 13:06:43.546586  7164 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:43.547029  7164 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:43.547286  7164 hybrid_clock.cc:648] HybridClock initialized: now 1783602403547263 us; error 0 us; skew 500 ppm
I20260709 13:06:43.548169  7164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:43.548612 19651 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50042
I20260709 13:06:43.551203  7164 webserver.cc:533] Webserver started at http://127.6.255.3:41607/ using document root <none> and password file <none>
I20260709 13:06:43.551731  7164 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:43.551899  7164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:43.552316  7164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:43.553562  7164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602046064212-7164-0/minicluster-data/ts-2-root/instance:
uuid: "e5b2e431c3204ee487c10f58d440707d"
format_stamp: "Formatted at 2026-07-09 13:06:43 on dist-test-slave-rnv4"
I20260709 13:06:43.558434  7164 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 13:06:43.562039 19864 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:43.562922  7164 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.002s
I20260709 13:06:43.563323  7164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602046064212-7164-0/minicluster-data/ts-2-root
uuid: "e5b2e431c3204ee487c10f58d440707d"
format_stamp: "Formatted at 2026-07-09 13:06:43 on dist-test-slave-rnv4"
I20260709 13:06:43.563637  7164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602046064212-7164-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602046064212-7164-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task5d6oYQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602046064212-7164-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:43.586930  7164 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:43.588198  7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:43.589790  7164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:43.592523  7164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:43.592777  7164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:43.593055  7164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:43.593245  7164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:43.637328  7164 rpc_server.cc:307] RPC server started. Bound to: 127.6.255.3:33743
I20260709 13:06:43.637488 19926 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.255.3:33743 every 8 connection(s)
I20260709 13:06:43.651695 19927 heartbeater.cc:344] Connected to a master server at 127.6.255.62:37417
I20260709 13:06:43.652058 19927 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:43.652899 19927 heartbeater.cc:507] Master 127.6.255.62:37417 requested a full tablet report, sending...
I20260709 13:06:43.654910 19651 ts_manager.cc:194] Registered new tserver with Master: e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:43.655812  7164 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014740969s
I20260709 13:06:43.656438 19651 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50046
I20260709 13:06:43.682356 19651 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50058:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:43.685248 19651 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:43.756763 19892 tablet_service.cc:1511] Processing CreateTablet for tablet aa0028f6eae14e1186d8464c0c12b33f (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:43.758291 19892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa0028f6eae14e1186d8464c0c12b33f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.761597 19891 tablet_service.cc:1511] Processing CreateTablet for tablet 34322023741e46f69328b109a5b54158 (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:43.763069 19891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34322023741e46f69328b109a5b54158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.766507 19890 tablet_service.cc:1511] Processing CreateTablet for tablet 8b5517dc489142e19aee3610500ee38e (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:43.768023 19890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b5517dc489142e19aee3610500ee38e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.770531 19888 tablet_service.cc:1511] Processing CreateTablet for tablet 46feb6ece88946fea8e12e1375c1e94c (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:43.772753 19887 tablet_service.cc:1511] Processing CreateTablet for tablet 59a482b1ce05463ea3ef901049d7c1e2 (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:43.773460 19818 tablet_service.cc:1511] Processing CreateTablet for tablet aa0028f6eae14e1186d8464c0c12b33f (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:43.774798 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa0028f6eae14e1186d8464c0c12b33f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.774639 19889 tablet_service.cc:1511] Processing CreateTablet for tablet bd4bd6f0d29248aab51d9a8bdcd054a5 (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:43.777067 19886 tablet_service.cc:1511] Processing CreateTablet for tablet 2348879030be45d49634020eace141f0 (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:43.778561 19817 tablet_service.cc:1511] Processing CreateTablet for tablet 34322023741e46f69328b109a5b54158 (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:43.779471 19888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46feb6ece88946fea8e12e1375c1e94c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.779923 19817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34322023741e46f69328b109a5b54158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.781669 19812 tablet_service.cc:1511] Processing CreateTablet for tablet 2348879030be45d49634020eace141f0 (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:43.783057 19812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2348879030be45d49634020eace141f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.783390 19889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd4bd6f0d29248aab51d9a8bdcd054a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.786629 19885 tablet_service.cc:1511] Processing CreateTablet for tablet 45faf2520a3e4d44bbfd74574b972b70 (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:43.788079 19885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45faf2520a3e4d44bbfd74574b972b70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.787827 19814 tablet_service.cc:1511] Processing CreateTablet for tablet 46feb6ece88946fea8e12e1375c1e94c (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:43.788782 19810 tablet_service.cc:1511] Processing CreateTablet for tablet 409f3b40dbc2457d8803d0d244488b59 (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:43.782689 19811 tablet_service.cc:1511] Processing CreateTablet for tablet 45faf2520a3e4d44bbfd74574b972b70 (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:43.786664 19813 tablet_service.cc:1511] Processing CreateTablet for tablet 59a482b1ce05463ea3ef901049d7c1e2 (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:43.790643 19815 tablet_service.cc:1511] Processing CreateTablet for tablet bd4bd6f0d29248aab51d9a8bdcd054a5 (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:43.791440 19886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2348879030be45d49634020eace141f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.791356 19884 tablet_service.cc:1511] Processing CreateTablet for tablet 409f3b40dbc2457d8803d0d244488b59 (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:43.792870 19884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 409f3b40dbc2457d8803d0d244488b59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.789286 19814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46feb6ece88946fea8e12e1375c1e94c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.796444 19811 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45faf2520a3e4d44bbfd74574b972b70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.792872 19816 tablet_service.cc:1511] Processing CreateTablet for tablet 8b5517dc489142e19aee3610500ee38e (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:43.797185 19815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd4bd6f0d29248aab51d9a8bdcd054a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.797824 19813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59a482b1ce05463ea3ef901049d7c1e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.801833 19816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b5517dc489142e19aee3610500ee38e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.805811 19810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 409f3b40dbc2457d8803d0d244488b59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.796026 19887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59a482b1ce05463ea3ef901049d7c1e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.836225 19942 tablet_bootstrap.cc:492] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d: Bootstrap starting.
I20260709 13:06:43.858847 19942 tablet_bootstrap.cc:654] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:43.862849 19743 tablet_service.cc:1511] Processing CreateTablet for tablet aa0028f6eae14e1186d8464c0c12b33f (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:43.864323 19743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa0028f6eae14e1186d8464c0c12b33f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.888178 19742 tablet_service.cc:1511] Processing CreateTablet for tablet 34322023741e46f69328b109a5b54158 (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:43.888898 19741 tablet_service.cc:1511] Processing CreateTablet for tablet 8b5517dc489142e19aee3610500ee38e (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:43.889767 19742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34322023741e46f69328b109a5b54158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.890532 19741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b5517dc489142e19aee3610500ee38e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.891989 19740 tablet_service.cc:1511] Processing CreateTablet for tablet bd4bd6f0d29248aab51d9a8bdcd054a5 (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:43.893355 19740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd4bd6f0d29248aab51d9a8bdcd054a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.909765 19941 tablet_bootstrap.cc:492] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0: Bootstrap starting.
I20260709 13:06:43.909718 19739 tablet_service.cc:1511] Processing CreateTablet for tablet 46feb6ece88946fea8e12e1375c1e94c (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:43.911310 19739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46feb6ece88946fea8e12e1375c1e94c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.916226 19941 tablet_bootstrap.cc:654] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:43.919621 19738 tablet_service.cc:1511] Processing CreateTablet for tablet 59a482b1ce05463ea3ef901049d7c1e2 (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:43.921392 19738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59a482b1ce05463ea3ef901049d7c1e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.926576 19735 tablet_service.cc:1511] Processing CreateTablet for tablet 409f3b40dbc2457d8803d0d244488b59 (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:43.928009 19735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 409f3b40dbc2457d8803d0d244488b59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.930526 19737 tablet_service.cc:1511] Processing CreateTablet for tablet 2348879030be45d49634020eace141f0 (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:43.931978 19737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2348879030be45d49634020eace141f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.935309 19736 tablet_service.cc:1511] Processing CreateTablet for tablet 45faf2520a3e4d44bbfd74574b972b70 (DEFAULT_TABLE table=test-workload [id=2e8f9d1b979741ba862cee51a5bf11c6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:43.937264 19736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45faf2520a3e4d44bbfd74574b972b70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.947324 19942 tablet_bootstrap.cc:492] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d: No bootstrap required, opened a new log
I20260709 13:06:43.947963 19942 ts_tablet_manager.cc:1403] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d: Time spent bootstrapping tablet: real 0.112s	user 0.021s	sys 0.011s
I20260709 13:06:43.948395 19944 tablet_bootstrap.cc:492] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8: Bootstrap starting.
I20260709 13:06:43.953073 19942 raft_consensus.cc:359] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:43.953857 19942 raft_consensus.cc:385] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:43.954156 19942 raft_consensus.cc:740] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Initialized, Role: FOLLOWER
I20260709 13:06:43.954949 19942 consensus_queue.cc:260] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:43.962152 19944 tablet_bootstrap.cc:654] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:43.965313 19927 heartbeater.cc:499] Master 127.6.255.62:37417 was elected leader, sending a full tablet report...
I20260709 13:06:43.974722 19942 ts_tablet_manager.cc:1434] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 13:06:43.974772 19944 tablet_bootstrap.cc:492] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8: No bootstrap required, opened a new log
I20260709 13:06:43.975490 19944 ts_tablet_manager.cc:1403] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent bootstrapping tablet: real 0.028s	user 0.008s	sys 0.005s
I20260709 13:06:43.975972 19942 tablet_bootstrap.cc:492] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d: Bootstrap starting.
I20260709 13:06:43.978562 19944 raft_consensus.cc:359] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } }
I20260709 13:06:43.979398 19944 raft_consensus.cc:385] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:43.979676 19944 raft_consensus.cc:740] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:43.980592 19944 consensus_queue.cc:260] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } }
I20260709 13:06:44.004168 19942 tablet_bootstrap.cc:654] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:44.004994 19941 tablet_bootstrap.cc:492] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0: No bootstrap required, opened a new log
I20260709 13:06:44.005712 19941 ts_tablet_manager.cc:1403] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0: Time spent bootstrapping tablet: real 0.096s	user 0.041s	sys 0.040s
I20260709 13:06:44.009124 19941 raft_consensus.cc:359] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.009980 19941 raft_consensus.cc:385] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.010390 19941 raft_consensus.cc:740] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf68973149544eae8929005c48bb84b0, State: Initialized, Role: FOLLOWER
I20260709 13:06:44.011502 19941 consensus_queue.cc:260] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.011852 19778 heartbeater.cc:499] Master 127.6.255.62:37417 was elected leader, sending a full tablet report...
I20260709 13:06:44.013589 19944 ts_tablet_manager.cc:1434] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent starting tablet: real 0.038s	user 0.018s	sys 0.019s
I20260709 13:06:44.014921 19944 tablet_bootstrap.cc:492] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8: Bootstrap starting.
I20260709 13:06:44.020706 19942 tablet_bootstrap.cc:492] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d: No bootstrap required, opened a new log
I20260709 13:06:44.021384 19942 ts_tablet_manager.cc:1403] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d: Time spent bootstrapping tablet: real 0.046s	user 0.016s	sys 0.004s
I20260709 13:06:44.023048 19944 tablet_bootstrap.cc:654] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:44.024752 19942 raft_consensus.cc:359] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.029115 19942 raft_consensus.cc:385] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.029646 19941 ts_tablet_manager.cc:1434] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.012s
I20260709 13:06:44.030615 19853 heartbeater.cc:499] Master 127.6.255.62:37417 was elected leader, sending a full tablet report...
W20260709 13:06:44.029930 19854 tablet.cc:2406] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:06:44.031173 19941 tablet_bootstrap.cc:492] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0: Bootstrap starting.
I20260709 13:06:44.029605 19942 raft_consensus.cc:740] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Initialized, Role: FOLLOWER
I20260709 13:06:44.036279 19942 consensus_queue.cc:260] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.040145 19944 tablet_bootstrap.cc:492] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8: No bootstrap required, opened a new log
I20260709 13:06:44.040850 19944 ts_tablet_manager.cc:1403] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent bootstrapping tablet: real 0.026s	user 0.018s	sys 0.002s
I20260709 13:06:44.042554 19950 raft_consensus.cc:493] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.043164 19950 raft_consensus.cc:515] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.045310 19941 tablet_bootstrap.cc:654] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:44.044795 19944 raft_consensus.cc:359] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.045869 19944 raft_consensus.cc:385] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.046269 19944 raft_consensus.cc:740] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:44.046996 19942 ts_tablet_manager.cc:1434] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:06:44.047318 19944 consensus_queue.cc:260] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.048192 19942 tablet_bootstrap.cc:492] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d: Bootstrap starting.
I20260709 13:06:44.049780 19944 ts_tablet_manager.cc:1434] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:06:44.050871 19944 tablet_bootstrap.cc:492] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8: Bootstrap starting.
I20260709 13:06:44.059082 19950 leader_election.cc:290] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:44.060479 19941 tablet_bootstrap.cc:492] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0: No bootstrap required, opened a new log
I20260709 13:06:44.061058 19941 ts_tablet_manager.cc:1403] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.011s
I20260709 13:06:44.064538 19941 raft_consensus.cc:359] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.065527 19941 raft_consensus.cc:385] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.065919 19941 raft_consensus.cc:740] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf68973149544eae8929005c48bb84b0, State: Initialized, Role: FOLLOWER
I20260709 13:06:44.066947 19941 consensus_queue.cc:260] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.068337 19944 tablet_bootstrap.cc:654] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:44.069578 19941 ts_tablet_manager.cc:1434] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0: Time spent starting tablet: real 0.008s	user 0.010s	sys 0.000s
I20260709 13:06:44.070626 19941 tablet_bootstrap.cc:492] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0: Bootstrap starting.
I20260709 13:06:44.077003 19944 tablet_bootstrap.cc:492] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8: No bootstrap required, opened a new log
I20260709 13:06:44.077198 19941 tablet_bootstrap.cc:654] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:44.077773 19944 ts_tablet_manager.cc:1403] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.004s
I20260709 13:06:44.080941 19944 raft_consensus.cc:359] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.081785 19944 raft_consensus.cc:385] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.082146 19944 raft_consensus.cc:740] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:44.083011 19944 consensus_queue.cc:260] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.083413 19753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34322023741e46f69328b109a5b54158" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" is_pre_election: true
I20260709 13:06:44.084187 19753 raft_consensus.cc:2468] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 0.
I20260709 13:06:44.085775 19793 leader_election.cc:304] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: 
I20260709 13:06:44.088482 19944 ts_tablet_manager.cc:1434] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:06:44.089824 19944 tablet_bootstrap.cc:492] T 45faf2520a3e4d44bbfd74574b972b70 P 7cc15333145b485ea8cdd6c1623ff0f8: Bootstrap starting.
I20260709 13:06:44.086639 19950 raft_consensus.cc:2804] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:44.090418 19950 raft_consensus.cc:493] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:44.090854 19950 raft_consensus.cc:3060] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.097196 19944 tablet_bootstrap.cc:654] T 45faf2520a3e4d44bbfd74574b972b70 P 7cc15333145b485ea8cdd6c1623ff0f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:44.099589 19950 raft_consensus.cc:515] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.071861 19942 tablet_bootstrap.cc:654] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:44.108484 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34322023741e46f69328b109a5b54158" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d" is_pre_election: true
I20260709 13:06:44.109367 19902 raft_consensus.cc:2468] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 0.
I20260709 13:06:44.109588 19950 leader_election.cc:290] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 election: Requested vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:44.110592 19944 tablet_bootstrap.cc:492] T 45faf2520a3e4d44bbfd74574b972b70 P 7cc15333145b485ea8cdd6c1623ff0f8: No bootstrap required, opened a new log
I20260709 13:06:44.110612 19942 tablet_bootstrap.cc:492] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d: No bootstrap required, opened a new log
I20260709 13:06:44.111330 19944 ts_tablet_manager.cc:1403] T 45faf2520a3e4d44bbfd74574b972b70 P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.008s
I20260709 13:06:44.111377 19942 ts_tablet_manager.cc:1403] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d: Time spent bootstrapping tablet: real 0.063s	user 0.010s	sys 0.004s
I20260709 13:06:44.111672 19753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34322023741e46f69328b109a5b54158" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
I20260709 13:06:44.112428 19753 raft_consensus.cc:3060] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.113068 19941 tablet_bootstrap.cc:492] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0: No bootstrap required, opened a new log
I20260709 13:06:44.113770 19941 ts_tablet_manager.cc:1403] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0: Time spent bootstrapping tablet: real 0.043s	user 0.021s	sys 0.018s
I20260709 13:06:44.114709 19944 raft_consensus.cc:359] T 45faf2520a3e4d44bbfd74574b972b70 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.114959 19942 raft_consensus.cc:359] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } }
I20260709 13:06:44.115589 19944 raft_consensus.cc:385] T 45faf2520a3e4d44bbfd74574b972b70 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.115777 19942 raft_consensus.cc:385] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.115975 19944 raft_consensus.cc:740] T 45faf2520a3e4d44bbfd74574b972b70 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:44.116196 19942 raft_consensus.cc:740] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Initialized, Role: FOLLOWER
I20260709 13:06:44.116959 19944 consensus_queue.cc:260] T 45faf2520a3e4d44bbfd74574b972b70 P 7cc15333145b485ea8cdd6c1623ff0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.117136 19942 consensus_queue.cc:260] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } }
I20260709 13:06:44.116954 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34322023741e46f69328b109a5b54158" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:44.118314 19902 raft_consensus.cc:3060] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.119565 19944 ts_tablet_manager.cc:1434] T 45faf2520a3e4d44bbfd74574b972b70 P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:06:44.120685 19944 tablet_bootstrap.cc:492] T 2348879030be45d49634020eace141f0 P 7cc15333145b485ea8cdd6c1623ff0f8: Bootstrap starting.
I20260709 13:06:44.125363 19753 raft_consensus.cc:2468] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 1.
I20260709 13:06:44.126957 19793 leader_election.cc:304] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: 
I20260709 13:06:44.127214 19944 tablet_bootstrap.cc:654] T 2348879030be45d49634020eace141f0 P 7cc15333145b485ea8cdd6c1623ff0f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:44.127964 19950 raft_consensus.cc:2804] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:44.135708 19942 ts_tablet_manager.cc:1434] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d: Time spent starting tablet: real 0.024s	user 0.003s	sys 0.003s
I20260709 13:06:44.117565 19941 raft_consensus.cc:359] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.138032 19941 raft_consensus.cc:385] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.138653 19941 raft_consensus.cc:740] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf68973149544eae8929005c48bb84b0, State: Initialized, Role: FOLLOWER
I20260709 13:06:44.139191 19944 tablet_bootstrap.cc:492] T 2348879030be45d49634020eace141f0 P 7cc15333145b485ea8cdd6c1623ff0f8: No bootstrap required, opened a new log
I20260709 13:06:44.139783 19944 ts_tablet_manager.cc:1403] T 2348879030be45d49634020eace141f0 P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 13:06:44.139919 19941 consensus_queue.cc:260] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.142773 19944 raft_consensus.cc:359] T 2348879030be45d49634020eace141f0 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.143639 19944 raft_consensus.cc:385] T 2348879030be45d49634020eace141f0 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.143946 19944 raft_consensus.cc:740] T 2348879030be45d49634020eace141f0 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:44.144946 19944 consensus_queue.cc:260] T 2348879030be45d49634020eace141f0 P 7cc15333145b485ea8cdd6c1623ff0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.148319 19942 tablet_bootstrap.cc:492] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d: Bootstrap starting.
I20260709 13:06:44.149302 19944 ts_tablet_manager.cc:1434] T 2348879030be45d49634020eace141f0 P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:06:44.150025 19902 raft_consensus.cc:2468] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 1.
I20260709 13:06:44.150449 19944 tablet_bootstrap.cc:492] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8: Bootstrap starting.
I20260709 13:06:44.157163 19944 tablet_bootstrap.cc:654] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:44.157685 19950 raft_consensus.cc:697] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Becoming Leader. State: Replica: cf68973149544eae8929005c48bb84b0, State: Running, Role: LEADER
I20260709 13:06:44.157161 19942 tablet_bootstrap.cc:654] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:44.158821 19950 consensus_queue.cc:237] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.160770 19941 ts_tablet_manager.cc:1434] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0: Time spent starting tablet: real 0.047s	user 0.011s	sys 0.004s
I20260709 13:06:44.161911 19946 raft_consensus.cc:493] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.163153 19941 tablet_bootstrap.cc:492] T 8b5517dc489142e19aee3610500ee38e P cf68973149544eae8929005c48bb84b0: Bootstrap starting.
I20260709 13:06:44.162631 19946 raft_consensus.cc:515] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.167239 19946 leader_election.cc:290] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553)
I20260709 13:06:44.172228 19941 tablet_bootstrap.cc:654] T 8b5517dc489142e19aee3610500ee38e P cf68973149544eae8929005c48bb84b0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:44.176375 19944 tablet_bootstrap.cc:492] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8: No bootstrap required, opened a new log
I20260709 13:06:44.177281 19944 ts_tablet_manager.cc:1403] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.006s
I20260709 13:06:44.180543 19944 raft_consensus.cc:359] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.181316 19942 tablet_bootstrap.cc:492] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d: No bootstrap required, opened a new log
I20260709 13:06:44.182029 19942 ts_tablet_manager.cc:1403] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.007s
I20260709 13:06:44.185664 19942 raft_consensus.cc:359] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.186444 19942 raft_consensus.cc:385] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.186753 19942 raft_consensus.cc:740] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Initialized, Role: FOLLOWER
I20260709 13:06:44.187613 19942 consensus_queue.cc:260] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.181293 19944 raft_consensus.cc:385] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.189155 19944 raft_consensus.cc:740] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:44.189213 19941 tablet_bootstrap.cc:492] T 8b5517dc489142e19aee3610500ee38e P cf68973149544eae8929005c48bb84b0: No bootstrap required, opened a new log
I20260709 13:06:44.190039 19941 ts_tablet_manager.cc:1403] T 8b5517dc489142e19aee3610500ee38e P cf68973149544eae8929005c48bb84b0: Time spent bootstrapping tablet: real 0.027s	user 0.016s	sys 0.005s
I20260709 13:06:44.191012 19944 consensus_queue.cc:260] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.193921 19942 ts_tablet_manager.cc:1434] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:06:44.193847 19941 raft_consensus.cc:359] T 8b5517dc489142e19aee3610500ee38e P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.194898 19941 raft_consensus.cc:385] T 8b5517dc489142e19aee3610500ee38e P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.195209 19942 tablet_bootstrap.cc:492] T bd4bd6f0d29248aab51d9a8bdcd054a5 P e5b2e431c3204ee487c10f58d440707d: Bootstrap starting.
I20260709 13:06:44.199069 19648 catalog_manager.cc:5697] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf68973149544eae8929005c48bb84b0 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "cf68973149544eae8929005c48bb84b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:44.195277 19941 raft_consensus.cc:740] T 8b5517dc489142e19aee3610500ee38e P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf68973149544eae8929005c48bb84b0, State: Initialized, Role: FOLLOWER
I20260709 13:06:44.202325 19941 consensus_queue.cc:260] T 8b5517dc489142e19aee3610500ee38e P cf68973149544eae8929005c48bb84b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.203541 19944 ts_tablet_manager.cc:1434] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 13:06:44.204782 19941 ts_tablet_manager.cc:1434] T 8b5517dc489142e19aee3610500ee38e P cf68973149544eae8929005c48bb84b0: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:06:44.205932 19941 tablet_bootstrap.cc:492] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0: Bootstrap starting.
I20260709 13:06:44.210734 19828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2348879030be45d49634020eace141f0" candidate_uuid: "e5b2e431c3204ee487c10f58d440707d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf68973149544eae8929005c48bb84b0" is_pre_election: true
I20260709 13:06:44.211671 19944 tablet_bootstrap.cc:492] T 409f3b40dbc2457d8803d0d244488b59 P 7cc15333145b485ea8cdd6c1623ff0f8: Bootstrap starting.
I20260709 13:06:44.212531 19941 tablet_bootstrap.cc:654] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:44.214265 19699 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:06:44.215394 19867 leader_election.cc:343] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:44.216853 19942 tablet_bootstrap.cc:654] T bd4bd6f0d29248aab51d9a8bdcd054a5 P e5b2e431c3204ee487c10f58d440707d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:44.218753 19753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2348879030be45d49634020eace141f0" candidate_uuid: "e5b2e431c3204ee487c10f58d440707d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" is_pre_election: true
I20260709 13:06:44.219620 19753 raft_consensus.cc:2468] T 2348879030be45d49634020eace141f0 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5b2e431c3204ee487c10f58d440707d in term 0.
I20260709 13:06:44.221170 19867 leader_election.cc:304] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, e5b2e431c3204ee487c10f58d440707d; no voters: cf68973149544eae8929005c48bb84b0
I20260709 13:06:44.222507 19946 raft_consensus.cc:2804] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:44.223049 19946 raft_consensus.cc:493] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:44.223536 19946 raft_consensus.cc:3060] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.227504 19944 tablet_bootstrap.cc:654] T 409f3b40dbc2457d8803d0d244488b59 P 7cc15333145b485ea8cdd6c1623ff0f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:44.232543 19942 tablet_bootstrap.cc:492] T bd4bd6f0d29248aab51d9a8bdcd054a5 P e5b2e431c3204ee487c10f58d440707d: No bootstrap required, opened a new log
I20260709 13:06:44.233415 19942 ts_tablet_manager.cc:1403] T bd4bd6f0d29248aab51d9a8bdcd054a5 P e5b2e431c3204ee487c10f58d440707d: Time spent bootstrapping tablet: real 0.038s	user 0.013s	sys 0.008s
I20260709 13:06:44.237074 19941 tablet_bootstrap.cc:492] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0: No bootstrap required, opened a new log
I20260709 13:06:44.236809 19942 raft_consensus.cc:359] T bd4bd6f0d29248aab51d9a8bdcd054a5 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.237962 19941 ts_tablet_manager.cc:1403] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.011s
I20260709 13:06:44.237906 19942 raft_consensus.cc:385] T bd4bd6f0d29248aab51d9a8bdcd054a5 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.238734 19942 raft_consensus.cc:740] T bd4bd6f0d29248aab51d9a8bdcd054a5 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Initialized, Role: FOLLOWER
I20260709 13:06:44.240711 19942 consensus_queue.cc:260] T bd4bd6f0d29248aab51d9a8bdcd054a5 P e5b2e431c3204ee487c10f58d440707d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.245805 19942 ts_tablet_manager.cc:1434] T bd4bd6f0d29248aab51d9a8bdcd054a5 P e5b2e431c3204ee487c10f58d440707d: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.004s
I20260709 13:06:44.245644 19946 raft_consensus.cc:515] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.247083 19942 tablet_bootstrap.cc:492] T 45faf2520a3e4d44bbfd74574b972b70 P e5b2e431c3204ee487c10f58d440707d: Bootstrap starting.
I20260709 13:06:44.247280 19941 raft_consensus.cc:359] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.247892 19946 leader_election.cc:290] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 1 election: Requested vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553)
I20260709 13:06:44.248145 19941 raft_consensus.cc:385] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.248561 19941 raft_consensus.cc:740] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf68973149544eae8929005c48bb84b0, State: Initialized, Role: FOLLOWER
I20260709 13:06:44.249370 19828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2348879030be45d49634020eace141f0" candidate_uuid: "e5b2e431c3204ee487c10f58d440707d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf68973149544eae8929005c48bb84b0"
I20260709 13:06:44.249617 19944 tablet_bootstrap.cc:492] T 409f3b40dbc2457d8803d0d244488b59 P 7cc15333145b485ea8cdd6c1623ff0f8: No bootstrap required, opened a new log
I20260709 13:06:44.250265 19944 ts_tablet_manager.cc:1403] T 409f3b40dbc2457d8803d0d244488b59 P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent bootstrapping tablet: real 0.039s	user 0.015s	sys 0.003s
I20260709 13:06:44.250633 19753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2348879030be45d49634020eace141f0" candidate_uuid: "e5b2e431c3204ee487c10f58d440707d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
I20260709 13:06:44.251266 19753 raft_consensus.cc:3060] T 2348879030be45d49634020eace141f0 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.253742 19944 raft_consensus.cc:359] T 409f3b40dbc2457d8803d0d244488b59 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.254603 19944 raft_consensus.cc:385] T 409f3b40dbc2457d8803d0d244488b59 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.254750 19950 raft_consensus.cc:493] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.254884 19942 tablet_bootstrap.cc:654] T 45faf2520a3e4d44bbfd74574b972b70 P e5b2e431c3204ee487c10f58d440707d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:44.255012 19944 raft_consensus.cc:740] T 409f3b40dbc2457d8803d0d244488b59 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Initialized, Role: FOLLOWER
W20260709 13:06:44.255515 19867 leader_election.cc:343] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:44.256307 19950 raft_consensus.cc:515] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.256193 19944 consensus_queue.cc:260] T 409f3b40dbc2457d8803d0d244488b59 P 7cc15333145b485ea8cdd6c1623ff0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.261935 19753 raft_consensus.cc:2468] T 2348879030be45d49634020eace141f0 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5b2e431c3204ee487c10f58d440707d in term 1.
I20260709 13:06:44.264118 19867 leader_election.cc:304] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, e5b2e431c3204ee487c10f58d440707d; no voters: cf68973149544eae8929005c48bb84b0
I20260709 13:06:44.264675 19948 raft_consensus.cc:493] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.249402 19941 consensus_queue.cc:260] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.265478 19948 raft_consensus.cc:515] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.266494 19946 raft_consensus.cc:2804] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:44.267071 19946 raft_consensus.cc:697] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [term 1 LEADER]: Becoming Leader. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Running, Role: LEADER
I20260709 13:06:44.269493 19948 leader_election.cc:290] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:44.267939 19946 consensus_queue.cc:237] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.270535 19944 ts_tablet_manager.cc:1434] T 409f3b40dbc2457d8803d0d244488b59 P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.002s
I20260709 13:06:44.271324 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4bd6f0d29248aab51d9a8bdcd054a5" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d" is_pre_election: true
I20260709 13:06:44.272044 19902 raft_consensus.cc:2468] T bd4bd6f0d29248aab51d9a8bdcd054a5 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 0.
I20260709 13:06:44.272396 19944 tablet_bootstrap.cc:492] T 46feb6ece88946fea8e12e1375c1e94c P 7cc15333145b485ea8cdd6c1623ff0f8: Bootstrap starting.
I20260709 13:06:44.273546 19793 leader_election.cc:304] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf68973149544eae8929005c48bb84b0, e5b2e431c3204ee487c10f58d440707d; no voters: 
I20260709 13:06:44.273998 19753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4bd6f0d29248aab51d9a8bdcd054a5" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" is_pre_election: true
I20260709 13:06:44.274609 19959 raft_consensus.cc:2804] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:44.275714 19959 raft_consensus.cc:493] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:44.276161 19959 raft_consensus.cc:3060] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.275877 19753 raft_consensus.cc:2468] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 0.
I20260709 13:06:44.267999 19941 ts_tablet_manager.cc:1434] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.002s
I20260709 13:06:44.280197 19944 tablet_bootstrap.cc:654] T 46feb6ece88946fea8e12e1375c1e94c P 7cc15333145b485ea8cdd6c1623ff0f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:44.280385 19941 tablet_bootstrap.cc:492] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0: Bootstrap starting.
I20260709 13:06:44.269125 19950 leader_election.cc:290] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:44.286398 19959 raft_consensus.cc:515] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.289628 19753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4bd6f0d29248aab51d9a8bdcd054a5" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
I20260709 13:06:44.290297 19753 raft_consensus.cc:3060] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.297713 19941 tablet_bootstrap.cc:654] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:44.299894 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4bd6f0d29248aab51d9a8bdcd054a5" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:44.300848 19902 raft_consensus.cc:3060] T bd4bd6f0d29248aab51d9a8bdcd054a5 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.304690 19650 catalog_manager.cc:5697] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d reported cstate change: term changed from 0 to 1, leader changed from <none> to e5b2e431c3204ee487c10f58d440707d (127.6.255.3). New cstate: current_term: 1 leader_uuid: "e5b2e431c3204ee487c10f58d440707d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:44.307360 19942 tablet_bootstrap.cc:492] T 45faf2520a3e4d44bbfd74574b972b70 P e5b2e431c3204ee487c10f58d440707d: No bootstrap required, opened a new log
I20260709 13:06:44.307981 19942 ts_tablet_manager.cc:1403] T 45faf2520a3e4d44bbfd74574b972b70 P e5b2e431c3204ee487c10f58d440707d: Time spent bootstrapping tablet: real 0.061s	user 0.012s	sys 0.011s
I20260709 13:06:44.311803 19946 raft_consensus.cc:493] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.312413 19946 raft_consensus.cc:515] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } }
I20260709 13:06:44.314858 19946 leader_election.cc:290] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553), 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037)
I20260709 13:06:44.316624 19828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa0028f6eae14e1186d8464c0c12b33f" candidate_uuid: "e5b2e431c3204ee487c10f58d440707d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf68973149544eae8929005c48bb84b0" is_pre_election: true
I20260709 13:06:44.318066 19902 raft_consensus.cc:2468] T bd4bd6f0d29248aab51d9a8bdcd054a5 P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 1.
W20260709 13:06:44.319046 19867 leader_election.cc:343] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:44.319530 19793 leader_election.cc:304] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf68973149544eae8929005c48bb84b0, e5b2e431c3204ee487c10f58d440707d; no voters: 
I20260709 13:06:44.320562 19950 raft_consensus.cc:2804] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:44.320681 19959 leader_election.cc:290] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 election: Requested vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:44.321297 19950 raft_consensus.cc:697] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Becoming Leader. State: Replica: cf68973149544eae8929005c48bb84b0, State: Running, Role: LEADER
I20260709 13:06:44.322466 19950 consensus_queue.cc:237] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.327466 19942 raft_consensus.cc:359] T 45faf2520a3e4d44bbfd74574b972b70 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.328740 19942 raft_consensus.cc:385] T 45faf2520a3e4d44bbfd74574b972b70 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.332231 19753 raft_consensus.cc:2468] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 1.
I20260709 13:06:44.329160 19942 raft_consensus.cc:740] T 45faf2520a3e4d44bbfd74574b972b70 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Initialized, Role: FOLLOWER
I20260709 13:06:44.336953 19942 consensus_queue.cc:260] T 45faf2520a3e4d44bbfd74574b972b70 P e5b2e431c3204ee487c10f58d440707d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.339797 19944 tablet_bootstrap.cc:492] T 46feb6ece88946fea8e12e1375c1e94c P 7cc15333145b485ea8cdd6c1623ff0f8: No bootstrap required, opened a new log
I20260709 13:06:44.339730 19752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa0028f6eae14e1186d8464c0c12b33f" candidate_uuid: "e5b2e431c3204ee487c10f58d440707d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" is_pre_election: true
I20260709 13:06:44.340626 19944 ts_tablet_manager.cc:1403] T 46feb6ece88946fea8e12e1375c1e94c P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent bootstrapping tablet: real 0.068s	user 0.022s	sys 0.011s
I20260709 13:06:44.340873 19752 raft_consensus.cc:2468] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5b2e431c3204ee487c10f58d440707d in term 0.
I20260709 13:06:44.342936 19942 ts_tablet_manager.cc:1434] T 45faf2520a3e4d44bbfd74574b972b70 P e5b2e431c3204ee487c10f58d440707d: Time spent starting tablet: real 0.035s	user 0.011s	sys 0.000s
I20260709 13:06:44.344044 19942 tablet_bootstrap.cc:492] T 46feb6ece88946fea8e12e1375c1e94c P e5b2e431c3204ee487c10f58d440707d: Bootstrap starting.
I20260709 13:06:44.343704 19944 raft_consensus.cc:359] T 46feb6ece88946fea8e12e1375c1e94c P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.344404 19944 raft_consensus.cc:385] T 46feb6ece88946fea8e12e1375c1e94c P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.344668 19944 raft_consensus.cc:740] T 46feb6ece88946fea8e12e1375c1e94c P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:44.345337 19944 consensus_queue.cc:260] T 46feb6ece88946fea8e12e1375c1e94c P 7cc15333145b485ea8cdd6c1623ff0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.347743 19944 ts_tablet_manager.cc:1434] T 46feb6ece88946fea8e12e1375c1e94c P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:44.348778 19944 tablet_bootstrap.cc:492] T 59a482b1ce05463ea3ef901049d7c1e2 P 7cc15333145b485ea8cdd6c1623ff0f8: Bootstrap starting.
I20260709 13:06:44.350972 19942 tablet_bootstrap.cc:654] T 46feb6ece88946fea8e12e1375c1e94c P e5b2e431c3204ee487c10f58d440707d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:44.351438 19867 leader_election.cc:304] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, e5b2e431c3204ee487c10f58d440707d; no voters: cf68973149544eae8929005c48bb84b0
I20260709 13:06:44.352751 19946 raft_consensus.cc:2804] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:44.353134 19946 raft_consensus.cc:493] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:44.353474 19946 raft_consensus.cc:3060] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.354704 19944 tablet_bootstrap.cc:654] T 59a482b1ce05463ea3ef901049d7c1e2 P 7cc15333145b485ea8cdd6c1623ff0f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:44.362463 19946 raft_consensus.cc:515] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } }
I20260709 13:06:44.364121 19942 tablet_bootstrap.cc:492] T 46feb6ece88946fea8e12e1375c1e94c P e5b2e431c3204ee487c10f58d440707d: No bootstrap required, opened a new log
I20260709 13:06:44.364825 19942 ts_tablet_manager.cc:1403] T 46feb6ece88946fea8e12e1375c1e94c P e5b2e431c3204ee487c10f58d440707d: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.011s
I20260709 13:06:44.365237 19944 tablet_bootstrap.cc:492] T 59a482b1ce05463ea3ef901049d7c1e2 P 7cc15333145b485ea8cdd6c1623ff0f8: No bootstrap required, opened a new log
I20260709 13:06:44.365885 19828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa0028f6eae14e1186d8464c0c12b33f" candidate_uuid: "e5b2e431c3204ee487c10f58d440707d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf68973149544eae8929005c48bb84b0"
I20260709 13:06:44.365911 19944 ts_tablet_manager.cc:1403] T 59a482b1ce05463ea3ef901049d7c1e2 P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.002s
W20260709 13:06:44.367599 19867 leader_election.cc:343] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:44.367686 19828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4bd6f0d29248aab51d9a8bdcd054a5" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf68973149544eae8929005c48bb84b0" is_pre_election: true
I20260709 13:06:44.367956 19942 raft_consensus.cc:359] T 46feb6ece88946fea8e12e1375c1e94c P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.368688 19942 raft_consensus.cc:385] T 46feb6ece88946fea8e12e1375c1e94c P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.369026 19942 raft_consensus.cc:740] T 46feb6ece88946fea8e12e1375c1e94c P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Initialized, Role: FOLLOWER
I20260709 13:06:44.370008 19946 leader_election.cc:290] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 1 election: Requested vote from peers cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553), 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037)
I20260709 13:06:44.370674 19944 raft_consensus.cc:359] T 59a482b1ce05463ea3ef901049d7c1e2 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.370822 19942 consensus_queue.cc:260] T 46feb6ece88946fea8e12e1375c1e94c P e5b2e431c3204ee487c10f58d440707d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.373411 19942 ts_tablet_manager.cc:1434] T 46feb6ece88946fea8e12e1375c1e94c P e5b2e431c3204ee487c10f58d440707d: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.006s
I20260709 13:06:44.374549 19942 tablet_bootstrap.cc:492] T 8b5517dc489142e19aee3610500ee38e P e5b2e431c3204ee487c10f58d440707d: Bootstrap starting.
I20260709 13:06:44.374994 19648 catalog_manager.cc:5697] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf68973149544eae8929005c48bb84b0 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "cf68973149544eae8929005c48bb84b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:44.381104 19942 tablet_bootstrap.cc:654] T 8b5517dc489142e19aee3610500ee38e P e5b2e431c3204ee487c10f58d440707d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:44.381851 19752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa0028f6eae14e1186d8464c0c12b33f" candidate_uuid: "e5b2e431c3204ee487c10f58d440707d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
I20260709 13:06:44.382489 19752 raft_consensus.cc:3060] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.385157 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4bd6f0d29248aab51d9a8bdcd054a5" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d" is_pre_election: true
I20260709 13:06:44.386029 19902 raft_consensus.cc:2393] T bd4bd6f0d29248aab51d9a8bdcd054a5 P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7cc15333145b485ea8cdd6c1623ff0f8 in current term 1: Already voted for candidate cf68973149544eae8929005c48bb84b0 in this term.
I20260709 13:06:44.387256 19718 leader_election.cc:304] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8; no voters: cf68973149544eae8929005c48bb84b0, e5b2e431c3204ee487c10f58d440707d
I20260709 13:06:44.371461 19944 raft_consensus.cc:385] T 59a482b1ce05463ea3ef901049d7c1e2 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.388516 19948 raft_consensus.cc:2749] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:44.388921 19942 tablet_bootstrap.cc:492] T 8b5517dc489142e19aee3610500ee38e P e5b2e431c3204ee487c10f58d440707d: No bootstrap required, opened a new log
I20260709 13:06:44.389467 19942 ts_tablet_manager.cc:1403] T 8b5517dc489142e19aee3610500ee38e P e5b2e431c3204ee487c10f58d440707d: Time spent bootstrapping tablet: real 0.015s	user 0.006s	sys 0.005s
I20260709 13:06:44.390403 19941 tablet_bootstrap.cc:492] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0: No bootstrap required, opened a new log
I20260709 13:06:44.390964 19941 ts_tablet_manager.cc:1403] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0: Time spent bootstrapping tablet: real 0.111s	user 0.021s	sys 0.020s
I20260709 13:06:44.391516 19752 raft_consensus.cc:2468] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5b2e431c3204ee487c10f58d440707d in term 1.
I20260709 13:06:44.392431 19942 raft_consensus.cc:359] T 8b5517dc489142e19aee3610500ee38e P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.388509 19944 raft_consensus.cc:740] T 59a482b1ce05463ea3ef901049d7c1e2 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:44.393203 19942 raft_consensus.cc:385] T 8b5517dc489142e19aee3610500ee38e P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.392720 19867 leader_election.cc:304] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, e5b2e431c3204ee487c10f58d440707d; no voters: cf68973149544eae8929005c48bb84b0
I20260709 13:06:44.393635 19942 raft_consensus.cc:740] T 8b5517dc489142e19aee3610500ee38e P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Initialized, Role: FOLLOWER
I20260709 13:06:44.393806 19944 consensus_queue.cc:260] T 59a482b1ce05463ea3ef901049d7c1e2 P 7cc15333145b485ea8cdd6c1623ff0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.394223 19941 raft_consensus.cc:359] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.394982 19941 raft_consensus.cc:385] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.394531 19942 consensus_queue.cc:260] T 8b5517dc489142e19aee3610500ee38e P e5b2e431c3204ee487c10f58d440707d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.395313 19941 raft_consensus.cc:740] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf68973149544eae8929005c48bb84b0, State: Initialized, Role: FOLLOWER
I20260709 13:06:44.396127 19944 ts_tablet_manager.cc:1434] T 59a482b1ce05463ea3ef901049d7c1e2 P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.001s
I20260709 13:06:44.396196 19941 consensus_queue.cc:260] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.394505 19946 raft_consensus.cc:2804] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:44.397769 19946 raft_consensus.cc:697] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [term 1 LEADER]: Becoming Leader. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Running, Role: LEADER
I20260709 13:06:44.396848 19942 ts_tablet_manager.cc:1434] T 8b5517dc489142e19aee3610500ee38e P e5b2e431c3204ee487c10f58d440707d: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:06:44.398588 19946 consensus_queue.cc:237] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } }
I20260709 13:06:44.403119 19941 ts_tablet_manager.cc:1434] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:06:44.404366 19941 tablet_bootstrap.cc:492] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0: Bootstrap starting.
I20260709 13:06:44.404868 19942 tablet_bootstrap.cc:492] T 59a482b1ce05463ea3ef901049d7c1e2 P e5b2e431c3204ee487c10f58d440707d: Bootstrap starting.
I20260709 13:06:44.410128 19941 tablet_bootstrap.cc:654] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:44.411000 19942 tablet_bootstrap.cc:654] T 59a482b1ce05463ea3ef901049d7c1e2 P e5b2e431c3204ee487c10f58d440707d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:44.419795 19941 tablet_bootstrap.cc:492] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0: No bootstrap required, opened a new log
I20260709 13:06:44.420387 19941 ts_tablet_manager.cc:1403] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.003s
I20260709 13:06:44.420475 19942 tablet_bootstrap.cc:492] T 59a482b1ce05463ea3ef901049d7c1e2 P e5b2e431c3204ee487c10f58d440707d: No bootstrap required, opened a new log
I20260709 13:06:44.423316 19942 ts_tablet_manager.cc:1403] T 59a482b1ce05463ea3ef901049d7c1e2 P e5b2e431c3204ee487c10f58d440707d: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.004s
I20260709 13:06:44.423700 19941 raft_consensus.cc:359] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.424624 19941 raft_consensus.cc:385] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.424968 19941 raft_consensus.cc:740] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf68973149544eae8929005c48bb84b0, State: Initialized, Role: FOLLOWER
I20260709 13:06:44.425771 19941 consensus_queue.cc:260] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.426265 19942 raft_consensus.cc:359] T 59a482b1ce05463ea3ef901049d7c1e2 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.426991 19942 raft_consensus.cc:385] T 59a482b1ce05463ea3ef901049d7c1e2 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.427342 19942 raft_consensus.cc:740] T 59a482b1ce05463ea3ef901049d7c1e2 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Initialized, Role: FOLLOWER
I20260709 13:06:44.426765 19648 catalog_manager.cc:5697] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d reported cstate change: term changed from 0 to 1, leader changed from <none> to e5b2e431c3204ee487c10f58d440707d (127.6.255.3). New cstate: current_term: 1 leader_uuid: "e5b2e431c3204ee487c10f58d440707d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:44.427983 19942 consensus_queue.cc:260] T 59a482b1ce05463ea3ef901049d7c1e2 P e5b2e431c3204ee487c10f58d440707d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.428465 19941 ts_tablet_manager.cc:1434] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:06:44.429656 19941 tablet_bootstrap.cc:492] T aa0028f6eae14e1186d8464c0c12b33f P cf68973149544eae8929005c48bb84b0: Bootstrap starting.
I20260709 13:06:44.430183 19942 ts_tablet_manager.cc:1434] T 59a482b1ce05463ea3ef901049d7c1e2 P e5b2e431c3204ee487c10f58d440707d: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:06:44.436561 19941 tablet_bootstrap.cc:654] T aa0028f6eae14e1186d8464c0c12b33f P cf68973149544eae8929005c48bb84b0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:44.441574 19950 raft_consensus.cc:493] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.441807 19941 tablet_bootstrap.cc:492] T aa0028f6eae14e1186d8464c0c12b33f P cf68973149544eae8929005c48bb84b0: No bootstrap required, opened a new log
I20260709 13:06:44.442337 19941 ts_tablet_manager.cc:1403] T aa0028f6eae14e1186d8464c0c12b33f P cf68973149544eae8929005c48bb84b0: Time spent bootstrapping tablet: real 0.013s	user 0.012s	sys 0.000s
I20260709 13:06:44.442148 19950 raft_consensus.cc:515] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.444386 19950 leader_election.cc:290] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:44.444635 19941 raft_consensus.cc:359] T aa0028f6eae14e1186d8464c0c12b33f P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } }
I20260709 13:06:44.445401 19941 raft_consensus.cc:385] T aa0028f6eae14e1186d8464c0c12b33f P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.445560 19752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46feb6ece88946fea8e12e1375c1e94c" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" is_pre_election: true
I20260709 13:06:44.445734 19941 raft_consensus.cc:740] T aa0028f6eae14e1186d8464c0c12b33f P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf68973149544eae8929005c48bb84b0, State: Initialized, Role: FOLLOWER
I20260709 13:06:44.446007 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46feb6ece88946fea8e12e1375c1e94c" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d" is_pre_election: true
I20260709 13:06:44.446270 19752 raft_consensus.cc:2468] T 46feb6ece88946fea8e12e1375c1e94c P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 0.
I20260709 13:06:44.446677 19902 raft_consensus.cc:2468] T 46feb6ece88946fea8e12e1375c1e94c P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 0.
I20260709 13:06:44.446784 19950 raft_consensus.cc:493] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.446545 19941 consensus_queue.cc:260] T aa0028f6eae14e1186d8464c0c12b33f P cf68973149544eae8929005c48bb84b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } }
I20260709 13:06:44.447379 19950 raft_consensus.cc:515] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.448808 19941 ts_tablet_manager.cc:1434] T aa0028f6eae14e1186d8464c0c12b33f P cf68973149544eae8929005c48bb84b0: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.002s
I20260709 13:06:44.448669 19793 leader_election.cc:304] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: 
I20260709 13:06:44.449461 19950 leader_election.cc:290] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:44.449882 19941 tablet_bootstrap.cc:492] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0: Bootstrap starting.
I20260709 13:06:44.449745 19961 raft_consensus.cc:2804] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:44.450343 19961 raft_consensus.cc:493] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:44.450700 19961 raft_consensus.cc:3060] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.452502 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45faf2520a3e4d44bbfd74574b972b70" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d" is_pre_election: true
I20260709 13:06:44.453184 19902 raft_consensus.cc:2468] T 45faf2520a3e4d44bbfd74574b972b70 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 0.
I20260709 13:06:44.454619 19752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45faf2520a3e4d44bbfd74574b972b70" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" is_pre_election: true
I20260709 13:06:44.455272 19752 raft_consensus.cc:2468] T 45faf2520a3e4d44bbfd74574b972b70 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 0.
I20260709 13:06:44.456261 19941 tablet_bootstrap.cc:654] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:44.456581 19793 leader_election.cc:304] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: 
I20260709 13:06:44.458057 19950 raft_consensus.cc:2804] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:44.458511 19950 raft_consensus.cc:493] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:44.458280 19961 raft_consensus.cc:515] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.459002 19950 raft_consensus.cc:3060] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.461294 19752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46feb6ece88946fea8e12e1375c1e94c" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
I20260709 13:06:44.461925 19752 raft_consensus.cc:3060] T 46feb6ece88946fea8e12e1375c1e94c P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.465703 19961 leader_election.cc:290] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 election: Requested vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:44.466782 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46feb6ece88946fea8e12e1375c1e94c" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:44.467356 19902 raft_consensus.cc:3060] T 46feb6ece88946fea8e12e1375c1e94c P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.468729 19941 tablet_bootstrap.cc:492] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0: No bootstrap required, opened a new log
I20260709 13:06:44.469240 19941 ts_tablet_manager.cc:1403] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0: Time spent bootstrapping tablet: real 0.020s	user 0.004s	sys 0.013s
I20260709 13:06:44.471915 19941 raft_consensus.cc:359] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.472663 19941 raft_consensus.cc:385] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.472114 19950 raft_consensus.cc:515] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.472994 19941 raft_consensus.cc:740] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf68973149544eae8929005c48bb84b0, State: Initialized, Role: FOLLOWER
I20260709 13:06:44.474231 19941 consensus_queue.cc:260] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.474588 19902 raft_consensus.cc:2468] T 46feb6ece88946fea8e12e1375c1e94c P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 1.
I20260709 13:06:44.475714 19950 leader_election.cc:290] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 election: Requested vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:44.476207 19753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45faf2520a3e4d44bbfd74574b972b70" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
I20260709 13:06:44.476605 19793 leader_election.cc:304] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf68973149544eae8929005c48bb84b0, e5b2e431c3204ee487c10f58d440707d; no voters: 
I20260709 13:06:44.476873 19753 raft_consensus.cc:3060] T 45faf2520a3e4d44bbfd74574b972b70 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.477945 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45faf2520a3e4d44bbfd74574b972b70" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:44.478521 19902 raft_consensus.cc:3060] T 45faf2520a3e4d44bbfd74574b972b70 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.483733 19941 ts_tablet_manager.cc:1434] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.001s
I20260709 13:06:44.484402 19753 raft_consensus.cc:2468] T 45faf2520a3e4d44bbfd74574b972b70 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 1.
I20260709 13:06:44.484970 19950 raft_consensus.cc:2804] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:44.484427 19752 raft_consensus.cc:2468] T 46feb6ece88946fea8e12e1375c1e94c P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 1.
I20260709 13:06:44.485757 19950 raft_consensus.cc:697] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Becoming Leader. State: Replica: cf68973149544eae8929005c48bb84b0, State: Running, Role: LEADER
I20260709 13:06:44.486093 19793 leader_election.cc:304] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: 
I20260709 13:06:44.487011 19961 raft_consensus.cc:2804] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:44.486781 19950 consensus_queue.cc:237] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.487540 19961 raft_consensus.cc:697] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Becoming Leader. State: Replica: cf68973149544eae8929005c48bb84b0, State: Running, Role: LEADER
I20260709 13:06:44.485742 19902 raft_consensus.cc:2468] T 45faf2520a3e4d44bbfd74574b972b70 P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 1.
I20260709 13:06:44.488411 19961 consensus_queue.cc:237] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.498430 19651 catalog_manager.cc:5697] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf68973149544eae8929005c48bb84b0 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "cf68973149544eae8929005c48bb84b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:44.499766 19651 catalog_manager.cc:5697] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf68973149544eae8929005c48bb84b0 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "cf68973149544eae8929005c48bb84b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:44.551501 19948 raft_consensus.cc:493] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.552176 19948 raft_consensus.cc:515] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.554184 19948 leader_election.cc:290] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:44.554976 19828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5517dc489142e19aee3610500ee38e" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf68973149544eae8929005c48bb84b0" is_pre_election: true
I20260709 13:06:44.555150 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5517dc489142e19aee3610500ee38e" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d" is_pre_election: true
I20260709 13:06:44.555662 19828 raft_consensus.cc:2468] T 8b5517dc489142e19aee3610500ee38e P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 0.
I20260709 13:06:44.555845 19902 raft_consensus.cc:2468] T 8b5517dc489142e19aee3610500ee38e P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 0.
I20260709 13:06:44.556167 19961 raft_consensus.cc:493] T 8b5517dc489142e19aee3610500ee38e P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.556607 19961 raft_consensus.cc:515] T 8b5517dc489142e19aee3610500ee38e P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.556794 19718 leader_election.cc:304] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: 
I20260709 13:06:44.557734 19948 raft_consensus.cc:2804] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:44.558095 19948 raft_consensus.cc:493] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:44.558429 19948 raft_consensus.cc:3060] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.558748 19961 leader_election.cc:290] T 8b5517dc489142e19aee3610500ee38e P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:44.560060 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5517dc489142e19aee3610500ee38e" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d" is_pre_election: true
I20260709 13:06:44.560184 19752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5517dc489142e19aee3610500ee38e" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" is_pre_election: true
I20260709 13:06:44.560724 19902 raft_consensus.cc:2468] T 8b5517dc489142e19aee3610500ee38e P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 0.
I20260709 13:06:44.561848 19793 leader_election.cc:304] T 8b5517dc489142e19aee3610500ee38e P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf68973149544eae8929005c48bb84b0, e5b2e431c3204ee487c10f58d440707d; no voters: 
I20260709 13:06:44.562554 19961 raft_consensus.cc:2804] T 8b5517dc489142e19aee3610500ee38e P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:44.562923 19961 raft_consensus.cc:493] T 8b5517dc489142e19aee3610500ee38e P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:44.563251 19961 raft_consensus.cc:3060] T 8b5517dc489142e19aee3610500ee38e P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.566083 19948 raft_consensus.cc:515] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.567306 19752 raft_consensus.cc:2393] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf68973149544eae8929005c48bb84b0 in current term 1: Already voted for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in this term.
I20260709 13:06:44.567842 19950 raft_consensus.cc:493] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.568261 19948 leader_election.cc:290] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 election: Requested vote from peers cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:44.568384 19950 raft_consensus.cc:515] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.570132 19961 raft_consensus.cc:515] T 8b5517dc489142e19aee3610500ee38e P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.570587 19950 leader_election.cc:290] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:44.570997 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5517dc489142e19aee3610500ee38e" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:44.571420 19828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5517dc489142e19aee3610500ee38e" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf68973149544eae8929005c48bb84b0"
I20260709 13:06:44.572001 19961 leader_election.cc:290] T 8b5517dc489142e19aee3610500ee38e P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 election: Requested vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:44.572222 19828 raft_consensus.cc:2393] T 8b5517dc489142e19aee3610500ee38e P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7cc15333145b485ea8cdd6c1623ff0f8 in current term 1: Already voted for candidate cf68973149544eae8929005c48bb84b0 in this term.
I20260709 13:06:44.571583 19902 raft_consensus.cc:3060] T 8b5517dc489142e19aee3610500ee38e P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.572973 19753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "409f3b40dbc2457d8803d0d244488b59" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" is_pre_election: true
I20260709 13:06:44.573570 19753 raft_consensus.cc:2468] T 409f3b40dbc2457d8803d0d244488b59 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 0.
I20260709 13:06:44.573967 19751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5517dc489142e19aee3610500ee38e" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
I20260709 13:06:44.574815 19751 raft_consensus.cc:2393] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cf68973149544eae8929005c48bb84b0 in current term 1: Already voted for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in this term.
I20260709 13:06:44.574955 19793 leader_election.cc:304] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: 
I20260709 13:06:44.576122 19901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "409f3b40dbc2457d8803d0d244488b59" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d" is_pre_election: true
I20260709 13:06:44.576512 19961 raft_consensus.cc:2804] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:44.576601 19900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5517dc489142e19aee3610500ee38e" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:44.576840 19901 raft_consensus.cc:2468] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 0.
I20260709 13:06:44.577057 19961 raft_consensus.cc:493] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:44.577476 19961 raft_consensus.cc:3060] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.579137 19793 leader_election.cc:304] T 8b5517dc489142e19aee3610500ee38e P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf68973149544eae8929005c48bb84b0; no voters: 7cc15333145b485ea8cdd6c1623ff0f8, e5b2e431c3204ee487c10f58d440707d
I20260709 13:06:44.579998 19950 raft_consensus.cc:2749] T 8b5517dc489142e19aee3610500ee38e P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:44.580318 19902 raft_consensus.cc:2468] T 8b5517dc489142e19aee3610500ee38e P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 1.
I20260709 13:06:44.581466 19718 leader_election.cc:304] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, e5b2e431c3204ee487c10f58d440707d; no voters: cf68973149544eae8929005c48bb84b0
I20260709 13:06:44.582674 19948 raft_consensus.cc:2804] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:44.583695 19950 raft_consensus.cc:493] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.583787 19948 raft_consensus.cc:697] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Becoming Leader. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Running, Role: LEADER
I20260709 13:06:44.583916 19961 raft_consensus.cc:515] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.584663 19950 raft_consensus.cc:515] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.585177 19948 consensus_queue.cc:237] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.586638 19950 leader_election.cc:290] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:44.586645 19961 leader_election.cc:290] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 election: Requested vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:44.587442 19751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "409f3b40dbc2457d8803d0d244488b59" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
I20260709 13:06:44.588044 19751 raft_consensus.cc:3060] T 409f3b40dbc2457d8803d0d244488b59 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.588718 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a482b1ce05463ea3ef901049d7c1e2" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d" is_pre_election: true
I20260709 13:06:44.589375 19902 raft_consensus.cc:2468] T 59a482b1ce05463ea3ef901049d7c1e2 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 0.
I20260709 13:06:44.589630 19900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "409f3b40dbc2457d8803d0d244488b59" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:44.590219 19900 raft_consensus.cc:3060] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.590636 19793 leader_election.cc:304] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf68973149544eae8929005c48bb84b0, e5b2e431c3204ee487c10f58d440707d; no voters: 
I20260709 13:06:44.591533 19961 raft_consensus.cc:2804] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:44.591898 19961 raft_consensus.cc:493] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:44.592216 19961 raft_consensus.cc:3060] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.592939 19753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a482b1ce05463ea3ef901049d7c1e2" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" is_pre_election: true
I20260709 13:06:44.594134 19753 raft_consensus.cc:2468] T 59a482b1ce05463ea3ef901049d7c1e2 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 0.
I20260709 13:06:44.597937 19900 raft_consensus.cc:2468] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 1.
I20260709 13:06:44.599562 19793 leader_election.cc:304] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf68973149544eae8929005c48bb84b0, e5b2e431c3204ee487c10f58d440707d; no voters: 
I20260709 13:06:44.599686 19961 raft_consensus.cc:515] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.600466 19950 raft_consensus.cc:2804] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:44.600957 19950 raft_consensus.cc:697] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Becoming Leader. State: Replica: cf68973149544eae8929005c48bb84b0, State: Running, Role: LEADER
I20260709 13:06:44.602052 19961 leader_election.cc:290] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 election: Requested vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:44.601816 19950 consensus_queue.cc:237] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.602427 19751 raft_consensus.cc:2468] T 409f3b40dbc2457d8803d0d244488b59 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 1.
I20260709 13:06:44.607187 19751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a482b1ce05463ea3ef901049d7c1e2" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
I20260709 13:06:44.607801 19751 raft_consensus.cc:3060] T 59a482b1ce05463ea3ef901049d7c1e2 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.608449 19900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a482b1ce05463ea3ef901049d7c1e2" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:44.609681 19900 raft_consensus.cc:3060] T 59a482b1ce05463ea3ef901049d7c1e2 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.611083 19651 catalog_manager.cc:5697] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf68973149544eae8929005c48bb84b0 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "cf68973149544eae8929005c48bb84b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:44.614342 19648 catalog_manager.cc:5697] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:44.616108 19900 raft_consensus.cc:2468] T 59a482b1ce05463ea3ef901049d7c1e2 P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 1.
I20260709 13:06:44.617471 19793 leader_election.cc:304] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf68973149544eae8929005c48bb84b0, e5b2e431c3204ee487c10f58d440707d; no voters: 
I20260709 13:06:44.618394 19950 raft_consensus.cc:2804] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:44.618844 19950 raft_consensus.cc:697] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Becoming Leader. State: Replica: cf68973149544eae8929005c48bb84b0, State: Running, Role: LEADER
I20260709 13:06:44.620129 19751 raft_consensus.cc:2468] T 59a482b1ce05463ea3ef901049d7c1e2 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 1.
I20260709 13:06:44.619669 19950 consensus_queue.cc:237] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.628134 19651 catalog_manager.cc:5697] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf68973149544eae8929005c48bb84b0 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "cf68973149544eae8929005c48bb84b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:44.665409 19950 consensus_queue.cc:1048] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:44.678740 19961 consensus_queue.cc:1048] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:44.706911 19900 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aa0028f6eae14e1186d8464c0c12b33f"
dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
mode: GRACEFUL
new_leader_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
 from {username='slave'} at 127.0.0.1:40138
I20260709 13:06:44.707530 19900 raft_consensus.cc:606] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [term 1 LEADER]: Received request to transfer leadership to TS 7cc15333145b485ea8cdd6c1623ff0f8
I20260709 13:06:44.721606 19828 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34322023741e46f69328b109a5b54158"
dest_uuid: "cf68973149544eae8929005c48bb84b0"
mode: GRACEFUL
new_leader_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
 from {username='slave'} at 127.0.0.1:54272
I20260709 13:06:44.722206 19828 raft_consensus.cc:606] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Received request to transfer leadership to TS 7cc15333145b485ea8cdd6c1623ff0f8
I20260709 13:06:44.731568 19828 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd4bd6f0d29248aab51d9a8bdcd054a5"
dest_uuid: "cf68973149544eae8929005c48bb84b0"
mode: GRACEFUL
new_leader_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
 from {username='slave'} at 127.0.0.1:54272
I20260709 13:06:44.732148 19828 raft_consensus.cc:606] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Received request to transfer leadership to TS 7cc15333145b485ea8cdd6c1623ff0f8
I20260709 13:06:44.734413 19961 consensus_queue.cc:1048] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:44.742378 19950 raft_consensus.cc:993] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0: : Instructing follower 7cc15333145b485ea8cdd6c1623ff0f8 to start an election
I20260709 13:06:44.742880 19950 raft_consensus.cc:1081] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Signalling peer 7cc15333145b485ea8cdd6c1623ff0f8 to start an election
I20260709 13:06:44.744992 19753 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bd4bd6f0d29248aab51d9a8bdcd054a5"
dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
 from {username='slave'} at 127.0.0.1:41402
I20260709 13:06:44.746306 19900 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2348879030be45d49634020eace141f0"
dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
mode: GRACEFUL
new_leader_uuid: "cf68973149544eae8929005c48bb84b0"
 from {username='slave'} at 127.0.0.1:40138
I20260709 13:06:44.746822 19753 raft_consensus.cc:493] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:44.746956 19900 raft_consensus.cc:606] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [term 1 LEADER]: Received request to transfer leadership to TS cf68973149544eae8929005c48bb84b0
I20260709 13:06:44.747287 19753 raft_consensus.cc:3060] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.754752 19753 raft_consensus.cc:515] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.755028 19961 consensus_queue.cc:1048] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:44.758771 19828 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "409f3b40dbc2457d8803d0d244488b59"
dest_uuid: "cf68973149544eae8929005c48bb84b0"
mode: GRACEFUL
new_leader_uuid: "e5b2e431c3204ee487c10f58d440707d"
 from {username='slave'} at 127.0.0.1:54272
I20260709 13:06:44.759423 19828 raft_consensus.cc:606] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Received request to transfer leadership to TS e5b2e431c3204ee487c10f58d440707d
I20260709 13:06:44.760687 19827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4bd6f0d29248aab51d9a8bdcd054a5" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cf68973149544eae8929005c48bb84b0"
I20260709 13:06:44.761536 19827 raft_consensus.cc:3055] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:44.761843 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4bd6f0d29248aab51d9a8bdcd054a5" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:44.761894 19827 raft_consensus.cc:740] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cf68973149544eae8929005c48bb84b0, State: Running, Role: LEADER
I20260709 13:06:44.764642 19827 consensus_queue.cc:260] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.766163 19827 raft_consensus.cc:3060] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.771199 19753 leader_election.cc:290] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 2 election: Requested vote from peers cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:44.772274 19827 raft_consensus.cc:2468] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 2.
I20260709 13:06:44.773628 19718 leader_election.cc:304] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: e5b2e431c3204ee487c10f58d440707d
I20260709 13:06:44.774334 19948 raft_consensus.cc:2804] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:44.774834 19948 raft_consensus.cc:697] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 LEADER]: Becoming Leader. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Running, Role: LEADER
I20260709 13:06:44.775625 19948 consensus_queue.cc:237] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.782124 19651 catalog_manager.cc:5697] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8 reported cstate change: term changed from 1 to 2, leader changed from cf68973149544eae8929005c48bb84b0 (127.6.255.2) to 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1). New cstate: current_term: 2 leader_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:44.800928 19961 raft_consensus.cc:493] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.801559 19961 raft_consensus.cc:515] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.803812 19961 leader_election.cc:290] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:44.803905 19959 raft_consensus.cc:493] T aa0028f6eae14e1186d8464c0c12b33f P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.804509 19959 raft_consensus.cc:515] T aa0028f6eae14e1186d8464c0c12b33f P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } }
I20260709 13:06:44.805068 19900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2348879030be45d49634020eace141f0" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d" is_pre_election: true
I20260709 13:06:44.805483 19751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2348879030be45d49634020eace141f0" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" is_pre_election: true
I20260709 13:06:44.805580 19946 consensus_queue.cc:1048] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:44.806645 19959 leader_election.cc:290] T aa0028f6eae14e1186d8464c0c12b33f P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743), 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037)
I20260709 13:06:44.806656 19751 raft_consensus.cc:2393] T 2348879030be45d49634020eace141f0 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf68973149544eae8929005c48bb84b0 in current term 1: Already voted for candidate e5b2e431c3204ee487c10f58d440707d in this term.
I20260709 13:06:44.807511 19900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa0028f6eae14e1186d8464c0c12b33f" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d" is_pre_election: true
I20260709 13:06:44.808575 19793 leader_election.cc:304] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf68973149544eae8929005c48bb84b0; no voters: 7cc15333145b485ea8cdd6c1623ff0f8, e5b2e431c3204ee487c10f58d440707d
I20260709 13:06:44.808872 19751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa0028f6eae14e1186d8464c0c12b33f" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" is_pre_election: true
I20260709 13:06:44.809728 19959 raft_consensus.cc:3060] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.810185 19751 raft_consensus.cc:2393] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf68973149544eae8929005c48bb84b0 in current term 1: Already voted for candidate e5b2e431c3204ee487c10f58d440707d in this term.
I20260709 13:06:44.811684 19793 leader_election.cc:304] T aa0028f6eae14e1186d8464c0c12b33f P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf68973149544eae8929005c48bb84b0; no voters: 7cc15333145b485ea8cdd6c1623ff0f8, e5b2e431c3204ee487c10f58d440707d
I20260709 13:06:44.813022 19961 raft_consensus.cc:3060] T aa0028f6eae14e1186d8464c0c12b33f P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.817548 19959 raft_consensus.cc:2749] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:44.819809 19971 consensus_queue.cc:1048] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:44.822074 19961 raft_consensus.cc:2749] T aa0028f6eae14e1186d8464c0c12b33f P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:44.830935 19946 raft_consensus.cc:993] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d: : Instructing follower cf68973149544eae8929005c48bb84b0 to start an election
I20260709 13:06:44.831486 19971 raft_consensus.cc:1081] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [term 1 LEADER]: Signalling peer cf68973149544eae8929005c48bb84b0 to start an election
I20260709 13:06:44.833182 19827 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2348879030be45d49634020eace141f0"
dest_uuid: "cf68973149544eae8929005c48bb84b0"
 from {username='slave'} at 127.0.0.1:54254
I20260709 13:06:44.833791 19827 raft_consensus.cc:493] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:44.834141 19827 raft_consensus.cc:3060] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.839185 19827 raft_consensus.cc:515] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.841149 19827 leader_election.cc:290] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 2 election: Requested vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:44.842170 19753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2348879030be45d49634020eace141f0" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
I20260709 13:06:44.842506 19900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2348879030be45d49634020eace141f0" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:44.842973 19753 raft_consensus.cc:3060] T 2348879030be45d49634020eace141f0 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.843153 19900 raft_consensus.cc:3055] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:44.843609 19900 raft_consensus.cc:740] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Running, Role: LEADER
I20260709 13:06:44.844537 19900 consensus_queue.cc:260] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.846004 19900 raft_consensus.cc:3060] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.851263 19753 raft_consensus.cc:2468] T 2348879030be45d49634020eace141f0 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 2.
I20260709 13:06:44.852821 19793 leader_election.cc:304] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: 
I20260709 13:06:44.853662 19961 raft_consensus.cc:2804] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:44.853876 19900 raft_consensus.cc:2468] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 2.
I20260709 13:06:44.854305 19961 raft_consensus.cc:697] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [term 2 LEADER]: Becoming Leader. State: Replica: cf68973149544eae8929005c48bb84b0, State: Running, Role: LEADER
I20260709 13:06:44.855283 19961 consensus_queue.cc:237] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:44.862320 19650 catalog_manager.cc:5697] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 reported cstate change: term changed from 1 to 2, leader changed from e5b2e431c3204ee487c10f58d440707d (127.6.255.3) to cf68973149544eae8929005c48bb84b0 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "cf68973149544eae8929005c48bb84b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:44.889618 19961 consensus_queue.cc:1048] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:44.901636 20000 consensus_queue.cc:1048] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:44.945199 19971 consensus_queue.cc:1048] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:44.956784 20003 consensus_queue.cc:1048] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:44.963961 20000 consensus_queue.cc:1048] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:44.964737 20003 raft_consensus.cc:993] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d: : Instructing follower 7cc15333145b485ea8cdd6c1623ff0f8 to start an election
I20260709 13:06:44.965293 20003 raft_consensus.cc:1081] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [term 1 LEADER]: Signalling peer 7cc15333145b485ea8cdd6c1623ff0f8 to start an election
I20260709 13:06:44.967569 19753 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aa0028f6eae14e1186d8464c0c12b33f"
dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
 from {username='slave'} at 127.0.0.1:41410
I20260709 13:06:44.968123 19753 raft_consensus.cc:493] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:44.968534 19753 raft_consensus.cc:3060] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.976158 19753 raft_consensus.cc:515] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } }
I20260709 13:06:44.977041 19950 consensus_queue.cc:1048] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:44.982329 19753 leader_election.cc:290] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 2 election: Requested vote from peers e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743), cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553)
I20260709 13:06:44.986153 19827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa0028f6eae14e1186d8464c0c12b33f" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cf68973149544eae8929005c48bb84b0"
I20260709 13:06:44.985400 19900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa0028f6eae14e1186d8464c0c12b33f" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:44.986927 19827 raft_consensus.cc:3060] T aa0028f6eae14e1186d8464c0c12b33f P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.987504 19900 raft_consensus.cc:3055] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:44.987887 19900 raft_consensus.cc:740] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Running, Role: LEADER
I20260709 13:06:44.988736 19900 consensus_queue.cc:260] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } }
I20260709 13:06:44.990077 19900 raft_consensus.cc:3060] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.996186 19827 raft_consensus.cc:2468] T aa0028f6eae14e1186d8464c0c12b33f P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 2.
I20260709 13:06:44.997462 19718 leader_election.cc:304] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: 
I20260709 13:06:44.998354 19948 raft_consensus.cc:2804] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:44.998842 19948 raft_consensus.cc:697] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 LEADER]: Becoming Leader. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Running, Role: LEADER
I20260709 13:06:44.999671 19948 consensus_queue.cc:237] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } }
I20260709 13:06:45.001621 19900 raft_consensus.cc:2468] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 2.
I20260709 13:06:45.007514 19651 catalog_manager.cc:5697] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8 reported cstate change: term changed from 1 to 2, leader changed from e5b2e431c3204ee487c10f58d440707d (127.6.255.3) to 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1). New cstate: current_term: 2 leader_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:45.023106 19948 consensus_queue.cc:1048] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:45.031284 19950 consensus_queue.cc:1048] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:45.043313 19948 consensus_queue.cc:1048] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:45.053942 19959 raft_consensus.cc:993] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0: : Instructing follower e5b2e431c3204ee487c10f58d440707d to start an election
I20260709 13:06:45.054497 19961 raft_consensus.cc:1081] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Signalling peer e5b2e431c3204ee487c10f58d440707d to start an election
I20260709 13:06:45.056275 19900 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "409f3b40dbc2457d8803d0d244488b59"
dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
 from {username='slave'} at 127.0.0.1:40114
I20260709 13:06:45.056859 19900 raft_consensus.cc:493] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:45.057262 19900 raft_consensus.cc:3060] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:45.060068 19961 consensus_queue.cc:1048] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:45.066357 19900 raft_consensus.cc:515] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:45.067637 19902 raft_consensus.cc:1240] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Rejecting Update request from peer cf68973149544eae8929005c48bb84b0 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:45.068902 19900 leader_election.cc:290] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 2 election: Requested vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553)
I20260709 13:06:45.071045 19751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "409f3b40dbc2457d8803d0d244488b59" candidate_uuid: "e5b2e431c3204ee487c10f58d440707d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
I20260709 13:06:45.071404 19827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "409f3b40dbc2457d8803d0d244488b59" candidate_uuid: "e5b2e431c3204ee487c10f58d440707d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cf68973149544eae8929005c48bb84b0"
I20260709 13:06:45.071740 19751 raft_consensus.cc:3060] T 409f3b40dbc2457d8803d0d244488b59 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:45.072180 19827 raft_consensus.cc:3055] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:45.072541 19827 raft_consensus.cc:740] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cf68973149544eae8929005c48bb84b0, State: Running, Role: LEADER
I20260709 13:06:45.073457 19827 consensus_queue.cc:260] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:45.074317 19961 consensus_queue.cc:1059] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:45.078514 19827 raft_consensus.cc:3060] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:45.079380 19751 raft_consensus.cc:2468] T 409f3b40dbc2457d8803d0d244488b59 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5b2e431c3204ee487c10f58d440707d in term 2.
I20260709 13:06:45.080709 19867 leader_election.cc:304] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, e5b2e431c3204ee487c10f58d440707d; no voters: 
I20260709 13:06:45.081669 19971 raft_consensus.cc:2804] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:45.082216 19971 raft_consensus.cc:697] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d [term 2 LEADER]: Becoming Leader. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Running, Role: LEADER
I20260709 13:06:45.083122 19971 consensus_queue.cc:237] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:45.086119 19827 raft_consensus.cc:2468] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5b2e431c3204ee487c10f58d440707d in term 2.
I20260709 13:06:45.091238 19650 catalog_manager.cc:5697] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d reported cstate change: term changed from 1 to 2, leader changed from cf68973149544eae8929005c48bb84b0 (127.6.255.2) to e5b2e431c3204ee487c10f58d440707d (127.6.255.3). New cstate: current_term: 2 leader_uuid: "e5b2e431c3204ee487c10f58d440707d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:45.109769 19961 consensus_queue.cc:1048] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:45.121853 19961 consensus_queue.cc:1048] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:45.305696 19827 raft_consensus.cc:1275] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Refusing update from remote peer 7cc15333145b485ea8cdd6c1623ff0f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:45.305788 19902 raft_consensus.cc:3060] T bd4bd6f0d29248aab51d9a8bdcd054a5 P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:45.307650 20012 consensus_queue.cc:1048] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:45.323100 19902 raft_consensus.cc:1275] T bd4bd6f0d29248aab51d9a8bdcd054a5 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Refusing update from remote peer 7cc15333145b485ea8cdd6c1623ff0f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:45.330925 19989 consensus_queue.cc:1048] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:06:45.387941 19959 raft_consensus.cc:993] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0: : Instructing follower 7cc15333145b485ea8cdd6c1623ff0f8 to start an election
I20260709 13:06:45.388550 19959 raft_consensus.cc:1081] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Signalling peer 7cc15333145b485ea8cdd6c1623ff0f8 to start an election
I20260709 13:06:45.390738 19751 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34322023741e46f69328b109a5b54158"
dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
 from {username='slave'} at 127.0.0.1:41402
I20260709 13:06:45.391214 19751 raft_consensus.cc:493] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:45.391507 19751 raft_consensus.cc:3060] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:45.401450 19751 raft_consensus.cc:515] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:45.406108 19827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34322023741e46f69328b109a5b54158" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cf68973149544eae8929005c48bb84b0"
I20260709 13:06:45.406363 19751 leader_election.cc:290] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 2 election: Requested vote from peers cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:45.406930 19827 raft_consensus.cc:3055] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:45.407363 19827 raft_consensus.cc:740] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cf68973149544eae8929005c48bb84b0, State: Running, Role: LEADER
I20260709 13:06:45.407541 19901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34322023741e46f69328b109a5b54158" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:45.408299 19901 raft_consensus.cc:3060] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:45.409265 19827 consensus_queue.cc:260] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:45.410897 19827 raft_consensus.cc:3060] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:45.415414 19900 raft_consensus.cc:1275] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Refusing update from remote peer cf68973149544eae8929005c48bb84b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:45.417137 19950 consensus_queue.cc:1048] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:45.420389 19827 raft_consensus.cc:2468] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 2.
I20260709 13:06:45.421627 19718 leader_election.cc:304] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: 
I20260709 13:06:45.422439 19948 raft_consensus.cc:2804] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:45.422931 19948 raft_consensus.cc:697] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 LEADER]: Becoming Leader. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Running, Role: LEADER
I20260709 13:06:45.423743 19948 consensus_queue.cc:237] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:45.433619 19751 raft_consensus.cc:1275] T 2348879030be45d49634020eace141f0 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Refusing update from remote peer cf68973149544eae8929005c48bb84b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:45.435084 19959 consensus_queue.cc:1048] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:45.438700 19901 raft_consensus.cc:2468] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 2.
I20260709 13:06:45.437005 19651 catalog_manager.cc:5697] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 reported cstate change: term changed from 1 to 2, leader changed from cf68973149544eae8929005c48bb84b0 (127.6.255.2) to 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1). New cstate: current_term: 2 leader_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:45.486336 19901 raft_consensus.cc:1275] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Refusing update from remote peer 7cc15333145b485ea8cdd6c1623ff0f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:45.487522 19948 consensus_queue.cc:1048] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:45.496376 19827 raft_consensus.cc:1275] T aa0028f6eae14e1186d8464c0c12b33f P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Refusing update from remote peer 7cc15333145b485ea8cdd6c1623ff0f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:45.498000 20012 consensus_queue.cc:1048] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:45.534660 19751 raft_consensus.cc:1275] T 409f3b40dbc2457d8803d0d244488b59 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Refusing update from remote peer e5b2e431c3204ee487c10f58d440707d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:45.536038 19971 consensus_queue.cc:1048] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:45.545147 19828 raft_consensus.cc:1275] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Refusing update from remote peer e5b2e431c3204ee487c10f58d440707d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:45.547387 19971 consensus_queue.cc:1048] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:45.819303 19828 raft_consensus.cc:1275] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Refusing update from remote peer 7cc15333145b485ea8cdd6c1623ff0f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:45.820812 20012 consensus_queue.cc:1048] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:45.831591 19902 raft_consensus.cc:1275] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Refusing update from remote peer 7cc15333145b485ea8cdd6c1623ff0f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:45.833264 19948 consensus_queue.cc:1048] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:06:46.333240 19849 debug-util.cc:398] Leaking SignalData structure 0x7b0800245080 after lost signal to thread 7167
W20260709 13:06:46.334465 19849 debug-util.cc:398] Leaking SignalData structure 0x7b0800387a60 after lost signal to thread 19685
W20260709 13:06:46.335798 19849 debug-util.cc:398] Leaking SignalData structure 0x7b08003076a0 after lost signal to thread 19777
W20260709 13:06:46.336869 19849 debug-util.cc:398] Leaking SignalData structure 0x7b080011aac0 after lost signal to thread 19852
W20260709 13:06:46.337967 19849 debug-util.cc:398] Leaking SignalData structure 0x7b0800233560 after lost signal to thread 19926
I20260709 13:06:47.919099 19648 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50070:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:47.923202 19648 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:47.990371 19738 tablet_service.cc:1511] Processing CreateTablet for tablet 021f87e2ac864b89a9563513db8a389d (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:47.991645 19738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 021f87e2ac864b89a9563513db8a389d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.992906 19812 tablet_service.cc:1511] Processing CreateTablet for tablet 021f87e2ac864b89a9563513db8a389d (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:47.994271 19812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 021f87e2ac864b89a9563513db8a389d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.998453 19739 tablet_service.cc:1511] Processing CreateTablet for tablet d50ac34758d044dbbf7ec338fff5a9ac (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:47.999884 19739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d50ac34758d044dbbf7ec338fff5a9ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.002372 19887 tablet_service.cc:1511] Processing CreateTablet for tablet 021f87e2ac864b89a9563513db8a389d (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:48.003736 19887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 021f87e2ac864b89a9563513db8a389d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.007069 19818 tablet_service.cc:1511] Processing CreateTablet for tablet d50ac34758d044dbbf7ec338fff5a9ac (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:48.008596 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d50ac34758d044dbbf7ec338fff5a9ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.018016 19890 tablet_service.cc:1511] Processing CreateTablet for tablet d50ac34758d044dbbf7ec338fff5a9ac (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:48.018802 19888 tablet_service.cc:1511] Processing CreateTablet for tablet edb7db72b50647c09980ff9b8cb188be (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:48.019718 19890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d50ac34758d044dbbf7ec338fff5a9ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.021162 19885 tablet_service.cc:1511] Processing CreateTablet for tablet 191b9b12c85b43db99997d1560cd18da (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:48.021610 19817 tablet_service.cc:1511] Processing CreateTablet for tablet 191b9b12c85b43db99997d1560cd18da (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:48.022725 19885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 191b9b12c85b43db99997d1560cd18da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.022532 19735 tablet_service.cc:1511] Processing CreateTablet for tablet edb7db72b50647c09980ff9b8cb188be (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:48.023999 19735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edb7db72b50647c09980ff9b8cb188be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.019745 19737 tablet_service.cc:1511] Processing CreateTablet for tablet 53bdb8e606e240dea3740473550cc59a (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:48.026594 19741 tablet_service.cc:1511] Processing CreateTablet for tablet 191b9b12c85b43db99997d1560cd18da (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:48.027086 19737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53bdb8e606e240dea3740473550cc59a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.028021 19741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 191b9b12c85b43db99997d1560cd18da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.023056 19817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 191b9b12c85b43db99997d1560cd18da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.030628 19813 tablet_service.cc:1511] Processing CreateTablet for tablet edb7db72b50647c09980ff9b8cb188be (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:48.032181 19813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edb7db72b50647c09980ff9b8cb188be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.033987 19888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edb7db72b50647c09980ff9b8cb188be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.075999 19810 tablet_service.cc:1511] Processing CreateTablet for tablet 53bdb8e606e240dea3740473550cc59a (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:48.079054 19810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53bdb8e606e240dea3740473550cc59a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.097015 19889 tablet_service.cc:1511] Processing CreateTablet for tablet 53bdb8e606e240dea3740473550cc59a (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:48.098655 19889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53bdb8e606e240dea3740473550cc59a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.103935 19816 tablet_service.cc:1511] Processing CreateTablet for tablet 2a9a23c67df64d0793abcb340fe4e985 (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:48.104678 19884 tablet_service.cc:1511] Processing CreateTablet for tablet 2a9a23c67df64d0793abcb340fe4e985 (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:48.106155 19884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a9a23c67df64d0793abcb340fe4e985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.106273 19738 tablet_service.cc:1511] Processing CreateTablet for tablet 2a9a23c67df64d0793abcb340fe4e985 (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:48.107859 19738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a9a23c67df64d0793abcb340fe4e985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.109774 19816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a9a23c67df64d0793abcb340fe4e985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.117563 19739 tablet_service.cc:1511] Processing CreateTablet for tablet 54491751f68f4ee5b5b9000c225b906e (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:48.120857 19814 tablet_service.cc:1511] Processing CreateTablet for tablet 54491751f68f4ee5b5b9000c225b906e (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:48.121307 19886 tablet_service.cc:1511] Processing CreateTablet for tablet 54491751f68f4ee5b5b9000c225b906e (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:48.123698 19886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54491751f68f4ee5b5b9000c225b906e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.126336 19814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54491751f68f4ee5b5b9000c225b906e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.127133 20033 tablet_bootstrap.cc:492] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8: Bootstrap starting.
I20260709 13:06:48.136198 19739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54491751f68f4ee5b5b9000c225b906e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.150707 19741 tablet_service.cc:1511] Processing CreateTablet for tablet bad2e092b8464dca8a6a73f54bf5f934 (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:48.152240 19741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bad2e092b8464dca8a6a73f54bf5f934. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.160197 20033 tablet_bootstrap.cc:654] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.169852 19812 tablet_service.cc:1511] Processing CreateTablet for tablet bad2e092b8464dca8a6a73f54bf5f934 (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:48.171716 19812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bad2e092b8464dca8a6a73f54bf5f934. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.174438 19887 tablet_service.cc:1511] Processing CreateTablet for tablet bad2e092b8464dca8a6a73f54bf5f934 (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:48.194279 19887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bad2e092b8464dca8a6a73f54bf5f934. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.190271 19818 tablet_service.cc:1511] Processing CreateTablet for tablet 5a6ea8c60bdb40fc902a427e79c155f8 (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:48.192415 19737 tablet_service.cc:1511] Processing CreateTablet for tablet 5a6ea8c60bdb40fc902a427e79c155f8 (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:48.200423 19737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a6ea8c60bdb40fc902a427e79c155f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.201403 19888 tablet_service.cc:1511] Processing CreateTablet for tablet 5a6ea8c60bdb40fc902a427e79c155f8 (DEFAULT_TABLE table=soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:48.202530 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a6ea8c60bdb40fc902a427e79c155f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.202745 19888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a6ea8c60bdb40fc902a427e79c155f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.230887 20036 tablet_bootstrap.cc:492] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d: Bootstrap starting.
I20260709 13:06:48.231341 20034 tablet_bootstrap.cc:492] T edb7db72b50647c09980ff9b8cb188be P cf68973149544eae8929005c48bb84b0: Bootstrap starting.
I20260709 13:06:48.238035 20036 tablet_bootstrap.cc:654] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.240054 20033 tablet_bootstrap.cc:492] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8: No bootstrap required, opened a new log
I20260709 13:06:48.240672 20033 ts_tablet_manager.cc:1403] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent bootstrapping tablet: real 0.114s	user 0.010s	sys 0.021s
I20260709 13:06:48.244719 20033 raft_consensus.cc:359] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.245510 20033 raft_consensus.cc:385] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.245839 20033 raft_consensus.cc:740] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.246585 20033 consensus_queue.cc:260] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.248884 20033 ts_tablet_manager.cc:1434] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:06:48.249902 20033 tablet_bootstrap.cc:492] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8: Bootstrap starting.
I20260709 13:06:48.256456 20033 tablet_bootstrap.cc:654] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.262041 20034 tablet_bootstrap.cc:654] T edb7db72b50647c09980ff9b8cb188be P cf68973149544eae8929005c48bb84b0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.281643 19989 raft_consensus.cc:493] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.283301 19989 raft_consensus.cc:515] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.285202 20036 tablet_bootstrap.cc:492] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d: No bootstrap required, opened a new log
I20260709 13:06:48.285454 19989 leader_election.cc:290] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:48.285943 20036 ts_tablet_manager.cc:1403] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d: Time spent bootstrapping tablet: real 0.055s	user 0.017s	sys 0.011s
I20260709 13:06:48.286486 20034 tablet_bootstrap.cc:492] T edb7db72b50647c09980ff9b8cb188be P cf68973149544eae8929005c48bb84b0: No bootstrap required, opened a new log
I20260709 13:06:48.287092 20034 ts_tablet_manager.cc:1403] T edb7db72b50647c09980ff9b8cb188be P cf68973149544eae8929005c48bb84b0: Time spent bootstrapping tablet: real 0.056s	user 0.013s	sys 0.000s
I20260709 13:06:48.287902 19827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "021f87e2ac864b89a9563513db8a389d" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf68973149544eae8929005c48bb84b0" is_pre_election: true
I20260709 13:06:48.289301 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "021f87e2ac864b89a9563513db8a389d" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d" is_pre_election: true
W20260709 13:06:48.290920 19718 leader_election.cc:343] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:48.291685 19718 leader_election.cc:343] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.292105 19718 leader_election.cc:304] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8; no voters: cf68973149544eae8929005c48bb84b0, e5b2e431c3204ee487c10f58d440707d
I20260709 13:06:48.292347 20033 tablet_bootstrap.cc:492] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8: No bootstrap required, opened a new log
I20260709 13:06:48.292904 19989 raft_consensus.cc:2749] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:48.290452 20034 raft_consensus.cc:359] T edb7db72b50647c09980ff9b8cb188be P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.289346 20036 raft_consensus.cc:359] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.294122 20034 raft_consensus.cc:385] T edb7db72b50647c09980ff9b8cb188be P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.293733 20033 ts_tablet_manager.cc:1403] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent bootstrapping tablet: real 0.044s	user 0.010s	sys 0.005s
I20260709 13:06:48.294375 20036 raft_consensus.cc:385] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.294546 20034 raft_consensus.cc:740] T edb7db72b50647c09980ff9b8cb188be P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf68973149544eae8929005c48bb84b0, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.294808 20036 raft_consensus.cc:740] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.295624 20034 consensus_queue.cc:260] T edb7db72b50647c09980ff9b8cb188be P cf68973149544eae8929005c48bb84b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.297135 20033 raft_consensus.cc:359] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.297916 20033 raft_consensus.cc:385] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.298192 20033 raft_consensus.cc:740] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.298893 20034 ts_tablet_manager.cc:1434] T edb7db72b50647c09980ff9b8cb188be P cf68973149544eae8929005c48bb84b0: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:06:48.298935 20033 consensus_queue.cc:260] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.300309 20034 tablet_bootstrap.cc:492] T 191b9b12c85b43db99997d1560cd18da P cf68973149544eae8929005c48bb84b0: Bootstrap starting.
I20260709 13:06:48.302688 20036 consensus_queue.cc:260] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.308765 20034 tablet_bootstrap.cc:654] T 191b9b12c85b43db99997d1560cd18da P cf68973149544eae8929005c48bb84b0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.310974 20036 ts_tablet_manager.cc:1434] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.003s
I20260709 13:06:48.312256 20036 tablet_bootstrap.cc:492] T 021f87e2ac864b89a9563513db8a389d P e5b2e431c3204ee487c10f58d440707d: Bootstrap starting.
I20260709 13:06:48.308759 20033 ts_tablet_manager.cc:1434] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent starting tablet: real 0.014s	user 0.001s	sys 0.005s
I20260709 13:06:48.325655 20033 tablet_bootstrap.cc:492] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8: Bootstrap starting.
I20260709 13:06:48.326390 20036 tablet_bootstrap.cc:654] T 021f87e2ac864b89a9563513db8a389d P e5b2e431c3204ee487c10f58d440707d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.327622 20034 tablet_bootstrap.cc:492] T 191b9b12c85b43db99997d1560cd18da P cf68973149544eae8929005c48bb84b0: No bootstrap required, opened a new log
I20260709 13:06:48.328156 20034 ts_tablet_manager.cc:1403] T 191b9b12c85b43db99997d1560cd18da P cf68973149544eae8929005c48bb84b0: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.009s
I20260709 13:06:48.331357 20034 raft_consensus.cc:359] T 191b9b12c85b43db99997d1560cd18da P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.332229 20034 raft_consensus.cc:385] T 191b9b12c85b43db99997d1560cd18da P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.332579 20034 raft_consensus.cc:740] T 191b9b12c85b43db99997d1560cd18da P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf68973149544eae8929005c48bb84b0, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.333333 20033 tablet_bootstrap.cc:654] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.333578 20034 consensus_queue.cc:260] T 191b9b12c85b43db99997d1560cd18da P cf68973149544eae8929005c48bb84b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.340384 20034 ts_tablet_manager.cc:1434] T 191b9b12c85b43db99997d1560cd18da P cf68973149544eae8929005c48bb84b0: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:06:48.341492 20034 tablet_bootstrap.cc:492] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0: Bootstrap starting.
I20260709 13:06:48.346035 20035 raft_consensus.cc:493] T edb7db72b50647c09980ff9b8cb188be P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.346849 20035 raft_consensus.cc:515] T edb7db72b50647c09980ff9b8cb188be P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.348578 20034 tablet_bootstrap.cc:654] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.349247 20035 leader_election.cc:290] T edb7db72b50647c09980ff9b8cb188be P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:48.351274 20036 tablet_bootstrap.cc:492] T 021f87e2ac864b89a9563513db8a389d P e5b2e431c3204ee487c10f58d440707d: No bootstrap required, opened a new log
I20260709 13:06:48.351832 20036 ts_tablet_manager.cc:1403] T 021f87e2ac864b89a9563513db8a389d P e5b2e431c3204ee487c10f58d440707d: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.015s
I20260709 13:06:48.352195 19751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb7db72b50647c09980ff9b8cb188be" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" is_pre_election: true
I20260709 13:06:48.354216 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb7db72b50647c09980ff9b8cb188be" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d" is_pre_election: true
I20260709 13:06:48.355262 20036 raft_consensus.cc:359] T 021f87e2ac864b89a9563513db8a389d P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.356158 20036 raft_consensus.cc:385] T 021f87e2ac864b89a9563513db8a389d P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.356484 20036 raft_consensus.cc:740] T 021f87e2ac864b89a9563513db8a389d P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.357215 20036 consensus_queue.cc:260] T 021f87e2ac864b89a9563513db8a389d P e5b2e431c3204ee487c10f58d440707d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
W20260709 13:06:48.361109 19699 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:06:48.361696 19793 leader_election.cc:343] T edb7db72b50647c09980ff9b8cb188be P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:48.362826 19793 leader_election.cc:343] T edb7db72b50647c09980ff9b8cb188be P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.362077 20036 ts_tablet_manager.cc:1434] T 021f87e2ac864b89a9563513db8a389d P e5b2e431c3204ee487c10f58d440707d: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:06:48.363232 19793 leader_election.cc:304] T edb7db72b50647c09980ff9b8cb188be P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf68973149544eae8929005c48bb84b0; no voters: 7cc15333145b485ea8cdd6c1623ff0f8, e5b2e431c3204ee487c10f58d440707d
I20260709 13:06:48.364058 19959 raft_consensus.cc:2749] T edb7db72b50647c09980ff9b8cb188be P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:48.365412 20034 tablet_bootstrap.cc:492] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0: No bootstrap required, opened a new log
I20260709 13:06:48.365947 20036 tablet_bootstrap.cc:492] T 53bdb8e606e240dea3740473550cc59a P e5b2e431c3204ee487c10f58d440707d: Bootstrap starting.
I20260709 13:06:48.366267 20034 ts_tablet_manager.cc:1403] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.004s
I20260709 13:06:48.369984 20034 raft_consensus.cc:359] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.370893 20034 raft_consensus.cc:385] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.371276 20034 raft_consensus.cc:740] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf68973149544eae8929005c48bb84b0, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.372233 20034 consensus_queue.cc:260] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.375113 20034 ts_tablet_manager.cc:1434] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:06:48.376209 20036 tablet_bootstrap.cc:654] T 53bdb8e606e240dea3740473550cc59a P e5b2e431c3204ee487c10f58d440707d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.376318 20034 tablet_bootstrap.cc:492] T d50ac34758d044dbbf7ec338fff5a9ac P cf68973149544eae8929005c48bb84b0: Bootstrap starting.
I20260709 13:06:48.384455 20034 tablet_bootstrap.cc:654] T d50ac34758d044dbbf7ec338fff5a9ac P cf68973149544eae8929005c48bb84b0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.346153 20033 tablet_bootstrap.cc:492] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8: No bootstrap required, opened a new log
I20260709 13:06:48.389982 20033 ts_tablet_manager.cc:1403] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent bootstrapping tablet: real 0.065s	user 0.019s	sys 0.002s
I20260709 13:06:48.393483 20033 raft_consensus.cc:359] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.394369 20033 raft_consensus.cc:385] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.394774 20033 raft_consensus.cc:740] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.396014 20033 consensus_queue.cc:260] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.399154 20033 ts_tablet_manager.cc:1434] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:06:48.402710 20033 tablet_bootstrap.cc:492] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8: Bootstrap starting.
I20260709 13:06:48.413113 20034 tablet_bootstrap.cc:492] T d50ac34758d044dbbf7ec338fff5a9ac P cf68973149544eae8929005c48bb84b0: No bootstrap required, opened a new log
I20260709 13:06:48.413785 20034 ts_tablet_manager.cc:1403] T d50ac34758d044dbbf7ec338fff5a9ac P cf68973149544eae8929005c48bb84b0: Time spent bootstrapping tablet: real 0.038s	user 0.006s	sys 0.014s
I20260709 13:06:48.415728 20033 tablet_bootstrap.cc:654] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.422909 20036 tablet_bootstrap.cc:492] T 53bdb8e606e240dea3740473550cc59a P e5b2e431c3204ee487c10f58d440707d: No bootstrap required, opened a new log
I20260709 13:06:48.423592 20036 ts_tablet_manager.cc:1403] T 53bdb8e606e240dea3740473550cc59a P e5b2e431c3204ee487c10f58d440707d: Time spent bootstrapping tablet: real 0.058s	user 0.028s	sys 0.010s
I20260709 13:06:48.423815 20034 raft_consensus.cc:359] T d50ac34758d044dbbf7ec338fff5a9ac P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.424819 20034 raft_consensus.cc:385] T d50ac34758d044dbbf7ec338fff5a9ac P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.425204 20034 raft_consensus.cc:740] T d50ac34758d044dbbf7ec338fff5a9ac P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf68973149544eae8929005c48bb84b0, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.426136 20034 consensus_queue.cc:260] T d50ac34758d044dbbf7ec338fff5a9ac P cf68973149544eae8929005c48bb84b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.428776 20034 ts_tablet_manager.cc:1434] T d50ac34758d044dbbf7ec338fff5a9ac P cf68973149544eae8929005c48bb84b0: Time spent starting tablet: real 0.015s	user 0.000s	sys 0.007s
I20260709 13:06:48.430027 20034 tablet_bootstrap.cc:492] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0: Bootstrap starting.
I20260709 13:06:48.431684 20036 raft_consensus.cc:359] T 53bdb8e606e240dea3740473550cc59a P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.432904 20036 raft_consensus.cc:385] T 53bdb8e606e240dea3740473550cc59a P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.433249 20036 raft_consensus.cc:740] T 53bdb8e606e240dea3740473550cc59a P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.434268 20036 consensus_queue.cc:260] T 53bdb8e606e240dea3740473550cc59a P e5b2e431c3204ee487c10f58d440707d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.438225 20034 tablet_bootstrap.cc:654] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.441208 20036 ts_tablet_manager.cc:1434] T 53bdb8e606e240dea3740473550cc59a P e5b2e431c3204ee487c10f58d440707d: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 13:06:48.442306 20036 tablet_bootstrap.cc:492] T 191b9b12c85b43db99997d1560cd18da P e5b2e431c3204ee487c10f58d440707d: Bootstrap starting.
I20260709 13:06:48.447739 20033 tablet_bootstrap.cc:492] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8: No bootstrap required, opened a new log
I20260709 13:06:48.448514 20033 ts_tablet_manager.cc:1403] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent bootstrapping tablet: real 0.046s	user 0.014s	sys 0.010s
I20260709 13:06:48.449045 20036 tablet_bootstrap.cc:654] T 191b9b12c85b43db99997d1560cd18da P e5b2e431c3204ee487c10f58d440707d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.452159 20033 raft_consensus.cc:359] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.453219 20033 raft_consensus.cc:385] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.453642 20033 raft_consensus.cc:740] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.454566 20033 consensus_queue.cc:260] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.457255 20033 ts_tablet_manager.cc:1434] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:06:48.459069 20033 tablet_bootstrap.cc:492] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8: Bootstrap starting.
I20260709 13:06:48.471457 19989 raft_consensus.cc:493] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.472018 19989 raft_consensus.cc:515] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.474284 19989 leader_election.cc:290] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:48.476380 19827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53bdb8e606e240dea3740473550cc59a" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf68973149544eae8929005c48bb84b0" is_pre_election: true
I20260709 13:06:48.478297 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53bdb8e606e240dea3740473550cc59a" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d" is_pre_election: true
I20260709 13:06:48.482488 19902 raft_consensus.cc:2468] T 53bdb8e606e240dea3740473550cc59a P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 0.
W20260709 13:06:48.483486 19718 leader_election.cc:343] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.499277 19718 leader_election.cc:304] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, e5b2e431c3204ee487c10f58d440707d; no voters: cf68973149544eae8929005c48bb84b0
I20260709 13:06:48.500885 19989 raft_consensus.cc:2804] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:48.501394 19989 raft_consensus.cc:493] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.501850 19989 raft_consensus.cc:3060] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.502776 20033 tablet_bootstrap.cc:654] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.505554 20034 tablet_bootstrap.cc:492] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0: No bootstrap required, opened a new log
I20260709 13:06:48.506299 20034 ts_tablet_manager.cc:1403] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0: Time spent bootstrapping tablet: real 0.077s	user 0.031s	sys 0.013s
I20260709 13:06:48.506824 20045 raft_consensus.cc:493] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.507447 20045 raft_consensus.cc:515] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.508435 20036 tablet_bootstrap.cc:492] T 191b9b12c85b43db99997d1560cd18da P e5b2e431c3204ee487c10f58d440707d: No bootstrap required, opened a new log
I20260709 13:06:48.509004 20036 ts_tablet_manager.cc:1403] T 191b9b12c85b43db99997d1560cd18da P e5b2e431c3204ee487c10f58d440707d: Time spent bootstrapping tablet: real 0.067s	user 0.034s	sys 0.010s
I20260709 13:06:48.509836 20045 leader_election.cc:290] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:48.511471 19827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191b9b12c85b43db99997d1560cd18da" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf68973149544eae8929005c48bb84b0" is_pre_election: true
I20260709 13:06:48.514124 19827 raft_consensus.cc:2468] T 191b9b12c85b43db99997d1560cd18da P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 0.
I20260709 13:06:48.510948 20034 raft_consensus.cc:359] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.515024 20034 raft_consensus.cc:385] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.515434 20034 raft_consensus.cc:740] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf68973149544eae8929005c48bb84b0, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.515678 19718 leader_election.cc:304] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: 
I20260709 13:06:48.513141 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191b9b12c85b43db99997d1560cd18da" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d" is_pre_election: true
I20260709 13:06:48.516155 20034 consensus_queue.cc:260] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.516659 20045 raft_consensus.cc:2804] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:48.518909 20034 ts_tablet_manager.cc:1434] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 13:06:48.519923 20034 tablet_bootstrap.cc:492] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0: Bootstrap starting.
I20260709 13:06:48.522079 19989 raft_consensus.cc:515] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.517160 20045 raft_consensus.cc:493] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.523272 20045 raft_consensus.cc:3060] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.526573 20034 tablet_bootstrap.cc:654] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.529112 19827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53bdb8e606e240dea3740473550cc59a" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf68973149544eae8929005c48bb84b0"
I20260709 13:06:48.529848 19827 raft_consensus.cc:3060] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.515101 20033 tablet_bootstrap.cc:492] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8: No bootstrap required, opened a new log
I20260709 13:06:48.537281 20033 ts_tablet_manager.cc:1403] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent bootstrapping tablet: real 0.078s	user 0.018s	sys 0.002s
I20260709 13:06:48.541887 20033 raft_consensus.cc:359] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.546514 20033 raft_consensus.cc:385] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.547104 20033 raft_consensus.cc:740] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.548216 20033 consensus_queue.cc:260] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.512439 20036 raft_consensus.cc:359] T 191b9b12c85b43db99997d1560cd18da P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.549731 20036 raft_consensus.cc:385] T 191b9b12c85b43db99997d1560cd18da P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.550006 19901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53bdb8e606e240dea3740473550cc59a" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:48.550231 20036 raft_consensus.cc:740] T 191b9b12c85b43db99997d1560cd18da P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.550884 19901 raft_consensus.cc:3060] T 53bdb8e606e240dea3740473550cc59a P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.551874 20036 consensus_queue.cc:260] T 191b9b12c85b43db99997d1560cd18da P e5b2e431c3204ee487c10f58d440707d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.553627 19902 raft_consensus.cc:2468] T 191b9b12c85b43db99997d1560cd18da P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 0.
I20260709 13:06:48.559161 20036 ts_tablet_manager.cc:1434] T 191b9b12c85b43db99997d1560cd18da P e5b2e431c3204ee487c10f58d440707d: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.002s
I20260709 13:06:48.562980 20036 tablet_bootstrap.cc:492] T edb7db72b50647c09980ff9b8cb188be P e5b2e431c3204ee487c10f58d440707d: Bootstrap starting.
I20260709 13:06:48.545400 20045 raft_consensus.cc:515] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.566644 19828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191b9b12c85b43db99997d1560cd18da" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf68973149544eae8929005c48bb84b0"
I20260709 13:06:48.567502 19828 raft_consensus.cc:3060] T 191b9b12c85b43db99997d1560cd18da P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.577033 19827 raft_consensus.cc:2468] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 1.
I20260709 13:06:48.580173 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191b9b12c85b43db99997d1560cd18da" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:48.581218 19902 raft_consensus.cc:3060] T 191b9b12c85b43db99997d1560cd18da P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.589578 19989 leader_election.cc:290] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 election: Requested vote from peers cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:48.592252 20045 leader_election.cc:290] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 election: Requested vote from peers cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:48.613886 19901 raft_consensus.cc:2468] T 53bdb8e606e240dea3740473550cc59a P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 1.
I20260709 13:06:48.617444 19828 raft_consensus.cc:2468] T 191b9b12c85b43db99997d1560cd18da P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 1.
I20260709 13:06:48.614488 19718 leader_election.cc:304] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: 
I20260709 13:06:48.620018 19902 raft_consensus.cc:2468] T 191b9b12c85b43db99997d1560cd18da P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 1.
I20260709 13:06:48.620747 19718 leader_election.cc:304] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: 
I20260709 13:06:48.622682 20045 raft_consensus.cc:2804] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:48.623387 20045 raft_consensus.cc:697] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Becoming Leader. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Running, Role: LEADER
I20260709 13:06:48.624550 20045 consensus_queue.cc:237] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.620843 20054 raft_consensus.cc:2804] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:48.629748 20054 raft_consensus.cc:697] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Becoming Leader. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Running, Role: LEADER
I20260709 13:06:48.630935 20054 consensus_queue.cc:237] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.634148 20034 tablet_bootstrap.cc:492] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0: No bootstrap required, opened a new log
I20260709 13:06:48.634740 20034 ts_tablet_manager.cc:1403] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0: Time spent bootstrapping tablet: real 0.115s	user 0.050s	sys 0.060s
I20260709 13:06:48.635588 20033 ts_tablet_manager.cc:1434] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent starting tablet: real 0.098s	user 0.005s	sys 0.006s
I20260709 13:06:48.637020 20033 tablet_bootstrap.cc:492] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8: Bootstrap starting.
I20260709 13:06:48.638646 20034 raft_consensus.cc:359] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.639750 20034 raft_consensus.cc:385] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.640136 20034 raft_consensus.cc:740] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf68973149544eae8929005c48bb84b0, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.641245 20053 raft_consensus.cc:493] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.641885 20053 raft_consensus.cc:515] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.641189 20034 consensus_queue.cc:260] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.644212 20053 leader_election.cc:290] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:48.645763 19828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb7db72b50647c09980ff9b8cb188be" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf68973149544eae8929005c48bb84b0" is_pre_election: true
I20260709 13:06:48.646445 19828 raft_consensus.cc:2468] T edb7db72b50647c09980ff9b8cb188be P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 0.
I20260709 13:06:48.647559 19901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb7db72b50647c09980ff9b8cb188be" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d" is_pre_election: true
I20260709 13:06:48.648584 20033 tablet_bootstrap.cc:654] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.649094 20036 tablet_bootstrap.cc:654] T edb7db72b50647c09980ff9b8cb188be P e5b2e431c3204ee487c10f58d440707d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.649739 19718 leader_election.cc:304] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: 
W20260709 13:06:48.650697 19718 leader_election.cc:343] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.651746 20053 raft_consensus.cc:2804] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:48.652225 20053 raft_consensus.cc:493] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.652684 20053 raft_consensus.cc:3060] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.653257 20034 ts_tablet_manager.cc:1434] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.002s
I20260709 13:06:48.661939 19971 raft_consensus.cc:493] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.662478 19971 raft_consensus.cc:515] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.663364 19650 catalog_manager.cc:5697] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:48.664624 19971 leader_election.cc:290] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553)
I20260709 13:06:48.665568 20035 raft_consensus.cc:493] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.666067 20035 raft_consensus.cc:515] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.667790 20034 tablet_bootstrap.cc:492] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0: Bootstrap starting.
I20260709 13:06:48.668123 20035 leader_election.cc:290] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:48.670468 19751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "021f87e2ac864b89a9563513db8a389d" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" is_pre_election: true
I20260709 13:06:48.671182 19751 raft_consensus.cc:2468] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 0.
I20260709 13:06:48.673921 19793 leader_election.cc:304] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: 
I20260709 13:06:48.675304 19901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "021f87e2ac864b89a9563513db8a389d" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d" is_pre_election: true
I20260709 13:06:48.676051 19901 raft_consensus.cc:2468] T 021f87e2ac864b89a9563513db8a389d P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 0.
I20260709 13:06:48.677762 19828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50ac34758d044dbbf7ec338fff5a9ac" candidate_uuid: "e5b2e431c3204ee487c10f58d440707d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf68973149544eae8929005c48bb84b0" is_pre_election: true
I20260709 13:06:48.678699 19828 raft_consensus.cc:2468] T d50ac34758d044dbbf7ec338fff5a9ac P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5b2e431c3204ee487c10f58d440707d in term 0.
I20260709 13:06:48.680482 19867 leader_election.cc:304] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf68973149544eae8929005c48bb84b0, e5b2e431c3204ee487c10f58d440707d; no voters: 
I20260709 13:06:48.682993 19751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50ac34758d044dbbf7ec338fff5a9ac" candidate_uuid: "e5b2e431c3204ee487c10f58d440707d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" is_pre_election: true
I20260709 13:06:48.683508 20034 tablet_bootstrap.cc:654] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.683699 19751 raft_consensus.cc:2468] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5b2e431c3204ee487c10f58d440707d in term 0.
I20260709 13:06:48.684963 20035 raft_consensus.cc:2804] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:48.685654 20035 raft_consensus.cc:493] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.686182 20035 raft_consensus.cc:3060] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.690968 19971 raft_consensus.cc:2804] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:48.691619 19971 raft_consensus.cc:493] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.692121 19971 raft_consensus.cc:3060] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.716225 20035 raft_consensus.cc:515] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.720708 19751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "021f87e2ac864b89a9563513db8a389d" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
I20260709 13:06:48.721647 19751 raft_consensus.cc:3060] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.660746 20053 raft_consensus.cc:515] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.741060 19828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb7db72b50647c09980ff9b8cb188be" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf68973149544eae8929005c48bb84b0"
I20260709 13:06:48.741935 19828 raft_consensus.cc:3060] T edb7db72b50647c09980ff9b8cb188be P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.780715 19828 raft_consensus.cc:2468] T edb7db72b50647c09980ff9b8cb188be P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 1.
I20260709 13:06:48.801396 19718 leader_election.cc:304] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: 
I20260709 13:06:48.802400 20045 raft_consensus.cc:2804] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:48.803146 20045 raft_consensus.cc:697] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Becoming Leader. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Running, Role: LEADER
I20260709 13:06:48.803959 20033 tablet_bootstrap.cc:492] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8: No bootstrap required, opened a new log
I20260709 13:06:48.804082 20045 consensus_queue.cc:237] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.805547 20033 ts_tablet_manager.cc:1403] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent bootstrapping tablet: real 0.169s	user 0.053s	sys 0.100s
I20260709 13:06:48.809924 20033 raft_consensus.cc:359] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.811226 20033 raft_consensus.cc:385] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.811779 20033 raft_consensus.cc:740] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.713014 19971 raft_consensus.cc:515] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.791478 20036 tablet_bootstrap.cc:492] T edb7db72b50647c09980ff9b8cb188be P e5b2e431c3204ee487c10f58d440707d: No bootstrap required, opened a new log
I20260709 13:06:48.791960 20034 tablet_bootstrap.cc:492] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0: No bootstrap required, opened a new log
I20260709 13:06:48.792471 20053 leader_election.cc:290] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 election: Requested vote from peers cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:48.792687 19901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb7db72b50647c09980ff9b8cb188be" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:48.793327 20035 leader_election.cc:290] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 election: Requested vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:48.813037 20033 consensus_queue.cc:260] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.733054 19751 raft_consensus.cc:2468] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 1.
I20260709 13:06:48.794588 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "021f87e2ac864b89a9563513db8a389d" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:48.815507 19902 raft_consensus.cc:3060] T 021f87e2ac864b89a9563513db8a389d P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.816318 20033 ts_tablet_manager.cc:1434] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
W20260709 13:06:48.814869 19718 leader_election.cc:343] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.816138 19793 leader_election.cc:304] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: 
I20260709 13:06:48.817920 20033 tablet_bootstrap.cc:492] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8: Bootstrap starting.
I20260709 13:06:48.813113 20034 ts_tablet_manager.cc:1403] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0: Time spent bootstrapping tablet: real 0.146s	user 0.047s	sys 0.041s
I20260709 13:06:48.818029 20035 raft_consensus.cc:2804] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:48.819677 20035 raft_consensus.cc:697] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Becoming Leader. State: Replica: cf68973149544eae8929005c48bb84b0, State: Running, Role: LEADER
I20260709 13:06:48.820537 20035 consensus_queue.cc:237] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.812973 20036 ts_tablet_manager.cc:1403] T edb7db72b50647c09980ff9b8cb188be P e5b2e431c3204ee487c10f58d440707d: Time spent bootstrapping tablet: real 0.250s	user 0.032s	sys 0.042s
I20260709 13:06:48.821700 20034 raft_consensus.cc:359] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.822631 20034 raft_consensus.cc:385] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.823009 20034 raft_consensus.cc:740] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf68973149544eae8929005c48bb84b0, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.824026 20034 consensus_queue.cc:260] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.824426 20036 raft_consensus.cc:359] T edb7db72b50647c09980ff9b8cb188be P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.825344 20036 raft_consensus.cc:385] T edb7db72b50647c09980ff9b8cb188be P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.825767 20036 raft_consensus.cc:740] T edb7db72b50647c09980ff9b8cb188be P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.827154 20033 tablet_bootstrap.cc:654] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.827442 20034 ts_tablet_manager.cc:1434] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:06:48.826756 20036 consensus_queue.cc:260] T edb7db72b50647c09980ff9b8cb188be P e5b2e431c3204ee487c10f58d440707d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.828790 20034 tablet_bootstrap.cc:492] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0: Bootstrap starting.
I20260709 13:06:48.830778 19902 raft_consensus.cc:2468] T 021f87e2ac864b89a9563513db8a389d P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 1.
I20260709 13:06:48.832052 19751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50ac34758d044dbbf7ec338fff5a9ac" candidate_uuid: "e5b2e431c3204ee487c10f58d440707d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
I20260709 13:06:48.832698 19751 raft_consensus.cc:3060] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.845254 20036 ts_tablet_manager.cc:1434] T edb7db72b50647c09980ff9b8cb188be P e5b2e431c3204ee487c10f58d440707d: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.004s
I20260709 13:06:48.845876 20034 tablet_bootstrap.cc:654] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.846400 20036 tablet_bootstrap.cc:492] T 2a9a23c67df64d0793abcb340fe4e985 P e5b2e431c3204ee487c10f58d440707d: Bootstrap starting.
I20260709 13:06:48.843918 19827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50ac34758d044dbbf7ec338fff5a9ac" candidate_uuid: "e5b2e431c3204ee487c10f58d440707d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf68973149544eae8929005c48bb84b0"
I20260709 13:06:48.849740 19827 raft_consensus.cc:3060] T d50ac34758d044dbbf7ec338fff5a9ac P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.864230 20036 tablet_bootstrap.cc:654] T 2a9a23c67df64d0793abcb340fe4e985 P e5b2e431c3204ee487c10f58d440707d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.869179 19827 raft_consensus.cc:2468] T d50ac34758d044dbbf7ec338fff5a9ac P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5b2e431c3204ee487c10f58d440707d in term 1.
I20260709 13:06:48.870998 19867 leader_election.cc:304] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf68973149544eae8929005c48bb84b0, e5b2e431c3204ee487c10f58d440707d; no voters: 
I20260709 13:06:48.871940 20065 raft_consensus.cc:2804] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:48.872090 19971 leader_election.cc:290] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 1 election: Requested vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553)
I20260709 13:06:48.873049 19751 raft_consensus.cc:2468] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5b2e431c3204ee487c10f58d440707d in term 1.
I20260709 13:06:48.882457 19959 raft_consensus.cc:493] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.883288 19959 raft_consensus.cc:515] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.885802 20065 raft_consensus.cc:697] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [term 1 LEADER]: Becoming Leader. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Running, Role: LEADER
I20260709 13:06:48.886826 20065 consensus_queue.cc:237] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.888726 19959 leader_election.cc:290] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:48.889235 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bad2e092b8464dca8a6a73f54bf5f934" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d" is_pre_election: true
I20260709 13:06:48.889817 19751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bad2e092b8464dca8a6a73f54bf5f934" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" is_pre_election: true
W20260709 13:06:48.890863 19793 leader_election.cc:343] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:48.891672 19793 leader_election.cc:343] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.892081 19793 leader_election.cc:304] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf68973149544eae8929005c48bb84b0; no voters: 7cc15333145b485ea8cdd6c1623ff0f8, e5b2e431c3204ee487c10f58d440707d
I20260709 13:06:48.892891 19959 raft_consensus.cc:2749] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:48.894003 19651 catalog_manager.cc:5697] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf68973149544eae8929005c48bb84b0 (127.6.255.2). New cstate: current_term: 1 leader_uuid: "cf68973149544eae8929005c48bb84b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:48.896584 20033 tablet_bootstrap.cc:492] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8: No bootstrap required, opened a new log
I20260709 13:06:48.905009 20033 ts_tablet_manager.cc:1403] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent bootstrapping tablet: real 0.087s	user 0.022s	sys 0.050s
I20260709 13:06:48.908452 20036 tablet_bootstrap.cc:492] T 2a9a23c67df64d0793abcb340fe4e985 P e5b2e431c3204ee487c10f58d440707d: No bootstrap required, opened a new log
I20260709 13:06:48.908231 20033 raft_consensus.cc:359] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.909130 20036 ts_tablet_manager.cc:1403] T 2a9a23c67df64d0793abcb340fe4e985 P e5b2e431c3204ee487c10f58d440707d: Time spent bootstrapping tablet: real 0.063s	user 0.009s	sys 0.007s
I20260709 13:06:48.909525 20069 raft_consensus.cc:493] T edb7db72b50647c09980ff9b8cb188be P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.911563 20069 raft_consensus.cc:515] T edb7db72b50647c09980ff9b8cb188be P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.909487 20033 raft_consensus.cc:385] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.913897 20033 raft_consensus.cc:740] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.914697 20033 consensus_queue.cc:260] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.914954 19753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb7db72b50647c09980ff9b8cb188be" candidate_uuid: "e5b2e431c3204ee487c10f58d440707d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" is_pre_election: true
I20260709 13:06:48.916663 20033 ts_tablet_manager.cc:1434] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:06:48.916770 19650 catalog_manager.cc:5697] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:48.919555 20033 tablet_bootstrap.cc:492] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8: Bootstrap starting.
I20260709 13:06:48.920460 20045 raft_consensus.cc:493] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.923727 20045 raft_consensus.cc:515] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.919669 19828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb7db72b50647c09980ff9b8cb188be" candidate_uuid: "e5b2e431c3204ee487c10f58d440707d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf68973149544eae8929005c48bb84b0" is_pre_election: true
I20260709 13:06:48.926076 20045 leader_election.cc:290] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:48.926211 19828 raft_consensus.cc:2393] T edb7db72b50647c09980ff9b8cb188be P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5b2e431c3204ee487c10f58d440707d in current term 1: Already voted for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in this term.
I20260709 13:06:48.918908 19650 catalog_manager.cc:5697] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:48.919374 20036 raft_consensus.cc:359] T 2a9a23c67df64d0793abcb340fe4e985 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.927153 20036 raft_consensus.cc:385] T 2a9a23c67df64d0793abcb340fe4e985 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.927510 20036 raft_consensus.cc:740] T 2a9a23c67df64d0793abcb340fe4e985 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.929015 19867 leader_election.cc:304] T edb7db72b50647c09980ff9b8cb188be P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5b2e431c3204ee487c10f58d440707d; no voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0
I20260709 13:06:48.930824 19971 raft_consensus.cc:3060] T edb7db72b50647c09980ff9b8cb188be P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.936934 19828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a9a23c67df64d0793abcb340fe4e985" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf68973149544eae8929005c48bb84b0" is_pre_election: true
I20260709 13:06:48.937721 20034 tablet_bootstrap.cc:492] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0: No bootstrap required, opened a new log
I20260709 13:06:48.938450 20034 ts_tablet_manager.cc:1403] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0: Time spent bootstrapping tablet: real 0.110s	user 0.016s	sys 0.014s
I20260709 13:06:48.937678 19828 raft_consensus.cc:2468] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 0.
I20260709 13:06:48.940599 20033 tablet_bootstrap.cc:654] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.940733 19718 leader_election.cc:304] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: 
I20260709 13:06:48.941668 20034 raft_consensus.cc:359] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.941702 20045 raft_consensus.cc:2804] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:48.941926 20036 consensus_queue.cc:260] T 2a9a23c67df64d0793abcb340fe4e985 P e5b2e431c3204ee487c10f58d440707d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.942634 20045 raft_consensus.cc:493] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.942459 20034 raft_consensus.cc:385] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.943045 20045 raft_consensus.cc:3060] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.943509 20034 raft_consensus.cc:740] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf68973149544eae8929005c48bb84b0, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.944341 20036 ts_tablet_manager.cc:1434] T 2a9a23c67df64d0793abcb340fe4e985 P e5b2e431c3204ee487c10f58d440707d: Time spent starting tablet: real 0.033s	user 0.002s	sys 0.003s
I20260709 13:06:48.944398 20034 consensus_queue.cc:260] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.945633 20036 tablet_bootstrap.cc:492] T 54491751f68f4ee5b5b9000c225b906e P e5b2e431c3204ee487c10f58d440707d: Bootstrap starting.
I20260709 13:06:48.946884 20034 ts_tablet_manager.cc:1434] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:06:48.950532 20034 tablet_bootstrap.cc:492] T 5a6ea8c60bdb40fc902a427e79c155f8 P cf68973149544eae8929005c48bb84b0: Bootstrap starting.
I20260709 13:06:48.956029 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a9a23c67df64d0793abcb340fe4e985" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d" is_pre_election: true
I20260709 13:06:48.956782 19902 raft_consensus.cc:2468] T 2a9a23c67df64d0793abcb340fe4e985 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 0.
I20260709 13:06:48.959378 19971 raft_consensus.cc:2749] T edb7db72b50647c09980ff9b8cb188be P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:48.963372 20045 raft_consensus.cc:515] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.965509 20045 leader_election.cc:290] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 election: Requested vote from peers cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:48.966656 20036 tablet_bootstrap.cc:654] T 54491751f68f4ee5b5b9000c225b906e P e5b2e431c3204ee487c10f58d440707d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.967085 19828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a9a23c67df64d0793abcb340fe4e985" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf68973149544eae8929005c48bb84b0"
I20260709 13:06:48.967707 19828 raft_consensus.cc:3060] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.967998 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a9a23c67df64d0793abcb340fe4e985" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:48.968786 19902 raft_consensus.cc:3060] T 2a9a23c67df64d0793abcb340fe4e985 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.973495 20069 leader_election.cc:290] T edb7db72b50647c09980ff9b8cb188be P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553)
I20260709 13:06:48.972440 19651 catalog_manager.cc:5697] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d reported cstate change: term changed from 0 to 1, leader changed from <none> to e5b2e431c3204ee487c10f58d440707d (127.6.255.3). New cstate: current_term: 1 leader_uuid: "e5b2e431c3204ee487c10f58d440707d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:48.979439 20034 tablet_bootstrap.cc:654] T 5a6ea8c60bdb40fc902a427e79c155f8 P cf68973149544eae8929005c48bb84b0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.983603 19902 raft_consensus.cc:2468] T 2a9a23c67df64d0793abcb340fe4e985 P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 1.
I20260709 13:06:48.990262 20036 tablet_bootstrap.cc:492] T 54491751f68f4ee5b5b9000c225b906e P e5b2e431c3204ee487c10f58d440707d: No bootstrap required, opened a new log
I20260709 13:06:48.990861 20036 ts_tablet_manager.cc:1403] T 54491751f68f4ee5b5b9000c225b906e P e5b2e431c3204ee487c10f58d440707d: Time spent bootstrapping tablet: real 0.045s	user 0.012s	sys 0.001s
I20260709 13:06:48.995009 20036 raft_consensus.cc:359] T 54491751f68f4ee5b5b9000c225b906e P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.996057 20036 raft_consensus.cc:385] T 54491751f68f4ee5b5b9000c225b906e P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.996425 20036 raft_consensus.cc:740] T 54491751f68f4ee5b5b9000c225b906e P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.998580 19718 leader_election.cc:304] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, e5b2e431c3204ee487c10f58d440707d; no voters: 
I20260709 13:06:49.001101 20054 raft_consensus.cc:2804] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:49.001609 20054 raft_consensus.cc:697] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Becoming Leader. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Running, Role: LEADER
I20260709 13:06:49.002434 20054 consensus_queue.cc:237] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:48.997318 20036 consensus_queue.cc:260] T 54491751f68f4ee5b5b9000c225b906e P e5b2e431c3204ee487c10f58d440707d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:49.020092 19650 catalog_manager.cc:5697] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:49.021915 19828 raft_consensus.cc:2468] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 1.
I20260709 13:06:49.027506 20034 tablet_bootstrap.cc:492] T 5a6ea8c60bdb40fc902a427e79c155f8 P cf68973149544eae8929005c48bb84b0: No bootstrap required, opened a new log
I20260709 13:06:49.028173 20034 ts_tablet_manager.cc:1403] T 5a6ea8c60bdb40fc902a427e79c155f8 P cf68973149544eae8929005c48bb84b0: Time spent bootstrapping tablet: real 0.078s	user 0.025s	sys 0.007s
I20260709 13:06:49.028573 20033 tablet_bootstrap.cc:492] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8: No bootstrap required, opened a new log
I20260709 13:06:49.023921 20036 ts_tablet_manager.cc:1434] T 54491751f68f4ee5b5b9000c225b906e P e5b2e431c3204ee487c10f58d440707d: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.005s
I20260709 13:06:49.031232 20034 raft_consensus.cc:359] T 5a6ea8c60bdb40fc902a427e79c155f8 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:49.032090 20034 raft_consensus.cc:385] T 5a6ea8c60bdb40fc902a427e79c155f8 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.032433 20034 raft_consensus.cc:740] T 5a6ea8c60bdb40fc902a427e79c155f8 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf68973149544eae8929005c48bb84b0, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.032827 20036 tablet_bootstrap.cc:492] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d: Bootstrap starting.
I20260709 13:06:49.033308 20034 consensus_queue.cc:260] T 5a6ea8c60bdb40fc902a427e79c155f8 P cf68973149544eae8929005c48bb84b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:49.035974 20034 ts_tablet_manager.cc:1434] T 5a6ea8c60bdb40fc902a427e79c155f8 P cf68973149544eae8929005c48bb84b0: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:49.039602 20036 tablet_bootstrap.cc:654] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.040053 20033 ts_tablet_manager.cc:1403] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent bootstrapping tablet: real 0.121s	user 0.021s	sys 0.008s
I20260709 13:06:49.043540 20033 raft_consensus.cc:359] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:49.044420 20033 raft_consensus.cc:385] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.044795 20033 raft_consensus.cc:740] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.046607 20033 consensus_queue.cc:260] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:49.050362 20033 ts_tablet_manager.cc:1434] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:06:49.051584 20033 tablet_bootstrap.cc:492] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8: Bootstrap starting.
I20260709 13:06:49.052815 20036 tablet_bootstrap.cc:492] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d: No bootstrap required, opened a new log
I20260709 13:06:49.053371 20036 ts_tablet_manager.cc:1403] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d: Time spent bootstrapping tablet: real 0.021s	user 0.005s	sys 0.011s
I20260709 13:06:49.056067 20036 raft_consensus.cc:359] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:49.056809 20036 raft_consensus.cc:385] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.057082 20036 raft_consensus.cc:740] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.057904 20036 consensus_queue.cc:260] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:49.058979 20033 tablet_bootstrap.cc:654] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.060192 20036 ts_tablet_manager.cc:1434] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:06:49.061232 20036 tablet_bootstrap.cc:492] T bad2e092b8464dca8a6a73f54bf5f934 P e5b2e431c3204ee487c10f58d440707d: Bootstrap starting.
I20260709 13:06:49.069164 20036 tablet_bootstrap.cc:654] T bad2e092b8464dca8a6a73f54bf5f934 P e5b2e431c3204ee487c10f58d440707d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.071425 20033 tablet_bootstrap.cc:492] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8: No bootstrap required, opened a new log
I20260709 13:06:49.072186 20033 ts_tablet_manager.cc:1403] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.001s
I20260709 13:06:49.075487 20036 tablet_bootstrap.cc:492] T bad2e092b8464dca8a6a73f54bf5f934 P e5b2e431c3204ee487c10f58d440707d: No bootstrap required, opened a new log
I20260709 13:06:49.075312 20033 raft_consensus.cc:359] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:49.076020 20036 ts_tablet_manager.cc:1403] T bad2e092b8464dca8a6a73f54bf5f934 P e5b2e431c3204ee487c10f58d440707d: Time spent bootstrapping tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 13:06:49.079022 20033 raft_consensus.cc:385] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.078887 20036 raft_consensus.cc:359] T bad2e092b8464dca8a6a73f54bf5f934 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:49.079483 20033 raft_consensus.cc:740] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.079618 20036 raft_consensus.cc:385] T bad2e092b8464dca8a6a73f54bf5f934 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.079994 20036 raft_consensus.cc:740] T bad2e092b8464dca8a6a73f54bf5f934 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.080407 20033 consensus_queue.cc:260] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:49.080686 20036 consensus_queue.cc:260] T bad2e092b8464dca8a6a73f54bf5f934 P e5b2e431c3204ee487c10f58d440707d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:49.082888 20033 ts_tablet_manager.cc:1434] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:06:49.083312 20036 ts_tablet_manager.cc:1434] T bad2e092b8464dca8a6a73f54bf5f934 P e5b2e431c3204ee487c10f58d440707d: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:06:49.089776 20045 consensus_queue.cc:1048] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:49.111315 20053 consensus_queue.cc:1048] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:49.118489 20054 consensus_queue.cc:1048] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:49.127867 20054 raft_consensus.cc:493] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.128381 20054 raft_consensus.cc:515] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:49.135660 20045 consensus_queue.cc:1048] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:49.139318 20054 leader_election.cc:290] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:49.141739 19900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54491751f68f4ee5b5b9000c225b906e" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d" is_pre_election: true
I20260709 13:06:49.142671 19900 raft_consensus.cc:2468] T 54491751f68f4ee5b5b9000c225b906e P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 0.
I20260709 13:06:49.143194 19827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54491751f68f4ee5b5b9000c225b906e" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf68973149544eae8929005c48bb84b0" is_pre_election: true
I20260709 13:06:49.143774 19827 raft_consensus.cc:2468] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 0.
I20260709 13:06:49.144097 19718 leader_election.cc:304] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, e5b2e431c3204ee487c10f58d440707d; no voters: 
I20260709 13:06:49.145071 20054 raft_consensus.cc:2804] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:49.145816 20054 raft_consensus.cc:493] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.146203 20054 raft_consensus.cc:3060] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.161746 20054 raft_consensus.cc:515] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:49.165733 19827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54491751f68f4ee5b5b9000c225b906e" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf68973149544eae8929005c48bb84b0"
I20260709 13:06:49.166399 19827 raft_consensus.cc:3060] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.169940 19900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54491751f68f4ee5b5b9000c225b906e" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:49.170688 19900 raft_consensus.cc:3060] T 54491751f68f4ee5b5b9000c225b906e P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.173907 20054 leader_election.cc:290] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 election: Requested vote from peers cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:49.179765 19827 raft_consensus.cc:2468] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 1.
I20260709 13:06:49.181246 19718 leader_election.cc:304] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: 
I20260709 13:06:49.182117 20054 raft_consensus.cc:2804] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:49.182701 20054 raft_consensus.cc:697] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Becoming Leader. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Running, Role: LEADER
I20260709 13:06:49.183950 20054 consensus_queue.cc:237] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:49.182168 19900 raft_consensus.cc:2468] T 54491751f68f4ee5b5b9000c225b906e P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 1.
I20260709 13:06:49.193385 20045 consensus_queue.cc:1048] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:49.204823 19648 catalog_manager.cc:5697] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:49.212524 19989 consensus_queue.cc:1048] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:49.287959 19959 consensus_queue.cc:1048] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:06:49.306970 20035 consensus_queue.cc:1048] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:49.308154 20053 raft_consensus.cc:493] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.316864 20053 raft_consensus.cc:515] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:49.323040 19827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a6ea8c60bdb40fc902a427e79c155f8" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf68973149544eae8929005c48bb84b0" is_pre_election: true
I20260709 13:06:49.323333 19900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a6ea8c60bdb40fc902a427e79c155f8" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d" is_pre_election: true
I20260709 13:06:49.323972 20053 leader_election.cc:290] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:49.324172 19827 raft_consensus.cc:2468] T 5a6ea8c60bdb40fc902a427e79c155f8 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 0.
I20260709 13:06:49.324385 19900 raft_consensus.cc:2468] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 0.
I20260709 13:06:49.326691 19718 leader_election.cc:304] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: 
I20260709 13:06:49.327823 20053 raft_consensus.cc:2804] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:49.328226 20053 raft_consensus.cc:493] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.328636 20053 raft_consensus.cc:3060] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.355767 20053 raft_consensus.cc:515] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:49.358381 20053 leader_election.cc:290] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 election: Requested vote from peers cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:49.359427 19827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a6ea8c60bdb40fc902a427e79c155f8" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf68973149544eae8929005c48bb84b0"
I20260709 13:06:49.360109 19827 raft_consensus.cc:3060] T 5a6ea8c60bdb40fc902a427e79c155f8 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.361971 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a6ea8c60bdb40fc902a427e79c155f8" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:49.362713 19902 raft_consensus.cc:3060] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.371696 19827 raft_consensus.cc:2468] T 5a6ea8c60bdb40fc902a427e79c155f8 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 1.
I20260709 13:06:49.373201 19718 leader_election.cc:304] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: 
I20260709 13:06:49.374091 20053 raft_consensus.cc:2804] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:49.375583 20053 raft_consensus.cc:697] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Becoming Leader. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Running, Role: LEADER
I20260709 13:06:49.376883 19902 raft_consensus.cc:2468] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 1.
W20260709 13:06:49.377902 19699 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:49.376958 20053 consensus_queue.cc:237] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:49.397042 19989 consensus_queue.cc:1048] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:49.412079 20069 consensus_queue.cc:1048] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:49.436121 19650 catalog_manager.cc:5697] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:49.447839 20045 consensus_queue.cc:1048] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:49.460685 20069 consensus_queue.cc:1048] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:49.460932 20045 raft_consensus.cc:493] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.471952 20045 raft_consensus.cc:515] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:49.478410 20045 leader_election.cc:290] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:49.480335 19827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bad2e092b8464dca8a6a73f54bf5f934" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf68973149544eae8929005c48bb84b0" is_pre_election: true
I20260709 13:06:49.482823 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bad2e092b8464dca8a6a73f54bf5f934" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d" is_pre_election: true
I20260709 13:06:49.483935 19902 raft_consensus.cc:2468] T bad2e092b8464dca8a6a73f54bf5f934 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 0.
I20260709 13:06:49.482607 19827 raft_consensus.cc:2468] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 0.
I20260709 13:06:49.490444 19718 leader_election.cc:304] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, e5b2e431c3204ee487c10f58d440707d; no voters: 
I20260709 13:06:49.491451 19989 raft_consensus.cc:2804] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:49.491916 19989 raft_consensus.cc:493] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.492340 19989 raft_consensus.cc:3060] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.514635 19989 raft_consensus.cc:515] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:49.516829 19989 leader_election.cc:290] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 election: Requested vote from peers cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:49.518471 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bad2e092b8464dca8a6a73f54bf5f934" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:49.518800 19828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bad2e092b8464dca8a6a73f54bf5f934" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf68973149544eae8929005c48bb84b0"
I20260709 13:06:49.519279 19902 raft_consensus.cc:3060] T bad2e092b8464dca8a6a73f54bf5f934 P e5b2e431c3204ee487c10f58d440707d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.519654 19828 raft_consensus.cc:3060] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.529245 19902 raft_consensus.cc:2468] T bad2e092b8464dca8a6a73f54bf5f934 P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 1.
I20260709 13:06:49.530764 19718 leader_election.cc:304] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, e5b2e431c3204ee487c10f58d440707d; no voters: 
I20260709 13:06:49.531538 19989 raft_consensus.cc:2804] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:49.532038 19989 raft_consensus.cc:697] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Becoming Leader. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Running, Role: LEADER
I20260709 13:06:49.533167 19989 consensus_queue.cc:237] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:49.533753 19828 raft_consensus.cc:2468] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 1.
I20260709 13:06:49.543092 19650 catalog_manager.cc:5697] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1). New cstate: current_term: 1 leader_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:49.621840 20045 consensus_queue.cc:1048] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:49.636291 19989 consensus_queue.cc:1048] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:49.811645 19827 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "021f87e2ac864b89a9563513db8a389d"
dest_uuid: "cf68973149544eae8929005c48bb84b0"
mode: GRACEFUL
new_leader_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
 from {username='slave'} at 127.0.0.1:54272
I20260709 13:06:49.812438 19827 raft_consensus.cc:606] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Received request to transfer leadership to TS 7cc15333145b485ea8cdd6c1623ff0f8
I20260709 13:06:49.819613 19900 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d50ac34758d044dbbf7ec338fff5a9ac"
dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
mode: GRACEFUL
new_leader_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
 from {username='slave'} at 127.0.0.1:40138
I20260709 13:06:49.820325 19900 raft_consensus.cc:606] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [term 1 LEADER]: Received request to transfer leadership to TS 7cc15333145b485ea8cdd6c1623ff0f8
I20260709 13:06:49.850273 19751 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53bdb8e606e240dea3740473550cc59a"
dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
mode: GRACEFUL
new_leader_uuid: "cf68973149544eae8929005c48bb84b0"
 from {username='slave'} at 127.0.0.1:41420
I20260709 13:06:49.850984 19751 raft_consensus.cc:606] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Received request to transfer leadership to TS cf68973149544eae8929005c48bb84b0
I20260709 13:06:49.856614 19751 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a9a23c67df64d0793abcb340fe4e985"
dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
mode: GRACEFUL
new_leader_uuid: "cf68973149544eae8929005c48bb84b0"
 from {username='slave'} at 127.0.0.1:41420
I20260709 13:06:49.857295 19751 raft_consensus.cc:606] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Received request to transfer leadership to TS cf68973149544eae8929005c48bb84b0
I20260709 13:06:49.862488 19751 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54491751f68f4ee5b5b9000c225b906e"
dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
mode: GRACEFUL
new_leader_uuid: "cf68973149544eae8929005c48bb84b0"
 from {username='slave'} at 127.0.0.1:41420
I20260709 13:06:49.863009 19751 raft_consensus.cc:606] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Received request to transfer leadership to TS cf68973149544eae8929005c48bb84b0
I20260709 13:06:49.868402 19751 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bad2e092b8464dca8a6a73f54bf5f934"
dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
mode: GRACEFUL
new_leader_uuid: "cf68973149544eae8929005c48bb84b0"
 from {username='slave'} at 127.0.0.1:41420
I20260709 13:06:49.868971 19751 raft_consensus.cc:606] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Received request to transfer leadership to TS cf68973149544eae8929005c48bb84b0
I20260709 13:06:49.874317 19751 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a6ea8c60bdb40fc902a427e79c155f8"
dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
mode: GRACEFUL
new_leader_uuid: "e5b2e431c3204ee487c10f58d440707d"
 from {username='slave'} at 127.0.0.1:41420
I20260709 13:06:49.874856 19751 raft_consensus.cc:606] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Received request to transfer leadership to TS e5b2e431c3204ee487c10f58d440707d
I20260709 13:06:49.923972 19989 consensus_queue.cc:1048] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:49.944406 20053 consensus_queue.cc:1048] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:49.953986 20053 raft_consensus.cc:993] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8: : Instructing follower cf68973149544eae8929005c48bb84b0 to start an election
I20260709 13:06:49.954540 19989 raft_consensus.cc:1081] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Signalling peer cf68973149544eae8929005c48bb84b0 to start an election
I20260709 13:06:49.956425 19827 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bad2e092b8464dca8a6a73f54bf5f934"
dest_uuid: "cf68973149544eae8929005c48bb84b0"
 from {username='slave'} at 127.0.0.1:54264
I20260709 13:06:49.957059 19827 raft_consensus.cc:493] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:49.957383 19827 raft_consensus.cc:3060] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:49.963472 19827 raft_consensus.cc:515] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:49.965484 19827 leader_election.cc:290] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 2 election: Requested vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:49.967711 19751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bad2e092b8464dca8a6a73f54bf5f934" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
I20260709 13:06:49.967959 19900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bad2e092b8464dca8a6a73f54bf5f934" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:49.968407 19751 raft_consensus.cc:3055] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:49.968506 19900 raft_consensus.cc:3060] T bad2e092b8464dca8a6a73f54bf5f934 P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:49.968793 19751 raft_consensus.cc:740] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Running, Role: LEADER
I20260709 13:06:49.969686 19751 consensus_queue.cc:260] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:49.971122 19751 raft_consensus.cc:3060] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:49.979364 19751 raft_consensus.cc:2468] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 2.
I20260709 13:06:49.980862 19989 consensus_queue.cc:1048] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:49.984880 19989 consensus_queue.cc:1048] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:49.991618 20045 raft_consensus.cc:993] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8: : Instructing follower cf68973149544eae8929005c48bb84b0 to start an election
I20260709 13:06:49.992280 20045 raft_consensus.cc:1081] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Signalling peer cf68973149544eae8929005c48bb84b0 to start an election
I20260709 13:06:49.994434 19827 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2a9a23c67df64d0793abcb340fe4e985"
dest_uuid: "cf68973149544eae8929005c48bb84b0"
 from {username='slave'} at 127.0.0.1:54264
I20260709 13:06:49.995649 19827 raft_consensus.cc:493] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:49.996052 19827 raft_consensus.cc:3060] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:50.005769 19900 raft_consensus.cc:2468] T bad2e092b8464dca8a6a73f54bf5f934 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 2.
I20260709 13:06:50.007682 19793 leader_election.cc:304] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf68973149544eae8929005c48bb84b0, e5b2e431c3204ee487c10f58d440707d; no voters: 
I20260709 13:06:50.008942 19959 raft_consensus.cc:2804] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:50.009526 19959 raft_consensus.cc:697] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [term 2 LEADER]: Becoming Leader. State: Replica: cf68973149544eae8929005c48bb84b0, State: Running, Role: LEADER
I20260709 13:06:50.010699 19959 consensus_queue.cc:237] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:50.019779 20054 raft_consensus.cc:993] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8: : Instructing follower e5b2e431c3204ee487c10f58d440707d to start an election
I20260709 13:06:50.020409 20054 raft_consensus.cc:1081] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Signalling peer e5b2e431c3204ee487c10f58d440707d to start an election
I20260709 13:06:50.024732 19827 raft_consensus.cc:515] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:50.024122 19648 catalog_manager.cc:5697] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 reported cstate change: term changed from 1 to 2, leader changed from 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1) to cf68973149544eae8929005c48bb84b0 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "cf68973149544eae8929005c48bb84b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.027901 19751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a9a23c67df64d0793abcb340fe4e985" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
I20260709 13:06:50.028471 19751 raft_consensus.cc:3055] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:50.028790 19751 raft_consensus.cc:740] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Running, Role: LEADER
I20260709 13:06:50.029659 19751 consensus_queue.cc:260] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:50.030793 19751 raft_consensus.cc:3060] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:50.037564 19900 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a6ea8c60bdb40fc902a427e79c155f8"
dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
 from {username='slave'} at 127.0.0.1:40122
I20260709 13:06:50.038100 19827 leader_election.cc:290] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 2 election: Requested vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:50.038296 19900 raft_consensus.cc:493] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:50.038764 19900 raft_consensus.cc:3060] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:50.039565 19902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a9a23c67df64d0793abcb340fe4e985" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:50.040302 19902 raft_consensus.cc:3060] T 2a9a23c67df64d0793abcb340fe4e985 P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:50.040680 19751 raft_consensus.cc:2468] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 2.
I20260709 13:06:50.041889 19793 leader_election.cc:304] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: 
I20260709 13:06:50.042977 19959 raft_consensus.cc:2804] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:50.045780 19959 raft_consensus.cc:697] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0 [term 2 LEADER]: Becoming Leader. State: Replica: cf68973149544eae8929005c48bb84b0, State: Running, Role: LEADER
I20260709 13:06:50.046849 19900 raft_consensus.cc:515] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:50.047235 19959 consensus_queue.cc:237] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:50.051815 19902 raft_consensus.cc:2468] T 2a9a23c67df64d0793abcb340fe4e985 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 2.
I20260709 13:06:50.059175 19751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a6ea8c60bdb40fc902a427e79c155f8" candidate_uuid: "e5b2e431c3204ee487c10f58d440707d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
I20260709 13:06:50.059732 19751 raft_consensus.cc:3055] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:50.059959 19751 raft_consensus.cc:740] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Running, Role: LEADER
I20260709 13:06:50.060571 19751 consensus_queue.cc:260] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:50.061666 19751 raft_consensus.cc:3060] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:50.063155 19900 leader_election.cc:290] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 2 election: Requested vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553)
I20260709 13:06:50.063069 19827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a6ea8c60bdb40fc902a427e79c155f8" candidate_uuid: "e5b2e431c3204ee487c10f58d440707d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cf68973149544eae8929005c48bb84b0"
I20260709 13:06:50.064199 19827 raft_consensus.cc:3060] T 5a6ea8c60bdb40fc902a427e79c155f8 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:50.063030 19650 catalog_manager.cc:5697] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0 reported cstate change: term changed from 1 to 2, leader changed from 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1) to cf68973149544eae8929005c48bb84b0 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "cf68973149544eae8929005c48bb84b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.073922 19827 raft_consensus.cc:2468] T 5a6ea8c60bdb40fc902a427e79c155f8 P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5b2e431c3204ee487c10f58d440707d in term 2.
I20260709 13:06:50.075443 19867 leader_election.cc:304] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf68973149544eae8929005c48bb84b0, e5b2e431c3204ee487c10f58d440707d; no voters: 
I20260709 13:06:50.077243 20098 raft_consensus.cc:2804] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:50.077822 20098 raft_consensus.cc:697] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d [term 2 LEADER]: Becoming Leader. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Running, Role: LEADER
I20260709 13:06:50.078910 20098 consensus_queue.cc:237] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:50.080983 20108 raft_consensus.cc:993] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d: : Instructing follower 7cc15333145b485ea8cdd6c1623ff0f8 to start an election
I20260709 13:06:50.082448 20109 raft_consensus.cc:1081] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [term 1 LEADER]: Signalling peer 7cc15333145b485ea8cdd6c1623ff0f8 to start an election
I20260709 13:06:50.084739 19753 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d50ac34758d044dbbf7ec338fff5a9ac"
dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
 from {username='slave'} at 127.0.0.1:41410
I20260709 13:06:50.085196 19751 raft_consensus.cc:2468] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5b2e431c3204ee487c10f58d440707d in term 2.
I20260709 13:06:50.085367 19753 raft_consensus.cc:493] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:50.085975 19753 raft_consensus.cc:3060] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:50.095906 19753 raft_consensus.cc:515] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:50.099032 19753 leader_election.cc:290] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 2 election: Requested vote from peers cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:50.100322 19900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50ac34758d044dbbf7ec338fff5a9ac" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:50.100898 19827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50ac34758d044dbbf7ec338fff5a9ac" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cf68973149544eae8929005c48bb84b0"
I20260709 13:06:50.101123 19900 raft_consensus.cc:3055] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:50.101527 19827 raft_consensus.cc:3060] T d50ac34758d044dbbf7ec338fff5a9ac P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:50.101588 19900 raft_consensus.cc:740] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Running, Role: LEADER
I20260709 13:06:50.103274 19900 consensus_queue.cc:260] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:50.112005 19648 catalog_manager.cc:5697] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d reported cstate change: term changed from 1 to 2, leader changed from 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1) to e5b2e431c3204ee487c10f58d440707d (127.6.255.3). New cstate: current_term: 2 leader_uuid: "e5b2e431c3204ee487c10f58d440707d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:50.113061 19827 raft_consensus.cc:2468] T d50ac34758d044dbbf7ec338fff5a9ac P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 2.
I20260709 13:06:50.114493 19718 leader_election.cc:304] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: 
I20260709 13:06:50.104823 19900 raft_consensus.cc:3060] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:50.115370 20045 raft_consensus.cc:2804] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:50.115911 20045 raft_consensus.cc:697] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 LEADER]: Becoming Leader. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Running, Role: LEADER
I20260709 13:06:50.123735 19900 raft_consensus.cc:2468] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 2.
I20260709 13:06:50.117009 20045 consensus_queue.cc:237] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:50.137622 19648 catalog_manager.cc:5697] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8 reported cstate change: term changed from 1 to 2, leader changed from e5b2e431c3204ee487c10f58d440707d (127.6.255.3) to 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1). New cstate: current_term: 2 leader_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.202219 20054 raft_consensus.cc:993] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8: : Instructing follower cf68973149544eae8929005c48bb84b0 to start an election
I20260709 13:06:50.202694 20045 raft_consensus.cc:1081] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Signalling peer cf68973149544eae8929005c48bb84b0 to start an election
I20260709 13:06:50.204310 19827 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53bdb8e606e240dea3740473550cc59a"
dest_uuid: "cf68973149544eae8929005c48bb84b0"
 from {username='slave'} at 127.0.0.1:54264
I20260709 13:06:50.204770 19827 raft_consensus.cc:493] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:50.205045 19827 raft_consensus.cc:3060] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:50.210950 19827 raft_consensus.cc:515] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:50.214035 19827 leader_election.cc:290] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 2 election: Requested vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:50.215245 19900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53bdb8e606e240dea3740473550cc59a" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:50.215404 19753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53bdb8e606e240dea3740473550cc59a" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
I20260709 13:06:50.215966 19900 raft_consensus.cc:3060] T 53bdb8e606e240dea3740473550cc59a P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:50.216238 19753 raft_consensus.cc:3055] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:50.216701 19753 raft_consensus.cc:740] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Running, Role: LEADER
I20260709 13:06:50.217774 19753 consensus_queue.cc:260] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:50.219125 19753 raft_consensus.cc:3060] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:50.224750 19900 raft_consensus.cc:2468] T 53bdb8e606e240dea3740473550cc59a P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 2.
I20260709 13:06:50.226415 19793 leader_election.cc:304] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf68973149544eae8929005c48bb84b0, e5b2e431c3204ee487c10f58d440707d; no voters: 
I20260709 13:06:50.227030 19753 raft_consensus.cc:2468] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 2.
I20260709 13:06:50.227387 19959 raft_consensus.cc:2804] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:50.228459 19959 raft_consensus.cc:697] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0 [term 2 LEADER]: Becoming Leader. State: Replica: cf68973149544eae8929005c48bb84b0, State: Running, Role: LEADER
I20260709 13:06:50.229496 19959 consensus_queue.cc:237] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:50.239630 19648 catalog_manager.cc:5697] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0 reported cstate change: term changed from 1 to 2, leader changed from 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1) to cf68973149544eae8929005c48bb84b0 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "cf68973149544eae8929005c48bb84b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.242707 20045 raft_consensus.cc:993] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8: : Instructing follower cf68973149544eae8929005c48bb84b0 to start an election
I20260709 13:06:50.243338 20045 raft_consensus.cc:1081] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Signalling peer cf68973149544eae8929005c48bb84b0 to start an election
I20260709 13:06:50.248659 19827 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "54491751f68f4ee5b5b9000c225b906e"
dest_uuid: "cf68973149544eae8929005c48bb84b0"
 from {username='slave'} at 127.0.0.1:54264
I20260709 13:06:50.249210 19827 raft_consensus.cc:493] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:50.249571 19827 raft_consensus.cc:3060] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:50.259080 19827 raft_consensus.cc:515] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:50.260634 19827 leader_election.cc:290] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 2 election: Requested vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:50.262158 19753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54491751f68f4ee5b5b9000c225b906e" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
I20260709 13:06:50.262844 19753 raft_consensus.cc:3055] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:50.262898 19900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54491751f68f4ee5b5b9000c225b906e" candidate_uuid: "cf68973149544eae8929005c48bb84b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:50.263311 19753 raft_consensus.cc:740] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Running, Role: LEADER
I20260709 13:06:50.263600 19900 raft_consensus.cc:3060] T 54491751f68f4ee5b5b9000c225b906e P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:50.264268 19753 consensus_queue.cc:260] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:50.265599 19753 raft_consensus.cc:3060] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:50.270684 19753 raft_consensus.cc:2468] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 2.
I20260709 13:06:50.271178 19900 raft_consensus.cc:2468] T 54491751f68f4ee5b5b9000c225b906e P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf68973149544eae8929005c48bb84b0 in term 2.
I20260709 13:06:50.271698 19793 leader_election.cc:304] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: 
I20260709 13:06:50.272599 19959 raft_consensus.cc:2804] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:50.273090 19959 raft_consensus.cc:697] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0 [term 2 LEADER]: Becoming Leader. State: Replica: cf68973149544eae8929005c48bb84b0, State: Running, Role: LEADER
I20260709 13:06:50.273994 19959 consensus_queue.cc:237] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:50.280555 19648 catalog_manager.cc:5697] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0 reported cstate change: term changed from 1 to 2, leader changed from 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1) to cf68973149544eae8929005c48bb84b0 (127.6.255.2). New cstate: current_term: 2 leader_uuid: "cf68973149544eae8929005c48bb84b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.388017 20089 raft_consensus.cc:993] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0: : Instructing follower 7cc15333145b485ea8cdd6c1623ff0f8 to start an election
I20260709 13:06:50.388619 19959 raft_consensus.cc:1081] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Signalling peer 7cc15333145b485ea8cdd6c1623ff0f8 to start an election
I20260709 13:06:50.390640 19753 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "021f87e2ac864b89a9563513db8a389d"
dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
 from {username='slave'} at 127.0.0.1:41402
I20260709 13:06:50.391296 19753 raft_consensus.cc:493] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:50.391660 19753 raft_consensus.cc:3060] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:50.397974 19753 raft_consensus.cc:515] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:50.399880 19753 leader_election.cc:290] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 2 election: Requested vote from peers cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553), e5b2e431c3204ee487c10f58d440707d (127.6.255.3:33743)
I20260709 13:06:50.401511 19827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "021f87e2ac864b89a9563513db8a389d" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cf68973149544eae8929005c48bb84b0"
I20260709 13:06:50.402082 19827 raft_consensus.cc:3055] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:50.402359 19827 raft_consensus.cc:740] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cf68973149544eae8929005c48bb84b0, State: Running, Role: LEADER
I20260709 13:06:50.403041 19827 consensus_queue.cc:260] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:50.404003 19827 raft_consensus.cc:3060] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:50.405887 19900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "021f87e2ac864b89a9563513db8a389d" candidate_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
I20260709 13:06:50.406522 19900 raft_consensus.cc:3060] T 021f87e2ac864b89a9563513db8a389d P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:50.411042 19827 raft_consensus.cc:2468] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 2.
I20260709 13:06:50.412442 19718 leader_election.cc:304] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, cf68973149544eae8929005c48bb84b0; no voters: 
I20260709 13:06:50.413231 20045 raft_consensus.cc:2804] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:50.413681 19900 raft_consensus.cc:2468] T 021f87e2ac864b89a9563513db8a389d P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc15333145b485ea8cdd6c1623ff0f8 in term 2.
I20260709 13:06:50.413791 20045 raft_consensus.cc:697] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 LEADER]: Becoming Leader. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Running, Role: LEADER
I20260709 13:06:50.415089 20045 consensus_queue.cc:237] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:50.424502 19648 catalog_manager.cc:5697] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 reported cstate change: term changed from 1 to 2, leader changed from cf68973149544eae8929005c48bb84b0 (127.6.255.2) to 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1). New cstate: current_term: 2 leader_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.448877 19753 raft_consensus.cc:1275] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Refusing update from remote peer cf68973149544eae8929005c48bb84b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:50.450616 19959 consensus_queue.cc:1048] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:50.459520 19900 raft_consensus.cc:1275] T bad2e092b8464dca8a6a73f54bf5f934 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Refusing update from remote peer cf68973149544eae8929005c48bb84b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:50.461031 19959 consensus_queue.cc:1048] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:50.511000 19902 raft_consensus.cc:1275] T 2a9a23c67df64d0793abcb340fe4e985 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Refusing update from remote peer cf68973149544eae8929005c48bb84b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:50.512452 20089 consensus_queue.cc:1048] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:50.523201 19753 raft_consensus.cc:1275] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Refusing update from remote peer cf68973149544eae8929005c48bb84b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:50.525890 19959 consensus_queue.cc:1048] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:50.629343 19827 raft_consensus.cc:1275] T 5a6ea8c60bdb40fc902a427e79c155f8 P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Refusing update from remote peer e5b2e431c3204ee487c10f58d440707d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:50.630906 20110 consensus_queue.cc:1048] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:50.644657 19753 raft_consensus.cc:1275] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Refusing update from remote peer e5b2e431c3204ee487c10f58d440707d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:50.646490 20110 consensus_queue.cc:1048] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:50.702394 19753 raft_consensus.cc:1275] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Refusing update from remote peer cf68973149544eae8929005c48bb84b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:50.702935 19827 raft_consensus.cc:1275] T d50ac34758d044dbbf7ec338fff5a9ac P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Refusing update from remote peer 7cc15333145b485ea8cdd6c1623ff0f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:50.703758 20113 consensus_queue.cc:1048] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:50.704259 20045 consensus_queue.cc:1048] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:50.713109 19902 raft_consensus.cc:1275] T 53bdb8e606e240dea3740473550cc59a P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Refusing update from remote peer cf68973149544eae8929005c48bb84b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:50.715656 20113 consensus_queue.cc:1048] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:50.716168 19902 raft_consensus.cc:1275] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Refusing update from remote peer 7cc15333145b485ea8cdd6c1623ff0f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:50.717723 20045 consensus_queue.cc:1048] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:50.813275 19827 raft_consensus.cc:1275] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Refusing update from remote peer 7cc15333145b485ea8cdd6c1623ff0f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:50.814586 20122 consensus_queue.cc:1048] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:50.825950 19753 raft_consensus.cc:1275] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Refusing update from remote peer cf68973149544eae8929005c48bb84b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:50.827497 20113 consensus_queue.cc:1048] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:50.829620 19901 raft_consensus.cc:1275] T 021f87e2ac864b89a9563513db8a389d P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Refusing update from remote peer 7cc15333145b485ea8cdd6c1623ff0f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:50.832993 20045 consensus_queue.cc:1048] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:50.839344 19902 raft_consensus.cc:1275] T 54491751f68f4ee5b5b9000c225b906e P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Refusing update from remote peer cf68973149544eae8929005c48bb84b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:50.840864 20113 consensus_queue.cc:1048] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:52.878355 19651 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50070:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:06:52.899698 19651 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:50070:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:06:53.028569  7164 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:06:53.117136 20165 log.cc:927] Time spent T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
W20260709 13:06:53.097142 19718 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.255.2:43553, user_credentials={real_user=slave}} blocked reactor thread for 62309us
I20260709 13:06:53.184038 20179 tablet.cc:1724] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.187860 20183 tablet.cc:1724] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.165791 20185 tablet.cc:1724] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.213624 20190 tablet.cc:1724] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.260850 20195 tablet.cc:1724] T 53bdb8e606e240dea3740473550cc59a P e5b2e431c3204ee487c10f58d440707d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.263631 20197 tablet.cc:1724] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.264252 20192 tablet.cc:1724] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:06:53.260977 20188 log.cc:927] Time spent T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:06:53.210247 20181 tablet.cc:1724] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.280771 20194 tablet.cc:1724] T d50ac34758d044dbbf7ec338fff5a9ac P cf68973149544eae8929005c48bb84b0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.284806 20179 tablet.cc:1724] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.302238 20179 tablet.cc:1724] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.310074 20197 tablet.cc:1724] T edb7db72b50647c09980ff9b8cb188be P e5b2e431c3204ee487c10f58d440707d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.342816 20190 tablet.cc:1724] T edb7db72b50647c09980ff9b8cb188be P cf68973149544eae8929005c48bb84b0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.345054 20189 tablet.cc:1724] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.347424 20185 tablet.cc:1724] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.350095 20198 tablet.cc:1724] T 191b9b12c85b43db99997d1560cd18da P e5b2e431c3204ee487c10f58d440707d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.351366 20199 tablet.cc:1724] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.359022 20181 tablet.cc:1724] T 191b9b12c85b43db99997d1560cd18da P cf68973149544eae8929005c48bb84b0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.375344 20190 tablet.cc:1724] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.377594 20185 tablet.cc:1724] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.386075 20199 tablet.cc:1724] T 021f87e2ac864b89a9563513db8a389d P e5b2e431c3204ee487c10f58d440707d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.390601 20183 tablet.cc:1724] T 5a6ea8c60bdb40fc902a427e79c155f8 P cf68973149544eae8929005c48bb84b0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.394299 20194 tablet.cc:1724] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.403935 19828 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2348879030be45d49634020eace141f0"
dest_uuid: "cf68973149544eae8929005c48bb84b0"
mode: GRACEFUL
new_leader_uuid: "e5b2e431c3204ee487c10f58d440707d"
 from {username='slave'} at 127.0.0.1:43090
I20260709 13:06:53.404719 19828 raft_consensus.cc:606] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [term 2 LEADER]: Received request to transfer leadership to TS e5b2e431c3204ee487c10f58d440707d
I20260709 13:06:53.377060 20192 tablet.cc:1724] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.408033 20195 tablet.cc:1724] T 2a9a23c67df64d0793abcb340fe4e985 P e5b2e431c3204ee487c10f58d440707d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.418926 20197 tablet.cc:1724] T 54491751f68f4ee5b5b9000c225b906e P e5b2e431c3204ee487c10f58d440707d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.428750 20199 tablet.cc:1724] T bad2e092b8464dca8a6a73f54bf5f934 P e5b2e431c3204ee487c10f58d440707d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:53.462386 19751 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34322023741e46f69328b109a5b54158"
dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
mode: GRACEFUL
new_leader_uuid: "e5b2e431c3204ee487c10f58d440707d"
 from {username='slave'} at 127.0.0.1:45712
I20260709 13:06:53.462984 19751 raft_consensus.cc:606] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 LEADER]: Received request to transfer leadership to TS e5b2e431c3204ee487c10f58d440707d
I20260709 13:06:53.464598  7164 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:06:53.465231  7164 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:53.466805 19649 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef: soft_deleted_table [id=7ce7c19aea414849924e2c89ce639096] alter complete (version 1)
I20260709 13:06:53.682772 20182 raft_consensus.cc:993] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8: : Instructing follower e5b2e431c3204ee487c10f58d440707d to start an election
I20260709 13:06:53.683328 20053 raft_consensus.cc:1081] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 LEADER]: Signalling peer e5b2e431c3204ee487c10f58d440707d to start an election
I20260709 13:06:53.684902 19901 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34322023741e46f69328b109a5b54158"
dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
 from {username='slave'} at 127.0.0.1:40122
I20260709 13:06:53.685490 19901 raft_consensus.cc:493] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:53.685828 19901 raft_consensus.cc:3060] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:53.692206 19901 raft_consensus.cc:515] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:53.694353 19901 leader_election.cc:290] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 3 election: Requested vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553)
I20260709 13:06:53.694878 19751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34322023741e46f69328b109a5b54158" candidate_uuid: "e5b2e431c3204ee487c10f58d440707d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
I20260709 13:06:53.695499 19751 raft_consensus.cc:3055] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:53.695417 19828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34322023741e46f69328b109a5b54158" candidate_uuid: "e5b2e431c3204ee487c10f58d440707d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cf68973149544eae8929005c48bb84b0"
I20260709 13:06:53.695897 19751 raft_consensus.cc:740] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7cc15333145b485ea8cdd6c1623ff0f8, State: Running, Role: LEADER
I20260709 13:06:53.696174 19828 raft_consensus.cc:3060] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:53.696799 19751 consensus_queue.cc:260] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:53.697937 19751 raft_consensus.cc:3060] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:53.702025 19828 raft_consensus.cc:2468] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5b2e431c3204ee487c10f58d440707d in term 3.
I20260709 13:06:53.703341 19867 leader_election.cc:304] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf68973149544eae8929005c48bb84b0, e5b2e431c3204ee487c10f58d440707d; no voters: 
I20260709 13:06:53.704332 20128 raft_consensus.cc:2804] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:53.704874 20128 raft_consensus.cc:697] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d [term 3 LEADER]: Becoming Leader. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Running, Role: LEADER
I20260709 13:06:53.705963 20128 consensus_queue.cc:237] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:53.707541 19751 raft_consensus.cc:2468] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5b2e431c3204ee487c10f58d440707d in term 3.
I20260709 13:06:53.716944 19650 catalog_manager.cc:5697] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d reported cstate change: term changed from 2 to 3, leader changed from 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1) to e5b2e431c3204ee487c10f58d440707d (127.6.255.3). New cstate: current_term: 3 leader_uuid: "e5b2e431c3204ee487c10f58d440707d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:53.859251 20167 raft_consensus.cc:993] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0: : Instructing follower e5b2e431c3204ee487c10f58d440707d to start an election
I20260709 13:06:53.859941 20169 raft_consensus.cc:1081] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [term 2 LEADER]: Signalling peer e5b2e431c3204ee487c10f58d440707d to start an election
I20260709 13:06:53.861545 19898 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2348879030be45d49634020eace141f0"
dest_uuid: "e5b2e431c3204ee487c10f58d440707d"
 from {username='slave'} at 127.0.0.1:40114
I20260709 13:06:53.861991 19898 raft_consensus.cc:493] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:53.862250 19898 raft_consensus.cc:3060] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:53.868146 19898 raft_consensus.cc:515] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:53.870121 19898 leader_election.cc:290] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 3 election: Requested vote from peers 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037), cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553)
I20260709 13:06:53.870857 19751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2348879030be45d49634020eace141f0" candidate_uuid: "e5b2e431c3204ee487c10f58d440707d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7cc15333145b485ea8cdd6c1623ff0f8"
I20260709 13:06:53.871469 19824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2348879030be45d49634020eace141f0" candidate_uuid: "e5b2e431c3204ee487c10f58d440707d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cf68973149544eae8929005c48bb84b0"
I20260709 13:06:53.871652 19751 raft_consensus.cc:3060] T 2348879030be45d49634020eace141f0 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:53.872210 19824 raft_consensus.cc:3055] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:53.872579 19824 raft_consensus.cc:740] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cf68973149544eae8929005c48bb84b0, State: Running, Role: LEADER
I20260709 13:06:53.873549 19824 consensus_queue.cc:260] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:53.875021 19824 raft_consensus.cc:3060] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:53.879719 19751 raft_consensus.cc:2468] T 2348879030be45d49634020eace141f0 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5b2e431c3204ee487c10f58d440707d in term 3.
I20260709 13:06:53.881136 19867 leader_election.cc:304] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cc15333145b485ea8cdd6c1623ff0f8, e5b2e431c3204ee487c10f58d440707d; no voters: 
I20260709 13:06:53.882138 20128 raft_consensus.cc:2804] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:53.882650 20128 raft_consensus.cc:697] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [term 3 LEADER]: Becoming Leader. State: Replica: e5b2e431c3204ee487c10f58d440707d, State: Running, Role: LEADER
I20260709 13:06:53.883617 19824 raft_consensus.cc:2468] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5b2e431c3204ee487c10f58d440707d in term 3.
I20260709 13:06:53.883507 20128 consensus_queue.cc:237] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } }
I20260709 13:06:53.894984 19650 catalog_manager.cc:5697] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d reported cstate change: term changed from 2 to 3, leader changed from cf68973149544eae8929005c48bb84b0 (127.6.255.2) to e5b2e431c3204ee487c10f58d440707d (127.6.255.3). New cstate: current_term: 3 leader_uuid: "e5b2e431c3204ee487c10f58d440707d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5b2e431c3204ee487c10f58d440707d" member_type: VOTER last_known_addr { host: "127.6.255.3" port: 33743 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:54.131191 19824 raft_consensus.cc:1275] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [term 3 FOLLOWER]: Refusing update from remote peer e5b2e431c3204ee487c10f58d440707d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:54.132712 20128 consensus_queue.cc:1048] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:54.146440 19751 raft_consensus.cc:1275] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 3 FOLLOWER]: Refusing update from remote peer e5b2e431c3204ee487c10f58d440707d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:54.148164 20168 consensus_queue.cc:1048] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:54.329129 19751 raft_consensus.cc:1275] T 2348879030be45d49634020eace141f0 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 3 FOLLOWER]: Refusing update from remote peer e5b2e431c3204ee487c10f58d440707d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:54.330399 20209 consensus_queue.cc:1048] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc15333145b485ea8cdd6c1623ff0f8" member_type: VOTER last_known_addr { host: "127.6.255.1" port: 46037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:54.339316 19824 raft_consensus.cc:1275] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [term 3 FOLLOWER]: Refusing update from remote peer e5b2e431c3204ee487c10f58d440707d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:54.341238 20168 consensus_queue.cc:1048] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf68973149544eae8929005c48bb84b0" member_type: VOTER last_known_addr { host: "127.6.255.2" port: 43553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:54.533680  7164 tablet_server.cc:179] TabletServer@127.6.255.1:0 shutting down...
W20260709 13:06:54.570645 19793 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 13 similar messages]
W20260709 13:06:54.574257 19793 consensus_peers.cc:597] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0 -> Peer 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037): Couldn't send request to peer 7cc15333145b485ea8cdd6c1623ff0f8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:54.576807  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:54.577682  7164 tablet_replica.cc:333] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8: stopping tablet replica
I20260709 13:06:54.578437  7164 raft_consensus.cc:2243] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.578974  7164 raft_consensus.cc:2272] T 54491751f68f4ee5b5b9000c225b906e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.581743  7164 tablet_replica.cc:333] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8: stopping tablet replica
I20260709 13:06:54.582522  7164 raft_consensus.cc:2243] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.583009  7164 raft_consensus.cc:2272] T 2a9a23c67df64d0793abcb340fe4e985 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:54.583019 19793 consensus_peers.cc:597] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0 -> Peer 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037): Couldn't send request to peer 7cc15333145b485ea8cdd6c1623ff0f8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:54.585513  7164 tablet_replica.cc:333] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8: stopping tablet replica
I20260709 13:06:54.586223  7164 raft_consensus.cc:2243] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:54.587127  7164 raft_consensus.cc:2272] T 191b9b12c85b43db99997d1560cd18da P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:54.589406 19793 consensus_peers.cc:597] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0 -> Peer 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037): Couldn't send request to peer 7cc15333145b485ea8cdd6c1623ff0f8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:54.589990  7164 tablet_replica.cc:333] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8: stopping tablet replica
I20260709 13:06:54.590648  7164 raft_consensus.cc:2243] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:54.591483  7164 raft_consensus.cc:2272] T d50ac34758d044dbbf7ec338fff5a9ac P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.594281  7164 tablet_replica.cc:333] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8: stopping tablet replica
I20260709 13:06:54.594974  7164 raft_consensus.cc:2243] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.595409  7164 raft_consensus.cc:2272] T bad2e092b8464dca8a6a73f54bf5f934 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.597364  7164 tablet_replica.cc:333] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8: stopping tablet replica
I20260709 13:06:54.597995  7164 raft_consensus.cc:2243] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.598400  7164 raft_consensus.cc:2272] T 34322023741e46f69328b109a5b54158 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.600956  7164 tablet_replica.cc:333] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8: stopping tablet replica
I20260709 13:06:54.601671  7164 raft_consensus.cc:2243] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:54.602449  7164 raft_consensus.cc:2272] T 021f87e2ac864b89a9563513db8a389d P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.604557  7164 tablet_replica.cc:333] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8: stopping tablet replica
I20260709 13:06:54.605121  7164 raft_consensus.cc:2243] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.605602  7164 raft_consensus.cc:2272] T 53bdb8e606e240dea3740473550cc59a P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.607622  7164 tablet_replica.cc:333] T 2348879030be45d49634020eace141f0 P 7cc15333145b485ea8cdd6c1623ff0f8: stopping tablet replica
I20260709 13:06:54.608219  7164 raft_consensus.cc:2243] T 2348879030be45d49634020eace141f0 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.608678  7164 raft_consensus.cc:2272] T 2348879030be45d49634020eace141f0 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.612027  7164 tablet_replica.cc:333] T 59a482b1ce05463ea3ef901049d7c1e2 P 7cc15333145b485ea8cdd6c1623ff0f8: stopping tablet replica
I20260709 13:06:54.612700  7164 raft_consensus.cc:2243] T 59a482b1ce05463ea3ef901049d7c1e2 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.613191  7164 raft_consensus.cc:2272] T 59a482b1ce05463ea3ef901049d7c1e2 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.615571  7164 tablet_replica.cc:333] T 46feb6ece88946fea8e12e1375c1e94c P 7cc15333145b485ea8cdd6c1623ff0f8: stopping tablet replica
I20260709 13:06:54.616184  7164 raft_consensus.cc:2243] T 46feb6ece88946fea8e12e1375c1e94c P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.616565  7164 raft_consensus.cc:2272] T 46feb6ece88946fea8e12e1375c1e94c P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.619493  7164 tablet_replica.cc:333] T 409f3b40dbc2457d8803d0d244488b59 P 7cc15333145b485ea8cdd6c1623ff0f8: stopping tablet replica
I20260709 13:06:54.620059  7164 raft_consensus.cc:2243] T 409f3b40dbc2457d8803d0d244488b59 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.620522  7164 raft_consensus.cc:2272] T 409f3b40dbc2457d8803d0d244488b59 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.622493  7164 tablet_replica.cc:333] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8: stopping tablet replica
I20260709 13:06:54.622983  7164 raft_consensus.cc:2243] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:54.623805  7164 raft_consensus.cc:2272] T 8b5517dc489142e19aee3610500ee38e P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.625962  7164 tablet_replica.cc:333] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8: stopping tablet replica
I20260709 13:06:54.626479  7164 raft_consensus.cc:2243] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.626864  7164 raft_consensus.cc:2272] T 5a6ea8c60bdb40fc902a427e79c155f8 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.628767  7164 tablet_replica.cc:333] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8: stopping tablet replica
I20260709 13:06:54.629257  7164 raft_consensus.cc:2243] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:54.630070  7164 raft_consensus.cc:2272] T edb7db72b50647c09980ff9b8cb188be P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.632040  7164 tablet_replica.cc:333] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8: stopping tablet replica
I20260709 13:06:54.632619  7164 raft_consensus.cc:2243] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:54.633352  7164 raft_consensus.cc:2272] T bd4bd6f0d29248aab51d9a8bdcd054a5 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.635542  7164 tablet_replica.cc:333] T 45faf2520a3e4d44bbfd74574b972b70 P 7cc15333145b485ea8cdd6c1623ff0f8: stopping tablet replica
I20260709 13:06:54.636042  7164 raft_consensus.cc:2243] T 45faf2520a3e4d44bbfd74574b972b70 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.636425  7164 raft_consensus.cc:2272] T 45faf2520a3e4d44bbfd74574b972b70 P 7cc15333145b485ea8cdd6c1623ff0f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.638542  7164 tablet_replica.cc:333] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8: stopping tablet replica
I20260709 13:06:54.638985  7164 raft_consensus.cc:2243] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:54.639811  7164 raft_consensus.cc:2272] T aa0028f6eae14e1186d8464c0c12b33f P 7cc15333145b485ea8cdd6c1623ff0f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.684417  7164 tablet_server.cc:196] TabletServer@127.6.255.1:0 shutdown complete.
W20260709 13:06:54.698351 19867 consensus_peers.cc:597] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d -> Peer 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037): Couldn't send request to peer 7cc15333145b485ea8cdd6c1623ff0f8. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:46037: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:54.724409  7164 tablet_server.cc:179] TabletServer@127.6.255.2:0 shutting down...
W20260709 13:06:54.735272 19867 consensus_peers.cc:597] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d -> Peer 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037): Couldn't send request to peer 7cc15333145b485ea8cdd6c1623ff0f8. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:46037: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:54.759080  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:54.760087  7164 tablet_replica.cc:333] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0: stopping tablet replica
I20260709 13:06:54.760838  7164 raft_consensus.cc:2243] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:54.761735  7164 raft_consensus.cc:2272] T 54491751f68f4ee5b5b9000c225b906e P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:54.761834 19793 consensus_peers.cc:597] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0 -> Peer 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037): Couldn't send request to peer 7cc15333145b485ea8cdd6c1623ff0f8. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:46037: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:54.763957  7164 tablet_replica.cc:333] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0: stopping tablet replica
I20260709 13:06:54.764667  7164 raft_consensus.cc:2243] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:54.765564  7164 raft_consensus.cc:2272] T 2a9a23c67df64d0793abcb340fe4e985 P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.767807  7164 tablet_replica.cc:333] T d50ac34758d044dbbf7ec338fff5a9ac P cf68973149544eae8929005c48bb84b0: stopping tablet replica
I20260709 13:06:54.768409  7164 raft_consensus.cc:2243] T d50ac34758d044dbbf7ec338fff5a9ac P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.768869  7164 raft_consensus.cc:2272] T d50ac34758d044dbbf7ec338fff5a9ac P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.770974  7164 tablet_replica.cc:333] T 191b9b12c85b43db99997d1560cd18da P cf68973149544eae8929005c48bb84b0: stopping tablet replica
I20260709 13:06:54.771569  7164 raft_consensus.cc:2243] T 191b9b12c85b43db99997d1560cd18da P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.771999  7164 raft_consensus.cc:2272] T 191b9b12c85b43db99997d1560cd18da P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.773937  7164 tablet_replica.cc:333] T 5a6ea8c60bdb40fc902a427e79c155f8 P cf68973149544eae8929005c48bb84b0: stopping tablet replica
I20260709 13:06:54.774544  7164 raft_consensus.cc:2243] T 5a6ea8c60bdb40fc902a427e79c155f8 P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.774981  7164 raft_consensus.cc:2272] T 5a6ea8c60bdb40fc902a427e79c155f8 P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.776995  7164 tablet_replica.cc:333] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0: stopping tablet replica
I20260709 13:06:54.777849  7164 raft_consensus.cc:2243] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.778393  7164 raft_consensus.cc:2272] T bd4bd6f0d29248aab51d9a8bdcd054a5 P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.780654  7164 tablet_replica.cc:333] T edb7db72b50647c09980ff9b8cb188be P cf68973149544eae8929005c48bb84b0: stopping tablet replica
I20260709 13:06:54.781404  7164 raft_consensus.cc:2243] T edb7db72b50647c09980ff9b8cb188be P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.781962  7164 raft_consensus.cc:2272] T edb7db72b50647c09980ff9b8cb188be P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.784267  7164 tablet_replica.cc:333] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0: stopping tablet replica
I20260709 13:06:54.784943  7164 raft_consensus.cc:2243] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:54.785807  7164 raft_consensus.cc:2272] T 53bdb8e606e240dea3740473550cc59a P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:54.786044 19793 consensus_peers.cc:597] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0 -> Peer 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037): Couldn't send request to peer 7cc15333145b485ea8cdd6c1623ff0f8. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:46037: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:54.787792  7164 tablet_replica.cc:333] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0: stopping tablet replica
I20260709 13:06:54.788426  7164 raft_consensus.cc:2243] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:54.789216  7164 raft_consensus.cc:2272] T 59a482b1ce05463ea3ef901049d7c1e2 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.791352  7164 tablet_replica.cc:333] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0: stopping tablet replica
I20260709 13:06:54.791915  7164 raft_consensus.cc:2243] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.792328  7164 raft_consensus.cc:2272] T 2348879030be45d49634020eace141f0 P cf68973149544eae8929005c48bb84b0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.795022  7164 tablet_replica.cc:333] T aa0028f6eae14e1186d8464c0c12b33f P cf68973149544eae8929005c48bb84b0: stopping tablet replica
I20260709 13:06:54.795632  7164 raft_consensus.cc:2243] T aa0028f6eae14e1186d8464c0c12b33f P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.796084  7164 raft_consensus.cc:2272] T aa0028f6eae14e1186d8464c0c12b33f P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.798286  7164 tablet_replica.cc:333] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0: stopping tablet replica
I20260709 13:06:54.798823  7164 raft_consensus.cc:2243] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.799309  7164 raft_consensus.cc:2272] T 409f3b40dbc2457d8803d0d244488b59 P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.801319  7164 tablet_replica.cc:333] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0: stopping tablet replica
I20260709 13:06:54.801949  7164 raft_consensus.cc:2243] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:54.802783  7164 raft_consensus.cc:2272] T 46feb6ece88946fea8e12e1375c1e94c P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.804962  7164 tablet_replica.cc:333] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0: stopping tablet replica
I20260709 13:06:54.805539  7164 raft_consensus.cc:2243] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:54.806259  7164 raft_consensus.cc:2272] T 45faf2520a3e4d44bbfd74574b972b70 P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.808315  7164 tablet_replica.cc:333] T 8b5517dc489142e19aee3610500ee38e P cf68973149544eae8929005c48bb84b0: stopping tablet replica
I20260709 13:06:54.808830  7164 raft_consensus.cc:2243] T 8b5517dc489142e19aee3610500ee38e P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.809228  7164 raft_consensus.cc:2272] T 8b5517dc489142e19aee3610500ee38e P cf68973149544eae8929005c48bb84b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.811188  7164 tablet_replica.cc:333] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0: stopping tablet replica
I20260709 13:06:54.811751  7164 raft_consensus.cc:2243] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:54.812456  7164 raft_consensus.cc:2272] T bad2e092b8464dca8a6a73f54bf5f934 P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.814517  7164 tablet_replica.cc:333] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0: stopping tablet replica
I20260709 13:06:54.815037  7164 raft_consensus.cc:2243] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.815474  7164 raft_consensus.cc:2272] T 021f87e2ac864b89a9563513db8a389d P cf68973149544eae8929005c48bb84b0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.817612  7164 tablet_replica.cc:333] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0: stopping tablet replica
I20260709 13:06:54.818132  7164 raft_consensus.cc:2243] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.818663  7164 raft_consensus.cc:2272] T 34322023741e46f69328b109a5b54158 P cf68973149544eae8929005c48bb84b0 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:54.832484 19867 consensus_peers.cc:597] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d -> Peer cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553): Couldn't send request to peer cf68973149544eae8929005c48bb84b0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:54.884704  7164 tablet_server.cc:196] TabletServer@127.6.255.2:0 shutdown complete.
I20260709 13:06:54.919721  7164 tablet_server.cc:179] TabletServer@127.6.255.3:0 shutting down...
I20260709 13:06:54.947930  7164 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:54.948884  7164 tablet_replica.cc:333] T 54491751f68f4ee5b5b9000c225b906e P e5b2e431c3204ee487c10f58d440707d: stopping tablet replica
I20260709 13:06:54.949847  7164 raft_consensus.cc:2243] T 54491751f68f4ee5b5b9000c225b906e P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.950418  7164 raft_consensus.cc:2272] T 54491751f68f4ee5b5b9000c225b906e P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.953150  7164 tablet_replica.cc:333] T 2a9a23c67df64d0793abcb340fe4e985 P e5b2e431c3204ee487c10f58d440707d: stopping tablet replica
I20260709 13:06:54.954077  7164 raft_consensus.cc:2243] T 2a9a23c67df64d0793abcb340fe4e985 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:54.954427 19867 consensus_peers.cc:597] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d -> Peer cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553): Couldn't send request to peer cf68973149544eae8929005c48bb84b0. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:43553: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:54.954628  7164 raft_consensus.cc:2272] T 2a9a23c67df64d0793abcb340fe4e985 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.956975  7164 tablet_replica.cc:333] T 191b9b12c85b43db99997d1560cd18da P e5b2e431c3204ee487c10f58d440707d: stopping tablet replica
I20260709 13:06:54.957643  7164 raft_consensus.cc:2243] T 191b9b12c85b43db99997d1560cd18da P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.958098  7164 raft_consensus.cc:2272] T 191b9b12c85b43db99997d1560cd18da P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.960213  7164 tablet_replica.cc:333] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d: stopping tablet replica
I20260709 13:06:54.960777  7164 raft_consensus.cc:2243] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.961215  7164 raft_consensus.cc:2272] T d50ac34758d044dbbf7ec338fff5a9ac P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.963346  7164 tablet_replica.cc:333] T 8b5517dc489142e19aee3610500ee38e P e5b2e431c3204ee487c10f58d440707d: stopping tablet replica
I20260709 13:06:54.964171  7164 raft_consensus.cc:2243] T 8b5517dc489142e19aee3610500ee38e P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.964731  7164 raft_consensus.cc:2272] T 8b5517dc489142e19aee3610500ee38e P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.967478  7164 tablet_replica.cc:333] T bad2e092b8464dca8a6a73f54bf5f934 P e5b2e431c3204ee487c10f58d440707d: stopping tablet replica
I20260709 13:06:54.968127  7164 raft_consensus.cc:2243] T bad2e092b8464dca8a6a73f54bf5f934 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.968597  7164 raft_consensus.cc:2272] T bad2e092b8464dca8a6a73f54bf5f934 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.970932  7164 tablet_replica.cc:333] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d: stopping tablet replica
I20260709 13:06:54.971474  7164 raft_consensus.cc:2243] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:54.972280  7164 raft_consensus.cc:2272] T 34322023741e46f69328b109a5b54158 P e5b2e431c3204ee487c10f58d440707d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.974766  7164 tablet_replica.cc:333] T 021f87e2ac864b89a9563513db8a389d P e5b2e431c3204ee487c10f58d440707d: stopping tablet replica
I20260709 13:06:54.975301  7164 raft_consensus.cc:2243] T 021f87e2ac864b89a9563513db8a389d P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.975733  7164 raft_consensus.cc:2272] T 021f87e2ac864b89a9563513db8a389d P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.977790  7164 tablet_replica.cc:333] T 53bdb8e606e240dea3740473550cc59a P e5b2e431c3204ee487c10f58d440707d: stopping tablet replica
I20260709 13:06:54.978304  7164 raft_consensus.cc:2243] T 53bdb8e606e240dea3740473550cc59a P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.978701  7164 raft_consensus.cc:2272] T 53bdb8e606e240dea3740473550cc59a P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.980659  7164 tablet_replica.cc:333] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d: stopping tablet replica
I20260709 13:06:54.981177  7164 raft_consensus.cc:2243] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:54.982193  7164 raft_consensus.cc:2272] T 2348879030be45d49634020eace141f0 P e5b2e431c3204ee487c10f58d440707d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.985807  7164 tablet_replica.cc:333] T 59a482b1ce05463ea3ef901049d7c1e2 P e5b2e431c3204ee487c10f58d440707d: stopping tablet replica
I20260709 13:06:54.986516  7164 raft_consensus.cc:2243] T 59a482b1ce05463ea3ef901049d7c1e2 P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:54.986863 19867 consensus_peers.cc:597] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d -> Peer cf68973149544eae8929005c48bb84b0 (127.6.255.2:43553): Couldn't send request to peer cf68973149544eae8929005c48bb84b0. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.2:43553: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:54.987001  7164 raft_consensus.cc:2272] T 59a482b1ce05463ea3ef901049d7c1e2 P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.989372  7164 tablet_replica.cc:333] T 46feb6ece88946fea8e12e1375c1e94c P e5b2e431c3204ee487c10f58d440707d: stopping tablet replica
I20260709 13:06:54.990015  7164 raft_consensus.cc:2243] T 46feb6ece88946fea8e12e1375c1e94c P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.990447  7164 raft_consensus.cc:2272] T 46feb6ece88946fea8e12e1375c1e94c P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.992987  7164 tablet_replica.cc:333] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d: stopping tablet replica
I20260709 13:06:54.993827  7164 raft_consensus.cc:2243] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:54.994715  7164 raft_consensus.cc:2272] T 409f3b40dbc2457d8803d0d244488b59 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:54.995695 19867 consensus_peers.cc:597] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d -> Peer 7cc15333145b485ea8cdd6c1623ff0f8 (127.6.255.1:46037): Couldn't send request to peer 7cc15333145b485ea8cdd6c1623ff0f8. Status: Network error: Client connection negotiation failed: client connection to 127.6.255.1:46037: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:54.997013  7164 tablet_replica.cc:333] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d: stopping tablet replica
I20260709 13:06:54.997726  7164 raft_consensus.cc:2243] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.998188  7164 raft_consensus.cc:2272] T aa0028f6eae14e1186d8464c0c12b33f P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:55.000192  7164 tablet_replica.cc:333] T 45faf2520a3e4d44bbfd74574b972b70 P e5b2e431c3204ee487c10f58d440707d: stopping tablet replica
I20260709 13:06:55.000762  7164 raft_consensus.cc:2243] T 45faf2520a3e4d44bbfd74574b972b70 P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:55.001187  7164 raft_consensus.cc:2272] T 45faf2520a3e4d44bbfd74574b972b70 P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:55.003383  7164 tablet_replica.cc:333] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d: stopping tablet replica
I20260709 13:06:55.003885  7164 raft_consensus.cc:2243] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:55.004665  7164 raft_consensus.cc:2272] T 5a6ea8c60bdb40fc902a427e79c155f8 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:55.006795  7164 tablet_replica.cc:333] T edb7db72b50647c09980ff9b8cb188be P e5b2e431c3204ee487c10f58d440707d: stopping tablet replica
I20260709 13:06:55.007720  7164 raft_consensus.cc:2243] T edb7db72b50647c09980ff9b8cb188be P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:55.008186  7164 raft_consensus.cc:2272] T edb7db72b50647c09980ff9b8cb188be P e5b2e431c3204ee487c10f58d440707d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:55.010226  7164 tablet_replica.cc:333] T bd4bd6f0d29248aab51d9a8bdcd054a5 P e5b2e431c3204ee487c10f58d440707d: stopping tablet replica
I20260709 13:06:55.010694  7164 raft_consensus.cc:2243] T bd4bd6f0d29248aab51d9a8bdcd054a5 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:55.011129  7164 raft_consensus.cc:2272] T bd4bd6f0d29248aab51d9a8bdcd054a5 P e5b2e431c3204ee487c10f58d440707d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:55.054739  7164 tablet_server.cc:196] TabletServer@127.6.255.3:0 shutdown complete.
I20260709 13:06:55.090103  7164 master.cc:562] Master@127.6.255.62:37417 shutting down...
I20260709 13:06:55.112139  7164 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:55.112665  7164 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:55.112970  7164 tablet_replica.cc:333] T 00000000000000000000000000000000 P 1740df38a6644ce6aa8ad917ff3a7eef: stopping tablet replica
I20260709 13:06:55.134472  7164 master.cc:584] Master@127.6.255.62:37417 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (12123 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (25816 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (368964 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 13:06:55.195638  7164 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 414 messages since previous log ~15 seconds ago
I20260709 13:06:55.195866  7164 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 413 messages since previous log ~15 seconds ago
I20260709 13:06:55.196020  7164 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 411 messages since previous log ~15 seconds ago
I20260709 13:06:55.196197  7164 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~35 seconds ago
I20260709 13:06:55.196353  7164 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~35 seconds ago
I20260709 13:06:55.196540  7164 logging.cc:436] LogThrottler Service Queue Overflow: suppressed but not reported on 2 messages since previous log ~235 seconds ago
I20260709 13:06:55.196707  7164 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 4 messages since previous log ~132 seconds ago
I20260709 13:06:55.196931  7164 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 10 messages since previous log ~0 seconds ago
I20260709 13:06:55.197239  7164 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~11 seconds ago
